Building a distributed app with PHP and Net_Gearman, part 4

In this installment, we’ll cover more worker callbacks, and we’ll pull everything together into a robust client/server application.

First, if you haven’t already, download the source for the examples in this tutorial

7. More worker callbacks

The monitor callback gives us some useful information about how our jobs are running.  You could even do interesting things like have your worker bail out if it’s idle for some interval of time.  But you can’t easily distinguish between job starts and job completions.  So, for instance, if you wanted to have your worker exit after processing a fixed number of jobs, you would have a difficult time doing that.

You can attach specific callbacks to job start, complete, and fail.  Let’s look at the relevant lines in gm_worker7.php:

The parameters passed to the callbacks are as follows:

  • $handle – an identifier for the job within the gearman server (e.g., “H:localhost:399”)
  • $job – a string representing the job class that was called (e.g., “Example2”)
  • $args – the argument array passed to your job class’s run() function
  • $result – the same $result array passed to your client’s complete callback (an array with the result of the job class’s run() function; note that if the run() function does not return an array, the value that it does return will be put into a hashed array under the key ‘result’)
  • $error – if the job class’s run() function throws a Net_Gearman_Job_Exception, your fail callback will be called, and the exception will be passed in this parameter

Obviously, you can get a lot more information about your jobs via these callbacks (although there’s no idle callback, so to track the process during idle times, you’ll also need the monitor callback).

Run gm_worker7.php and gm_client7.php to see these callbacks in action.

8. Putting it all together

Our team has made massive investments in PHP, and we have a great deal of admiration for the technology.  But I’m not brave (or reckless) enough to run a PHP script as a true “always on” daemon.  There are just too many things that could go wrong in a scenario like that: memory or resource leaks, crashes, etc.  So I prefer to run PHP scripts for limited durations.

With the right set of callbacks, we can have our worker process service a fixed number of jobs and then exit.  A cron job can attempt to start a worker every minute (or even more frequently); if one is already running, the new process will exit immediately.  This keeps a worker available more or less all the time without the risks associated with an always-on process.

Have a look at gm_worker8.php, where we pull all these pieces together.  We process command-line arguments to set the number of jobs to run before exiting and  an instance number, which allows us to run more than one worker on a given machine, useful for multi-core worker hosts.

Part 1 · Part 2 · Part 3

Leave a comment

Your email address will not be published. Required fields are marked *