Laravel Job and Queue

This post shows the steps on how to run queue in database using background worker process.

  • Useful tutorials:
  • Queue Connections
    • In env file, there’s a variable called QUEUE_CONNECTION. If it is set to `sync’,  any jobs scheduled will be executed as soon as they are dispatched.  This is done synchronously.  This needs to be changed to QUEUE_CONNECTION=database
    • Don’t forget to run php artisan config:cache after for it to take effect if the env is cached
  • Some jobs running synchronous, some running async:
    • UsedispatchNow for synchronous jobs, use dispatch for async jobs.
  • Create a job
    php artisan make:job JobName

    A module will be created in

    App\Jobs\JobName
  • Add what you want to accomplish by the job to the handle() function of the generated module
  • Queue a job in database, add this to the .env file and restart the server
    QUEUE_CONNECTION=database
  • Create queue table in the database:
    php artisan queue:table

    then run migration

    php artisan migrate

    Jobs will queue in this table

  • To start a process that performs a task using the job information in the queue table
    php artisan queue:work
  • Multiple processes can take jobs from the queue and run in parallel. Just call
    php artisan queue:work

    again from another cmd or terminal.  For more processes, here’s an sample bash script

    for i in {1..20} do
        artisan do-command $i &
    done
  • queue:work also takes a –tries option that will indicate how many time the process will try before it was declared failed. Default value for tries is 1.
  • Failed jobs go to failed job table. The table must be created as follows:
    php artisan queue:failed-table

    Don’t forget to run migration.

  • Retry all jobs in failed-table:
    php artisan queue:retry all
  • Please note that queue workers are long-lived processes and store the booted application state in memory. As a result, they will not notice changes in your codebase after they have been started. So, during your deployment process, be sure to restart your queue workers.
  • To avoid having to restart after each code change:
    php artisan queue:listen

According to Laravel documentation:  Alternatively, you may run the queue:listen command. When using the queue:listen command, you don’t have to manually restart the worker when you want to reload your updated code or reset the application state; however, this command is significantly less efficient than the queue:work command: