Workers

Processes kept running beside your site, for as long as the environment is up: a queue consumer, a scheduler, a listener on a stream — anything that is neither a request nor a scheduled command. A worker is your application started on a different command: the same image at the same version, the same release, the same variables and the same settings as the site. It has no port and no hostname, and nothing routes to it.

Nothing processes your jobs until vallic.yaml declares a worker that does. The platform starts it, keeps it running, and restarts it on every deploy.

Declaring one

workers:
  - name: queue
    command: 'php bin/console messenger:consume async --time-limit=3600'
    replicas: 2

• name — lowercase letters, digits and hyphens, starting with a letter, and different for each worker. It names the containers: worker-queue-1, worker-queue-2.
• command — one line, run in the release at /var/www/html/current. It runs under sh, so $VALLIC_PRIVATE_DIR and your other variables expand in it. Your framework's own command, a script, a binary your build made, a shell loop: whatever your application runs.
• replicas — how many copies run, from 1 to 16. Leave it out for one.

The list is read at the commit being deployed and applied by that deploy, like everything else in the file. A worker added starts; one whose command changed starts again on the new one; one taken out of the file is stopped. Raising replicas adds copies beside the ones running, and lowering it stops the highest-numbered.

What it runs in

Your application's image, at its version. On a PHP project that is the PHP-FPM image, at the PHP your runtime names; on Node.js, Bun, Python, Ruby, Go or Rust, that application's image. The tools your build put in the release are on PATH, as they are in the site's container, and what the running image does not carry is not there either: a Go or Rust worker runs the binary you built, not go run or cargo run.

The site's variables and settings. Your team's variables and the platform's reach a worker as they reach the site, and so does every setting you give the application — on PHP, the memory limit and the rest of what you set on php. A worker given less than the site would be the one that fails. Applying changed variables restarts the workers with the site. An APM agent your project uses (Monitoring) traces the workers as it traces the site.

The site's files. The release is read-only, as it is for the site, and a worker writes where the site does: under /mnt/files, and its own log files in /var/log/app. What it prints is collected with the rest of your logs.

Kept running

If it exits, it is started again — whether it finished, failed or was killed. That is what makes a time limit safe: messenger:consume --time-limit=3600 exits after an hour and starts again fresh, without whatever it had been holding on to.

A command that stops when the work runs out needs something to wait on. Drush's queue:run returns as soon as its queue is empty, so on its own it is started over and over. Loop it, with a pause:

workers:
  - name: imports
    command: 'while true; do drush queue:run imports --time-limit=300; sleep 30; done'

A deploy does not wait on a worker. It checks the site, at its health path. A worker that cannot start says why in its logs, and one that was not running when its machine last reported is named under that machine, with its state, on the environment's overview.

Deploys restart them

Every deploy restarts every worker on the new release once it is live, so no worker goes on running code a deploy has replaced. Nothing like queue:restart or horizon:terminate is needed.

• On the machine that deploys first, workers restart before the deploy steps run: the release is live before the steps, for the workers as for the site.
• Every other machine, a worker machine included, takes the release after that first machine has run the steps.
• A deploy that rolls back puts the workers back on the release it returns to.

A worker has ten seconds to stop. A deploy, and applying variables, each stop it and start it again. Whatever it is in the middle of when the ten seconds run out is cut off, so a job should be safe to run again from the start.

How many, and where

One declared worker per machine that can run one. A long-running process competes with the site for the CPU that answers requests, so an environment has room for as many different workers as it has worker machines, or, without any, as many as it has web servers — never fewer than one. replicas does not count against it. A file that declares more refuses the deploy, and says so.

Staging, development and preview environments have room for one. They run on one machine, whatever production runs, so a second worker that production has room for is one they refuse. Fold related work into one worker instead: one consumer can read several queues — queue:work --queue=high,default, messenger:consume async high, Celery's -Q high,default.

Every machine that runs your workers runs all of them, replicas copies on each:

The environment has	Your workers run on
One machine	that machine, beside the site
A worker machine	the worker machine, and not the web machine
Two or more web servers and no worker machine	every web server
Two or more worker machines	every worker machine

The last two are the dedicated shape, where replicas: 2 is two copies on each of those machines. Consumers share a queue however many of them there are. A scheduler does not: two copies run every task twice. Keep a schedule in cron, which runs on one machine however many the environment has, or run it as a worker only where one machine runs your workers.

A machine for them

Without a worker machine, workers share a machine with the site, and a busy one shows up as a slow site rather than a slow job. A worker machine takes every worker off the web machine — they move, rather than split — and is a machine like any other: you choose its size, and it is billed as one. See Shapes.

• Flexible, in production: the environment's Resources tab offers Workers among what can move to a machine of its own, once a deploy has recorded some, and can bring them back to the web machine. See Upgrades.
• Dedicated, in production: the same tab sets how many worker machines there are, as it does web servers — up to eight, each the size of the others.
• A shared plan or a bare-metal server has no second machine to give them.

Giving them more room. A worker machine is resized like any other, and an upscale for seven days can reach it; autoscale moves only the machines that serve a page. Each copy of a worker takes the memory a copy of your application takes — on PHP, up to its memory limit — and the configurator's figures cannot know what your code holds, so leave room for every copy when you choose a size.

Where the jobs wait

A worker takes its work from a queue, which is either a service of its own or the database you already run:

• The database — Laravel's database driver, Symfony Messenger's Doctrine transport, Rails' Solid Queue — needs nothing added.
• Valkey or Redis keeps nothing on disk by default, so a restart empties the queue. Set it to store for a queue — see Keeping its data — and keep the queue on a database number of its own, apart from the cache: clearing a Redis cache empties its whole database, queue and all.
• RabbitMQ is a broker, with its queues and messages kept on disk.

Name the one you use under services in vallic.yaml.

Framework by framework

Project	Worker command	More
Laravel	php artisan queue:work, or php artisan horizon	The queue
Symfony	php bin/console messenger:consume async --time-limit=3600	Messenger
Drupal	drush queue:run <queue>, in a loop as above	—
Django and Python	python -m celery -A mysite worker, RQ, Dramatiq	Django, Python
Rails	bin/jobs for Solid Queue, bundle exec sidekiq	Rails
Node.js, Go, Rust	your own: node worker.js, bin/worker	Node.js

• Laravel: queue:work, not queue:listen. Running the scheduler as a worker, php artisan schedule:work, takes the place of the platform's schedule:run cron, which is then not added — and is a scheduler, so the rule above applies. The PHP image has pcntl, which queue:work needs to time a job out and Horizon needs to run.
• Python: run it as python -m …, so it is the release's own Python and packages that run it.
• Go and Rust: the worker is your binary; nothing is compiled where it runs.

Getting into one

Each copy has a container of its own, worker-<name>-<n>, on the machine that runs it. Open it from the CLI:

vallic ssh production --container worker-queue-1 -- php bin/console messenger:failed:show

or with ssh, as @worker-queue-1. The environment's Shell access card has the lines for yours. See Shell access.

When a deploy refuses

Message	What to do
"workers" should be a list of processes to keep running.	Start each worker with - name:
Each entry under "workers" needs a name and a command.	Add whichever is missing
Worker name "Queue" should be lowercase letters, digits and hyphens, starting with a letter.	Rename it
There is more than one worker called "queue".	Give each its own name
"workers.command" should be a single line of text.	Put it on one line, or in a script your release ships
Worker "queue" asks for 20 copies; between 1 and 16.	Ask for 16 or fewer
declares 2 worker(s) and this environment has room for 1	Add a worker machine or a web server, or fold the work into one worker

Next

• vallic.yaml — cron, and the rest of the file
• Shapes — a machine for background work
• Deployments — what a deploy does, in order