Sixteen minutes. Three bugs. Zero errors.
Or: How Symfony Messenger convinced us everything was fine while doing absolutely nothing
We deployed a Symfony Messenger worker to production, watched it start, saw the
familiar [OK] Consuming messages from transport "bulk_operations" in the logs,
and moved on. Messages piled up. The worker sat idle. No exceptions, no
warnings, no failed jobs. Just silence.
What followed was a 16-minute debugging session that uncovered three independent bugs, each one invisible in its own way. Here’s the timeline.
TL;DR: Three bugs, three silent failures. (1) auto_setup=0 in the DSN
meant the messenger_messages table was never created — the worker logs OK
anyway. (2) use_notify: true is PostgreSQL-only; MySQL ignores it and never
falls back to polling. (3) PHP-FPM wrote Europe/Berlin timestamps while PHP
CLI read them as UTC, delaying every background job by exactly one hour.
The Setup
Permalink to "The Setup"The environment: Symfony 6.2, PHP 8.1, MySQL 8.0 on a bare-metal Ubuntu server. All findings below are specific to this stack — behaviour may differ across Symfony versions or with the Redis/AMQP transports. The background worker runs as a systemd service (a standard Linux mechanism for managing long-running background processes):
# /etc/systemd/system/messenger.service
ExecStart=/usr/bin/php /var/www/app/bin/console messenger:consume bulk_operations --time-limit=3600 --memory-limit=128M
User=www-data
The transport uses Doctrine:
# config/packages/messenger.yaml
framework:
messenger:
transports:
bulk_operations:
dsn: "%env(MESSENGER_TRANSPORT_DSN)%"
options:
use_notify: true
check_delayed_interval: 60000
And the DSN in .env:
MESSENGER_TRANSPORT_DSN=doctrine://default?auto_setup=0
Three configuration decisions. Three bugs. Let’s walk through them in the order we found them.
Why Does Messenger Fail With a Missing ‘messenger_messages’ Table?
Permalink to "Why Does Messenger Fail With a Missing ‘messenger_messages’ Table?"The messenger_messages table didn’t exist.
The DSN had auto_setup=0, which tells Symfony’s Doctrine transport not to
create its queue table automatically. This is a reasonable production setting —
you don’t want your app creating database tables on the fly. But it assumes
you’ve created the table yourself. We hadn’t.
Here’s the thing: messenger_messages isn’t a Doctrine entity. There’s no
migration for it, no entity class, nothing in src/Entity/. It’s an internal
transport table that Symfony manages outside your ORM layer. If you don’t let
the transport create it and you don’t create it manually, it simply doesn’t
exist.
The worker’s response to a missing table?
[OK] Consuming messages from transport "bulk_operations". No error. No
warning. Just a green checkmark and infinite patience.
The fix was two lines:
# Remove auto_setup=0 from the DSN
# MESSENGER_TRANSPORT_DSN=doctrine://default?auto_setup=0
MESSENGER_TRANSPORT_DSN=doctrine://default
# Create the table immediately
php bin/console messenger:setup-transports
The command confirmed both bulk_operations and failed transports were set
up. Verify with SHOW TABLES LIKE 'messenger_messages' — you should see one
row. Table created, first bug squashed. We dispatched three test messages from
the admin UI.
Nothing happened.
What Happens When ‘use_notify: true’ Is Set on MySQL?
Permalink to "What Happens When ‘use_notify: true’ Is Set on MySQL?"The use_notify: true option enables PostgreSQL’s LISTEN/NOTIFY mechanism — a
push-based system where the database notifies the worker when new messages
arrive. It’s efficient and eliminates polling.
Our production database is MySQL.
MySQL doesn’t support LISTEN/NOTIFY. When you set use_notify: true on a
MySQL-backed transport, Symfony doesn’t throw an error, doesn’t log a warning,
doesn’t fall back to polling. The worker starts, reports OK, and waits for
notifications that will never come.
This was a leftover from an earlier stage of the project when the database was PostgreSQL. The migration to MySQL happened, the transport config didn’t follow.
The fix: remove the entire options block.
# config/packages/messenger.yaml
framework:
messenger:
transports:
bulk_operations:
dsn: "%env(MESSENGER_TRANSPORT_DSN)%"
Without use_notify, Symfony falls back to standard database polling. We
cleared the cache, restarted the worker, dispatched three more messages. Running
messenger:consume --limit=1 -vvv now at least showed the consumer querying the
database — so polling was working.
Still nothing.
How Does a PHP Timezone Mismatch Silently Delay Messenger Jobs?
Permalink to "How Does a PHP Timezone Mismatch Silently Delay Messenger Jobs?"This one took the longest to find and gave us a false positive along the way.
We ran a manual consumer with verbose output:
sudo -u www-data timeout 10 php bin/console messenger:consume bulk_operations --limit=1 -vvv
Zero messages consumed. No output at all — not even “no messages found.” We checked the database:
SELECT id, queue_name, available_at, delivered_at
FROM messenger_messages
WHERE delivered_at IS NULL;
Six messages sitting there, all with delivered_at = NULL. The queue_name was
default — which looked suspicious but turned out to be correct (Symfony’s
Doctrine transport matches on this by default). What caught our eye was
available_at:
available_at: 2026-02-27 14:30:20
Server time: 13:30 UTC. The messages were timestamped one hour in the future.
Two PHPs, Two Timezones
Permalink to "Two PHPs, Two Timezones"PHP on this server runs in two modes with two separate configurations:
# PHP-FPM (serves web requests)
# /etc/php/8.1/fpm/php.ini
date.timezone = Europe/Berlin
# PHP CLI (runs the worker)
# /etc/php/8.1/cli/php.ini
;date.timezone =
# Commented out → defaults to UTC
When a user dispatches a message through the web interface, PHP-FPM stamps
available_at with the current time in Europe/Berlin (CET, UTC+1). At 14:30
CET, that’s 2026-02-27 14:30:20. MySQL stores this in a datetime column — no
timezone information attached.
The worker runs via PHP CLI in UTC. It queries:
SELECT * FROM messenger_messages
WHERE available_at <= NOW()
AND delivered_at IS NULL
ORDER BY available_at ASC
LIMIT 1;
NOW() returns 13:30 (UTC). The message says 14:30. The worker skips it.
Every message waits exactly one hour before becoming “available.”
The False Positive
Permalink to "The False Positive"Here’s where we fooled ourselves. To test the theory, we manually adjusted the timestamps:
UPDATE messenger_messages
SET available_at = DATE_SUB(available_at, INTERVAL 1 HOUR)
WHERE delivered_at IS NULL;
Between our test runs, the background systemd worker — still running — picked up those time-corrected messages. We saw the messages disappear from the table and thought: “It’s working now.”
It wasn’t. A user dispatched a new bulk job minutes later. Same problem. The
available_at timestamp was again one hour ahead. Our manual UPDATE had fixed
the symptoms for six messages while the root cause kept producing new ones.
The Actual Fix
Permalink to "The Actual Fix"# Uncomment and set the timezone
sudo sed -i 's/^;date.timezone =$/date.timezone = Europe\/Berlin/' /etc/php/8.1/cli/php.ini
# Restart the worker (picks up new php.ini)
sudo systemctl restart messenger
Two pending messages were consumed within seconds.
You can verify the CLI timezone without restarting anything:
sudo -u www-data php -r "echo date_default_timezone_get();"
# Before: UTC
# After: Europe/Berlin
Long-term: The more robust fix is to standardize on UTC across the entire
stack — PHP-FPM, PHP CLI, and the MySQL server timezone. When everything speaks
UTC, the FPM/CLI disparity disappears as a bug category. Set
date.timezone = UTC in both php.ini files and default-time-zone = UTC in
my.cnf, and this class of bug can’t happen again.
The Pattern: Silence as Failure Mode
Permalink to "The Pattern: Silence as Failure Mode"All three bugs share a trait: Symfony Messenger reported success while doing
nothing. The worker’s [OK] message is a startup confirmation, not a health
check. It means “I’m listening,” not “I can hear anything.” This pattern isn’t
unique to this component — we ran into the same silence with a
Redis lock expiration bug in the Crunz scheduler,
a completely different component with the same failure mode: green output, no
work done.
If this is your first production deployment, run through this checklist:
-
Verify the table exists. Run
php bin/console messenger:setup-transportsand check withSHOW TABLES LIKE 'messenger_messages'. Don’t rely onauto_setup— be explicit. -
Match transport options to your database.
use_notifyis PostgreSQL-only. If you migrated databases, audit yourmessenger.yaml. -
Unify PHP timezones. Check both FPM and CLI configs. They’re separate
php.inifiles with separate defaults. A one-hour offset won’t throw an error — it’ll just delay every message. -
Test with verbose output.
messenger:consume --limit=1 -vvvshould show the message being received and handled. If it produces no output, the problem is upstream of your handler. -
Compare timestamps. After dispatching a test message, immediately check
available_atagainstSELECT NOW(). They should be within seconds of each other.
How to Monitor for Silent Messenger Failures
Permalink to "How to Monitor for Silent Messenger Failures"The [OK] startup message is useless as a health signal. To actually catch a
stuck queue, watch two metrics:
Queue depth — how many async jobs are waiting:
SELECT COUNT(*) AS queue_depth
FROM messenger_messages
WHERE delivered_at IS NULL;
Age of oldest pending message — how long the queue has been stuck:
SELECT MIN(available_at) AS oldest_pending
FROM messenger_messages
WHERE delivered_at IS NULL;
Alert if queue depth grows past a threshold, or if oldest_pending is more than
a few minutes in the past. A queue that grows but never shrinks means the worker
is broken, not busy.
Symfony Messenger’s failure mode is silence. The worker starts, logs a green checkmark, and waits. Whether it’s waiting for a table that doesn’t exist, notifications from a database that can’t send them, or timestamps that haven’t arrived yet — the output is the same. Your monitoring won’t catch it. Your logs won’t show it. The only signal is the absence of work getting done.
