Skip to main content
ArticlesProjects

Queues you can pause and buckets you can walk away from

Two weeks of Laravel 13: a read-through filesystem driver for zero-downtime storage migrations, a global queue pause switch, debounced listeners, and Redis cluster fixes worth upgrading for.

Steve McDougallAug 20269 min read

Two things landed in the last fortnight that change how you handle a specific kind of bad afternoon. The first is the afternoon you decide to move ten million files from S3 to somewhere cheaper. The second is the afternoon a downstream API starts returning garbage and every queue worker you own is cheerfully processing jobs into it. Both of those used to be problems you solved with your own scaffolding. Neither is now.

Everything else in v13.25.0, v13.26.0 and the 12.x backports is smaller, and I’ll get to the parts that are worth your attention. But those two are the ones that alter a decision you might make this month.

Migrating object storage without a migration

Moving buckets is one of those jobs where the difficulty is entirely in the tail. Copying the files is easy, you run a sync and wait. The hard part is the window: new uploads landing in the old bucket while the sync runs, references in the database pointing at a location that may or may not have been copied yet, and the fact that you cannot flip the switch until you are certain every last object made it across. So most teams end up writing a resolver that tries the new disk, falls back to the old one, and lives in the codebase forever because nobody is ever confident enough to delete it.

Taylor’s read-through filesystem in v13.26.0 is that resolver, in the framework, with the fallback logic already written. You configure a disk with a destination and a legacy source. Writes go to the destination. Reads check the destination first, and when the file isn’t there they fetch it from the source and promote it, so the object copies itself across on first access. The bucket drains as your users touch it, and the day the fallback stops being hit is the day you can remove it.

Aaron Francis wrote it up properly on the Laravel blog in Object storage migrations with Laravel’s read-through filesystem, and I’d go there for the configuration rather than take my word for the key names. What I want to flag is the follow-up PR, 61155, which lets a read-through disk serve from the source without copying. That sounds like a footnote and isn’t. Copy-on-read is the right default for a migration you intend to finish, but it’s the wrong behaviour if you’re reading from an archive you have no intention of duplicating, or if you’re pointing at a shared source that several applications read and none of them own. Having both modes is what makes this a filesystem feature rather than a one-off migration tool.

The cost is worth naming. A read that misses the destination is now two network round trips plus a write, and the promotion happens inline on the request that triggered it. If your legacy bucket is in another region, some unlucky user pays for that. It’s still a better trade than a maintenance window, but it isn’t free, and it’s the reason you want the migration to actually finish rather than leaving the fallback in place for two years.

A pause switch for queues

The other one is the global pause switch for queues in v13.25.0, with the supporting work in v13.26.0 that surfaces paused queues in the worker output and avoids cross-slot reads when checking them on a Redis cluster.

Until now, stopping work meant stopping workers. You either scaled the deployment to zero, ran queue:restart and hoped, or you built a feature flag that every job checked at the top of handle() and released itself on. All three are bad in the same way: the pause is somewhere other than the queue, so whether it takes effect depends on something you have to reason about separately from the thing you’re trying to stop.

A pause that the worker itself honours means the workers stay up, the jobs stay queued, and the backlog is visible in the place you’d normally look at a backlog. That matters more than it sounds. Killing workers during an incident gives you a system in an unfamiliar state at the exact moment you want fewer unknowns, and it makes the recovery a deploy rather than a toggle.

The release notes are terse and I’m not going to guess at the exact API surface from a PR title, so check the queue documentation for the calling convention. The behaviour is the interesting part: the worker knows, and it tells you it knows.

Also in this area, Queue::forward() arrived in 61188 and the note is a single line. I don’t know what it does well enough to explain it to you, so I’ll leave it rather than invent something plausible. There’s also a UniqueJobSkipped event, a JobReleased event on the worker, the timeout value attached to JobTimedOut, and a UUID on faked jobs for inspection. Individually dull, collectively the sign of someone systematically making the queue observable from the outside instead of from log lines.

Debounced listeners

Steve Bauman’s debounceable queued listeners is the change I expect to reach for soonest. The pattern it addresses is one everybody has hand-rolled: a model gets touched forty times in a burst, each touch fires an event, and the listener rebuilds a search index or busts a cache forty times when once would have done. The usual workarounds are a unique job with a short TTL, or a dispatch delay plus a cache key you check on the way in, and neither reads like what you meant.

Having it as a property of the listener puts the decision where the reasoning is. The listener is the thing that knows it’s idempotent and expensive, so the listener is the right place to say “collapse these”. I’d want to be careful about the window in anything user-facing, because debouncing is a promise that the work happens eventually and not a promise about when, but for indexing, cache warming and notification digests it’s exactly right.

The Redis cluster work

If you run phpredis against a cluster, v13.26.0 is the release to take, and it’s not one feature but a run of them from @tgivslife: keeping the connection usable when a pipeline or transaction fails, rebuilding the client after a cluster response error, passing a connector so the cluster connection can rebuild its client at all, scanning every master node rather than one, and fixing an infinite scan loop when pruning stale cache tags.

Read that list again as symptoms rather than fixes. A connection that becomes unusable after a failed pipeline is a process that works until the first blip and then quietly returns errors for the rest of its life. A scan that only visits one master is a cache tag flush that appears to work and leaves keys behind on every other node. An infinite scan loop during pruning is a worker pinned at full CPU for reasons that make no sense from the outside. These are the failures you spend a day and a half not reproducing locally, because none of them happen on a single-node Redis on your laptop.

Alongside that, Taylor added retries on transient failures for some Redis commands. Taken together this is the most valuable thing in the fortnight for anyone actually running Laravel at a size where clustering is on the table, and it’s the part that will get the least attention because none of it is a feature.

The fixes I’d take without reading the diff

Three in the 12.x line, all backported, all worth having.

The in validation rule could be bypassed via loose comparison. If you have ever used in to constrain a role, a status or a plan tier, and you accept JSON where a client controls the type of the value, that is a rule you believed was a boundary and wasn’t. Same category: temporary upload URLs now read the upload target strictly from the query string.

The third is escaping single quotes in Postgres JSON path attributes. I wrote recently about storing whole payloads in jsonb and querying them with data->currency style accessors, and this is precisely the seam in that approach where a user-supplied key could go somewhere it shouldn’t. If you build JSON paths from anything that came in over the wire, take the patch.

One more that isn’t security but is the kind of bug that eats an afternoon: throwUnless() did nothing when given a closure. Silently. A guard clause that has never once fired is worse than no guard clause, because you stopped thinking about that case the day you wrote it.

Everything else, briefly

There’s a quiet theme of enums finally behaving consistently. Backed enum queue names are respected when queueing mailables, enum keys are normalised in typed cache getters, inOrderOf() accepts enums, and the queue drivers now agree with the fake about them. If you’ve leaned on enums for queue names and been caught out by a fake that disagreed with production, that gap has closed.

foreignUlidFor() joins the schema helpers, which is the obvious counterpart to foreignIdFor() and saves the usual two lines:

$table->foreignUlidFor(User::class);

The Image class picked up fromStream(), a public toFormat(), and a Responsable implementation, so you can return one straight from a controller. Process pools are iterable, process fakes have assertion helpers, and idle process timeouts get their own exception class rather than being indistinguishable from a hard timeout. Guzzle 8 is supported. artisan dev now runs through @laravel/multiplex, which had a couple of rough edges in v13.25.0 — repeatedly asking to install multiplex, and starting Vite on projects with no Vite — fixed the following week.

And orWhereKey/orWhereKeyNot shipped in v13.26.0 and were reverted the same day in v13.26.1. If you happened to pin 13.26.0 and used them, that’s your upgrade note.

The other Laravel blog post of the fortnight is two field marketers building laracon.us/photos with Claude and Laravel Cloud. It’s a nice story and there’s nothing in it that changes how you write Laravel, which I mean as a description rather than a criticism.

Share

XLinkedIn

Related

Keep Reading

All posts →