Laravel Horizon Job Batching for Reliable Workflows | Mohamed Said [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://www.msaied.com) [ Home ](https://www.msaied.com) [ Projects ](https://www.msaied.com/projects) [ Articles ](https://www.msaied.com/articles) [ Certificates ](https://www.msaied.com/certificates) [ Contact ](https://www.msaied.com#contact-section) [ ](https://github.com/EG-Mohamed) [ Home ](https://www.msaied.com) [ Projects ](https://www.msaied.com/projects) [ Articles ](https://www.msaied.com/articles) [ Certificates ](https://www.msaied.com/certificates) [ Contact ](https://www.msaied.com#contact-section) [ home ](https://www.msaied.com) [ articles ](https://www.msaied.com/articles) Job Batching with Laravel Horizon: Reliable Async Workflows at Scale On this page 1. [ Why Job Batching Deserves More Attention ](#why-job-batching-deserves-more-attention) 2. [ Anatomy of a Batch ](#anatomy-of-a-batch) 3. [ Making Jobs Batchable ](#making-jobs-batchable) 4. [ Horizon Configuration for Batch Workloads ](#horizon-configuration-for-batch-workloads) 5. [ Pruning and Observability ](#pruning-and-observability) 6. [ Nested Batches and Dynamic Fan-Out ](#nested-batches-and-dynamic-fan-out) 7. [ Takeaways ](#takeaways) ![Job Batching with Laravel Horizon: Reliable Async Workflows at Scale](https://cdn.msaied.com/553/b794b736bfd84f3cbcc6218319916544.png) #laravel #queues #horizon #async Job Batching with Laravel Horizon: Reliable Async Workflows at Scale ====================================================================== 15 Aug 2026 3 min read ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said Table of contents 1. [ 01 Why Job Batching Deserves More Attention ](#why-job-batching-deserves-more-attention) 2. [ 02 Anatomy of a Batch ](#anatomy-of-a-batch) 3. [ 03 Making Jobs Batchable ](#making-jobs-batchable) 4. [ 04 Horizon Configuration for Batch Workloads ](#horizon-configuration-for-batch-workloads) 5. [ 05 Pruning and Observability ](#pruning-and-observability) 6. [ 06 Nested Batches and Dynamic Fan-Out ](#nested-batches-and-dynamic-fan-out) 7. [ 07 Takeaways ](#takeaways) Why Job Batching Deserves More Attention ---------------------------------------- Laravel's `Bus::batch()` API has been available since Laravel 8, yet most teams still reach for simple `dispatch()` calls or manual counters to coordinate parallel work. Paired with Horizon's real-time supervision, batching gives you a first-class primitive for fan-out/fan-in patterns — think bulk imports, report generation, or multi-tenant data migrations — without building your own orchestration layer. --- Anatomy of a Batch ------------------ A batch is a collection of jobs that share a lifecycle. Laravel tracks completion, failure counts, and pending jobs in the `job_batches` table. ```php use Illuminate\Bus\Batch; use Illuminate\Support\Facades\Bus; $batch = Bus::batch([ new ProcessChunk($chunk1), new ProcessChunk($chunk2), new ProcessChunk($chunk3), ]) ->name('nightly-import') ->allowFailures() // keep running even if some jobs fail ->onProgress(function (Batch $batch) { logger()->info('Batch progress', [ 'id' => $batch->id, 'pending' => $batch->pendingJobs, 'failed' => $batch->failedJobs, 'progress' => $batch->progress(), ]); }) ->then(function (Batch $batch) { // All jobs succeeded ImportCompleted::dispatch($batch->id); }) ->catch(function (Batch $batch, \Throwable $e) { // At least one job failed (called once per failure when allowFailures is on) ImportFailed::dispatch($batch->id, $e->getMessage()); }) ->finally(function (Batch $batch) { // Always runs — success or failure ImportFinished::dispatch($batch->id); }) ->dispatch(); ``` > **Key distinction:** `->catch()` fires for *each* failed job when `allowFailures()` is set. Without it, the first failure cancels the batch and `->catch()` fires once. --- Making Jobs Batchable --------------------- Add the `Batchable` trait and guard against cancelled batches: ```php use Illuminate\Bus\Batchable; use Illuminate\Contracts\Queue\ShouldQueue; class ProcessChunk implements ShouldQueue { use Batchable; public int $tries = 3; public int $backoff = 10; public function __construct(private readonly array $rows) {} public function handle(): void { if ($this->batch()?->cancelled()) { return; // bail early — another job may have triggered cancellation } foreach ($this->rows as $row) { // process row... } } } ``` --- Horizon Configuration for Batch Workloads ----------------------------------------- Batches benefit from dedicated queues so they don't starve interactive jobs. ```php // config/horizon.php 'environments' => [ 'production' => [ 'supervisor-default' => [ 'connection' => 'redis', 'queue' => ['high', 'default'], 'processes' => 5, ], 'supervisor-batch' => [ 'connection' => 'redis', 'queue' => ['batch'], 'processes' => 20, // scale independently 'timeout' => 300, ], ], ], ``` Dispatch batch jobs onto the dedicated queue: ```php Bus::batch($jobs) ->onQueue('batch') ->dispatch(); ``` --- Pruning and Observability ------------------------- Batch records accumulate. Schedule pruning and expose batch status via an API or Filament panel: ```php // routes/api.php Route::get('/imports/{batchId}', function (string $batchId) { $batch = Bus::findBatch($batchId); abort_unless($batch, 404); return response()->json([ 'progress' => $batch->progress(), 'pending' => $batch->pendingJobs, 'failed' => $batch->failedJobs, 'finished_at' => $batch->finishedAt, ]); }); ``` In `app/Console/Kernel.php` (or a scheduled command in Laravel 11+): ```php $schedule->command('queue:prune-batches --hours=48')->daily(); ``` --- Nested Batches and Dynamic Fan-Out ---------------------------------- You can add jobs to a running batch from within a job — useful when the total work isn't known upfront: ```php public function handle(): void { $subJobs = $this->discoverMoreWork(); if ($subJobs) { $this->batch()->add($subJobs); } } ``` Laravel increments `pendingJobs` atomically, so progress tracking stays accurate. --- Takeaways --------- - Use `allowFailures()` for resilient fan-out; omit it when partial success is unacceptable. - Always check `$this->batch()?->cancelled()` at the top of `handle()` to avoid wasted work. - Isolate batch queues in Horizon so throughput scales independently of interactive queues. - Prune `job_batches` on a schedule — unbounded growth will hurt query performance. - Expose batch progress via a lightweight endpoint or admin panel for operational visibility. - Dynamic `batch()->add()` enables adaptive fan-out when the total job count is unknown at dispatch time. Found this useful? [ ](https://twitter.com/intent/tweet?url=https%3A%2F%2Fwww.msaied.com%2Farticles%2Fjob-batching-with-laravel-horizon-reliable-async-workflows-at-scale&text=Job+Batching+with+Laravel+Horizon%3A+Reliable+Async+Workflows+at+Scale) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.msaied.com%2Farticles%2Fjob-batching-with-laravel-horizon-reliable-async-workflows-at-scale) Frequently Asked Questions ---------------------------- 3 questions Q01 What is the difference between `->catch()` and `->finally()` in a Laravel batch? `->catch()` is invoked when one or more jobs fail — once per failure if `allowFailures()` is active, or once on the first failure otherwise. `->finally()` always runs after the batch finishes, regardless of success or failure, making it the right place for cleanup or notification logic. Q02 Can I add jobs to a batch after it has already been dispatched? Yes. From within a batchable job you can call `$this->batch()->add($moreJobs)`. Laravel increments the pending job counter atomically, so progress reporting and completion callbacks remain accurate. Q03 How do I prevent the `job\_batches` table from growing indefinitely? Schedule `queue:prune-batches --hours=48` (or your preferred retention window) using Laravel's task scheduler. This removes finished and cancelled batch records older than the specified threshold. Continue reading More Articles --------------- [ View all ](https://www.msaied.com/articles) [ ![Octane Worker Lifecycle, State Leakage, and Memory Management in Production](https://cdn.msaied.com/554/8cc265358b47e59601a66d1e247eba9a.png) laravel octane performance ### Octane Worker Lifecycle, State Leakage, and Memory Management in Production Laravel Octane keeps workers alive across requests, which means static state, resolved singletons, and stale d... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 16 Aug 2026 3 min read Read ](https://www.msaied.com/articles/octane-worker-lifecycle-state-leakage-and-memory-management-in-production-2) [ ![Contextual Eloquent Casts: Custom Cast Classes, Value Objects, and Inbound-Only Transforms](https://cdn.msaied.com/552/a7825c0c6f53d934f84fce522573eafb.png) laravel eloquent value-objects ### Contextual Eloquent Casts: Custom Cast Classes, Value Objects, and Inbound-Only Transforms Go beyond primitive casts. Learn how to build custom Eloquent cast classes that hydrate value objects, handle... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 15 Aug 2026 1 min read Read ](https://www.msaied.com/articles/contextual-eloquent-casts-custom-cast-classes-value-objects-and-inbound-only-transforms) [ ![Contextual Eloquent Scopes: Binding Query Logic to Domain State at Runtime](https://cdn.msaied.com/551/dc00bc1e6fb2999c99a0b5b8fb42a8c3.png) laravel eloquent architecture ### Contextual Eloquent Scopes: Binding Query Logic to Domain State at Runtime Learn how to attach runtime-aware query scopes to Eloquent models using the service container, avoiding scatte... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 15 Aug 2026 3 min read Read ](https://www.msaied.com/articles/contextual-eloquent-scopes-binding-query-logic-to-domain-state-at-runtime) [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://www.msaied.com)Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow. Explore - [Home](https://www.msaied.com) - [Projects](https://www.msaied.com/projects) - [Articles](https://www.msaied.com/articles) - [Certificates](https://www.msaied.com/certificates) - [Contact](https://www.msaied.com#contact-section) Connect - [ hello@msaied.com ](mailto:hello@msaied.com) - [ +20 109 461 9204 ](tel:+201094619204) © 2026 Mohamed Said. All rights reserved. [ ](https://github.com/EG-Mohamed) [ ](https://www.linkedin.com/in/msaiedm/) [ ](https://wa.me/201094619204) [ ](mailto:hello@msaied.com) [ ](https://drive.google.com/file/u/0/d/1MF20IPRJyzfy32mhEutjL5EpSls0w2Q8/view)