Laravel URL AI Transformer by Spatie | Mohamed Said [Skip to content](#main) [ ![](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed SaidLaravel Backend Engineer ](https://www.msaied.com) - [Home](https://www.msaied.com) - [Projects](https://www.msaied.com/projects) - [Articles](https://www.msaied.com/articles) - [Certificates](https://www.msaied.com/certificates) - [About](https://www.msaied.com#about) [ Contact](https://www.msaied.com#contact) Menu Menu ---- Close - [HomeStart here](https://www.msaied.com) - [ProjectsCase studies](https://www.msaied.com/projects) - [ArticlesEngineering notes](https://www.msaied.com/articles) - [CertificatesCredentials](https://www.msaied.com/certificates) - [AboutHow I work](https://www.msaied.com#about) - [ContactGet in touch](https://www.msaied.com#contact) [Start a conversation](https://www.msaied.com#contact) [WhatsApp](https://wa.me/201094619204) [Email](mailto:hello@msaied.com) 1. [Home](https://www.msaied.com) 2. / 3. [Articles](https://www.msaied.com/articles) 4. / 5. [Composer Pacakge](https://www.msaied.com/articles?category=composer-pacakge) 6. / 7. Laravel URL AI Transformer: Extract Structured Data From URLs [Composer Pacakge](https://www.msaied.com/articles?category=composer-pacakge) [AI](https://www.msaied.com/articles?category=ai) Laravel URL AI Transformer: Extract Structured Data From URLs ============================================================== Spatie's Laravel URL AI Transformer fetches URLs, sends each page's content to an AI model, and stores structured results in your database — with built-in JSON-LD support, failure tracking, and testable transformers. ![](https://cdn.msaied.com/01M22N44A70A5MC2S599JP0MPH.webp) [Mohamed Said](https://www.msaied.com#person) Published 14 Oct 2026 · Updated 15 Oct 2026 · 4 min read ShareCopy linkCopied ![Laravel URL AI Transformer: Extract Structured Data From URLs](https://cdn.msaied.com/782/215fd94331f880a70b7bfaf16528790d.png) On this page +1. [What Is the Laravel URL AI Transformer?](#what-is-the-laravel-url-ai-transformer) 2. [Key Features at a Glance](#key-features-at-a-glance) 3. [Registering URLs](#registering-urls) 4. [Writing a Custom Transformer](#writing-a-custom-transformer) 5. [Structured Output](#structured-output) 6. [Choosing a Provider and Model](#choosing-a-provider-and-model) 7. [Testing](#testing) 8. [Installation](#installation) 9. [Key Takeaways](#key-takeaways) What Is the Laravel URL AI Transformer? --------------------------------------- Spatie's `laravel-url-ai-transformer` package automates a common but tedious task: fetching a list of URLs, passing each page's content to an AI model, and persisting the response to your database. Practical use cases include generating schema.org JSON-LD for your own pages, pulling product details into queryable fields, or summarising blog posts at scale. Version 2 rebuilds every transformer as a [Laravel AI agent](https://laravel-news.com/ship-ai-with-laravel-building-your-first-agent-with-laravel-13s-ai-sdk) and requires PHP 8.4 and Laravel 12 or 13. Key Features at a Glance ------------------------ - **Built-in `LdJsonTransformer`** — requests schema.org JSON-LD and validates the JSON before storage. - **Command filters** — `--url="*/blog/*"` and `--transformer="ldJson"` accept wildcards for partial reruns. - **Conditional runs** — `shouldRun()` skips recently processed URLs; `--force` overrides it. - **Failure tracking** — exception message and trace are stored on the result row, and a `TransformerFailed` event fires. - **Regeneration** — `regenerate()` queues a single result again; `regenerateNow()` runs it synchronously. - **Crawling integration** — pair with `spatie/crawler` to transform every internal URL on a site. - **AI-free transformers** — override `transform()` to compute results yourself (e.g. word count). Registering URLs ---------------- Register URLs and their transformers in a service provider using the `Transform` class: ```php use Spatie\LaravelUrlAiTransformer\Support\Transform; use Spatie\LaravelUrlAiTransformer\Transformers\LdJsonTransformer; Transform::urls( 'https://example.com/about', 'https://example.com/products', )->usingTransformers(new LdJsonTransformer, new SummaryTransformer); ``` `urls()` also accepts a closure, so database queries only run when the transformation executes — not on every request: ```php Transform::urls( fn () => Article::published()->pluck('url')->toArray() )->usingTransformers(new LdJsonTransformer); ``` Run `php artisan transform-urls` to dispatch a queued job per transformer, or add `--now` for synchronous execution. Results land in `transformation_results` and are retrieved with: ```php $ldJson = TransformationResult::forUrl('https://example.com/about', 'ldJson'); ``` Writing a Custom Transformer ---------------------------- Generate a transformer with `php artisan make:transformer SummaryTransformer`. The only required method is `instructions()`, which becomes the AI agent's system prompt: ```php class SummaryTransformer extends Transformer { public function instructions(): Stringable|string { return 'Summarize this webpage content in 3 concise bullet points.'; } } ``` Before the content is sent, the package strips scripts, styles, and HTML tags, collapses whitespace, and truncates at 6,000 characters. Structured Output ----------------- Implement `HasStructuredOutput` to validate AI responses against a JSON schema before storage: ```php class ProductTransformer extends Transformer implements HasStructuredOutput { public function instructions(): Stringable|string { return 'Extract the product details from this webpage.'; } public function schema(JsonSchema $schema): array { return [ 'name' => $schema->string()->required(), 'price' => $schema->number()->required(), 'in_stock' => $schema->boolean(), ]; } } ``` Choosing a Provider and Model ----------------------------- The config sets a default provider and model (defaulting to the cheapest OpenAI model). Individual transformers override this with PHP attributes: ```php #[Provider(Lab::Anthropic)] #[Model('claude-haiku-4-5-20251001')] #[MaxTokens(1000)] #[Temperature(0.1)] class FactsTransformer extends Transformer { ... } ``` Use `#[UseSmartestModel]` to keep the configured provider but upgrade the model, or define `provider()` and `model()` methods for runtime decisions. Testing ------- Fake HTTP responses and AI output in tests without hitting real endpoints: ```php Http::fake(['https://example.com/article' => Http::response('Content')]); SummaryTransformer::fake(["• Point one\n• Point two\n• Point three"]); ``` Installation ------------ ```bash composer require spatie/laravel-url-ai-transformer php artisan vendor:publish --tag="url-ai-transformer-migrations" php artisan migrate ``` Add your provider API key (e.g. `OPENAI_API_KEY`) to `.env`. Upgrading from v1 (which used Prism)? Check the official upgrade guide for config and migration changes. Key Takeaways ------------- - Transformers are Laravel AI agents — one `instructions()` method is all you need to get started. - Structured output schemas validate AI responses before they reach your database. - Built-in failure tracking, conditional runs, and wildcard filters make production use practical. - The package is fully testable with HTTP and AI fakes. - Requires PHP 8.4 and Laravel 12 or 13. --- *Source: [Laravel URL AI Transformer — Laravel News](https://laravel-news.com/laravel-url-ai-transformer)* - [Laravel](https://www.msaied.com/articles?search=Laravel) - [AI](https://www.msaied.com/articles?search=AI) - [Spatie](https://www.msaied.com/articles?search=Spatie) - [Structured Data](https://www.msaied.com/articles?search=Structured%20Data) - [Laravel Packages](https://www.msaied.com/articles?search=Laravel%20Packages) Frequently asked questions --------------------------- What does the Laravel URL AI Transformer package do?It fetches a list of URLs, sends each page's cleaned text content to an AI model, and stores the structured response in a `transformation\_results` database table. Common uses include generating JSON-LD markup and extracting product details. How do you prevent a transformer from reprocessing URLs it has already handled?Define a `shouldRun()` method on the transformer to return false for recently processed URLs. You can bypass this check by passing the `--force` flag to the `transform-urls` Artisan command. Can I use a different AI provider, such as Anthropic, instead of OpenAI?Yes. Apply the `#\[Provider(Lab::Anthropic)\]` and `#\[Model('...')\]` PHP attributes to your transformer class, or define `provider()` and `model()` methods when the choice depends on runtime data. ![Mohamed Said](https://cdn.msaied.com/01M22N44A70A5MC2S599JP0MPH.webp)About the author ---------------- [Mohamed Said](https://www.msaied.com#person)Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow. [About](https://www.msaied.com#about) [GitHub ↗](https://github.com/EG-Mohamed) [LinkedIn ↗](https://www.linkedin.com/in/msaiedm/) [WhatsApp ↗](https://wa.me/201094619204) [Email Address ↗](mailto:hello@msaied.com) [My CV ↗](https://drive.google.com/file/u/0/d/1MF20IPRJyzfy32mhEutjL5EpSls0w2Q8/view) [Previous articleRule Builders for Every Validation Rule in Laravel 13.36](https://www.msaied.com/articles/rule-builders-for-every-validation-rule-in-laravel-1336) On this page ------------- 1. [What Is the Laravel URL AI Transformer?](#what-is-the-laravel-url-ai-transformer) 2. [Key Features at a Glance](#key-features-at-a-glance) 3. [Registering URLs](#registering-urls) 4. [Writing a Custom Transformer](#writing-a-custom-transformer) 5. [Structured Output](#structured-output) 6. [Choosing a Provider and Model](#choosing-a-provider-and-model) 7. [Testing](#testing) 8. [Installation](#installation) 9. [Key Takeaways](#key-takeaways) ### Have a technical challenge? Tell me what you’re building. I reply within two working days. [Start a conversation](https://www.msaied.com#contact) Related articles ----------------- [ ![](https://cdn.msaied.com/781/c4463b384b964fd45e6f3cfa00f8e95c.png) Laravel · 4 min read### Rule Builders for Every Validation Rule in Laravel 13.36 14 Oct 2026 ](https://www.msaied.com/articles/rule-builders-for-every-validation-rule-in-laravel-1336) [ ![](https://cdn.msaied.com/776/8e8ee70ebeedf358233b53e19ecae4a1.png) Laravel · 4 min read### Filament v5.10.2 Released: Accessibility Fixes, RTL Support, and New Features 13 Oct 2026 ](https://www.msaied.com/articles/filament-v5102-released-accessibility-fixes-rtl-support-and-new-features) [ ![](https://cdn.msaied.com/780/484d115f641fe90ef3f2022aaff09526.png) Laravel · 4 min read### Filament v4.15.2 Released: Accessibility Overhaul, Bug Fixes & New Features 13 Oct 2026 ](https://www.msaied.com/articles/filament-v4152-released-accessibility-overhaul-bug-fixes-new-features) Have a technical challenge? ---------------------------- Tell me what you’re building. I reply within two working days. [Discuss your project ↗](https://www.msaied.com#contact) © 2026 Mohamed Said · Built with Laravel, meant to last.Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow. - [Home](https://www.msaied.com) - [Articles](https://www.msaied.com/articles) - [Certificates](https://www.msaied.com/certificates) - [GitHub](https://github.com/EG-Mohamed) - [LinkedIn](https://www.linkedin.com/in/msaiedm/) - [WhatsApp](https://wa.me/201094619204) - [Email Address](mailto:hello@msaied.com) - [My CV](https://drive.google.com/file/u/0/d/1MF20IPRJyzfy32mhEutjL5EpSls0w2Q8/view) - [Sitemap](https://www.msaied.com/sitemap.xml)