diff --git a/docs/docs.json b/docs/docs.json index 6ba6c67472..aeb5650ae9 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -333,6 +333,16 @@ } ] }, + { + "group": "Use cases", + "pages": [ + "guides/use-cases/overview", + "guides/use-cases/data-processing-etl", + "guides/use-cases/media-generation", + "guides/use-cases/media-processing", + "guides/use-cases/marketing" + ] + }, { "group": "Example projects", "pages": [ diff --git a/docs/guides/introduction.mdx b/docs/guides/introduction.mdx index 420494ea66..13d6f96431 100644 --- a/docs/guides/introduction.mdx +++ b/docs/guides/introduction.mdx @@ -15,6 +15,8 @@ mode: "center" +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + ## Guides Get set up fast using our detailed walk-through guides. @@ -39,6 +41,8 @@ Get set up fast using our detailed walk-through guides. | [Using webhooks in Next.js](/guides/frameworks/nextjs-webhooks) | Trigger tasks from a webhook in Next.js | | [Using webhooks in Remix](/guides/frameworks/remix-webhooks) | Trigger tasks from a webhook in Remix | + + ## Example projects Example projects are full projects with example repos you can fork and use. These are a great way of learning how to use Trigger.dev in your projects. diff --git a/docs/guides/use-cases/data-processing-etl.mdx b/docs/guides/use-cases/data-processing-etl.mdx new file mode 100644 index 0000000000..f6bfdd6581 --- /dev/null +++ b/docs/guides/use-cases/data-processing-etl.mdx @@ -0,0 +1,159 @@ +--- +title: "Data processing & ETL workflows" +sidebarTitle: "Data processing & ETL" +description: "Learn how to use Trigger.dev for data processing and ETL (Extract, Transform, Load), including web scraping, database synchronization, batch enrichment and more." +--- + +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + +## Overview + +Build complex data pipelines that process large datasets without timeouts. Handle streaming analytics, batch enrichment, web scraping, database sync, and file processing with automatic retries and progress tracking. + +## Featured examples + + + + Import CSV files with progress streamed live to frontend. + + + Scrape websites using BrowserBase and Puppeteer. + + + Trigger tasks from Supabase database webhooks. + + + +## Benefits of using Trigger.dev for data processing & ETL workflows + +**Process datasets for hours without timeouts:** Handle multi-hour transformations, large file processing, or complete database exports. No execution time limits. + +**Parallel processing with built-in rate limiting:** Process thousands of records simultaneously while respecting API rate limits. Scale efficiently without overwhelming downstream services. + +**Stream progress to your users in real-time:** Show row-by-row processing status updating live in your dashboard. Users see exactly where processing is and how long remains. + +## Production use cases + + + + +Read how MagicSchool AI uses Trigger.dev to generate insights from millions of student interactions. + + + + + +Read how Comp AI uses Trigger.dev to automate evidence collection at scale, powering their open source, AI-driven compliance platform. + + + + +Read how Midday use Trigger.dev to sync large volumes of bank transactions in their financial management platform. + + + + +## Example workflow patterns + + + + Simple CSV import pipeline. Receives file upload, parses CSV rows, validates data, imports to database with progress tracking. + +
+ +```mermaid +graph TB + A[importCSV] --> B[parseCSVFile] + B --> C[validateRows] + C --> D[bulkInsertToDB] + D --> E[notifyCompletion] +``` + +
+
+ + + **Coordinator pattern with parallel extraction**. Batch triggers parallel extraction from multiple sources (APIs, databases, S3), transforms and validates data, loads to data warehouse with monitoring. + +
+ +```mermaid +graph TB + A[runETLPipeline] --> B[coordinateExtraction] + B --> C[batchTriggerAndWait] + + C --> D[extractFromAPI] + C --> E[extractFromDatabase] + C --> F[extractFromS3] + + D --> G[transformData] + E --> G + F --> G + + G --> H[validateData] + H --> I[loadToWarehouse] +``` + +
+
+ + + **Coordinator pattern with browser automation**. Launches headless browsers in parallel to scrape multiple pages, extracts structured data, cleans and normalizes content, stores in database. + +
+ +```mermaid +graph TB + A[scrapeSite] --> B[coordinateScraping] + B --> C[batchTriggerAndWait] + + C --> D[scrapePage1] + C --> E[scrapePage2] + C --> F[scrapePageN] + + D --> G[cleanData] + E --> G + F --> G + + G --> H[normalizeData] + H --> I[storeInDatabase] +``` + +
+
+ + + **Coordinator pattern with rate limiting**. Fetches records needing enrichment, batch triggers parallel API calls with configurable concurrency to respect rate limits, validates enriched data, updates database. + +
+ +```mermaid +graph TB + A[enrichRecords] --> B[fetchRecordsToEnrich] + B --> C[coordinateEnrichment] + C --> D[batchTriggerAndWait] + + D --> E[enrichRecord1] + D --> F[enrichRecord2] + D --> G[enrichRecordN] + + E --> H[validateEnrichedData] + F --> H + G --> H + + H --> I[updateDatabase] +``` + +
+
+
+ + diff --git a/docs/guides/use-cases/marketing.mdx b/docs/guides/use-cases/marketing.mdx new file mode 100644 index 0000000000..83d08d95d9 --- /dev/null +++ b/docs/guides/use-cases/marketing.mdx @@ -0,0 +1,147 @@ +--- +title: "Marketing workflows" +sidebarTitle: "Marketing" +description: "Learn how to use Trigger.dev for marketing workflows, including drip campaigns, behavioral triggers, personalization engines, and AI-powered content workflows" +--- + +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + +## Overview + +Build marketing workflows from email drip sequences to orchestrating full multi-channel campaigns. Handle multi-day sequences, behavioral triggers, dynamic content generation, and build live analytics dashboards. + +## Featured examples + + + + Send multi-day email sequences with wait delays between messages. + + + Transform product photos into professional marketing images using Replicate. + + + Approve marketing content using a human-in-the-loop workflow. + + + +## Benefits of using Trigger.dev for marketing workflows + +**Delays without idle costs:** Wait hours or weeks between steps. Waits over 5 seconds are automatically checkpointed and don't count towards compute usage. Perfect for drip campaigns and scheduled follow-ups. + +**Guaranteed delivery:** Messages send exactly once, even after retries. Personalized content isn't regenerated on failure. + +**Scale without limits:** Process thousands in parallel while respecting rate limits. Send to entire segments without overwhelming APIs. + +## Production use cases + + + +Read how Icon uses Trigger.dev to process and generate thousands of videos per month for their AI-driven video creation platform. + + + +## Example workflow patterns + + + + Simple drip campaign. User signs up, waits specified delay, sends personalized email, tracks engagement. + +
+ +```mermaid +graph TB + A[userCreateAccount] --> B[sendWelcomeEmail] + B --> C[wait.for 24h] + C --> D[sendProductTipsEmail] + D --> E[wait.for 7d] + E --> F[sendFeedbackEmail] + +``` + +
+
+ + + **Router pattern with delay orchestration**. User action triggers campaign, router selects channel based on preferences (email/SMS/push), coordinates multi-day sequence with delays between messages, tracks engagement across channels. + +
+ +```mermaid +graph TB + A[startCampaign] --> B[fetchUserProfile] + B --> C[selectChannel] + C --> D{Preferred
Channel?} + + D -->|Email| E[sendEmail1] + D -->|SMS| F[sendSMS1] + D -->|Push| G[sendPush1] + + E --> H[wait.for 2d] + F --> H + G --> H + + H --> I[sendFollowUp] + I --> J[trackConversion] +``` + +
+
+ + + **Supervisor pattern with approval gate**. Generates AI marketing content (images, copy, assets), pauses with wait.forToken for human review, applies revisions if needed, publishes to channels after approval. + +
+ +```mermaid +graph TB + A[createCampaignAssets] --> B[generateAIContent] + B --> C[wait.forToken approval] + C --> D{Approved?} + + D -->|Yes| E[publishToChannels] + D -->|Needs revision| F[applyFeedback] + F --> B +``` + +
+
+ + + **Coordinator pattern with enrichment**. User completes survey, batch triggers parallel enrichment from CRM/analytics, analyzes and scores responses, updates customer profiles, triggers personalized follow-up campaigns. + +
+ +```mermaid +graph TB + A[processSurveyResponse] --> B[coordinateEnrichment] + B --> C[batchTriggerAndWait] + + C --> D[fetchCRMData] + C --> E[fetchAnalytics] + C --> F[fetchBehaviorData] + + D --> G[analyzeAndScore] + E --> G + F --> G + + G --> H[updateCRMProfile] + H --> I[triggerFollowUp] +``` + +
+
+
+ + diff --git a/docs/guides/use-cases/media-generation.mdx b/docs/guides/use-cases/media-generation.mdx new file mode 100644 index 0000000000..919440df5f --- /dev/null +++ b/docs/guides/use-cases/media-generation.mdx @@ -0,0 +1,144 @@ +--- +title: "AI media generation workflows" +sidebarTitle: "AI media generation" +description: "Learn how to use Trigger.dev for AI media generation including image creation, video synthesis, audio generation, and multi-modal content workflows" +--- + +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + +## Overview + +Build AI media generation pipelines that handle unpredictable API latencies and long-running operations. Generate images, videos, audio, and multi-modal content with automatic retries, progress tracking, and no timeout limits. + +## Featured examples + + + + Transform product photos into professional marketing images using Replicate. + + + Generate memes with DALL·E 3 and add human approval steps. + + + Generate images from text prompts using the Vercel AI SDK. + + + +## Benefits of using Trigger.dev for AI media generation workflows + +**Pay only for active compute, not AI inference time:** Checkpoint-resume pauses during AI API calls. Generate content that takes minutes or hours without paying for idle inference time. + +**No timeout limits for long generations:** Handle generations that take minutes or hours without execution limits. Perfect for high-quality video synthesis and complex multi-modal workflows. + +**Human approval gates for brand safety:** Add review steps before publishing AI-generated content. Pause workflows for human approval using waitpoint tokens. + +## Production use cases + + + + + +Read how Icon uses Trigger.dev to process and generate thousands of videos per month for their AI-driven video creation platform. + + + + + +Read how Papermark process thousands of documents per month using Trigger.dev. + + + + +## Example workflow patterns + + + + **Supervisor pattern with approval gate**. Generates AI content, pauses execution with wait.forToken to allow human review, applies feedback if needed, publishes approved content. + +
+ +```mermaid +graph TB + A[generateContent] --> B[createWithAI] + B --> C[wait.forToken approval] + C --> D{Approved?} + + D -->|Yes| E[publishContent] + D -->|Needs revision| F[applyFeedback] + F --> B +``` + +
+
+ + Simple AI image generation. Receives prompt and parameters, calls OpenAI DALL·E 3, post-processes result, uploads to storage. + +
+ +```mermaid +graph TB + A[generateImage] --> B[optimizeImage] + B --> C[uploadToStorage] + C --> D[updateDatabase] +``` + +
+
+ + + **Coordinator pattern with rate limiting**. Receives batch of generation requests, coordinates parallel processing with configurable concurrency to respect API rate limits, validates outputs, stores results. + +
+ +```mermaid +graph TB + A[processBatch] --> B[coordinateGeneration] + B --> C[batchTriggerAndWait] + + C --> D[generateImage1] + C --> E[generateImage2] + C --> F[generateImageN] + + D --> G[validateResults] + E --> G + F --> G + + G --> H[storeResults] + H --> I[notifyCompletion] +``` + +
+
+ + + **Coordinator pattern with sequential processing**. Generates initial content with AI, applies style transfer or enhancement, upscales resolution, optimizes and compresses for delivery. + +
+ +```mermaid +graph TB + A[processCreative] --> B[generateWithAI] + B --> C[applyStyleTransfer] + C --> D[upscaleResolution] + D --> E[optimizeAndCompress] + E --> F[uploadToStorage] +``` + +
+
+ +
+ + diff --git a/docs/guides/use-cases/media-processing.mdx b/docs/guides/use-cases/media-processing.mdx new file mode 100644 index 0000000000..e0f6aa8719 --- /dev/null +++ b/docs/guides/use-cases/media-processing.mdx @@ -0,0 +1,191 @@ +--- +title: "Media processing workflows" +sidebarTitle: "Media processing" +description: "Learn how to use Trigger.dev for media processing including video transcoding, image optimization, audio transformation, and document conversion." +--- + +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + +## Overview + +Build media processing pipelines that handle large files and long-running operations. Process videos, images, audio, and documents with automatic retries, progress tracking, and no timeout limits. + +## Featured examples + + + + Process videos and upload results to R2 storage using FFmpeg. + + + Transform product photos into professional marketing images using Replicate. + + + Convert documents to PDF using LibreOffice. + + + +## Benefits of using Trigger.dev for media processing workflows + +**Process multi-hour videos without timeouts:** Transcode videos, extract frames, or run CPU-intensive operations for hours. No execution time limits. + +**Stream progress to users in real-time:** Show processing status updating live in your UI. Users see exactly where encoding is and how long remains. + +**Parallel processing with resource control:** Process hundreds of files simultaneously with configurable concurrency limits. Control resource usage without overwhelming infrastructure. + +## Example workflow patterns + + + + Simple video transcoding pipeline. Downloads video from storage, batch triggers parallel transcoding to multiple formats and thumbnail extraction, uploads all results. + +
+ +```mermaid +graph TB + A[processVideo] --> B[downloadFromStorage] + B --> C[batchTriggerAndWait] + + C --> D[transcodeToHD] + C --> E[transcodeToSD] + C --> F[extractThumbnail] + + D --> G[uploadToStorage] + E --> G + F --> G +``` + +
+
+ + + **Router + Coordinator pattern**. Analyzes video metadata to determine source resolution, routes to appropriate transcoding preset, batch triggers parallel post-processing for thumbnails, preview clips, and chapter detection. + +
+ +```mermaid +graph TB + A[processVideoUpload] --> B[analyzeMetadata] + B --> C{Source
Resolution?} + + C -->|4K Source| D[transcode4K] + C -->|HD Source| E[transcodeHD] + C -->|SD Source| F[transcodeSD] + + D --> G[coordinatePostProcessing] + E --> G + F --> G + + G --> H[batchTriggerAndWait] + H --> I[extractThumbnails] + H --> J[generatePreview] + H --> K[detectChapters] + + I --> L[uploadToStorage] + J --> L + K --> L + + L --> M[notifyComplete] +``` + +
+
+ + + **Router + Coordinator pattern**. Analyzes image content to detect type, routes to specialized processing (background removal for products, face detection for portraits, scene analysis for landscapes), upscales with AI, batch triggers parallel variant generation. + +
+ +```mermaid +graph TB + A[processImageUpload] --> B[analyzeContent] + B --> C{Content
Type?} + + C -->|Product| D[removeBackground] + C -->|Portrait| E[detectFaces] + C -->|Landscape| F[analyzeScene] + + D --> G[upscaleWithAI] + E --> G + F --> G + + G --> H[batchTriggerAndWait] + H --> I[generateWebP] + H --> J[generateThumbnails] + H --> K[generateSocialCrops] + + I --> L[uploadToStorage] + J --> L + K --> L +``` + +
+
+ + + **Coordinator pattern**. Pre-processes raw audio with noise reduction and speaker diarization, batch triggers parallel tasks for transcription (Deepgram), audio enhancement, and chapter detection, aggregates results to generate show notes and publish. + +
+ +```mermaid +graph TB + A[processAudioUpload] --> B[cleanAudio] + B --> C[coordinateProcessing] + + C --> D[batchTriggerAndWait] + D --> E[transcribeWithDeepgram] + D --> F[enhanceAudio] + D --> G[detectChapters] + + E --> H[generateShowNotes] + F --> H + G --> H + + H --> I[publishToPlatforms] +``` + +
+
+ + + **Router pattern with human-in-the-loop**. Detects file type and routes to appropriate processor, classifies document with AI to determine type (invoice/contract/receipt), extracts structured data fields, optionally pauses with wait.forToken for human approval. + +
+ +```mermaid +graph TB + A[processDocumentUpload] --> B[detectFileType] + + B -->|PDF| C[extractText] + B -->|Word/Excel| D[convertToPDF] + B -->|Image| E[runOCR] + + C --> F[classifyDocument] + D --> F + E --> F + + F -->|Invoice| G[extractLineItems] + F -->|Contract| H[extractClauses] + F -->|Receipt| I[extractExpenses] + + G --> J{Needs
Review?} + H --> J + I --> J + + J -->|Yes| K[wait.forToken approval] + J -->|No| L[processAndIntegrate] + K --> L +``` + +
+
+
+ + diff --git a/docs/guides/use-cases/overview.mdx b/docs/guides/use-cases/overview.mdx new file mode 100644 index 0000000000..8bb9477abb --- /dev/null +++ b/docs/guides/use-cases/overview.mdx @@ -0,0 +1,11 @@ +--- +title: "Use cases" +sidebarTitle: "Overview" +description: "Explore common use cases for Trigger.dev including data processing, media workflows, marketing automation, and AI generation" +--- + +import UseCasesCards from "/snippets/use-cases-cards.mdx"; + +Trigger.dev handles workflows that traditional platforms struggle with: long-running operations, unpredictable API latencies, multi-hour processing, and complex orchestration patterns. Our platform provides no timeout limits, automatic retries, and real-time progress tracking built in. + + diff --git a/docs/snippets/use-cases-cards.mdx b/docs/snippets/use-cases-cards.mdx new file mode 100644 index 0000000000..b43f8e5c9d --- /dev/null +++ b/docs/snippets/use-cases-cards.mdx @@ -0,0 +1,24 @@ +## Featured use cases + + + + Build complex data pipelines that process large datasets without timeouts. + + + Batch process videos, images, audio, and documents with no execution time limits. + + + Generate images, videos, audio, documents and other media using AI models. + + + Build drip campaigns, create marketing content, and orchestrate multi-channel campaigns. + +