Content Creation Tasks
Relevant source files
The following files were used as context for generating this wiki page:
The Content Creation system is a high-level background processing pipeline designed to generate social-media-ready videos for artists. It integrates multiple AI services, primarily via fal.ai, to handle image generation, video animation, upscaling, and lip-syncing. The system follows a structured pipeline that transforms a template and artist-specific assets into a final rendered video file with captions and audio overlays.
The core of this system is implemented as a set of Trigger.dev v4 background workers. These tasks are decoupled from primary database storage, relying on external APIs for persistence and artist context. The pipeline is highly configurable, supporting variations in template styles, caption lengths, and quality settings like upscaling.
Sources: src/tasks/createContentTask.ts:35-50, README.md:20-25
Core Task: create-content
The create-content task represents the full execution pipeline. It is a schemaTask that validates incoming payloads against a strict Zod schema before executing an 11-step sequence to produce a final video.
The Content Pipeline
The pipeline executes the following logical steps:
- Template Loading: Retrieves style guides, reference images, and prompt configurations.
- Asset Resolution: Fetches artist-specific "face-guides" and selects appropriate audio clips.
- Context Gathering: Fetches artist and audience identity information for personalized captioning.
- Visual Generation:
- Generates a base image using a combination of face-guides and template prompts.
- (Optional) Performs 2x detail enhancement via upscaling.
- Motion Generation: Animates the image into a video or uses audio-to-video for lip-syncing.
- Post-Processing:
- Upscales the generated video (e.g., 720p to 1080p).
- Generates a TikTok-style caption using the Recoup Chat API.
- Final Rendering: Uses FFmpeg to crop the video to a 9:16 aspect ratio and overlay the final audio and captions.
Sources: src/tasks/createContentTask.ts:35-144, src/content/testPipeline.ts:70-160
Technical Workflow
The following diagram illustrates the flow of data through the content creation pipeline:
The diagram shows the sequential nature of the pipeline, including optional upscaling branches for both images and video.
Sources: src/tasks/createContentTask.ts:55-144
Data Models and Validation
Configuration Schema
The createContentPayloadSchema defines the requirements for initiating a content task. It ensures that essential artist and account identifiers are present along with configuration flags for the AI models.
| Field | Type | Description |
|---|---|---|
accountId | string | Unique identifier for the account owning the task. |
artistSlug | string | URL-friendly identifier for the artist. |
template | string | The name of the style template to apply. |
lipsync | boolean | If true, uses audio-to-video for synchronized mouth movement. Defaults to false. |
captionLength | enum | Length of generated text: short, medium, or long. |
upscale | boolean | Enables high-quality enhancement for image and video steps. |
githubRepo | string | URL to the repository containing artist assets (face-guides, songs). |
Sources: src/schemas/contentCreationSchema.ts:6-20, src/tasks/createContentTask.ts:55-60
Template System
Templates are loaded from a dedicated directory and contain JSON-based style and caption guides.
| Component | File | Purpose |
|---|---|---|
styleGuide | style-guide.json | Contains camera, environment, and subject rules for image generation. |
captionGuide | caption-guide.json | Defines linguistic style and formatting for captions. |
videoMoods | video-moods.json | Arrays of energy/mood descriptors for animation. |
videoMovements | video-movements.json | Specific movement instructions (e.g., "natural breathing"). |
Sources: src/content/loadTemplate.ts:40-70, src/content/loadTemplate.ts:133-150
Polling and Callbacks
Since content generation can take up to 30 minutes, the system employs a secondary task to monitor execution and notify downstream consumers.
poll-content-run
This task uses the Trigger.dev runs.poll() mechanism to wait for one or more create-content runs to finish. Once the runs complete, it aggregates the results and triggers a callback to the Recoup API.
The sequence diagram details the asynchronous coordination between the polling task, the content generator, and the external API callback.
Sources: src/tasks/pollContentRunTask.ts:16-33, src/tasks/createContentTask.ts:42-43
Rendering Logic
Final video assembly is performed locally within the task environment using FFmpeg. This step is critical for ensuring the output adheres to social media standards (9:16 aspect ratio) and properly integrates the AI-generated visual elements with selected audio.
- Cropping: Converts 16:9 AI output to 9:16 vertical video.
- Audio Integration: Overlays the selected song clip at specific timestamps.
- Caption Overlay: Draws the generated text onto the video frames with adaptive font sizing based on caption length.
Sources: src/tasks/createContentTask.ts:135-144, src/content/testPipeline.ts:285-350
Summary
The Content Creation Task system provides a robust, automated workflow for producing high-quality artist content. By chaining specialized AI models (fal.ai) and local media processing (FFmpeg), it minimizes manual effort in video production while ensuring brand consistency through a modular template system. The use of background workers and polling ensures that the long-running nature of AI generation does not block user-facing operations.
Sources: src/tasks/createContentTask.ts:35-50, README.md:1-10