Media Upscaling
Relevant source files
The following files were used as context for generating this wiki page:
Media Upscaling is a critical enhancement stage in the content creation pipeline designed to transform lower-resolution AI-generated assets into high-quality, social-ready media. This module provides specialized functions for upscaling both static images and video clips using fal.ai models, ensuring that final renders meet professional quality standards for platforms like TikTok and Instagram.
The upscaling process is optional and governed by the upscale flag in the content creation payload. When enabled, it adds approximately 2 minutes to the total pipeline duration but significantly improves visual detail and resolution, typically moving video from 720p to 1080p.
Sources: src/tasks/createContentTask.ts:32-41, src/schemas/contentCreationSchema.ts:14-15
Architecture and Workflow
The upscaling system operates as a middle-tier service between initial asset generation (image/video) and the final rendering process. It utilizes the fal.ai client to perform asynchronous processing.
Workflow Integration
In the standard content creation task, upscaling occurs at two specific checkpoints:
- Image Upscaling: Triggered immediately after the base image is generated and before video animation begins.
- Video Upscaling: Triggered after the video is generated (either via standard image-to-video or lipsync audio-to-video) but before the final FFmpeg render and caption overlay.
This diagram illustrates the conditional logic applied to both image and video assets during the pipeline execution.
Sources: src/tasks/createContentTask.ts:98-102, src/tasks/createContentTask.ts:130-134
Image Upscaling
Image upscaling enhances static AI-generated images by adding detail and increasing resolution. This is handled by the upscaleImage function, which utilizes the fal-ai/seedvr/upscale/image model by default.
Implementation Details
The process involves sending the generated imageUrl to the upscale model. The system uses a "subscription" pattern to wait for the processing to complete before extracting the new URL.
| Parameter | Default Value / Model | Description |
|---|---|---|
| Model | fal-ai/seedvr/upscale/image | The AI model responsible for 2x detail enhancement. |
| Output Format | png | Standard output for static upscaled assets. |
Sources: src/content/upscaleImage.ts:1-20, src/content/defaultPipelineConfig.ts:28
Video Upscaling
Video upscaling targets the improvement of motion clips, specifically moving resolution from 720p to a target of 1080p. This is managed by the upscaleVideo function.
Technical Configuration
The video upscaler uses the fal-ai/seedvr/upscale/video model. Unlike image upscaling which focuses on static detail, the video upscaler balances temporal consistency and noise.
This sequence shows the interaction between the internal task runner and the external AI processing service for video assets.
Sources: src/content/upscaleVideo.ts:1-30, src/content/defaultPipelineConfig.ts:25
Configuration Parameters
The following configuration is applied during the video upscale request:
- Upscale Mode:
target - Target Resolution:
1080p - Noise Scale:
0.1 - Output Quality:
high - Output Format:
X264 (.mp4)
Sources: src/content/upscaleVideo.ts:21-30
Pipeline Configuration
Upscaling models and settings are defined in the central pipeline configuration to allow for consistent behavior across different content types.
| Configuration Key | Value | Description |
|---|---|---|
upscaleModel | fal-ai/seedvr/upscale/image | Default model for static image enhancement. |
videoResolution | 720p | Standard resolution before upscaling occurs. |
resolution | 2K | Internal target resolution for high-quality assets. |
Sources: src/content/defaultPipelineConfig.ts:24-34
Local Testing
The project includes a testPipeline.ts utility that allows developers to test upscaling steps independently of the full Trigger.dev environment. This script manages state via a .pipeline-state.json file, allowing an upscaling test to pick up the URL generated by a previous image or video generation step.
- Command (Image):
npx tsx src/content/testPipeline.ts upscale-image - Command (Video):
npx tsx src/content/testPipeline.ts upscale-video
Sources: src/content/testPipeline.ts:1-25, src/content/testPipeline.ts:128-158
Summary
Media Upscaling serves as a quality-assurance layer within the tasks repository. By leveraging fal.ai's SeedVR models, the system ensures that AI-generated artifacts are polished and high-resolution before being sent to the final FFmpeg rendering stage. This modular approach allows for optional high-quality processing while maintaining a faster standard pipeline for rapid content iteration.
Sources: src/tasks/createContentTask.ts:32-41