Flamingo Music Analysis

Relevant source files

The following files were used as context for generating this wiki page:

• lib/flamingo/callFlamingoGenerate.ts
• app/api/songs/analyze/route.ts
• lib/flamingo/processAnalyzeMusicRequest.ts
• lib/flamingo/validateFlamingoGenerateBody.ts
• lib/flamingo/postFlamingoGenerateHandler.ts
• lib/flamingo/presets/types.ts
• lib/flamingo/isFlamingoGenerateResult.ts

Flamingo Music Analysis is a specialized subsystem within the Recoup API designed to analyze audio tracks and answer music-related questions using an Audio Language Model. The system can identify musical characteristics such as genre, tempo, and key, transcribe lyrics, and reason about musical structures based on provided audio URLs or text prompts.

The feature is exposed through a RESTful API and can also be utilized via internal toolchains. It supports both raw custom prompts and structured "presets" that provide curated analysis parameters for common music industry tasks.

Sources: app/api/songs/analyze/route.ts:12-18, lib/flamingo/processAnalyzeMusicRequest.ts:39-42

System Architecture

The Flamingo Analysis system follows a layered architecture that separates the API delivery, business logic, and external model invocation. It employs a DRY (Don't Repeat Yourself) pattern, allowing the same core logic to be shared between the public API and internal MCP (Model Context Protocol) tools.

Request Flow

The following diagram illustrates the lifecycle of a music analysis request from the initial API call to the final response.

The system ensures that requests are authenticated and validated before any intensive computation or external calls occur.

Sources: app/api/songs/analyze/route.ts:47, lib/flamingo/postFlamingoGenerateHandler.ts:16-56, lib/flamingo/processAnalyzeMusicRequest.ts:44-55

API Implementation

The primary entry point is the POST /api/songs/analyze route. This route requires authentication via either an x-api-key header or an Authorization: Bearer token.

Request Body Schema

The request body is validated using a Zod schema that enforces mutual exclusivity between custom prompts and presets.

Field	Type	Required	Description
preset	Enum	Partial	Curated preset name (e.g., catalog_metadata). Exactly one of preset or prompt is required.
prompt	String	Partial	Custom text prompt or question (max 24,000 chars).
audio_url	String	Optional	Public URL to an audio file (MP3, WAV, FLAC). Required for certain presets.
max_new_tokens	Number	No	Max tokens to generate (1–2048, default: 512).
temperature	Number	No	Controls creativity (0–2, default: 1.0).
top_p	Number	No	Nucleus sampling cutoff (0–1, default: 1.0).
do_sample	Boolean	No	Enables sampling (default: false).

Sources: lib/flamingo/validateFlamingoGenerateBody.ts:13-33, app/api/songs/analyze/route.ts:25-33

Validation Logic

The validateFlamingoGenerateBody function ensures that:

1. Either preset or prompt is provided, but not both.
2. The audio_url is a valid URL if provided.
3. Numeric parameters fall within specified ranges (e.g., max_new_tokens between 1 and 2048).

Sources: lib/flamingo/validateFlamingoGenerateBody.ts:31-41

Analysis Components

Presets and Post-processing

Presets are configurations that define a curated prompt, specific generation parameters, and an optional response parser. If a preset defines a parseResponse function, the system will attempt to transform the raw model string (e.g., into JSON) before returning it to the user.

export interface PresetConfig {
  name: string;
  prompt: string;
  params: {
    max_new_tokens: number;
    temperature: number;
    do_sample: boolean;
  };
  requiresAudio: boolean;
  responseFormat: "json" | "text";
  parseResponse?: (raw: string) => unknown;
}

Sources: lib/flamingo/presets/types.ts:7-35, lib/flamingo/processAnalyzeMusicRequest.ts:98-106

External Model Integration

The system communicates with the Music Flamingo model hosted on Modal. This integration is handled by callFlamingoGenerate, which performs a POST request to the FLAMINGO_GENERATE_URL.

The FlamingoGenerateResult interface strictly expects a response string and an elapsed_seconds number.

Sources: lib/flamingo/callFlamingoGenerate.ts:19-45, lib/flamingo/isFlamingoGenerateResult.ts:6-10

Error Handling

The system implements robust error handling at multiple levels:

• Authentication Errors: Managed by validateAuthContext, returning unauthorized responses if keys are missing or invalid.
• Validation Errors: Managed by Zod schemas, returning 400 status codes with details on missing_fields.
• Infrastructure Errors: If the Modal endpoint returns a non-OK status (e.g., 503), the system logs the error and returns a 500 status to the client.
• Processing Errors: If a parseResponse function fails for a specific preset, the system falls back to returning the raw model response rather than failing the entire request.

Sources: lib/flamingo/postFlamingoGenerateHandler.ts:32-47, lib/flamingo/callFlamingoGenerate.ts:37-40, lib/flamingo/processAnalyzeMusicRequest.ts:103-105

Conclusion

Flamingo Music Analysis provides a structured interface for complex audio reasoning. By combining curated presets with flexible custom prompting and a centralized processing logic, the system serves as a powerful tool for both automated catalog metadata generation and interactive musical analysis within the Recoup platform.