Webhook Idempotency & Delivery Tracking

Relevant source files

The following files were used as context for generating this wiki page:

• packages/db/prisma/schema.prisma
• packages/ingestion/src/process-webhook.ts
• apps/web/app/api/webhooks/github/route.ts
• README.md
• packages/review/src/orchestrator.ts
• apps/worker/src/index.ts

The AI Code Reviewer system implements a robust webhook handling architecture designed to ensure reliable, at-least-once delivery while maintaining strict idempotency. Because GitHub may re-deliver webhooks upon timeout or delivery failure, the system tracks every unique delivery ID to prevent duplicate processing of the same event.

The architecture decouples the high-traffic ingestion point (the web server) from the resource-intensive processing logic (the background worker). This ensures the web server can acknowledge GitHub webhooks within the required 5-second window while the actual AI review, which may take 30+ seconds, happens asynchronously.

Sources: apps/web/app/api/webhooks/github/route.ts:13-33, README.md:92-98

Delivery Tracking Architecture

All incoming GitHub webhooks are first persisted in the database before any processing occurs. This provides an audit trail and the primary mechanism for deduplication.

Data Model: WebhookDelivery

The WebhookDelivery model acts as the source of truth for all received events.

Field	Type	Description
deliveryId	String	Unique ID sent by GitHub in the X-GitHub-Delivery header.
event	String	The type of GitHub event (e.g., pull_request).
payload	Json	The full raw JSON body of the webhook.
receivedAt	DateTime	Timestamp when the webhook hit the web server.
processedAt	DateTime?	Timestamp when the worker successfully finished processing.

Sources: packages/db/prisma/schema.prisma:418-428, apps/web/app/api/webhooks/github/route.ts:50-53

The Ingestion Flow

When GitHub sends a POST request, the web server performs the following sequence:

1. Signature Verification: Validates the X-Hub-Signature-256 using HMAC-SHA256 to ensure the request originated from GitHub.
2. Deduplication Check: Performs an upsert on the WebhookDelivery table using the deliveryId.
3. Short-Circuiting: If the record already has a processedAt timestamp, the system returns a 200 OK without re-enqueuing the job.
4. Asynchronous Enqueue: If new or unprocessed, it enqueues a webhook.process job via pg-boss.

The following diagram illustrates the lifecycle of a webhook from arrival to ingestion:

Idempotency Strategies

Idempotency is enforced at multiple layers to handle both GitHub re-deliveries and job retries within the worker.

Webhook Layer Idempotency

At the entry point, the system uses the deliveryId to detect duplicates. GitHub re-delivers the exact same deliveryId if it does not receive a response in time. By checking processedAt, the web server avoids triggering a second background job for an event that has already been handled.

Sources: apps/web/app/api/webhooks/github/route.ts:88-92, README.md:468-468

Business Logic Idempotency (AiReview)

Even if a job is enqueued twice (e.g., due to a database race condition), the ingestion handler performs a secondary check before creating a review. For pull_request events, the system checks for an existing AiReview record matching the unique combination of repositoryId, pullRequestNumber, and commitSha.

// From packages/ingestion/src/process-webhook.ts
const existing = await prisma.aiReview.findUnique({
  where: {
    repositoryId_pullRequestNumber_commitSha: {
      repositoryId: repo.id,
      pullRequestNumber: pr.number,
      commitSha,
    },
  },
});

if (existing) {
  log.debug({ aiReviewId: existing.id, status: existing.status }, 'AiReview already exists — skipping');
  return;
}

Sources: packages/ingestion/src/process-webhook.ts:114-129, packages/db/prisma/schema.prisma:499-499

Orchestrator Idempotency

The ReviewPr job handler checks the current status of the AiReview record. If the status is not PENDING (e.g., it is already RUNNING or COMPLETED), the handler terminates immediately to prevent duplicate LLM calls or duplicate comments on GitHub.

Sources: packages/review/src/orchestrator.ts:47-52

Component Interaction

The system relies on pg-boss as the transactional job queue, which allows the database and the queue to stay in sync.

Summary of Delivery Statuses

The system tracks the lifecycle through the AiReviewStatus enum in the database.

Status	Description
PENDING	Review record created, job enqueued but not yet started by worker.
RUNNING	Worker has picked up the job and is currently calling LLMs.
COMPLETED	Review finished, comments posted to GitHub, and summary saved.
FAILED	An error occurred; the error message is stored in the error column.
SKIPPED	Configuration settings (e.g., ignore paths) caused the review to be aborted.

Sources: packages/db/prisma/schema.prisma:503-509, packages/review/src/orchestrator.ts:98-103

Conclusion

Webhook idempotency and delivery tracking are critical to the reliability of the AI Code Reviewer. By combining persistent delivery logs, unique database constraints on review entities, and state-aware job handlers, the system ensures that every pull request update is reviewed exactly once, regardless of network instability or GitHub delivery retries.