Load Balancing

Relevant source files

The following files were used as context for generating this wiki page:

• internal/server/load_balancer.go
• internal/server/target_weight.go
• internal/server/session_affinity.go
• internal/server/service.go
• internal/cmd/deploy.go
• internal/server/load_balancer_test.go
• internal/server/target_weight_test.go

Load balancing in kamal-proxy is responsible for distributing incoming network traffic across multiple healthy backend targets. It supports various strategies including weighted round-robin, reader/writer split, and session affinity to ensure high availability and optimal resource utilization. The system is designed to handle dynamic deployments, retries for idempotent requests, and target health monitoring.

Sources: internal/server/load_balancer.go, internal/server/service.go:230-245

Architecture and Components

The load balancing logic is encapsulated within the LoadBalancer struct, which manages a list of Target objects. Each Service within the proxy can have an active load balancer and a rollout load balancer for canary deployments.

Data Structures

Component	Description
LoadBalancer	Coordinates target selection, health tracking, and session affinity for a service.
Target	Represents a single backend instance, including its address, weight, and current health state.
TargetList	A collection of targets, split into writers (primary) and readers (replicas).
TargetOptions	Configuration for targets, including health check settings and connection limits.

Sources: internal/server/load_balancer.go:19-45, internal/server/service.go:82-168

Request Flow Diagram

The following diagram illustrates how a request is routed through the load balancer to a specific target.

Target Selection Strategies

Weighted Round-Robin

kamal-proxy implements a smooth weighted round-robin algorithm (similar to Nginx). Every target is assigned a weight (defaulting to 1). The proxy tracks "credits" for each target; the target with the most credit serves the request and then pays back the total weight of the pool. This ensures that a target with weight 5 receives exactly 5 times the traffic of a target with weight 1, with requests interleaved rather than sent in large bursts.

Sources: internal/server/target_weight.go:68-96, internal/server/target_weight_test.go:94-118

Reader/Writer Split

Targets are categorized as either writers or readers.

• Writers: Handle all mutating methods (POST, PUT, DELETE, PATCH) and WebSockets by default.
• Readers: Handle GET and HEAD requests.
  If no readers are configured, writers handle all traffic.

Sources: internal/server/load_balancer.go:216-250, internal/server/load_balancer_test.go:88-120

Session Affinity

Session affinity (sticky sessions) ensures a client remains pinned to the same backend target.

1. Selection: On the first request, a target is chosen via the standard algorithm.
2. Pinning: The proxy issues an HttpOnly cookie containing an HMAC-signed identifier of the target.
3. Persistence: Subsequent requests containing this cookie are routed to the same target if it remains healthy.
4. Fallback: If the pinned target becomes unhealthy, the proxy re-runs the selection algorithm and issues a new pin.

Sources: internal/server/session_affinity.go:12-48, internal/server/service.go:119-123

Configuration Options

Load balancing behavior is configured via deployment flags or RPC calls.

Flag	Type	Default	Description
--target	String Slice	[]	Target URLs with optional weight (e.g., web:3000;weight=5).
--read-target	String Slice	[]	Read-only replicas with optional weights.
--weight	Integer	1	Share of traffic (1-1000) relative to peers.
--session-affinity	Boolean	false	Enables sticky sessions via cookies.
--session-affinity-cookie	String	kamal-proxy-session	Custom name for the affinity cookie.
--target-try-duration	Duration	0	How long to retry placing a request on a healthy target.

Sources: internal/cmd/deploy.go:49-55, internal/cmd/deploy.go:102-108, internal/server/target_weight.go:18-22

Implementation Details

Smooth Weight Selection

The selection logic maintains currentWeight for each target to interleave traffic effectively.

func (tl TargetList) nextWeighted() *Target {
	var chosen *Target
	total := 0

	for _, target := range tl {
		target.currentWeight += target.weight
		total += target.weight

		if chosen == nil || target.currentWeight > chosen.currentWeight {
			chosen = target
		}
	}

	if chosen == nil {
		return nil
	}

	chosen.currentWeight -= total
	return chosen
}

Sources: internal/server/target_weight.go:80-96

Target Health and Retries

The load balancer only considers targets in the TargetStateHealthy state. For idempotent requests (methods without a body like GET or HEAD), the proxy can retry the request on a different target if the first attempt fails, governed by the TargetTryDuration configuration.

Sources: internal/server/load_balancer.go:180-210, internal/server/service.go:114-117

Load balancing in kamal-proxy provides a robust foundation for high-availability web applications by combining granular traffic control via weights with resilience features like automatic health-based retries and session persistence.