Relevant source files

The following files were used as context for generating this wiki page: - [contribute/architecture/k8s-inspired-backend-arch.md](https://github.com/ajit216/grafana/blob/main/contribute/architecture/k8s-inspired-backend-arch.md) - [pkg/services/apiserver/README.md](https://github.com/ajit216/grafana/blob/main/pkg/services/apiserver/README.md) - [pkg/registry/apis/provisioning/resources/resources.go](https://github.com/ajit216/grafana/blob/main/pkg/registry/apis/provisioning/resources/resources.go) - [apps/iam/pkg/apis/iam_manifest.go](https://github.com/ajit216/grafana/blob/main/apps/iam/pkg/apis/iam_manifest.go) - [devenv/scopes/scopes.go](https://github.com/ajit216/grafana/blob/main/devenv/scopes/scopes.go) - [pkg/apimachinery/apis/common/v0alpha1/types.go](https://github.com/ajit216/grafana/blob/main/pkg/apimachinery/apis/common/v0alpha1/types.go)

# Kubernetes-Inspired Resource Model Grafana is adopting a resource-oriented model for core resources like dashboards, folders, and alert rules. This model allows users to interact with resources by defining a desired state via structured objects sent to new Resource APIs (`/apis/...`). Grafana then reconciles the system's current state with this declared state, following Kubernetes patterns for declarative, eventually-consistent design. The architecture addresses limitations in legacy APIs, such as inconsistent structure, lack of versioning, and the absence of machine-readable schemas. By adopting Kubernetes conventions for URL structures, versioning (e.g., `v0alpha1`, `v1beta1`), and schemas, Grafana enables standardized observability-as-code tooling and improved extensibility for plugins. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:3-15](), [pkg/services/apiserver/README.md:42-45]() ## 1. Core Architecture and Components The backend architecture is structured into layers that mimic the Kubernetes API server while remaining integrated within the Grafana process. ### API Layer and URL Structure Resource APIs follow strict Kubernetes conventions for paths: `/apis///namespaces///`. Namespaces in this context map to Grafana Organization IDs or Grafana Cloud Stack IDs for scoping. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:24-35]() ### Component Interaction The following diagram illustrates the relationship between the legacy API paths and the new Kubernetes-inspired resource paths. ```mermaid graph TD User([User]) --> UI[Grafana UI] User -->|"Direct Call"| LegacyAPI["Legacy API (/api/...)"] User -->|"Direct Call"| ResourceAPI["Resource API (/apis/...)"] UI --> LegacyAPI UI --> ResourceAPI subgraph ServiceLayer["Service Layer"] Handlers["Resource Handlers"] K8sFacade["K8sHandler Facade"] Logic["Service Logic"] end ResourceAPI --> Handlers LegacyAPI --> Logic Logic -.->|"Feature Flag"| K8sFacade K8sFacade -.-> ResourceAPI subgraph StorageLayer["Storage Layer"] UnifiedStorage["Unified Storage"] SQLDB[("SQL Database")] end Handlers --> UnifiedStorage UnifiedStorage --> SQLDB Logic --> SQLDB ``` Explanation: This flowchart shows how users and the UI interact with both Legacy and Resource APIs, and how Unified Storage acts as the common persistence bridge to the SQL database. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:Request Flow Diagram]() ### Key System Components | Component | Description | | :--- | :--- | | **Resource APIs** | API endpoints mimicking Kubernetes structure, served directly by Grafana. | | **Unified Storage** | Internal abstraction for resource persistence, decoupling handlers from the backend. | | **K8sHandler** | A loopback client facade enabling legacy services to interact with Resource APIs. | | **App SDK** | Tools for defining resources using CUE and generating necessary Go/TS code. | Sources: [contribute/architecture/k8s-inspired-backend-arch.md:42-55]() ## 2. Resource Management Logic The model uses a schema-first approach, primarily implemented through the "Apps Approach" in the `apps/` directory. ### Resource Persistence and Versioning Unlike Kubernetes, which uses `etcd`, Grafana's Unified Storage utilizes existing SQL databases (PostgreSQL, MySQL, SQLite) for operational simplicity. It implements the Kubernetes `storage.Interface` to translate standard operations like `Create`, `Get`, `List`, and `Watch` into SQL queries. - **resourceVersion**: A monotonically increasing counter used for optimistic concurrency control. Updates fail with a `409 Conflict` if the provided version does not match the stored version. - **Native Watch**: Supported by writing modifications to a `resource_history` table, which internal pollers or Go channels monitor to stream events to clients. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:58-78]() ### Resource Definitions (IAM Example) Resources are defined with metadata and specific selectable fields. For example, the Identity and Access Management (IAM) app defines kinds like `User`, `Team`, and `Role`. ```go var appManifestData = app.ManifestData{ AppName: "iam", Group: "iam.grafana.app", PreferredVersion: "v0alpha1", Versions: []app.ManifestVersion{ { Name: "v0alpha1", Served: true, Kinds: []app.ManifestVersionKind{ { Kind: "User", Plural: "Users", Scope: "Namespaced", SelectableFields: []string{"spec.email", "spec.login"}, }, }, }, }, } ``` Sources: [apps/iam/pkg/apis/iam_manifest.go:24-85]() ## 3. Migration and Compatibility Grafana maintains a specific migration path to move resources from legacy handlers to the Kubernetes-inspired model without breaking existing workflows. ### Migration Sequence The migration for each resource type follows a phased approach: 1. **Schema Definition**: Define the resource kind using CUE. 2. **Handler Implementation**: Create Resource API handlers that interact with legacy storage. 3. **Feature Flagging**: Introduce a toggle (e.g., `kubernetesClient`) to route traffic. 4. **Unified Storage Migration**: Change the source of truth to Unified Storage under a second flag. 5. **Deprecation**: Remove legacy endpoints once the Resource API reaches GA. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:122-140]() ### Divergence from Kubernetes While the APIs are "Kubernetes-like," they are not strictly conformant. Notable differences include: - **Auth**: Uses Grafana standard auth (API keys, sessions, RBAC) instead of ServiceAccounts. - **Reconcilers**: Run internally within the Grafana process rather than as external operators. - **Data Size**: Grafana supports larger resources than standard `etcd` limits, impacting operations like `LIST`. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:98-118]() ### `kubectl` Integration For development, Grafana can generate a `grafana.kubeconfig` when the `grafanaAPIServerEnsureKubectlAccess` feature flag is enabled. This allows standard tools like `kubectl` to manage Grafana resources if the server is running over HTTPS. Sources: [contribute/architecture/k8s-inspired-backend-arch.md:146-160](), [pkg/services/apiserver/README.md:26-35]() ## 4. Resource Provisioning and Management The `ResourcesManager` handles the lifecycle of these objects, including file-based provisioning from Git repositories. ### Resource Identification Internal logic identifies resources using a combination of Name, Resource type, and Group. ```mermaid classDiagram class resourceID { +String Name +String Resource +String Group } class ResourcesManager { -ReaderWriter repo -FolderManager folders -Map resourcesLookup +WriteResourceFileFromObject() +WriteResourceFromFile() +RenameResourceFile() } ResourcesManager --> resourceID : uses ``` Explanation: This class diagram illustrates how the `ResourcesManager` uses a `resourceID` struct to track and manage resources in its internal lookup map. Sources: [pkg/registry/apis/provisioning/resources/resources.go:70-85]() ### Error Handling and Validation Provisioning logic wraps standard Kubernetes API errors (like `BadRequest`) and resource-specific errors into a `ResourceValidationError` to treat them as warnings during synchronization processes. Sources: [pkg/registry/apis/provisioning/resources/resources.go:30-60]() ## 5. Implementation Summary The Kubernetes-Inspired Resource Model is realized through a specific folder structure in the codebase: - `pkg/apimachinery`: Common types and utilities used by both clients and servers. - `pkg/apiserver`: Library code for the embedded API server. - `pkg/registry/apis`: Location where types are registered with the API server. - `apps/`: The modular implementation where resources are defined via CUE schemas. Sources: [pkg/services/apiserver/README.md:47-65]() This model signifies a fundamental shift in Grafana's backend, moving toward a unified, schema-driven, and declarative management system for all core features.