GraphQL API

Relevant source files

The following files were used as context for generating this wiki page:

• datahub-graphql-core/README.md
• datahub-web-react/README.md
• datahub-web-react/src/app/entityV2/document/changeHistory/ARCHITECTURE.md
• docs-website/sidebars.js
• docs-website/static/llms.txt
• metadata-ingestion/tests/unit/ingestion/source/confluence/fixtures/default_repo/DOCS/20006.md
• datahub-graphql-core/src/main/java/com/linkedin/datahub/graphql/GraphQLEngine.java
• datahub-graphql-core/src/main/resources/app.graphql

The DataHub GraphQL API serves as a flexible, graph-based abstraction layer built on top of the Generalized Metadata Service (GMS). It provides a unified interface for reading and writing metadata related to various entities such as Datasets, CorpUsers, Dashboards, and more. This API is designed to support both internal DataHub UI requirements and programmatic access from external non-Java environments.

The API is packaged as a shared library module (datahub-graphql-core), allowing it to be deployed in different modes depending on the architectural needs, such as a standalone facade or embedded within other Java services like datahub-frontend.

Sources: datahub-graphql-core/README.md:5-10, docs-website/static/llms.txt:60-65

Core Architecture and Components

The GraphQL implementation in DataHub consists of several decoupled components that work together to resolve complex metadata queries.

Architectural Overview

The following diagram illustrates how the GraphQL engine sits between client applications and the underlying metadata storage services.

The GraphQL engine abstracts the complexity of the GMS layer by providing a graph-based representation.
Sources: datahub-graphql-core/README.md:7-15, datahub-web-react/README.md:155-160

Primary Components

Component	Description
GMS Schema	A GraphQL schema definition based on GMS Pegasus models, located in the resources folder.
Data Fetchers	Also known as Resolvers, these components are responsible for resolving individual fields in the schema.
Data Loaders	Components used to fetch data from downstream sources efficiently by batching requests to reduce the N+1 problem.
GraphQLEngine	A wrapper for the standard graphql-java object, offering a Builder API for configuration.
GMSGraphQLEngine	A pre-configured engine capable of resolving the standard GMS schema using built-in fetchers and loaders.

Sources: datahub-graphql-core/README.md:12-23

Deployment Modes

The DataHub GraphQL API can be operated in two primary modes:

1. Standalone Mode: Acts as a GraphQL facade. This is primarily intended for programmatic access to the GMS graph from non-Java environments (e.g., Python scripts or Node.js applications).
2. Embedded Mode: Leveraged within another Java server (such as datahub-frontend) to surface an extended GraphQL schema. This allows the UI to fetch both core metadata and application-specific state in a single request.

Sources: datahub-graphql-core/README.md:25-30

Extending the GraphQL Model

Adding a new entity or field to the graph involves a structured process to ensure parity between the GMS Pegasus models and the GraphQL schema.

Implementation Flow for New Entities

This flow ensures that new metadata types are correctly resolved and exposed through the API.
Sources: datahub-graphql-core/README.md:34-75

Key Extension Steps

• Schema Extension: New types or inputs are added to entity.graphql. GraphQL models generally mirror GMS models but must transform Maps into lists of {key, value} objects.
• EntityType Implementation: Classes like DatasetType.java define how to load entities and map them to the GraphQL data model.
• Search Integration: To make a new entity searchable, it must be added to the SearchAcrossEntitiesResolver list and the UrnToEntityMapper.

Sources: datahub-graphql-core/README.md:37-45, datahub-graphql-core/README.md:80-90

Integration with React UI

The datahub-web-react application relies heavily on the GraphQL API for all data requirements. It uses the Apollo Client to issue queries defined in .graphql files.

Data Fetching Example: Change History

The Document Change History feature illustrates a practical application of the GraphQL API, using it to retrieve a timeline of document modifications.

The architecture leverages specific utility functions to handle the conversion of GraphQL StringMapEntry lists into usable key-value records for the UI.
Sources: datahub-web-react/src/app/entityV2/document/changeHistory/ARCHITECTURE.md:25-35, datahub-web-react/src/app/entityV2/document/changeHistory/ARCHITECTURE.md:105-125

React Entity Registry

The EntityRegistry in the React app acts as a layer of abstraction that maps GraphQL EntityType symbols to specific rendering components (profiles, previews).
Sources: datahub-web-react/README.md:148-155

Query Reference and Best Practices

Primary Key Lookups

The convention for entity retrieval is a top-level Query that accepts a single "urn" parameter.
Sources: datahub-graphql-core/README.md:46-50

Sample Query

query {
  user(urn: "urn:li:corpuser:datahub") {
    username
    info {
      email
      fullName
    }
  }
}

Sources: metadata-ingestion/tests/unit/ingestion/source/confluence/fixtures/default_repo/DOCS/20006.md:15-22

Conclusion

The GraphQL API is a cornerstone of the DataHub architecture, providing a flexible and performant way to interact with the Metadata Graph. By decoupling the schema from the underlying storage and providing clear patterns for extensibility, it enables both a rich user interface and robust programmatic integrations. Rebuilding the module via ./gradlew datahub-graphql-core:build ensures that schema changes are reflected in auto-generated Java POJOs used by resolvers.

Sources: datahub-graphql-core/README.md:5-10, datahub-graphql-core/README.md:52-55