End-to-End Testing
Relevant source files
The following files were used as context for generating this wiki page:
End-to-End (E2E) testing in the GitHub MCP Server project is designed to validate the "black box" behavior of the server artifacts. These tests ensure that the server, when built and running as a container, correctly interacts with the live GitHub API via the Model Context Protocol (MCP) using standard input/output (stdio).
The E2E suite complements unit tests by verifying the integration of multiple components, including Docker image building, container execution, and real-world network interactions with GitHub. Because these tests interact with live resources, they are gated behind specific build tags and require valid GitHub credentials.
Sources: e2e/README.md:3-8, e2e/e2e_test.go:1-25
Test Architecture and Components
The E2E testing framework supports two distinct modes of execution: a full black-box mode using Docker and a debug mode using an in-process server.
Execution Modes
- Standard Mode (Docker-based): This mode builds the
github-mcp-serverDocker image and runs it. The test client communicates with the containerized server over stdio. This provides the highest confidence in the final artifact but makes debugging internal server state more difficult. - Debug Mode (In-process): By setting the
GITHUB_MCP_SERVER_E2E_DEBUG=trueenvironment variable, the tests run an in-process version of the MCP server. This bypasses Docker and the Cobra/Viper configuration parsing, allowing developers to set breakpoints within the server code for easier troubleshooting.
Sources: e2e/README.md:10-15, e2e/README.md:52-58, e2e/e2e_test.go:110-150
Key Components
The following diagram illustrates the flow of a standard E2E test execution using the Docker-based approach:
The diagram shows the lifecycle of an E2E test, from the initial Docker build to the communication between the MCP client and the containerized server.
Sources: e2e/e2e_test.go:85-105, e2e/e2e_test.go:115-135
Environment Configuration
E2E tests require specific environment variables to function, primarily for authentication and targeting specific GitHub environments.
| Variable | Description | internal Mapping |
|---|---|---|
GITHUB_MCP_SERVER_E2E_TOKEN | A valid GitHub Personal Access Token (PAT). | GITHUB_PERSONAL_ACCESS_TOKEN |
GITHUB_MCP_SERVER_E2E_HOST | The GitHub host to target (e.g., for GHEC/GHES). | GITHUB_HOST |
GITHUB_MCP_SERVER_E2E_DEBUG | Enables in-process server execution for debugging. | N/A |
The tests use a sync.Once implementation to ensure the token and host are retrieved only once per test run, and the Docker image is built only once to optimize execution time.
Sources: e2e/README.md:17-25, e2e/e2e_test.go:33-80
Testing Lifecycle and Implementation
The E2E tests are implemented using the standard Go testing package but are isolated via the //go:build e2e constraint.
Setup and Teardown
Tests follow a strict lifecycle to ensure clean environments:
- Setup:
setupMCPClientinitializes the client, ensures the Docker image is ready, and handles server initialization via the MCPinitializerequest. - Execution: The test calls specific tools (e.g.,
get_me,create_repository) using theCallToolmethod of the MCP client. - Teardown: Since the project focuses on GitHub integration, some tests involve state mutation. The tests utilize
t.Cleanupto delete temporary resources, such as repositories created during the test run, using a direct GitHub REST client.
Sources: e2e/e2e_test.go:107-160, e2e/e2e_test.go:210-230
Sequence of a Tool Call Test
This sequence diagram represents the interaction during a tool call test like TestGetMe:
Sources: e2e/e2e_test.go:162-185, e2e/README.md:3-8
Limitations and Exclusions
To maintain manageable costs and reliability, the E2E suite has intentional limitations:
- Global State Mutation: Tools that change global user state (e.g., marking all notifications as read) are generally excluded from E2E tests because they are not idempotent and could interfere with the tester's account. These rely on unit tests and manual verification.
- Verbosity: The tests are intentionally verbose and repetitive to avoid premature abstractions, allowing the suite to mature before refactoring.
- Idempotency: Developers are encouraged to write tests that clean up after themselves to ensure the suite can be run repeatedly against the same account.
Sources: e2e/README.md:60-75, e2e/README.md:77-81
Comparison with Unit Testing
While E2E tests use live APIs, the project heavily utilizes unit tests with mocked HTTP clients for rapid feedback and comprehensive edge-case coverage.
| Feature | Unit Testing | End-to-End Testing |
|---|---|---|
| Backend | Mocked HTTP (go-github-mock) | Live GitHub API |
| Speed | Very Fast | Slow (builds & network) |
| Coverage | Internal logic, error handling | Integration, Auth, Docker |
| Requirements | None | PAT, Docker, Network |
Sources: pkg/github/pullrequests_test.go:35-60, pkg/github/issues_test.go:40-70, e2e/README.md:3-8
Summary
The End-to-End testing suite provides a critical final check for the GitHub MCP Server. By running the actual Docker artifact against the real GitHub API, it ensures that authentication, networking, and protocol implementation work correctly in a production-like environment. While limited in scope to reduce maintenance, it serves as the definitive validation for the server's black-box behavior.
Sources: e2e/README.md:3-10, e2e/e2e_test.go:10-30