Testing Strategy

Relevant source files

The following files were used as context for generating this wiki page:

• Packages/src/Cli~/src/tests/cli-e2e.test.ts
• Assets/Editor/TestRunnerMenu.cs
• Packages/src/TypeScriptServer~/src/tests/integration/execute-tool-use-case.integration.test.ts
• Assets/Tests/Editor/GetLogsUseCaseTests.cs
• Assets/Tests/Editor/DynamicCodeToolTests/RestrictedModeDangerousApiTests.cs
• Packages/src/TypeScriptServer~/src/tests/integration/refresh-tools-use-case.integration.test.ts
• Packages/src/Cli~/src/tests/skills-manager.test.ts
• Assets/Editor/UnitySearchTester.cs
• Assets/Tests/Editor/DynamicCodeToolTests/WrapCodeWithUsingStatementsAdvancedTests.cs
• Packages/src/ARCHITECTURE.md
• README.md

The uLoopMCP project employs a multi-layered testing strategy to ensure the reliability, correctness, and security of its components. The strategy spans from low-level unit tests for individual functions in both C# and TypeScript to high-level end-to-end (E2E) tests that validate the entire system. This comprehensive approach is critical for a system designed to allow AI agents to operate a Unity project autonomously.

The testing philosophy covers three main areas: C# testing within the Unity Editor, TypeScript testing for the server and CLI components, and E2E tests that simulate real-world usage. This ensures that each part of the architecture, from the core tool logic in Unity to the communication bridge and client-facing CLI, is thoroughly vetted.

Testing Layers Overview

The project's testing is divided into distinct layers, each with its own tools and scope.

Test Layer	Technology	Tools Used	Scope	Example Files
C# Unit & Integration	C# (Unity)	NUnit, Unity Test Runner	Individual classes, Use Cases, and service interactions within Unity.	GetLogsUseCaseTests.cs, RestrictedModeDangerousApiTests.cs
TypeScript Unit	TypeScript	Jest	Pure functions and isolated components in the CLI and server.	skills-manager.test.ts
TypeScript Integration	TypeScript	Jest	Use Case orchestration and interaction with service interfaces.	execute-tool-use-case.integration.test.ts
End-to-End (E2E)	Node.js, Shell	Jest, child_process	Full CLI command execution against a running Unity Editor instance.	cli-e2e.test.ts
Manual & Helper	C# (Unity)	Unity EditorWindow, MenuItem	Developer-facing tools for quick, manual validation of features.	UnitySearchTester.cs, TestRunnerMenu.cs

Sources: Assets/Tests/Editor/GetLogsUseCaseTests.cs, Packages/src/Cli~/src/tests/cli-e2e.test.ts, Assets/Editor/UnitySearchTester.cs

C# (Unity Editor) Testing

Testing on the C# side is performed within the Unity Editor using the integrated NUnit framework and Test Runner. This layer focuses on the core logic of the MCP tools and their underlying services.

Sources: Assets/Tests/Editor/GetLogsUseCaseTests.cs, Assets/Tests/Editor/DynamicCodeToolTests/RestrictedModeDangerousApiTests.cs

Unit and Integration Testing

Unit and integration tests for C# components are structured to validate individual Use Cases and their interactions with services, aligning with the project's DDD-inspired architecture.

The diagram below shows the dependencies for testing the GetLogsUseCase.

Key characteristics of C# testing include:

• Test Isolation: Each test sets up its own environment and cleans up afterward. For instance, GetLogsUseCaseTests clears the developer console before each test.
• Boundary Value Testing: Tests cover edge cases, such as MaxCount = 0 or negative values, to ensure robust behavior.
• Error and Cancellation Handling: Tests verify that the system handles null parameters, invalid inputs, and task cancellations gracefully.
• Security Validation: A critical component of the testing strategy is verifying the security model. RestrictedModeDangerousApiTests.cs contains a comprehensive suite of tests that ensure dangerous APIs (e.g., File.Delete, Process.Start, networking) are blocked in Restricted mode, while safe APIs remain accessible.

// Example from RestrictedModeDangerousApiTests.cs
[Test]
public async Task TestRestrictedMode_FileDelete_Blocked()
{
    string code = @"
        System.IO.File.Delete(""test.txt"");
        return ""File deleted"";
    ";
    
    ExecutionResult result = await executor.ExecuteCodeAsync(
        code, "TestCommand", null, CancellationToken.None, compileOnly: false
    );
    
    Assert.IsFalse(result.Success, "File.Delete should be blocked in Restricted mode");
    StringAssert.Contains("Dangerous", result.ErrorMessage, "Should report security violation");
}

Sources: Assets/Tests/Editor/DynamicCodeToolTests/RestrictedModeDangerousApiTests.cs:81-95

Manual and Helper Testing

To facilitate development and manual verification, the project includes custom Editor windows and menu items.

• TestRunnerMenu.cs: Adds menu items under uLoopMCP/Tools/Test Runner to execute specific test suites (e.g., all EditMode tests, CompileCommandTests) directly from the Unity menu. This allows for quick, targeted feedback during development.
• UnitySearchTester.cs: Provides a dedicated Editor window for manually testing the UnitySearchTool. Developers can input a search query, set parameters, and view the formatted results directly within Unity.

The flow for using the UnitySearchTester is as follows:

TypeScript (Server & CLI) Testing

The TypeScript server and CLI are tested using the Jest framework. The tests are divided into unit and integration tests, mirroring the Clean Architecture principles used in the TypeScript application.

Sources: Packages/src/TypeScriptServer~/src/tests/integration/execute-tool-use-case.integration.test.ts, Packages/src/Cli~/src/tests/skills-manager.test.ts

Unit Testing

Unit tests focus on pure functions with no external dependencies. For example, skills-manager.test.ts validates the parseFrontmatter function, which is responsible for parsing metadata from SKILL.md files. These tests cover various edge cases, such as missing frontmatter, boolean values, and malformed content.

Sources: Packages/src/Cli~/src/tests/skills-manager.test.ts:8-20

Integration Testing

Integration tests for TypeScript Use Cases validate the business logic and orchestration of services. These tests use mocked dependencies to isolate the Use Case from external infrastructure like the actual TCP connection.

The sequence diagram below illustrates the integration test for ExecuteToolUseCase.

This diagram shows how the test verifies the correct sequence of calls to ensure a tool is executed only if the connection is active and the tool exists.

Sources: Packages/src/TypeScriptServer~/src/tests/integration/execute-tool-use-case.integration.test.ts:50-77

End-to-End (E2E) Testing

E2E tests provide the highest level of confidence by testing the entire system from the command line to the Unity Editor. These tests require a running Unity instance with the uLoopMCP package installed.

Sources: [Packages/src/Cli~/src/tests/cli-e2e.test.ts]

CLI E2E Tests

The E2E tests are orchestrated by Jest and execute the bundled uloop CLI tool using Node.js's child_process.execSync. The tests cover a wide array of commands, including compile, get-logs, execute-menu-item, and skills management.

A key feature of the E2E test suite is its resilience to Unity's Domain Reloads. A wrapper function, runCliWithRetry, handles errors related to domain reloads by waiting and retrying the command.

The E2E test execution flow is as follows:

A typical E2E test involves multiple CLI calls to set up a state, perform an action, and then verify the result.

// Example from cli-e2e.test.ts
it('should clear console and verify logs are empty', () => {
  // First output some logs
  runCli(`execute-menu-item --menu-item-path "${TEST_LOG_MENU_PATH}"`);

  // Clear console
  const result = runCliJson<{ Success: boolean }>('clear-console');
  expect(result.Success).toBe(true);

  // Verify logs are cleared
  const logsAfterClear = runCliJson<{ TotalCount: number; Logs: unknown[] }>('get-logs');
  expect(logsAfterClear.TotalCount).toBe(0);
  expect(logsAfterClear.Logs.length).toBe(0);
});

Sources: Packages/src/Cli~/src/tests/cli-e2e.test.ts:192-206

Conclusion

The testing strategy for uLoopMCP is robust and multi-faceted, providing strong guarantees about the system's functionality, reliability, and security. By combining C# unit tests in Unity, TypeScript unit and integration tests for the server/CLI, and comprehensive E2E tests, the project ensures that developers and AI agents can confidently use the provided tools. The emphasis on security testing, particularly for the dynamic code execution feature, is a cornerstone of this strategy, making uLoopMCP a more trustworthy tool for AI-driven development.