Architecture Overview

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• client/cmd/main.go
• api/securitytest/securitytest.go
• cli/analysis/analysis.go
• cli/types/types.go
• client/analysis/output.go
• api/util/api/api.go
• client/integration/sonarqube/sonarqube.go
• api/securitytest/brakeman.go
• api/securitytest/safety.go

huskyCI is an open-source tool designed to orchestrate various static security analysis tools and centralize their results. It integrates into CI/CD pipelines, running security tests in isolated Docker containers for each specified language or tool. The system is composed of a client-side CLI, a backend API, and a fleet of Docker images, each containing a specific security tool.

The core principle is to provide a unified interface for running multiple security scanners and consolidating their findings into a standardized format. This allows developers to get immediate feedback on security issues within their CI process and enables security teams to gather metrics from a central database.

Sources: README.md

Core Components

The huskyCI ecosystem is built around three main components: the client, the API, and the security tool containers.

• huskyCI Client: A command-line application (huskyci-client) that developers execute within their CI/CD pipeline. Its primary role is to communicate with the huskyCI API to initiate and monitor security scans.
• huskyCI API: A backend service that receives analysis requests from the client. It orchestrates the execution of security tests by spinning up Docker or Kubernetes containers, processes the raw output from each tool, and stores the results.
• Security Tool Containers: Each security tool (e.g., gosec, bandit, brakeman) is packaged into a dedicated Docker image. The API uses these images to run scans against the target codebase in an isolated environment.
• Database: A MongoDB instance is used to store analysis results, security test configurations, and user data, centralizing all information.

This diagram illustrates the high-level interaction between the components.

Client-Side Workflow

The huskyci-client is the entry point for any analysis. It follows a clear, sequential process to manage the scan and report the results.

Sources: client/cmd/main.go

This sequence diagram shows the steps taken by the client application.

The client's main logic is orchestrated in client/cmd/main.go:

1. Initialization: Checks for the JSON argument to determine output format and verifies required environment variables (HUSKYCI_CLIENT_REPO_URL, HUSKYCI_CLIENT_REPO_BRANCH, HUSKYCI_CLIENT_API_ADDR).
2. Start Analysis: Sends a request to the huskyCI API to begin a new analysis and receives a unique Run ID (RID).
3. Monitor Analysis: Periodically polls the API using the RID to check if the analysis has finished.
4. Process and Print Results: Once the analysis is complete, it retrieves the full results. The analysis.PrintResults function formats and prints the vulnerabilities to standard output. It can produce either a human-readable summary or a full JSON object.
5. SonarQube Integration: It calls sonarqube.GenerateOutputFile to convert the huskyCI results into a sonarqube.json file, which can be used for integration with SonarQube.
6. Exit Status: The client exits with a specific code to signal the CI pipeline's status:
   • 0: No blocking (HIGH/MEDIUM) vulnerabilities found.
   • 190: HIGH or MEDIUM severity vulnerabilities were found.
   • 1: An error occurred during the client's execution.

Sources: client/cmd/main.go:21-135, client/analysis/output.go, client/integration/sonarqube/sonarqube.go

API and Security Test Orchestration

The huskyCI API is responsible for the heavy lifting of running the security scans.

Sources: api/securitytest/securitytest.go

Analysis Flow

When the API receives a request, it initiates a series of security tests.

1. Scan Initialization: A SecTestScanInfo struct is created for each security test to be run. This struct holds all context for the scan, including the repository URL, branch, and the security test configuration.
2. Container Execution: The Start() method is called, which determines the infrastructure to use (docker or kubernetes) from environment variables. It then executes the appropriate function (dockerRun or kubeRun) to start a container with the tool's image and command.
3. Output Analysis: After the container finishes, the analyze() method is called. It uses a map (securityTestAnalyze) to delegate the parsing of the raw output to a tool-specific function (e.g., analyzeBrakeman, analyzeSafety).
4. Vulnerability Standardization: Each analyze<Tool> function (e.g., in api/securitytest/brakeman.go) unmarshals the tool's native JSON output and maps the findings into the standardized types.HuskyCIVulnerability struct. This ensures all results, regardless of the source tool, have a consistent format.
5. Result Storage: The standardized vulnerabilities are stored within the SecTestScanInfo and eventually persisted to the database as part of the main Analysis document.

Sources: api/securitytest/securitytest.go:13-33, 95-136, api/securitytest/brakeman.go:37-52, api/securitytest/safety.go:32-68

Supported Security Tools

huskyCI determines which tools to run by detecting the languages present in the codebase. This mapping is defined on the client-side and implicitly handled by the API.

Language	Security Tool(s)	Docker Image
Go	Gosec	huskyci/gosec
Python	Bandit, Safety	huskyci/bandit, huskyci/safety
Ruby	Brakeman	huskyci/brakeman
JavaScript	NpmAudit, YarnAudit	huskyci/npmaudit, huskyci/yarnaudit
Java	SpotBugs	huskyci/spotbugs
HCL	TFSec	huskyci/tfsec
C#	SecurityCodeScan	huskyci/securitycodescan
Generic	Gitleaks	huskyci/gitleaks

Sources: cli/analysis/analysis.go:171-197, [README.md]

Data Structures

The system relies on a set of well-defined Go structs to pass data between the client, API, and database.

Sources: cli/types/types.go

Core Data Models

This diagram shows the relationship between the main data structures used in an analysis.

Analysis*: The top-level object representing a single huskyCI run. It contains the overall status, results, and a list of all containers that were executed.

• Container: Represents the execution of a single security tool. It holds the container ID, status (finished, error), result (passed, failed), and the raw output (COutput).
• SecurityTest: Defines a security tool, including its Docker image, command, and target language.
• HuskyCIResults: A collection of results, categorized by language (e.g., GoResults, PythonResults).
• HuskyCISecurityTestOutput: Contains arrays of vulnerabilities, categorized by severity (LowVulns, MediumVulns, HighVulns).
• HuskyCIVulnerability: The standardized vulnerability format, containing all relevant details like the affected file, line number, code snippet, and severity.

Sources: cli/types/types.go:28-175