Adding a New Security Tool

Relevant source files

The following files were used as context for generating this wiki page:

• api/securitytest/securitytest.go
• api/util/api/api.go
• api/securitytest/brakeman.go
• api/securitytest/safety.go
• api/securitytest/securitycodescan.go
• client/analysis/output.go
• client/types/types.go
• cli/analysis/analysis.go
• client/integration/sonarqube/sonarqube.go
• CONTRIBUTING.md
• README.md

Introduction

huskyCI is an open-source orchestration tool designed to run various static security analysis tools and centralize their results. Its architecture is modular, allowing for the straightforward integration of new security scanners. This extensibility is a core feature, enabling the platform to adapt to new technologies and security challenges.

Adding a new security tool involves three main stages: containerizing the tool, integrating it with the huskyCI API by implementing a results parser, and updating the client-side components to display the new results. This guide provides a comprehensive walkthrough of the process, using examples from the existing codebase. For a list of currently supported tools, see the project's README.md.

Sources: README.md, api/securitytest/securitytest.go

Core Architecture

Each security tool in huskyCI runs inside an isolated Docker container. The huskyCI API is responsible for orchestrating these containers, whether on a local Docker engine or a Kubernetes cluster. After a container finishes its execution, the API retrieves its stdout, parses it to identify vulnerabilities, and stores the results in a database.

This diagram illustrates the high-level flow of a single security test run.

Sources: api/securitytest/securitytest.go:81-104

The properties of each security tool, such as its Docker image and execution command, are defined by the SecurityTest struct.

Field	Type	Description
Name	string	The unique name of the security tool (e.g., "brakeman").
Image	string	The Docker image name (e.g., "huskyci/brakeman").
ImageTag	string	The tag of the Docker image (e.g., "4.6.1").
Cmd	string	The shell command to be executed inside the container.
Language	string	The programming language this tool targets.
Default	bool	Whether the tool is enabled by default.
TimeOutInSeconds	int	The timeout for the container execution.

Sources: client/types/types.go:73-82

Step 1: Creating the Docker Image

The first step is to create a Docker image that contains your security tool. This image is the self-contained environment where the analysis will run.

Requirements

• The Docker image must have the security tool and all its dependencies installed.
• It must be capable of cloning a Git repository, as huskyCI passes the repository URL and branch to the container.
• The entrypoint or command should execute the security tool against the cloned code.
• Crucially, the tool's output must be formatted as a single-line JSON string and printed to stdout. The huskyCI API only parses the stdout of the container.

The Cmd field in the SecurityTest definition contains placeholders like %GIT_REPO% and %GIT_BRANCH%, which huskyCI replaces at runtime. A typical command clones the repository, navigates into it, runs the scan, and prints the JSON output.

Sources: api/securitytest/securitytest.go:108-111, client/integration/sonarqube/testdata/analysis/vulnerable_go_project.json (for cmd examples)

Step 2: Integrating with the huskyCI API

Once the Docker image is ready, you need to teach the huskyCI API how to use it and how to understand its output.

Sources: api/util/api/api.go, api/securitytest/securitytest.go

2.1. Registering the New Tool

The API needs to be aware of the new tool. This is done during application startup.

1. Add to the Tool List: In api/util/api/api.go, add the name of your new tool to the securityTests slice within the checkEachSecurityTest function.

   // api/util/api/api.go:200-201
   func (cH *CheckUtils) checkEachSecurityTest(configAPI *apiContext.APIConfig) error {
   	securityTests := []string{"enry", "gitauthors", "gosec", "brakeman", "bandit", "npmaudit", "yarnaudit", "spotbugs", "gitleaks", "safety", "tfsec", "securitycodescan", "your-new-tool"} // Add here
       // ...
   }
   

   Sources: api/util/api/api.go:201-202

2. Define the Tool Configuration: In the same file, add a new case to the switch statement in the checkSecurityTest function. Here, you will define the SecurityTest properties for your tool, such as the image name, tag, and command. This configuration will be stored in the database.

   // api/util/api/api.go:230-261
   func checkSecurityTest(securityTestName string, configAPI *apiContext.APIConfig) error {
       // ...
       switch securityTestName {
       // ...
       case "securitycodescan":
           securityTestConfig = *configAPI.SecurityCodeScanSecurityTest
       case "your-new-tool": // Add a new case
           // This config can be loaded from environment variables and API context, similar to existing tools.
           securityTestConfig = types.SecurityTest{
               Name:             "your-new-tool",
               Image:            "huskyci/your-new-tool",
               ImageTag:         "1.0.0",
               Cmd:              "git clone %GIT_REPO% code && cd code && your-tool-binary --output=json .",
               Language:         "YourLanguage",
               Default:          true,
               TimeOutInSeconds: 360,
           }
       default:
           return errors.New("securityTest name not defined")
       }
       // ...
   }
   

   Sources: api/util/api/api.go:230-261

2.2. Implementing the Output Parser

The API needs a dedicated function to parse the JSON output from your tool's container.

1. Create a New Parser File: Create a new file, e.g., api/securitytest/newtool.go.

2. Define Output Structs: In this new file, define Go structs that match the structure of your tool's JSON output. For example, Brakeman has BrakemanOutput and WarningItem structs.

   // Example from api/securitytest/brakeman.go
   type BrakemanOutput struct {
       Warnings []WarningItem `json:"warnings"`
       IgnoredWarnings []WarningItem `json:"ignored_warnings"`
   }
   
   type WarningItem struct {
       Type       string `json:"warning_type"`
       Message    string `json:"message"`
       File       string `json:"file"`
       Line       int    `json:"line"`
       Confidence string `json:"confidence"`
   }
   

   Sources: api/securitytest/brakeman.go:16-27

3. Create the Analysis Function: Implement the main analysis function (e.g., analyzeNewTool). This function will unmarshal the container's output, iterate through the findings, and map them to huskyCI's standardized HuskyCIVulnerability struct.

4. Register the Analysis Function: In api/securitytest/securitytest.go, add your new function to the securityTestAnalyze map.

   // api/securitytest/securitytest.go:20-33
   var securityTestAnalyze = map[string]func(scanInfo *SecTestScanInfo) error{
       "bandit":           analyzeBandit,
       "brakeman":         analyzeBrakeman,
       // ...
       "securitycodescan": analyzeSecurityCodeScan,
       "your-new-tool":    analyzeNewTool, // Add your new function here
   }
   

   Sources: api/securitytest/securitytest.go:20-33

The following diagram shows the logic flow within a typical analysis function.

Sources: api/securitytest/safety.go:34-70, api/securitytest/brakeman.go:30-50

Step 3: Updating the Client-Side

To ensure the results from the new tool are correctly displayed to the user, the client-side application must be updated.

3.1. CLI Tool Discovery

The huskyci-cli tool can detect languages in a local directory and suggest which security tests are available. To make it aware of your new tool, you need to update cli/analysis/analysis.go.

Add a new case for the target language in the getAvailableSecurityTests function.

// cli/analysis/analysis.go:148-167
func (a *Analysis) getAvailableSecurityTests(languages []string) map[string][]string {
	// ...
	for _, language := range languages {
		switch language {
		// ...
		case "C#":
			list[language] = []string{"huskyci/securitycodescan"}
        case "YourLanguage": // Add a new case
            list[language] = []string{"huskyci/your-new-tool"}
		}
	}
	// ...
	return list
}

Sources: cli/analysis/analysis.go:148-167

3.2. Client Output Processing

The huskyci-client is responsible for fetching the final analysis from the API and printing it.

1. Update Data Structures: In client/types/types.go, you may need to add a new field to the HuskyCIResults struct to hold your tool's output. If the tool targets a new language, you'll need a new struct (e.g., YourLanguageResults).

   // client/types/types.go
   type HuskyCIResults struct {
       // ...
       CSharpResults     CSharpResults     `bson:"csharpresults,omitempty" json:"csharpresults,omitempty"`
       GenericResults    GenericResults    `bson:"genericresults,omitempty" json:"genericresults,omitempty"`
       YourLanguageResults // Add new results struct
   }
   

   Sources: client/types/types.go:50-58

2. Update Summary Logic: In client/analysis/output.go, update the prepareAllSummary and printAllSummary functions to calculate and display the summary of vulnerabilities found by your new tool.

3. Create a Print Function: Create a new function, printSTDOUTOutput<NewTool>, to format and print the vulnerabilities to the console. This function should be called from the main printSTDOUTOutput function.

   // client/analysis/output.go
   func printSTDOUTOutput(analysis types.Analysis) {
       // ...
       printSTDOUTOutputSecurityCodeScan(outputJSON.CSharpResults.HuskyCISecurityCodeScanOutput.HighVulns)
       printSTDOUTOutputNewTool(...) // Call your new print function
       // ...
   }
   

   Sources: client/analysis/output.go:20-63

Step 4: SonarQube Integration (Optional)

If you want the findings to be included in SonarQube reports, you must update the integration logic in client/integration/sonarqube/sonarqube.go.

In the GenerateOutputFile function, append the vulnerability slices from your new tool to the allVulns slice. The rest of the logic, which converts HuskyCIVulnerability into SonarIssue, will handle the rest.

// client/integration/sonarqube/sonarqube.go:49-52
func GenerateOutputFile(analysis types.Analysis, outputPath, outputFileName string) error {
    // ...
    // spotbugs
	allVulns = append(allVulns, analysis.HuskyCIResults.JavaResults.HuskyCISpotBugsOutput.LowVulns...)
	allVulns = append(allVulns, analysis.HuskyCIResults.JavaResults.HuskyCISpotBugsOutput.MediumVulns...)
	allVulns = append(allVulns, analysis.HuskyCIResults.JavaResults.HuskyCISpotBugsOutput.HighVulns...)

    // Append your new tool's vulnerabilities here
    allVulns = append(allVulns, analysis.HuskyCIResults.YourLanguageResults.YourNewToolOutput.LowVulns...)
    // ...
}

Sources: client/integration/sonarqube/sonarqube.go:20-52

Conclusion

Integrating a new security tool into huskyCI is a systematic process that leverages its container-based, extensible architecture. By following the steps of containerizing the tool, implementing a corresponding parser in the Go API, and updating the client for result presentation, developers can enhance huskyCI's capabilities to cover more languages and vulnerability types. Before submitting your changes, please review the project's contributing guide.

Sources: CONTRIBUTING.md