Deployment Scripts

Relevant source files

The following files were used as context for generating this wiki page:

• deployments/scripts/build-containers.sh
• deployments/scripts/push-containers.sh
• deployments/scripts/check-containers-version.sh
• deployments/docker-compose.yml
• api/config.yaml

The deployment scripts and configuration files are responsible for setting up the local development environment, building the various security tool containers, and managing their versions for distribution. These components are essential for both developing and deploying the huskyCI platform. The primary mechanism for local deployment is Docker Compose, while a collection of shell scripts handles the lifecycle of the security tool images.

The core of the system's deployment is defined in docker-compose.yml, which orchestrates the main services. The security tools themselves are packaged as individual Docker images, managed by scripts located in the deployments/scripts/ directory. These scripts automate building, versioning, and pushing the container images to a registry.

Local Environment Setup (docker-compose)

A local huskyCI environment can be provisioned using the docker-compose.yml file. This file defines the core services required to run the application, their configurations, and their interdependencies.

Sources: deployments/docker-compose.yml

Services

The docker-compose.yml file defines three main services: dockerapi, api, and mongodb.

Service	Image	Description
dockerapi	docker:24-rc-dind	Runs a Docker-in-Docker (dind) daemon, exposing the Docker API over TCP on port 2376. This allows the api service to orchestrate security scan containers.
api	Custom build (deployments/dockerfiles/api.Dockerfile)	The main huskyCI API server. It connects to the dockerapi to run scans and to mongodb for data persistence. It requires several environment variables for configuration.
mongodb	mongo:4.2.2	The database service used by the api to store analysis results and configuration data. It is initialized with a user and database via mongo-init.sh.

Sources: deployments/docker-compose.yml:2-85

Service Dependencies

The services are configured to start in a specific order to ensure proper initialization. The api service depends on both mongodb and dockerapi being ready before it starts.

This diagram illustrates the dependencies between the services defined in docker-compose.yml.

Configuration

Key configurations are managed through environment variables and volume mounts in the docker-compose.yml file.

API Service Environment Variables:

Variable	Example Value	Description
HUSKYCI_DATABASE_DB_ADDR	mongodb	Address of the MongoDB service.
HUSKYCI_DOCKERAPI_ADDR	dockerapi	Address of the Docker API service.
HUSKYCI_DOCKERAPI_TLS_VERIFY	0	Disables TLS verification for the Docker API connection.
HUSKYCI_INFRASTRUCTURE_USE	docker	Specifies that the infrastructure for running scans is Docker.

Sources: deployments/docker-compose.yml:22-31

Container Image Management

A set of shell scripts is provided to automate the build and release process for the security test container images. These scripts ensure that all tool containers are built consistently and tagged with appropriate versions before being pushed to a container registry.

Container Build, Versioning, and Push Workflow

The following diagram shows the end-to-end process for managing the security tool container images, from building to pushing them to a registry.

Building Containers (build-containers.sh)

This script is responsible for building all the Docker images for the security tools used by huskyCI. It uses docker buildx build to create multi-platform (linux/amd64) images and tags them with :latest.

The script builds the following tool containers:

• huskyci/bandit
• huskyci/brakeman
• huskyci/enry
• huskyci/gitauthors
• huskyci/gosec
• huskyci/npmaudit
• huskyci/yarnaudit
• huskyci/safety
• huskyci/gitleaks
• huskyci/spotbugs
• huskyci/tfsec
• huskyci/securitycodescan

Sources: deployments/scripts/build-containers.sh:9-20

Checking Container Versions (check-containers-version.sh)

After the images are built, this script is used to verify the version of the underlying tool packaged within each container. It does this by running the container and executing a command to print the version, which is then parsed from the output.

The following table shows some examples of the commands used to extract tool versions:

Container	Command to Get Version
huskyci/bandit	docker run --rm huskyci/bandit:latest bandit --version
huskyci/brakeman	docker run --rm huskyci/brakeman:latest brakeman --version
huskyci/gosec	docker run --rm huskyci/gosec:latest gosec --version
huskyci/safety	docker run --rm huskyci/safety:latest safety --version
huskyci/gitleaks	docker run --rm huskyci/gitleaks:latest gitleaks --version

Sources: deployments/scripts/check-containers-version.sh:9-20

Pushing Containers (push-containers.sh)

This script orchestrates the final steps of the release process. It performs two main actions:

1. Tagging: It first determines the specific version of each tool (similar to check-containers-version.sh). Then, it tags the existing :latest image with a specific version tag (e.g., huskyci/bandit:1.6.2).
2. Pushing: It pushes both the :latest tag and the new version-specific tag to the container registry.

Example of tagging and pushing for the bandit container:

# Get the version
banditVersion=$(docker run --rm huskyci/bandit:latest bandit --version | grep bandit | awk -F " " '{print $2}')

# Tag the image
docker tag "huskyci/bandit:latest" "huskyci/bandit:$banditVersion"

# Push both tags
docker push "huskyci/bandit:latest" && docker push "huskyci/bandit:$banditVersion"

Sources: deployments/scripts/push-containers.sh:9-50

Security Test Configuration

The configuration for each security test, including the Docker image to use and the command to execute, is defined in api/config.yaml. The huskyCI API reads this file to know how to run each security analysis.

Tool	Image	Image Tag	Language
gosec	huskyci/gosec	v2.3.0	Go
bandit	huskyci/bandit	1.6.2	Python
brakeman	huskyci/brakeman	4.8.2	Ruby
safety	huskyci/safety	1.9.0	Python
npmaudit	huskyci/npmaudit	9.6.6	JavaScript
yarnaudit	huskyci/yarnaudit	1.22.19	JavaScript
gitleaks	huskyci/gitleaks	v.7.6.1	Generic
tfsec	huskyci/tfsec	0.58.15	HCL

Sources: api/config.yaml

Each tool's entry contains a cmd field, which is a shell script executed inside the container. This command is responsible for cloning the target repository, running the security tool, and formatting the output as JSON. The commands use placeholders like %GIT_REPO% and %GIT_BRANCH% that are substituted by the API at runtime.

Sources: api/securitytest/securitytest.go:130-133