Deploying with Kubernetes CronJob

Relevant source files

The following files were used as context for generating this wiki page:

• iac-recert-engine/cronjob.yaml
• iac-recert-engine/example.config.yaml
• iac-recert-engine/config.yaml
• iac-recert-engine/cmd/ice/run.go
• iac-recert-engine/github-action.yaml
• iac-recert-engine/internal/engine/engine.go

The iac-recert-engine, also known as ICE, can be deployed as a Kubernetes CronJob to automate the recertification process on a recurring schedule. This method provides a platform-agnostic way to run the engine, contrasting with the tightly integrated GitHub Actions approach. The CronJob is responsible for creating a new Pod on a defined schedule, which then executes the recertification scan and creates pull requests as needed.

This deployment model relies on standard Kubernetes resources: a CronJob to manage the schedule, a ConfigMap to provide the application configuration, and a Secret to securely store the Git provider's personal access token. The schedule itself is defined within the CronJob manifest, but can also be documented within the application's configuration file for reference.

Sources: iac-recert-engine/cronjob.yaml, iac-recert-engine/example.config.yaml:63-65

Architecture Overview

The Kubernetes deployment consists of three main resources that work together to execute the iac-recert-engine.

The diagram below illustrates the relationship between the Kubernetes resources required for the deployment.

CronJob (iac-recert-engine):* The primary resource that triggers the execution based on a cron schedule.

• ConfigMap (ice-config): Contains the config.yaml file, which is mounted as a volume into the pod.
• Secret (ice-secrets): Securely stores the github-token, which is exposed to the container as an environment variable.

Sources: iac-recert-engine/cronjob.yaml:2-30

CronJob Manifest

The core of the deployment is the cronjob.yaml manifest. It defines the schedule, the job template, and the pod specification for running the engine.

Sources: iac-recert-engine/cronjob.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: iac-recert-engine
spec:
  schedule: "0 2 * * 0" # Weekly on Sunday at 2am
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: ice
            image: iac-recert-engine:latest
            args:
            - run
            env:
            - name: GITHUB_TOKEN
              valueFrom:
                secretKeyRef:
                  name: ice-secrets
                  key: github-token
            volumeMounts:
            - name: config-volume
              mountPath: /app/config.yaml
              subPath: config.yaml
          volumes:
          - name: config-volume
            configMap:
              name: ice-config
          restartPolicy: OnFailure

Sources: iac-recert-engine/cronjob.yaml:1-30

Key Configuration Fields

Field	Description	Source
spec.schedule	The cron schedule for running the job. The example runs weekly on Sunday at 2am. This value can be documented in config.yaml under the schedule block but the CronJob manifest is the source of truth for execution.	cronjob.yaml:6, example.config.yaml:63-65
spec.jobTemplate.spec.template.spec.containers[0].image	The Docker image for the iac-recert-engine.	cronjob.yaml:13
spec.jobTemplate.spec.template.spec.containers[0].args	The command to execute within the container. It is set to run, which triggers the main recertification process.	cronjob.yaml:14-15, cmd/ice/run.go:14-17
spec.jobTemplate.spec.template.spec.containers[0].env	Defines environment variables. GITHUB_TOKEN is sourced from a Kubernetes Secret named ice-secrets.	cronjob.yaml:16-21, config.yaml:8
spec.jobTemplate.spec.template.spec.volumes	Defines volumes to be mounted. The config-volume is sourced from a ConfigMap named ice-config.	cronjob.yaml:25-28
spec.jobTemplate.spec.template.spec.containers[0].volumeMounts	Mounts the config.yaml file from the config-volume into the container at /app/config.yaml.	cronjob.yaml:22-24

Execution Flow

When the Kubernetes scheduler triggers the CronJob, a new Job and corresponding Pod are created. The container within the pod starts and executes the ice run command.

This diagram shows the sequence of events from pod startup to engine execution.

The run command initializes the configuration, sets up a logger, and creates a new instance of the recertification Engine. It then calls the Run method on the engine, which orchestrates the entire process of scanning files, analyzing their history, checking for recertification needs, grouping files, and creating pull requests.

Sources: cmd/ice/run.go:14-41, internal/engine/engine.go:80-83

Required Kubernetes Resources

ConfigMap

A ConfigMap named ice-config is required to provide the config.yaml to the application. The data section of the ConfigMap must contain a key named config.yaml with the full configuration as its value.

Example ice-config ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ice-config
data:
  config.yaml: |
    version: "1.0"
    repository:
      url: "https://github.com/example/infrastructure%22
      provider: "github"
    auth:
      provider: "github"
      token_env: "GITHUB_TOKEN"
    global:
      dry_run: false
      verbose_logging: true
      default_base_branch: "main"
    patterns:
      - name: "terraform-prod"
        description: "Production Terraform"
        paths:
          - "terraform/prod/**/*.tf"
        recertification_days: 180
        enabled: true
    pr_strategy:
      type: "per_pattern"
    assignment:
      strategy: "last_committer"
      fallback_assignees:
        - "devops-team"
    pr_template:
      title: "🔄 Recertification: {pattern_name}"
      include_file_list: true
      include_checklist: true

Sources: cronjob.yaml:27-28, config.yaml:1-29

Secret

A Secret named ice-secrets is necessary to securely provide the Git provider's personal access token. The Secret must contain a key named github-token with the token as its value.

Example ice-secrets Secret (encoded):

apiVersion: v1
kind: Secret
metadata:
  name: ice-secrets
type: Opaque
data:
  github-token: <base64-encoded-github-token>

Sources: cronjob.yaml:19-21, example.config.yaml:14-16

Comparison with GitHub Actions

While the CronJob provides a generic deployment pattern, the project also includes a github-action.yaml for running the engine directly within GitHub Actions. Both methods use a cron schedule, but differ in their environment and setup.

Feature	Kubernetes CronJob	GitHub Actions
Environment	Any Kubernetes cluster	GitHub-hosted or self-hosted runner
Scheduling	cronjob.yaml spec.schedule	github-action.yaml on.schedule.cron
Configuration	ConfigMap mounted as a file	Config file checked into the repository
Secrets	Kubernetes Secret	GitHub repository/organization secrets
Execution	Runs a pre-built container image	Checks out code, builds the binary, and runs it

Sources: cronjob.yaml:6, github-action.yaml:5-6, github-action.yaml:15-18, github-action.yaml:21-23