Understanding Helm Hooks in AGH3
Relevant source files
The following files were used as context for generating this wiki page:
- charts/agh3/templates/hooks/pre/crds/10-actions-crd-job.yml
- charts/agh3/templates/hooks/post/sidecar-container-enabler/50-sidecar-container-feature-enabler.job.yml
- charts/agh3/values.yaml
- charts/agh3/templates/hooks/post/sidecar-container-enabler/0-sidecar-container-feature-enabler.cm.yml
- charts/agh3/crds/actions.crds.yaml
The AGH3 Helm chart utilizes Helm hooks to perform critical setup and configuration actions during the release lifecycle. These hooks are Kubernetes manifests with special annotations that allow them to run at specific points, such as before an installation begins or after an upgrade is completed. This ensures that the cluster environment is properly prepared with necessary Custom Resource Definitions (CRDs) and features before the main application components are deployed.
This document details the pre-install and post-install hooks used within the AGH3 chart, explaining their purpose, architecture, and configuration. The primary hooks are responsible for installing the Action CRD and enabling the SidecarContainers feature gate on k3s clusters.
Helm Hook Overview
Helm hooks are Kubernetes Jobs or other manifest types that are executed at specific points in a release's lifecycle. The AGH3 chart uses them for two main purposes:
- Pre-Install/Pre-Upgrade: To ensure that dependencies, like CRDs, are present in the cluster before the rest of the chart's resources are created.
- Post-Install/Post-Upgrade: To perform configuration on the cluster itself after the chart has been deployed, such as enabling a Kubernetes feature gate.
The execution order of hooks is controlled by helm.sh/hook-weight, with lower weights running first.
Sources: charts/agh3/templates/hooks/pre/crds/10-actions-crd-job.yml:5-6, charts/agh3/templates/hooks/post/sidecar-container-enabler/50-sidecar-container-feature-enabler.job.yml:5-6
Pre-Install Hook: Actions CRD Installation
Before any AGH3 application resources are installed, a pre-install,pre-upgrade hook runs to apply the actions.agh.lkc-lab.com Custom Resource Definition (CRD) to the cluster. This is essential because other components in the chart depend on this CRD being available.
Sources: charts/agh3/templates/hooks/pre/crds/10-actions-crd-job.yml:5
CRD Installation Job
The hook is implemented as a Kubernetes Job. This job runs a container with kubectl to apply the CRD definition. The CRD manifest itself is expected to be available within a ConfigMap, which is mounted as a volume into the Job's pod.
The following diagram illustrates the process of the pre-install hook.
The key properties of this hook job are:
- Hook Type:
pre-install,pre-upgrade - Hook Weight:
-5(runs early) - Delete Policy:
before-hook-creation,hook-succeeded(the old hook job is deleted before a new one runs, and successful jobs are cleaned up) - Image: A
kubectlimage, configured viavalues.yaml. - Command:
kubectl apply -f /crds/actions.crds.yaml
Sources: charts/agh3/templates/hooks/pre/crds/10-actions-crd-job.yml:5-7, 21
Configuration
The CRD installation job is configured in the values.yaml file under the crds section.
| Parameter | Description | Default Value |
|---|---|---|
crds.job.name | The name of the Kubernetes Job. | actions-crd-job |
crds.job.image.repository | The container image repository for kubectl. | docker/bitnamilegacy/kubectl |
crds.job.image.tag | The tag for the kubectl image. | 1.28.15-debian-12-r5 |
crds.cm.name | The name of the ConfigMap containing the CRD definition. | actions-crd |
crds.volume.name | The name of the volume mounted into the job. | crd-volume |
Sources: charts/agh3/values.yaml:30-47
Post-Install Hook: Sidecar Container Feature Enabler
After the main chart resources are installed or upgraded, a post-install,post-upgrade hook runs to ensure the SidecarContainers feature gate is enabled on the cluster nodes. This is particularly relevant for k3s environments. This hook is conditional and only runs if clusterFeatureEnabler.jobs.sidecarContainers.enabled is set to true.
Sources: charts/agh3/templates/hooks/post/sidecar-container-enabler/50-sidecar-container-feature-enabler.job.yml:1, 5, charts/agh3/values.yaml:801-802
Feature Enabler Job and Script
This process involves two resources, both annotated as hooks:
- ConfigMap (
0-sidecar-container-feature-enabler.cm.yml): Created with ahook-weightof "0", this ConfigMap holds the shell script that performs the feature enablement logic. - Job (
50-sidecar-container-feature-enabler.job.yml): Created with ahook-weightof "50", this Job mounts the script from the ConfigMap and executes it.
The script logic is as follows:
- It accepts a node name as an argument.
- It uses
kubectl get --raw /metricsto check if theSidecarContainersfeature is already enabled. - If not enabled, it runs
kubectl debugon the target node. - Inside the debug container, it uses
chroot /hostto access the node's filesystem. - It modifies the
/etc/systemd/system/k3s.servicefile to add the feature gate argument:--kube-apiserver-arg=feature-gates=SidecarContainers=true. - Finally, it reloads the systemd daemon and restarts the
k3s.serviceto apply the change.
The following diagram shows the workflow for the sidecar feature enabler hook.
Here is a snippet of the script that modifies the k3s service:
kubectl debug "node/$NODE_NAME" -it --image=ubuntu -- chroot /host /bin/bash -c 'cat /etc/systemd/system/k3s.service | sed /^\$/d | tee /tmp/patched-k3s.service && printf "\t\x27--kube-apiserver-arg=feature-gates=SidecarContainers=true\x27 \\" | tee -a /tmp/patched-k3s.service && mv /tmp/patched-k3s.service /etc/systemd/system/k3s.service && systemctl daemon-reload && systemctl restart k3s.service'
Configuration
The sidecar container feature enabler is configured in values.yaml under the clusterFeatureEnabler section.
| Parameter | Description | Default Value |
|---|---|---|
clusterFeatureEnabler.enabled | Master switch to enable all feature enabler jobs. | true |
clusterFeatureEnabler.jobs.sidecarContainers.enabled | Enables the specific job for sidecar containers. | true |
clusterFeatureEnabler.image.repository | The container image repository for the job. | docker/bitnamilegacy/kubectl |
clusterFeatureEnabler.image.tag | The tag for the job's image. | 1.28.15-debian-12-r5 |
clusterFeatureEnabler.serviceAccount.name | The ServiceAccount used by the job. | cluster-feature-enabler-sa |
Sources: charts/agh3/values.yaml:789-802
Summary
The AGH3 Helm chart effectively uses pre-install and post-install hooks to manage the deployment environment. The pre-install hook ensures the Action CRD is available before other resources are created, preventing dependency-related errors. The post-install hook dynamically configures the underlying Kubernetes cluster by enabling the SidecarContainers feature gate, demonstrating a powerful pattern for handling cluster-level prerequisites directly from a Helm release. These hooks are configurable via values.yaml, providing flexibility for different deployment scenarios.