Flux – oci

Blog: Verify the integrity of the Helm Charts stored in OCI-compliant registries as OCI artifacts

Mon, 14 Nov 2022 10:30:00 +0000

Cosign integration was one of the most important features we shipped in the Flux v0.35 release. After that, we wrote a blog post which explains how to use the feature with OCIRepository resources which enables fetching OCI artifacts from container registries. If you haven’t read it yet, we highly encourage you to go and check it out first.

Flux v0.36.0 allows you to prove the authenticity of HelmChart resources with the help of the cosign integration. Here we will demonstrate how to use the cosign integration to verify the integrity of the Helm charts stored in OCI-compliant registries as OCI artifacts.

Not at #kubecon so I had time to prepare Flux and Flagger releases. Flagger v1.24 comes with signed releases & OCI Helm charts. Flux v0.36 adds support for verifying Helm charts with Cosign. https://t.co/6MYwzfMA3W
— Stefan Prodan (@stefanprodan) October 27, 2022

Starting with Helm v3.8.0, Helm supports the OCI registry as a one of the storage option for Helm charts as an alternative to Helm repositories. The Helm CLI can push and pull Helm charts to and from OCI-compliant registries.

Note: Prior to Helm v3.8.0, OCI support was experimental. To use it there, you need to enable the feature by setting the HELM_EXPERIMENTAL_OCI environment variable to 1.

As we store Helm charts in OCI-compliant registries as OCI artifacts, we can now use the cosign integration to sign and verify them. Also, thanks to Flux, you can reconcile resources such as plain-text Kubernetes YAML manifests, Terraform modules, etc. from OCI-compliant registries with the help of OCIRepository resources. You can achieve the same thing for Helm charts with HelmRepository resources. This means that you can store Helm charts in OCI-compliant registries as OCI artifacts and use Flux to reconcile them like the following:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
 name: podinfo
 namespace: default
spec:
 type: "oci"
 interval: 5m0s
 url: oci://ghcr.io/stefanprodan/charts

Note:

Here you can review the complete registries lists that support the OCI artifact specification: OCI-Conformant Products.

You will notice that when you open the list, DockerHub is not included into the list but it will be added soon because they recently announced OCI Artifacts support, and you can read more about it from here.

Let’s jump right into the details of how we can actually use it.

We will deploy Prometheus by using its community Helm charts stored as OCI artifacts in OCI registry. Recently, Prometheus’ community started to publish their Helm charts to OCI registries and sign them with cosign using the keyless approach, you can learn more the process here. Then we are going to verify it with cosign and configure Flux to verify the Helm chart’s signatures before they are downloaded and reconciled. As the Prometheus community signed their Helm Charts without providing a key pair, we do not need to specify any key in the HelmChart resource’ provider.cosign spec to enable keyless verification for Flux.

For the sake of simplicity, we’ve deployed Prometheus alone but if you want to learn more about installing the Prometheus stack including Grafana, Alertmanager, etc., please refer to the official Flux page that can help you to do that.

You need three things to complete this demo;

cosign CLI
- https://docs.sigstore.dev/cosign/installation/
A Kubernetes cluster
- https://kind.sigs.k8s.io/#installation-and-usage
Flux CLI
- https://fluxcd.io/flux/cmd/

Let’s start by creating a simple Kubernetes cluster:

kind create cluster

Use the Flux CLI to do pre-flight checks:

$ flux check --pre
► checking prerequisites
✔ Kubernetes 1.25.3 >=1.20.6-0
✔ prerequisites checks passed

If the checks are successful, you can install Flux on the cluster.

Let’s install Flux on it - if you need to use other options, check out the installation page.

export GITHUB_USER=developer-guy
export GITHUB_TOKEN=$GITHUB_TOKEN

flux bootstrap github \
 --owner=$GITHUB_USER \
 --repository=flux-cosign-helm-oci-demo \
 --branch=main \
 --path=./clusters/my-cluster \
 --personal

Note: Don’t forget to change the values with your own details!

As we stick to GitOps practices, we only create files that contain the HelmRepository and HelmRelease resources. After committing and pushing those changes into the upstream repository, Flux will watch for changes and use them as source-of-truth for the configuration:

git clone git@github.com:developer-guy/flux-cosign-helm-oci-demo.git
cd flux-cosign-helm-oci-demo

Let’s create the HelmRepository resource first:

flux create source helm prometheus-community \
 --url=oci://ghcr.io/prometheus-community/charts \
 --interval=10m \
 --export > ./clusters/my-cluster/prometheus-community-helmrepository.yaml

Now, let’s move on with creating the HelmRelease resource:

flux create helmrelease prometheus \
 --source=HelmRepository/prometheus-community \
 --chart=prometheus \
 --interval=10m \
 --release-name prometheus \
 --target-namespace=monitoring \
 --create-target-namespace \
 --export > ./clusters/my-cluster/prometheus-helmrelease.yaml

and run the following command to add the verify section to the HelmRelease resource’ .spec.chart.spec section to enable keylesss verification:

yq e '.spec.chart.spec|=({"verify": { "provider": "cosign" } } +.)' ./clusters/my-cluster/prometheus-helmrelease.yaml

This command above will add the following part to the HelmRelease resource’s .spec.chart.spec section:

verify:
 provider: cosign

then, commit and push the changes:

git commit -m "Add prometheus HelmRelease and HelmRepository resources"
git push

After a couple of seconds, Flux will have applied these changes. Now let’s check the status of them:

Or, you can trigger the reconciliation immediately by running the simple command: flux reconcile source git flux-system

For the HelmRepository resource:

$ flux get sources helm
NAME REVISION SUSPENDED READY MESSAGE
prometheus-community False True Helm repository is ready

For the HelmRelease resource:

$ flux get helmreleases
NAME REVISION SUSPENDED READY MESSAGE
prometheus 15.18.0 False True Release reconciliation succeeded

If everything is fine, you can check the pods in the monitoring namespace:

$ kubectl get pods -n monitoring
NAME READY STATUS RESTARTS AGE
prometheus-alertmanager-54b7d7cf45-2b7zf 2/2 Running 0 115s
prometheus-kube-state-metrics-67f68d64bb-vlmvd 1/1 Running 0 115s
prometheus-node-exporter-46gm6 1/1 Running 0 115s
prometheus-pushgateway-596cd99697-t79zt 1/1 Running 0 115s
prometheus-server-c458cf6f9-nvstw 2/2 Running 0 115s

Great! Now, you have installed Prometheus with Flux by using the Helm chart stored in the OCI registry and verified it with cosign.

We can assume that the Helm chart’s signature is verified as we let it be deployed in a cluster but let’s do have double check and check the status of the HelmRelease to see whether the verification is successful or not:

$ kubectl get helmcharts -n flux-system flux-system-prometheus -ojsonpath='{.status.conditions[?(@.type=="SourceVerified")]}'
{"lastTransitionTime":"2022-11-09T13:27:38Z","message":"verified signature of version 15.18.0","observedGeneration":1,"reason":"Succeeded","status":"True","type":"SourceVerified"}

That’s super cool! Because Flux is going to add a condition to the HelmChart resource’s status section to show the verification status. If the verification is successful, it will add a condition like the one above.

DIY (Do it yourself) Approach

The Prometheus community Helm charts only serve as an example. Here is how you can do the same thing with your own Helm charts.

Create a Helm chart
Package the Helm chart as .tar.gz file
Login to the OCI-compliant registry that you want to use to store your Helm chart
Push the Helm chart as OCI artifact

Let’s create a sample directory that will contain our Helm chart:

mkdir -p helm-oci-demo
cd helm-oci demo

Now package the Helm chart as .tar.gz file:

 helm create nginx

Let’s package the Helm chart as .tar.gz file:

helm package nginx

Now, let’s login to the OCI-compliant registry that you want to use to store your Helm chart. In this example, we’ll be using the GitHub Container Registry (ghcr.io):

echo $GHCR_PAT | helm registry login ghcr.io -u $USER --password-stdin

Note: Don’t forget to change the values with your own details!

At this point, we are ready to push the Helm chart as OCI artifact:

$ helm push nginx-0.1.0.tgz oci://ghcr.io/$USER
Pushed: ghcr.io/developer-guy/nginx:0.1.0
Digest: sha256:21f92cbd63ab495d8fc44d54dabc4815c88d37697b3f8b757ca8e51ef178a2e7

So, the Helm chart is pushed to the OCI registry. It’s time to sign it with cosign. As cosign recommends we should always sign images based on their digests (@sha256:) rather than a tag. So, we should grab the digest from the command output above which is 21f92cbd63ab495d8fc44d54dabc4815c88d37697b3f8b757ca8e51ef178a2e7 in this case, and use that digest while signing the image:

As we saw the keyless approach before, let’s try the key-based approach this time. To do that, we should create public/private key pairs first:

cosign generate-key-pair

This command will generate two files, a cosign.pub which is a publickey and cosign.key which is a private key pair and store them in the current directory directory.

Now, let’s sign the image with the private key:

cosign sign --key cosign.key ghcr.io/$USER/nginx@21f92cbd63ab495d8fc44d54dabc4815c88d37697b3f8b757ca8e51ef178a2e7

Cool! Now we have signed the image with the private key. Let’s check the signature:

cosign verify --key cosign.key ghcr.io/$USER/nginx@21f92cbd63ab495d8fc44d54dabc4815c88d37697b3f8b757ca8e51ef178a2e7

Yay! It’s verified. But in order to make the public key accessible by Flux, we need to create a Kubernetes secret to store the public key:

kubectl -n flux-system create secret generic cosign-pub \
 --from-file=cosign.pub=cosign.pub

Now, we can use it in our Flux configuration. The rest of the steps are the same as the previous section. For the sake of simplicity, we won’t repeat them here other than the HelmRepository and HelmRelease resources’ creation steps.

Let’s create the HelmRepository resource first:

flux create source helm $USER-charts\
 --url=oci://ghcr.io/$USER \
 --interval=10m \
 --export > ./clusters/my-cluster/nginx-helmrepository.yaml

Let’s move on with creating the HelmRelease resource:

flux create helmrelease nginx \
 --source=HelmRepository/$USER-charts \
 --chart=nginx \
 --interval=10m \
 --release-name nginx \
 --target-namespace=default \
 --export > ./clusters/my-cluster/nginx-helmrelease.yaml

Don’t forget to run the following command to add the verify section to the HelmRelease resource’ .spec.chart.spec section to enable verification:

yq e '.spec.chart.spec|=({"verify": { "provider": "cosign", "secretRef": { "name": "cosign-pub" } } } +.)' ./clusters/my-cluster/nginx-helmrelease.yaml

This command above will add the following part to the HelmRelease resource’s .spec.chart.spec section:

verify:
 provider: cosign
 secretRef:
 name: cosign-pub

That’s all you need to do folks!

Congratulations! You have successfully signed your Helm chart with cosign with key-based approach and used it with Flux.

Blog: CNCF Talk: Increased security and scalability with OCI

Wed, 26 Oct 2022 13:20:00 +0000

Integrating OCI into Flux was one of the most-requested features of all times. We listened to your feedback and in the past couple of releases, OCI was integrated more deeply into Flux. Here is a brief summary of what landed when:

v0.31 (Jun 2022): Support for Helm repositories of type OCI
v0.32 (Aug 2022): Kubernetes manifests, Kustomize overlays and Terraform code as OCI artifacts
v0.33 (Aug 2022): More configurability of OCI settings
v0.34 (Sep 2022): More flexibility when interacting with OCI artifacts/repositories
v0.35 (Sep 2022): verify OCI artifacts signed by cosign
v0.36 (Oct 2022): verify OCI helm charts signed by cosign plus lots of new tooling to interact with OCI using the Flux CLI

To bring you up to speed with what’s possible, Max Jonas Werner, Flux Core Maintainer and Senior Software Engineer at Weaveworks, gave a talk in the CNCF Online Programme series to give some background and do a practical demo.

First off, Max explained the core GitOps concepts and gave an overview of the architecture of Flux. In the next step, he dived into how Docker and others created the Open Containers Initiative (OCI) which is a part of the Linux Foundation.

One of the key points Max is making is that we went through a transformation from Docker containers to generic application and configuration containers. More and more OCI is becoming an application delivery format.

OCI registries (which implement the distribution spec) are a commodity in the cloud space. This means that it’s very easy to get enhanced scalability this way, because pulling an OCI image is much less resource-intensive compared to a full or shallow Git clone. Additionally, high available registries are available everywhere.

It also provides many ways to secure your infrastructure. Flux leverages Kubernetes workload identity and IAM when pulling OCI artifacts from managed registries. So no more key management, no more SSH keys to generate, no more proprietary API usage for token generation. You use the same mechanism that is used for pulling container images. You might also want to check out this post about verifying authenticity of artifacts with cosign.

Max spends more than half of his presentation time for the demo, so you get a good idea of how to use these new features and integrate them into your setup.

Check out the video here:

Thanks a lot Max for taking the time to walk us through this!

Start your journey and start using Flux’s OCI features today.

Blog: Prove the Authenticity of OCI Artifacts

Mon, 17 Oct 2022 12:30:00 +0000

Software supply chain attacks are one of the most critical risks threatening today’s software and have begun to collapse like a dark cloud over the software industry. For the Flux family of projects we are taking precautions against these threats. Apart from implementing security features and best practices, it is important to us to educate our users. You can find all Flux’s security articles here. Today we will talk about a new security feature.

Let’s start with a brief historical explanation of how we got to this point. It all started with the following sentence:

Flux should be able to distribute and reconcile Kubernetes configuration packaged as OCI artifacts.

RFC-0003: Flux OCI support for Kubernetes manifests.

From then on, the Flux community worked hard and brought this feature with Flux v0.32. So with that, you can store and distribute various sources such as Kubernetes manifests, Kustomize overlays, and Terraform modules as OCI (Open Container Initiative) artifacts with Flux CLI and tell Flux to reconcile your sources that are stored in OCI Artifacts, and Flux will do that for you. 🕺🏻

But this only covered the first stage of the entire implementation. There is more than that. ☝️

One of the most exciting features of this RFC is the verification of artifacts. But why, what is it, is it really necessary or just a hype thing? This is a long topic that we need to discuss. Suppose you store the cluster desired state as OCI artifacts in a container registry. How can you be one hundred percent sure that the resources that Flux reconciles are the same as the resources that you’ve pushed to the OCI registry? This is where the verification of artifacts comes into play. But, how can we do that? 🤔

Thanks to the Sigstore community we have a great set of services and tools for signing and verifying authenticity. One of the tools is cosign which can be used for container signing, verification, and storage in an OCI registry. We will use it to verify the authenticity of the OCI Artifacts in Flux. Starting with v0.35, Flux comes with support for verifying OCI artifacts signed with Sigstore Cosign. Documentation for setting it up can be found here.

Let’s jump right into the details of how we can actually use it.

We will deploy cert-manager by storing its manifests in OCI registry packaged as an OCI Artifacts, using the Flux CLI. Then we are going to sign it with cosign and configure Flux to verify the artifacts’ signatures before they are downloaded and reconciled.

You need three things to complete this demo;

cosign CLI
- https://docs.sigstore.dev/cosign/installation/
A Kubernetes cluster
- https://kind.sigs.k8s.io/#installation-and-usage
Flux CLI
- https://fluxcd.io/flux/cmd/

Let’s start by creating a simple Kubernetes cluster:

kind create cluster

Let’s install Flux on it - if you need to use other options, check out the installation page.

export GITHUB_USER=developer-guy
export GITHUB_TOKEN=$GITHUB_TOKEN

flux bootstrap github \
 --owner=$GITHUB_USER \
 --repository=flux-cosign-demo \
 --branch=main \
 --path=./clusters/my-cluster \
 --personal

⚠️ Note: Don’t forget to change the values with your own details!

First we download the cert-manager install manifests from GitHub:

curl -sSLO https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml

https://github.com/cert-manager/cert-manager/releases/tag/v1.9.1

Next we push the manifests to GitHub container registry with Flux CLI:

mkdir -p ./manifests
 cp cert-manager.yaml ./manifests
$ flux push artifact oci://ghcr.io/$GITHUB_USER/manifests/cert-manager:v1.9.1 \
 --path="./manifests" \
 --source="https://github.com/cert-manager/cert-manager.git" \
 --revision="v1.9.1/4486c01f726f17d2790a8a563ae6bc6e98465505"
► pushing artifact to ghcr.io/developer-guy/manifests/cert-manager:v1.9.1
✔ artifact successfully pushed to ghcr.io/developer-guy/manifests/cert-manager@sha256:d1fb0442865148a4e9b4c3431c71d8e44af56c3eb658ea495c5ec48d48c6638b

Before signing the OCI artifact with Cosign, we need to create a set of key pairs, a public and private one:

cosign generate-key-pair

This command above outputs two files to disk: cosign.pub and cosign.key. The cosign.pub file is the public key and the cosign.key file is the private key. You can use the cosign.pub file to verify the container image and the cosign.key file to sign the container image.

To let Flux to verify the signature of the OCI artifact, we should create a secret that contains the public key::

kubectl -n flux-system create secret generic cosign-pub \
 --from-file=cosign.pub=cosign.pub

Now, let’s sign it:

$ cosign sign --key cosign.key ghcr.io/$GITHUB_USER/manifests/cert-manager:v1.9.1
Enter password for private key:
Pushing signature to: ghcr.io/developer-guy/manifests/cert-manager

As we stick into the GitOps practices, we should create a file that contains the OCIRepository resource, then commit and push those changes into the upstream repository that Flux watches for changes:

git clone git@github.com:developer-guy/flux-cosign-demo.git
cd flux-cosign-demo

Let’s create a secret with the GitHub token:

$ flux create secret oci ghcr-auth \
 --url=ghcr.io \
 --username=${GITHUB_USER} \
 --password=${GITHUB_TOKEN}
► oci secret 'ghcr-auth' created in 'flux-system' namespace

Configure Flux to pull the cert-manager artifact, verify its signature and apply its contents:

cat << EOF | tee ./clusters/my-cluster/cert-manager-sync.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
 name: cert-manager
 namespace: flux-system
spec:
 interval: 5m
 url: oci://ghcr.io/${GITHUB_USER}/manifests/cert-manager
 ref:
 semver: "*"
 secretRef:
 name: ghcr-auth
 verify:
 provider: cosign
 secretRef:
 name: cosign-pub
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
 name: cert-manager
 namespace: flux-system
spec:
 interval: 1h
 timeout: 5m
 sourceRef:
 kind: OCIRepository
 name: cert-manager
 path: "."
 prune: true
 wait: true
EOF

Let’s commit and push these changes:

git add .
git commit -m"Add cert-manager OCIRepository and Kustomization"
git push

After couple of seconds for Flux will have applied these changes. Now let’s check the status of them:

Or, you can trigger the reconcilation immediately by running the simple command: flux reconcile source git flux-system

For Kustomization resources:

$ flux get kustomizations
NAME REVISION SUSPENDED READY MESSAGE
cert-manager v1.9.1/d1fb0442865148a4e9b4c3431c71d8e44af56c3eb658ea495c5ec48d48c6638b False True Applied revision: v1.9.1/d1fb0442865148a4e9b4c3431c71d8e44af56c3eb658ea495c5ec48d48c6638b
flux-system main/14f1e66 False True Applied revision: main/14f1e66

For OCIRepository resources:

$ flux get sources oci
NAME REVISION SUSPENDED READY MESSAGE
cert-manager v1.9.1/d1fb0442865148a4e9b4c3431c71d8e44af56c3eb658ea495c5ec48d48c6638b False True stored artifact for digest 'v1.9.1/d1fb0442865148a4e9b4c3431c71d8e44af56c3eb658ea495c5ec48d48c6638b'

If you see the status of the OCIRepository is True, it means that Flux has successfully verified the signature of the container image. Because Flux adds a condition with the following attributes to the OCIRepository’s .status.conditions:

type: SourceVerified
status: “True”
reason: Succeeded

If the verification fails, Flux will set the SourceVerified status to False and will not fetch the artifact contents from the registry. If you see the status of the Kustomization is True, it means that Flux has successfully applied the manifests that are stored in the container image.

Let’s check the status of the cert-manager deployment:

$ kubectl get pods --namespace cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-cainjector-857ff8f7cb-l469h 1/1 Running 0 76s
cert-manager-d58554549-9fbgj 1/1 Running 0 76s
cert-manager-webhook-76fdf7c485-9v82g 1/1 Running 0 76s

Furthermore

As we can store Helm Charts in OCI registries with the release of Helm v3.8.0 which means that we can also sign them with cosign and verify them with Flux. The Flux community is already working on it and want to add support for verifying the Helm charts stored in OCI registries as OCI Artifacts in the next releases of Flux. You can follow the progress of this feature in the following issue: fluxcd/source-controller#914.

The Sigstore community is aware of the risks and the toil of managing public/private key pairs, so cosign offers another mode for signing and verification called Keyless, which do not require managing any keys manually. Flux also supports that. If you omit the .verify.secretRef field, Flux will try to verify the signature using the Keyless mode. It’s worth mentioning keyless verification is an experimental feature, using custom root CAs or self-hosted Rekor instances are currently not supported.

Blog: Managing Kyverno Policies as OCI Artifacts with OCIRepository Sources

Thu, 01 Sep 2022 11:30:00 +0000

The Flux team has released a new version of Flux v0.32 that includes fantastic features. One of them is OCI Repositories feature that allows us to store and distribute a wide variety of sources such as Kubernetes manifests, Kustomize overlays, and Terraform modules as OCI (Open Container Initiative) artifacts. Furthermore, the Flux team got us even more excited because they are planning to verify the authenticity of the OCI artifacts before they get applied into Kubernetes by integrating Cosign, which is one of the most significant projects from the @projectsigstore community that help us to sign and verify OCI images, blobs, etc. please see the issue to get more details about the plan.

⚠️ Note: You can read the RFC of this feature here.

I'm super excited to announce that @fluxcd support for distributing #Kubernetes manifests, Kustomize overlays and Terraform code as OCI artifacts has finally shipped in v0.32. https://t.co/144HY6LUTy
— Stefan Prodan (@stefanprodan) August 11, 2022

Today’s blog post is all about a quick tour of this feature and will give you a real-world example of it to show you how you can leverage this feature to manage Kyverno policies as OCI Artifacts. It is worth saying that this topic has been discussed for a while in the Kyverno community, too. There is an ongoing issue about packaging and distributing Kyverno policies as OCI Artifacts through its CLI. Also, there is a chance to move that logic into Kyverno’s core.

But for those who might not be familiar enough with OCI artifacts (including me), it’s worth explaining what the OCI Artifacts are before jumping into the details. OCI Artifacts gives you the power of storing and distributing other types of data (nearly anything), such as Kubernetes deployment files, Helm Charts, and CNAB, in addition to container images via OCI registries. And today, we’ll be using this feature for Kyverno policies. To be more precise, OCI Artifacts are not a new specification, format, or API. It just utilizes the existent OCI manifest and OCI index definitions. Hence, we can quickly start using the same client tooling, such as a crane, skopeo, etc., and distribute them using OCI registries, thanks to the OCI distribution-spec. Because OCI Artifacts does not change anything related to the specs, it only expands them to give people (artifact authors) power to define their content types. It is more like a generic definition for determining what can be stored in an OCI registry and consumed by clients.

The Flux CLI generates a single layer OCI image for storing things. As you can use some other tools to generate an OCI image with multiple layers in it, you can use the Layer Selection feature that Flux provides to select the layers you want to use in the OCI image. If the layer selector matches more than one layer, the first layer matching the specified media type will be used. Note that Flux requires that the OCI layer is compressed in the tar+gzip format.

Today, we’ll leverage the OCI Repositories feature to apply Kyverno policies stored in an OCI registry into the Kubernetes cluster.

First, we need to install Flux CLI, please see the installation page for more details.

Next, we should have a Kubernetes cluster running. We’ll be using KinD for this purpose.

kind create cluster

Once the cluster has been provisioned successfully, we need to install Flux components into it by simply running the command below:

$ flux bootstrap github \
 --owner=developer-guy \
 --repository=flux-kyverno-policies \
 --path=clusters/local \
 --personal

⚠️ Note: Don’t forget to change the values with your own details!

This command will install Flux and create necessary files for us and push them into the repository.

Next, we should install Kyverno by using a GitOps approach with Flux. In order to do that, we use the following resources:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
 name: kyverno-controller
 namespace: flux-system
spec:
 interval: 30m
 url: https://github.com/kyverno/kyverno
 ignore: |
 /*
 !/config/
 ref:
 semver: "1.x"
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
 name: kyverno-controller
 namespace: flux-system
spec:
 interval: 30m
 sourceRef:
 kind: GitRepository
 name: kyverno-controller
 serviceAccountName: kustomize-controller
 path: ./config/release
 prune: true
 wait: true
 timeout: 5m

Do not forget to check whether everything works fine before moving into the next steps:

$ flux get kustomizations kyverno-controller
NAME REVISION SUSPENDED READY MESSAGE
kyverno-controller v1.7.3/f2b63ce False True Applied revision: v1.7.3/f2b63ce

Now, we are ready to create an OCI image to store my Kyverno policies.

⚠️ You can find all the code examples in GitHub.

In order to do that, we will clone our repository that holds the Kyverno policies and create an OCI artifact to store them.

⚠️ We are expecting that some other team like DevSecOps will be responsible for maintaining and publishing the policies to our registry.

$ git clone https://github.com/developer-guy/my-kyverno-policies.git
$ cd my-kyverno-policies
$ flux push artifact oci://ghcr.io/developer-guy/policies:v1.0.0 \
 --path="." \
 --source="$(git config --get remote.origin.url)" \
 --revision="$(git branch --show-current)/$(git rev-parse HEAD)"
► pushing artifact to ghcr.io/developer-guy/policies:v1.0.0
✔ artifact successfully pushed to ghcr.io/developer-guy/policies@sha256:56e853e3c5c02139c840b7f5c89a02f63ede8dc498ed3925a52360032aa49e60

⚠️ Note: Don’t forget to change the values with your own details!

Last but not least, we need to create an OCIRepository resource that points to my OCI artifact:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
 name: kyverno-policies
 namespace: flux-system
spec:
 interval: 5m
 url: oci://ghcr.io/developer-guy/policies
 ref:
 semver: "v1.x"
 secretRef:
 name: ghcr-auth
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
 name: kyverno-policies
 namespace: flux-system
spec:
 sourceRef:
 kind: OCIRepository
 name: kyverno-policies
 interval: 60m
 retryInterval: 5m
 path: ./
 prune: true
 wait: true
 timeout: 2m
 dependsOn:
 - name: kyverno-controller
 patches: # enforce all policies
 - patch: |
 - op: replace
 path: /spec/validationFailureAction
 value: enforce
 target:
 kind: ClusterPolicy

I’d like to highlight some key points about the resources above. Here in OCIRepository resource, we are using SemVer to select the policies that we want to apply. .spec.ref is an optional field to specify the OCI reference to resolve and watch for changes. If not specified, the latest version of the repository will be used. You can reach out to the complete list of references supported in Flux, here is the link for you.

Also, in the Kustomization resource, we are using .spec.patches to apply patches to the policies that we want to enforce. We are using op: replace to replace the existing value of the field with the new one. path is the path to the field that we want to replace. value is the value of the field that we want to replace. To get more detail about the Patches, please see the link.

Last but not least, we are specifying an explicit dependencies for the Kustomization resource by using dependsOn keyword that ensures the Kyverno deployment is ready before applying the policies. This is important because Kyverno needs to be installed before applying the policies. Otherwise, the policies won’t be used because CRD (Custom Resource Definitions) won’t exist until Kyverno works. You can learn more about the dependencies of Kustomization resource, here.

Now, we can apply these manifests by committing and pushing them to the repository and letting Flux take care of the rest but still, one little step left that we need to do, which is authentication.

⚠️ Don’t forget, the authentication part is only needed when the OCI artifact is not publicly accessible. If your image has publicy available, you can skip that part.

You might notice a secretRef section in the OCIRepository resource. We should create this secret because Flux should be able to pull my container image. To do that, we should follow the documentation.

$ flux create secret oci ghcr-auth \
 --url=ghcr.io \
 --username=developer-guy \
 --password=${GITHUB_PAT}
► oci secret 'ghcr-auth' created in 'flux-system' namespace

Once everything is completed, you should be able to see the following output:

$ kubectl get clusterpolicies
NAME BACKGROUND ACTION READY
require-base-image true enforce true

This is what we expected to happen, whee!🕺🏻

This is an exciting policy, though, if you want to learn more about it, I wrote a blog post that explains what the base image concept refers to and how we can enforce policies related to them.

As you can see, this feature is quite promising and easy to use. I hope you enjoyed it, and please stay tuned because there are more features on the way you don’t want to miss.

Thanks for reading.