Flux – Source Controllers

Flux: Controller Options

Mon, 01 Jan 0001 00:00:00 +0000

To customise the controller options at install time, please see the bootstrap customization guide.

Source controller flags

Name	Type	Description
`--artifact-retention-records`	int	The maximum number of artifacts to be kept in storage after a garbage collection. (default 2)
`--artifact-retention-ttl`	duration	The duration of time that artifacts from previous reconciliations will be kept in storage before being garbage collected. (default 1m0s)
`--concurrent`	int	The number of concurrent reconciles per controller. (default 2)
`--default-service-account`	string	Default service account to use for workload identity when not specified in resources.
`--enable-leader-election`	boolean	Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
`--events-addr`	string	The address of the events receiver.
`--health-addr`	string	The address the health endpoint binds to. (default “:9440”)
`--helm-cache-max-size`	int	The maximum size of the cache in number of indexes.
`--helm-cache-purge-interval`	string	The interval at which the cache is purged. Valid time units are ms, s, m. (default “1m”)
`--helm-cache-ttl`	string	The TTL of an index in the cache. Valid time units are ms, s, m. (default “15m”)
`--helm-chart-file-max-size`	int	The max allowed size in bytes of a file in a Helm chart. (default 5242880)
`--helm-chart-max-size`	int	The max allowed size in bytes of a Helm chart file. (default 10485760)
`--helm-index-max-size`	int	The max allowed size in bytes of a Helm repository index file. (default 52428800)
`--interval-jitter-percentage`	uint8	Percentage of jitter to apply to interval durations. A value of 10 will apply a jitter of +/-10% to the interval duration. It cannot be negative, and must be less than 100. (default 5)
`--leader-election-lease-duration`	duration	Interval at which non-leader candidates will wait to force acquire leadership (duration string). (default 35s)
`--leader-election-release-on-cancel`	boolean	Defines if the leader should step down voluntarily on controller manager shutdown. (default true)
`--leader-election-renew-deadline`	duration	Duration that the leading controller manager will retry refreshing leadership before giving up (duration string). (default 30s)
`--leader-election-retry-period`	duration	Duration the LeaderElector clients should wait between tries of actions (duration string). (default 5s)
`--log-encoding`	string	Log encoding format. Can be ‘json’ or ‘console’. (default “json”)
`--log-level`	string	Log verbosity level. Can be one of ’trace’, ‘debug’, ‘info’, ’error’. (default “info”)
`--max-retry-delay`	duration	The maximum amount of time for which an object being reconciled will have to wait before a retry. (default 15m0s)
`--metrics-addr`	string	The address the metric endpoint binds to. (default “:8080”)
`--min-retry-delay`	duration	The minimum amount of time for which an object being reconciled will have to wait before a retry. (default 750ms)
`--requeue-dependency`	duration	The interval at which failing dependencies are reevaluated. (default 30s)
`--ssh-hostkey-algos`	strings	The list of hostkey algorithms to use for ssh connections, arranged from most preferred to the least.
`--ssh-kex-algos`	strings	The list of key exchange algorithms to use for ssh connections, arranged from most preferred to the least.
`--storage-addr`	string	The address the static file server binds to. (default “:9090”)
`--storage-adv-addr`	string	The advertised address of the static file server.
`--storage-path`	string	The local storage path.
`--token-cache-max-size`	int	The maximum amount of entries in the LRU cache used for tokens. (default 100, enabled)
`--token-cache-max-duration`	duration	The maximum duration for which a token would be considered unexpired. This is capped at 1h. (default 1h)
`--watch-all-namespaces`	boolean	Watch for custom resources in all namespaces, if set to false it will only watch the runtime namespace. (default true)
`--watch-label-selector`	string	Watch for resources with matching labels e.g. ‘sharding.fluxcd.io/key=shard1’.
`--feature-gates`	mapStringBool	A comma separated list of key=value pairs defining the state of experimental features.

Feature Gates

Name	Default Value	Description
`CacheSecretsAndConfigMaps`	`false`	Configures the caching of Secrets and ConfigMaps by the controller-runtime client. When enabled, it will cache both object types, resulting in increased memory usage and cluster-wide RBAC permissions (list and watch).
`ObjectLevelWorkloadIdentity`	`false`	Enables the use of object-level workload identity for the controller.
`OptimizedGitClones`	`true`	Optimises Git resource usage by only cloning repositories when the HEAD commit changed since last reconciliation.

Source watcher flags

Name	Type	Description
`--artifact-digest-algo`	string	The hashing algorithm used to calculate the digest of artifacts. (default “sha256”)
`--artifact-retention-records`	int	The maximum number of artifacts to be kept in storage after a garbage collection. (default 2)
`--artifact-retention-ttl`	duration	The duration of time that artifacts from previous reconciliations will be kept in storage before being garbage collected. (default 1m0s)
`--concurrent`	int	The number of concurrent reconciles per controller. (default 10)
`--enable-leader-election`	boolean	Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
`--events-addr`	string	The address of the events receiver.
`--health-addr`	string	The address the health endpoint binds to. (default “:9440”)
`--http-retry`	int	The maximum number of retries when failing to fetch artifacts over HTTP. (default 9)
`--interval-jitter-percentage`	uint8	Percentage of jitter to apply to interval durations. A value of 10 will apply a jitter of +/-10% to the interval duration. It cannot be negative, and must be less than 100. (default 5)
`--leader-election-lease-duration`	duration	Interval at which non-leader candidates will wait to force acquire leadership (duration string). (default 35s)
`--leader-election-release-on-cancel`	boolean	Defines if the leader should step down voluntarily on controller manager shutdown. (default true)
`--leader-election-renew-deadline`	duration	Duration that the leading controller manager will retry refreshing leadership before giving up (duration string). (default 30s)
`--leader-election-retry-period`	duration	Duration the LeaderElector clients should wait between tries of actions (duration string). (default 5s)
`--log-encoding`	string	Log encoding format. Can be ‘json’ or ‘console’. (default “json”)
`--log-level`	string	Log verbosity level. Can be one of ’trace’, ‘debug’, ‘info’, ’error’. (default “info”)
`--max-retry-delay`	duration	The maximum amount of time for which an object being reconciled will have to wait before a retry. (default 15m0s)
`--metrics-addr`	string	The address the metric endpoint binds to. (default “:8080”)
`--min-retry-delay`	duration	The minimum amount of time for which an object being reconciled will have to wait before a retry. (default 750ms)
`--no-cross-namespace-refs`	boolean	When set to true, references between custom resources are allowed only if the reference and the referee are in the same namespace. (default false)
`--reconciliation-timeout`	duration	The maximum duration of a reconciliation. (default 10m0s)
`--requeue-dependency`	duration	The interval at which failing dependencies are reevaluated. (default 5s)
`--storage-addr`	string	The address the static file server binds to. (default “:9090”)
`--storage-adv-addr`	string	The advertised address of the static file server.
`--storage-path`	string	The local storage path. (default “/data”)
`--watch-all-namespaces`	boolean	Watch for resources in all namespaces, if set to false it will only watch the runtime namespace. (default true)
`--feature-gates`	mapStringBool	A comma separated list of key=value pairs defining the state of experimental features.

Feature Gates

No feature gates are currently available for source-watcher.

Flux: Git Repositories

Mon, 01 Jan 0001 00:00:00 +0000

The GitRepository API defines a Source to produce an Artifact for a Git repository revision.

Example

The following is an example of a GitRepository. It creates a tarball (.tar.gz) Artifact with the fetched data from a Git repository for the resolved reference.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 url: https://github.com/stefanprodan/podinfo
 ref:
 branch: master

In the above example:

A GitRepository named podinfo is created, indicated by the .metadata.name field.
The source-controller checks the Git repository every five minutes, indicated by the .spec.interval field.
It clones the master branch of the https://github.com/stefanprodan/podinfo repository, indicated by the .spec.ref.branch and .spec.url fields.
The specified branch and resolved HEAD revision are used as the Artifact revision, reported in-cluster in the .status.artifact.revision field.
When the current GitRepository revision differs from the latest fetched revision, a new Artifact is archived.
The new Artifact is reported in the .status.artifact field.

You can run this example by saving the manifest into gitrepository.yaml.

Apply the resource on the cluster:
```
kubectl apply -f gitrepository.yaml
```

Run kubectl get gitrepository to see the GitRepository:

NAME URL AGE READY STATUS
podinfo https://github.com/stefanprodan/podinfo 5s True stored artifact for revision 'master@sha1:132f4e719209eb10b9485302f8593fc0e680f4fc'

Run kubectl describe gitrepository podinfo to see the Artifact and Conditions in the GitRepository’s Status:

...
Status:
 Artifact:
 Digest: sha256:95e386f421272710c4cedbbd8607dbbaa019d500e7a5a0b6720bc7bebefc7bf2
 Last Update Time: 2022-02-14T11:23:36Z
 Path: gitrepository/default/podinfo/132f4e719209eb10b9485302f8593fc0e680f4fc.tar.gz
 Revision: master@sha1:132f4e719209eb10b9485302f8593fc0e680f4fc
 Size: 91318
 URL: http://source-controller.source-system.svc.cluster.local./gitrepository/default/podinfo/132f4e719209eb10b9485302f8593fc0e680f4fc.tar.gz
 Conditions:
 Last Transition Time: 2022-02-14T11:23:36Z
 Message: stored artifact for revision 'master@sha1:132f4e719209eb10b9485302f8593fc0e680f4fc'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: Ready
 Last Transition Time: 2022-02-14T11:23:36Z
 Message: stored artifact for revision 'master@sha1:132f4e719209eb10b9485302f8593fc0e680f4fc'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: ArtifactInStorage
 Observed Generation: 1
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal NewArtifact 62s source-controller stored artifact for commit 'Merge pull request #160 from stefanprodan/release-6.0.3'

Writing a GitRepository spec

As with all other Kubernetes config, a GitRepository needs apiVersion, kind, and metadata fields. The name of a GitRepository object must be a valid DNS subdomain name.

A GitRepository also needs a .spec section.

URL

.spec.url is a required field that specifies the HTTP/S or SSH address of the Git repository.

Note: Unlike using git, the shorter scp-like syntax is not supported for SSH addresses (e.g. user@example.com:repository.git). Instead, the valid URL format is ssh://user@example.com:22/repository.git.

Secret reference

.spec.secretRef.name is an optional field to specify a name reference to a Secret in the same namespace as the GitRepository, containing authentication credentials for the Git repository.

The required fields in the Secret depend on the specified protocol in the URL.

Basic access authentication

To authenticate towards a Git repository over HTTPS using basic access authentication (in other words: using a username and password), the referenced Secret is expected to contain .data.username and .data.password values.

---
apiVersion: v1
kind: Secret
metadata:
 name: basic-access-auth
type: Opaque
data:
 username: <BASE64>
 password: <BASE64>

Bearer token authentication

To authenticate towards a Git repository over HTTPS using bearer token authentication (in other words: using a Authorization: Bearer header), the referenced Secret is expected to contain the token in .data.bearerToken.

Note: If you are looking to use OAuth tokens with popular servers (e.g. GitHub, Bitbucket, GitLab), you should use basic access authentication instead. These servers use basic HTTP authentication, with the OAuth token as the password. Check the documentation of your Git server for details.

---
apiVersion: v1
kind: Secret
metadata:
 name: bearer-token-auth
type: Opaque
data:
 bearerToken: <BASE64>

HTTPS Certificate Authority

To provide a Certificate Authority to trust while connecting with a Git repository over HTTPS, the referenced Secret’s .data can contain a ca.crt or caFile key. ca.crt takes precedence over caFile, i.e. if both keys are present, the value of ca.crt will be taken into consideration.

---
apiVersion: v1
kind: Secret
metadata:
 name: https-ca-credentials
 namespace: default
type: Opaque
data:
 ca.crt: <BASE64>

HTTPS Mutual TLS authentication

To authenticate towards a Git repository over HTTPS using mutual TLS, the referenced Secret’s .data should contain the following keys:

tls.crt and tls.key, to specify the client certificate and private key used for TLS client authentication. These must be used in conjunction, i.e. specifying one without the other will lead to an error.
ca.crt, to specify the CA certificate used to verify the server, which is required if the server is using a self-signed certificate.

---
apiVersion: v1
kind: Secret
metadata:
 name: https-tls-certs
 namespace: default
type: Opaque
data:
 tls.crt: <BASE64>
 tls.key: <BASE64>
 ca.crt: <BASE64>

SSH authentication

To authenticate towards a Git repository over SSH, the referenced Secret is expected to contain identity and known_hosts fields. With the respective private key of the SSH key pair, and the host keys of the Git repository.

---
apiVersion: v1
kind: Secret
metadata:
 name: ssh-credentials
type: Opaque
stringData:
 identity: |
 -----BEGIN OPENSSH PRIVATE KEY-----
 ...
 -----END OPENSSH PRIVATE KEY-----
 known_hosts: |
 github.com ecdsa-sha2-nistp256 AAAA...

Alternatively, the Flux CLI can be used to automatically create the secret, and also populate the known_hosts:

flux create secret git podinfo-auth \
 --url=ssh://git@github.com/stefanprodan/podinfo \
 --private-key-file=./identity

For password-protected SSH private keys, the password must be provided via an additional password field in the secret. Flux CLI also supports this via the --password flag.

Provider

.spec.provider is an optional field that allows specifying an OIDC provider used for authentication purposes.

Supported options are:

generic
azure
github

When provider is not specified, it defaults to generic indicating that mechanisms using spec.secretRef are used for authentication.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

Azure

The azure provider can be used to authenticate to Azure DevOps repositories automatically using Workload Identity.

Pre-requisites

Ensure that your Azure DevOps Organization is connected to Microsoft Entra.
Ensure Workload Identity is properly set up on your cluster.

Configure Flux controller

Create a managed identity to access Azure DevOps. Establish a federated identity credential between the managed identity and the source-controller service account. In the default installation, the source-controller service account is located in the flux-system namespace with name source-controller. Ensure the federated credential uses the correct namespace and name of the source-controller service account. For more details, please refer to this guide.
Add the managed identity to the Azure DevOps organization as a user. Ensure that the managed identity has the necessary permissions to access the Azure DevOps repository as described here.
Add the following patch to your bootstrap repository in flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |-
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 namespace: flux-system
 annotations:
 azure.workload.identity/client-id: <AZURE_CLIENT_ID>
 labels:
 azure.workload.identity/use: "true"
 - patch: |-
 apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: source-controller
 namespace: flux-system
 labels:
 azure.workload.identity/use: "true"
 spec:
 template:
 metadata:
 labels:
 azure.workload.identity/use: "true"

Note: When azure provider is used with GitRepository, the .spec.url must follow this format:

https://dev.azure.com/{your-organization}/{your-project}/_git/{your-repository}

GitHub

The github provider can be used to authenticate to Git repositories using GitHub Apps.

Pre-requisites

Register the GitHub App with the necessary permissions and generate a private key for the app.
Install the app in the organization/account configuring access to the necessary repositories.

Configure GitHub App secret

The GitHub App information is specified in .spec.secretRef in the format specified below:

Get the App ID from the app settings page at https://github.com/settings/apps/<app-name>.
Get the App Installation ID from the app installations page at https://github.com/settings/installations. Click the installed app, the URL will contain the installation ID https://github.com/settings/installations/<installation-id>. For organizations, the first part of the URL may be different, but it follows the same pattern.
The private key that was generated in the pre-requisites.
(Optional) GitHub Enterprise Server users can set the base URL to http(s)://HOSTNAME/api/v3.
(Optional) If GitHub Enterprise Server uses a private CA, include its bundle (root and any intermediates) in ca.crt. If the ca.crt is specified, then it will be used for TLS verification for all API / Git over HTTPS requests to the GitHub Enterprise Server.

NOTE: If the secret contains tls.crt, tls.key then mutual TLS configuration will be automatically enabled. Omit these keys if the GitHub server does not support mutual TLS.

apiVersion: v1
kind: Secret
metadata:
 name: github-sa
type: Opaque
stringData:
 githubAppID: "<app-id>"
 githubAppInstallationID: "<app-installation-id>"
 githubAppPrivateKey: |
 -----BEGIN RSA PRIVATE KEY-----
 ...
 -----END RSA PRIVATE KEY-----
 githubAppBaseURL: "<github-enterprise-api-url>" #optional, required only for GitHub Enterprise Server users
 ca.crt: | #optional, for GitHub Enterprise Server users
 -----BEGIN CERTIFICATE-----
 ...
 -----END CERTIFICATE-----

Alternatively, the Flux CLI can be used to automatically create the secret with the github app authentication information.

flux create secret githubapp ghapp-secret \
 --app-id=1 \
 --app-installation-id=3 \
 --app-private-key=~/private-key.pem

Service Account reference

.spec.serviceAccountName is an optional field to specify a Service Account in the same namespace as GitRepository with purpose depending on the value of the .spec.provider field:

When .spec.provider is set to azure, the Service Account will be used for Workload Identity authentication. In this case, the controller feature gate ObjectLevelWorkloadIdentity must be enabled, otherwise the controller will error out. For Azure DevOps specific setup, see the Azure DevOps integration guide.

Note: that for a publicly accessible git repository, you don’t need to provide a secretRef nor serviceAccountName.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

Interval

.spec.interval is a required field that specifies the interval at which the Git repository must be fetched.

After successfully reconciling the object, the source-controller requeues it for inspection after the specified interval. The value must be in a Go recognized duration string format, e.g. 10m0s to reconcile the object every 10 minutes.

If the .metadata.generation of a resource changes (due to e.g. a change to the spec), this is handled instantly outside the interval window.

Note: The controller can be configured to apply a jitter to the interval in order to distribute the load more evenly when multiple GitRepository objects are set up with the same interval. For more information, please refer to the source-controller configuration options.

Timeout

.spec.timeout is an optional field to specify a timeout for Git operations like cloning. The value must be in a Go recognized duration string format, e.g. 1m30s for a timeout of one minute and thirty seconds. The default value is 60s.

Reference

.spec.ref is an optional field to specify the Git reference to resolve and watch for changes. References are specified in one or more subfields (.branch, .tag, .semver, .name, .commit), with latter listed fields taking precedence over earlier ones. If not specified, it defaults to a master branch reference.

Branch example

To Git checkout a specified branch, use .spec.ref.branch:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 branch: <branch-name>

This will perform a shallow clone to only fetch the specified branch.

Tag example

To Git checkout a specified tag, use .spec.ref.tag:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 tag: <tag-name>

This field takes precedence over .branch.

SemVer example

To Git checkout a tag based on a SemVer range, use .spec.ref.semver:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 # SemVer range reference: https://github.com/Masterminds/semver#checking-version-constraints
 semver: "<semver-range>"

This field takes precedence over .branch and .tag.

Name example

To Git checkout a specified reference, use .spec.ref.name:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 # Ref name format reference: https://git-scm.com/docs/git-check-ref-format#_description
 name: <reference-name>

Valid examples are: refs/heads/main, refs/tags/v0.1.0, refs/pull/420/head, refs/merge-requests/1/head.

This field takes precedence over .branch, .tag, and .semver.

Note: Azure DevOps and AWS CodeCommit do not support fetching the HEAD of a pull request. While Azure DevOps allows you to fetch the merge commit that will be created after merging a PR (using refs/pull/<id>/merge), this field can only be used to fetch references that exist in the current state of the Git repository and not references that will be created in the future.

Commit example

To Git checkout a specified commit, use .spec.ref.commit:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 commit: "<commit SHA>"

This field takes precedence over all other fields. It can be combined with .spec.ref.branch to perform a shallow clone of the branch, in which the commit must exist:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ref:
 branch: <branch>
 commit: "<commit SHA within branch>"

Verification

.spec.verify is an optional field to enable the verification of Git commit signatures. The field offers two subfields:

.mode, to specify what Git object(s) should be verified. Supported values are:
- HEAD: Verifies the commit object pointed to by the HEAD of the repository after performing a checkout via .spec.ref.
- head: Same as HEAD, supported for backwards compatibility purposes.
- Tag: Verifies the tag object pointed to by the specified/inferred tag reference in .spec.ref.tag, .spec.ref.semver or .spec.ref.name.
- TagAndHEAD: Verifies the tag object pointed to by the specified/inferred tag reference in .spec.ref.tag, .spec.ref.semver or .spec.ref.name and the commit object pointed to by the tag.
.secretRef.name, to specify a reference to a Secret in the same namespace as the GitRepository. Containing the (PGP) public keys of trusted Git authors.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 1m
 url: https://github.com/stefanprodan/podinfo
 ref:
 branch: master
 verify:
 mode: HEAD
 secretRef:
 name: pgp-public-keys

When the verification succeeds, the controller adds a Condition with the following attributes to the GitRepository’s .status.conditions:

type: SourceVerifiedCondition
status: "True"
reason: Succeeded

Verification Secret example

---
apiVersion: v1
kind: Secret
metadata:
 name: pgp-public-keys
 namespace: default
type: Opaque
data:
 author1.asc: <BASE64>
 author2.asc: <BASE64>

Exporting armored public keys (.asc files) using gpg, and generating a Secret:

# Export armored public keys
gpg --export --armor 3CB12BA185C47B67 > author1.asc
gpg --export --armor 6A7436E8790F8689 > author2.asc
# Generate secret
kubectl create secret generic pgp-public-keys \
 --from-file=author1.asc \
 --from-file=author2.asc \
 -o yaml

Ignore

.spec.ignore is an optional field to specify rules in the .gitignore pattern format. Paths matching the defined rules are excluded while archiving.

When specified, .spec.ignore overrides the default exclusion list, and may overrule the .sourceignore file exclusions. See excluding files for more information.

Sparse checkout

.spec.sparseCheckout is an optional field to specify list of directories to checkout when cloning the repository. If specified, only the specified directory contents will be present in the artifact produced for this repository.

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m
 url: https://github.com/stefanprodan/podinfo
 ref:
 branch: master
 sparseCheckout:
 - charts
 - kustomize

Suspend

.spec.suspend is an optional field to suspend the reconciliation of a GitRepository. When set to true, the controller will stop reconciling the GitRepository, and changes to the resource or in the Git repository will not result in a new Artifact. When the field is set to false or removed, it will resume.

Proxy secret reference

.spec.proxySecretRef.name is an optional field used to specify the name of a Secret that contains the proxy settings for the object. These settings are used for all remote Git operations related to the GitRepository. The Secret can contain three keys:

address, to specify the address of the proxy server. This is a required key.
username, to specify the username to use if the proxy server is protected by basic authentication. This is an optional key.
password, to specify the password to use if the proxy server is protected by basic authentication. This is an optional key.

The proxy server must be either HTTP/S or SOCKS5. You can use a SOCKS5 proxy with a HTTP/S Git repository url.

Examples:

---
apiVersion: v1
kind: Secret
metadata:
 name: http-proxy
type: Opaque
stringData:
 address: http://proxy.com
 username: mandalorian
 password: grogu

---
apiVersion: v1
kind: Secret
metadata:
 name: ssh-proxy
type: Opaque
stringData:
 address: socks5://proxy.com
 username: mandalorian
 password: grogu

Proxying can also be configured in the source-controller Deployment directly by using the standard environment variables such as HTTPS_PROXY, ALL_PROXY, etc.

.spec.proxySecretRef.name takes precedence over all environment variables.

Recurse submodules

.spec.recurseSubmodules is an optional field to enable the initialization of all submodules within the cloned Git repository, using their default settings. This option defaults to false.

Note that for most Git providers (e.g. GitHub and GitLab), deploy keys can not be used as reusing a key across multiple repositories is not allowed. You have to use either HTTPS token-based authentication, or an SSH key belonging to a (bot) user who has access to the main repository and all submodules.

Include

.spec.include is an optional field to map the contents of GitRepository Artifacts into another. This may look identical to Git submodules but has multiple benefits over regular submodules:

Including a GitRepository allows you to use different authentication methods for different repositories.
A change in the included repository will trigger an update of the including repository.
Multiple GitRepository objects could include the same repository, which decreases the amount of cloning done compared to using submodules.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: include-example
spec:
 include:
 - repository:
 name: other-repository
 fromPath: deploy/kubernetes
 toPath: base/app

The .fromPath and .toPath fields allow you to limit the files included, and where they will be copied to. If you do not specify a value for .fromPath, all files from the referenced GitRepository Artifact will be included. The .toPath defaults to the .repository.name (e.g. ./other-repository/*).

Working with GitRepositories

Excluding files

By default, files which match the default exclusion rules are excluded while archiving the Git repository contents as an Artifact. It is possible to overwrite and/or overrule the default exclusions using a file in the Git repository and/or an in-spec set of rules.

`.sourceignore` file

Excluding files is possible by adding a .sourceignore file in the Git repository. The .sourceignore file follows the .gitignore pattern format, and pattern entries may overrule default exclusions.

The controller recursively loads ignore files so a .sourceignore can be placed in the repository root or in subdirectories.

Ignore spec

Another option is to define the exclusions within the GitRepository spec, using the .spec.ignore field. Specified rules override the default exclusion list, and may overrule .sourceignore file exclusions.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 ignore: |
 # exclude all
 /*
 # include deploy dir
 !/deploy
 # exclude file extensions from deploy dir
 /deploy/**/*.md
 /deploy/**/*.txt

Triggering a reconcile

To manually tell the source-controller to reconcile a GitRepository outside the specified interval window, a GitRepository can be annotated with reconcile.fluxcd.io/requestedAt: <arbitrary value>. Annotating the resource queues the GitRepository for reconciliation if the <arbitrary-value> differs from the last value the controller acted on, as reported in .status.lastHandledReconcileAt.

Using kubectl:

kubectl annotate --field-manager=flux-client-side-apply --overwrite gitrepository/<repository-name> reconcile.fluxcd.io/requestedAt="$(date +%s)"

Using flux:

flux reconcile source git <repository-name>

Waiting for `Ready`

When a change is applied, it is possible to wait for the GitRepository to reach a ready state using kubectl:

kubectl wait gitrepository/<repository-name> --for=condition=ready --timeout=1m

Suspending and resuming

When you find yourself in a situation where you temporarily want to pause the reconciliation of a GitRepository, you can suspend it using the .spec.suspend field.

Suspend a GitRepository

In your YAML declaration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 suspend: true

Using kubectl:

kubectl patch gitrepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\": {\"suspend\" : true }}'

Using flux:

flux suspend source git <repository-name>

Note: When a GitRepository has an Artifact and is suspended, and this Artifact later disappears from the storage due to e.g. the source-controller Pod being evicted from a Node, this will not be reflected in the GitRepository’s Status until it is resumed.

Resume a GitRepository

In your YAML declaration, comment out (or remove) the field:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
spec:
 # suspend: true

Note: Setting the field value to false has the same effect as removing it, but does not allow for “hot patching” using e.g. kubectl while practicing GitOps; as the manually applied patch would be overwritten by the declared state in Git.

Using kubectl:

kubectl patch gitrepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\" : {\"suspend\" : false }}'

Using flux:

flux resume source git <repository-name>

Debugging a GitRepository

There are several ways to gather information about a GitRepository for debugging purposes.

Describe the GitRepository

Describing a GitRepository using kubectl describe gitrepository <repository-name> displays the latest recorded information for the resource in the Status and Events sections:

...
Status:
...
 Conditions:
 Last Transition Time: 2022-02-14T09:40:27Z
 Message: processing object: new generation 1 -> 2
 Observed Generation: 2
 Reason: ProgressingWithRetry
 Status: True
 Type: Reconciling
 Last Transition Time: 2022-02-14T09:40:27Z
 Message: failed to checkout and determine revision: unable to clone 'https://github.com/stefanprodan/podinfo': couldn't find remote ref "refs/heads/invalid"
 Observed Generation: 2
 Reason: GitOperationFailed
 Status: False
 Type: Ready
 Last Transition Time: 2022-02-14T09:40:27Z
 Message: failed to checkout and determine revision: unable to clone 'https://github.com/stefanprodan/podinfo': couldn't find remote ref "refs/heads/invalid"
 Observed Generation: 2
 Reason: GitOperationFailed
 Status: True
 Type: FetchFailed
 Observed Generation: 1
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Warning GitOperationFailed 2s (x9 over 4s) source-controller failed to checkout and determine revision: unable to clone 'https://github.com/stefanprodan/podinfo': couldn't find remote ref "refs/heads/invalid"

Trace emitted Events

To view events for specific GitRepository(s), kubectl events can be used in combination with --for to list the Events for specific objects. For example, running

kubectl events --for GitRepository/<repository-name>

lists

LAST SEEN TYPE REASON OBJECT MESSAGE
2m14s Normal NewArtifact gitrepository/<repository-name> stored artifact for commit 'Merge pull request #160 from stefanprodan/release-6.0.3'
36s Normal ArtifactUpToDate gitrepository/<repository-name> artifact up-to-date with remote revision: 'master@sha1:132f4e719209eb10b9485302f8593fc0e680f4fc'
94s Warning GitOperationFailed gitrepository/<repository-name> failed to checkout and determine revision: unable to clone 'https://github.com/stefanprodan/podinfo': couldn't find remote ref "refs/heads/invalid"

Besides being reported in Events, the reconciliation errors are also logged by the controller. The Flux CLI offer commands for filtering the logs for a specific GitRepository, e.g. flux logs --level=error --kind=GitRepository --name=<repository-name>.

GitRepository Status

Artifact

The GitRepository reports the latest synchronized state from the Git repository as an Artifact object in the .status.artifact of the resource.

The Artifact file is a gzip compressed TAR archive (<commit sha>.tar.gz), and can be retrieved in-cluster from the .status.artifact.url HTTP address.

Artifact example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
 name: <repository-name>
status:
 artifact:
 digest: sha256:e750c7a46724acaef8f8aa926259af30bbd9face2ae065ae8896ba5ee5ab832b
 lastUpdateTime: "2022-01-29T06:59:23Z"
 path: gitrepository/<namespace>/<repository-name>/c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2.tar.gz
 revision: master@sha1:363a6a8fe6a7f13e05d34c163b0ef02a777da20a
 size: 91318
 url: http://source-controller.<namespace>.svc.cluster.local./gitrepository/<namespace>/<repository-name>/363a6a8fe6a7f13e05d34c163b0ef02a777da20a.tar.gz

Default exclusions

The following files and extensions are excluded from the Artifact by default:

Git files (.git/, .gitignore, .gitmodules, .gitattributes)
File extensions (.jpg, .jpeg, .gif, .png, .wmv, .flv, .tar.gz, .zip)
CI configs (.github/, .circleci/, .travis.yml, .gitlab-ci.yml, appveyor.yml, .drone.yml, cloudbuild.yaml, codeship-services.yml, codeship-steps.yml)
CLI configs (.goreleaser.yml, .sops.yaml)
Flux v1 config (.flux.yaml)

To define your own exclusion rules, see excluding files.

Conditions

A GitRepository enters various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching the Git state, it can be ready, or it can fail during reconciliation.

The GitRepository API is compatible with the kstatus specification, and reports Reconciling and Stalled conditions where applicable to provide better (timeout) support to solutions polling the GitRepository to become Ready.

Reconciling GitRepository

The source-controller marks a GitRepository as reconciling when one of the following is true:

There is no current Artifact for the GitRepository, or the reported Artifact is determined to have disappeared from the storage.
The generation of the GitRepository is newer than the Observed Generation.
The newly resolved Artifact revision differs from the current Artifact.

When the GitRepository is “reconciling”, the Ready Condition status becomes Unknown when the controller detects drift, and the controller adds a Condition with the following attributes to the GitRepository’s .status.conditions:

type: Reconciling
status: "True"
reason: Progressing | reason: ProgressingWithRetry

If the reconciling state is due to a new revision, an additional Condition is added with the following attributes:

type: ArtifactOutdated
status: "True"
reason: NewRevision

Both Conditions have a “negative polarity”, and are only present on the GitRepository while their status value is "True".

Ready GitRepository

The source-controller marks a GitRepository as ready when it has the following characteristics:

The GitRepository reports an Artifact.
The reported Artifact exists in the controller’s Artifact storage.
The controller was able to communicate with the remote Git repository using the current spec.
The revision of the reported Artifact is up-to-date with the latest resolved revision of the remote Git repository.

When the GitRepository is “ready”, the controller sets a Condition with the following attributes in the GitRepository’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the GitRepository is marked as reconciling, or e.g. a transient error occurs due to a temporary network issue.

When the GitRepository Artifact is archived in the controller’s Artifact storage, the controller sets a Condition with the following attributes in the GitRepository’s .status.conditions:

type: ArtifactInStorage
status: "True"
reason: Succeeded

This ArtifactInStorage Condition will retain a status value of "True" until the Artifact in the storage no longer exists.

Failed GitRepository

The source-controller may get stuck trying to produce an Artifact for a GitRepository without completing. This can occur due to some of the following factors:

The remote Git repository URL is temporarily unavailable.
The Git repository does not exist.
The Secret reference contains a reference to a non-existing Secret.
A specified Include is unavailable.
The verification of the Git commit signature failed.
The credentials in the referenced Secret are invalid.
The GitRepository spec contains a generic misconfiguration.
A storage related failure when storing the artifact.

When this happens, the controller sets the Ready Condition status to False, and adds a Condition with the following attributes to the GitRepository’s .status.conditions:

type: FetchFailed | type: IncludeUnavailable | type: StorageOperationFailed
status: "True"
reason: AuthenticationFailed | reason: GitOperationFailed

This condition has a “negative polarity”, and is only present on the GitRepository while the status value is "True". There may be more arbitrary values for the reason field to provide accurate reason for a condition.

In addition to the above Condition types, when the verification of a Git commit signature fails. A condition with the following attributes is added to the GitRepository’s .status.conditions:

type: SourceVerifiedCondition
status: "False"
reason: Failed

While the GitRepository has one or more of these Conditions, the controller will continue to attempt to produce an Artifact for the resource with an exponential backoff, until it succeeds and the GitRepository is marked as ready.

Note that a GitRepository can be reconciling while failing at the same time, for example due to a newly introduced configuration issue in the GitRepository spec. When a reconciliation fails, the Reconciling Condition reason would be ProgressingWithRetry. When the reconciliation is performed again after the failure, the reason is updated to Progressing.

Observed Ignore

The source-controller reports an observed ignore in the GitRepository’s .status.observedIgnore. The observed ignore is the latest .spec.ignore value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the ignore in spec. It indicates the ignore rules used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedIgnore: |
 cue
 pkg
 ...

Observed Recurse Submodules

The source-controller reports an observed recurse submodule in the GitRepository’s .status.observedRecurseSubmodules. The observed recurse submodules is the latest .spec.recurseSubmodules value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the recurse submodules in spec. It indicates the recurse submodules configuration used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedRecurseSubmodules: true
 ...

Observed Include

The source-controller reports observed include in the GitRepository’s .status.observedInclude. The observed include is the latest .spec.recurseSubmodules value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the include in spec. It indicates the include configuration used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedInclude:
 - fromPath: deploy/webapp
 repository:
 name: repo1
 toPath: foo
 - fromPath: deploy/secure
 repository:
 name: repo2
 toPath: bar
 ...

Observed Sparse Checkout

The source-controller reports observed sparse checkout in the GitRepository’s .status.observedSparseCheckout. The observed sparse checkout is the latest .spec.sparseCheckout value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the sparseCheckout in spec. It indicates the sparse checkout configuration used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedSparseCheckout:
 - charts
 - kustomize
 ...

Source Verification Mode

The source-controller reports the Git object(s) it verified in the Git repository to create an artifact in the GitRepository’s .status.sourceVerificationMode. This value is the same as the verification mode in spec. The verification status is applicable only to the latest Git repository revision used to successfully build and store an artifact.

Observed Generation

The source-controller reports an observed generation in the GitRepository’s .status.observedGeneration. The observed generation is the latest .metadata.generation which resulted in either a ready state, or stalled due to error it can not recover from without human intervention.

Last Handled Reconcile At

The source-controller reports the last reconcile.fluxcd.io/requestedAt annotation value it acted on in the .status.lastHandledReconcileAt field.

For practical information about this field, see triggering a reconcile.

Flux: OCI Repositories

Mon, 01 Jan 0001 00:00:00 +0000

The OCIRepository API defines a Source to produce an Artifact for an OCI repository.

Example

The following is an example of an OCIRepository. It creates a tarball (.tar.gz) Artifact with the fetched data from an OCI repository for the resolved digest.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 url: oci://ghcr.io/stefanprodan/manifests/podinfo
 ref:
 tag: latest

In the above example:

An OCIRepository named podinfo is created, indicated by the .metadata.name field.
The source-controller checks the OCI repository every five minutes, indicated by the .spec.interval field.
It pulls the latest tag of the ghcr.io/stefanprodan/manifests/podinfo repository, indicated by the .spec.ref.tag and .spec.url fields.
The resolved tag and SHA256 digest is used as the Artifact revision, reported in-cluster in the .status.artifact.revision field.
When the current OCIRepository digest differs from the latest fetched digest, a new Artifact is archived.
The new Artifact is reported in the .status.artifact field.

You can run this example by saving the manifest into ocirepository.yaml.

Apply the resource on the cluster:
```
kubectl apply -f ocirepository.yaml
```

Run kubectl get ocirepository to see the OCIRepository:

NAME URL AGE READY STATUS
podinfo oci://ghcr.io/stefanprodan/manifests/podinfo 5s True stored artifact with revision 'latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de'

Run kubectl describe ocirepository podinfo to see the Artifact and Conditions in the OCIRepository’s Status:

...
Status:
 Artifact:
 Digest: sha256:d7e924b4882e55b97627355c7b3d2e711e9b54303afa2f50c25377f4df66a83b
 Last Update Time: 2025-06-14T11:23:36Z
 Path: ocirepository/default/podinfo/3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de.tar.gz
 Revision: latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de
 Size: 1105
 URL: http://source-controller.flux-system.svc.cluster.local./ocirepository/oci/podinfo/3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de.tar.gz
 Conditions:
 Last Transition Time: 2025-06-14T11:23:36Z
 Message: stored artifact for revision 'latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: Ready
 Last Transition Time: 2025-06-14T11:23:36Z
 Message: stored artifact for revision 'latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: ArtifactInStorage
 Observed Generation: 1
 URL: http://source-controller.source-system.svc.cluster.local./gitrepository/default/podinfo/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal NewArtifact 62s source-controller stored artifact with revision 'latest/3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de' from 'oci://ghcr.io/stefanprodan/manifests/podinfo'

Writing an OCIRepository spec

As with all other Kubernetes config, an OCIRepository needs apiVersion, kind, and metadata fields. The name of an OCIRepository object must be a valid DNS subdomain name.

An OCIRepository also needs a .spec section.

URL

.spec.url is a required field that specifies the address of the container image repository in the format oci://<host>:<port>/<org-name>/<repo-name>.

Note: that specifying a tag or digest is not acceptable for this field.

Provider

.spec.provider is an optional field that allows specifying an OIDC provider used for authentication purposes.

Supported options are:

generic
aws
azure
gcp

The generic provider can be used for public repositories or when static credentials are used for authentication, either with spec.secretRef or spec.serviceAccountName. If you do not specify .spec.provider, it defaults to generic.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

AWS

The aws provider can be used to authenticate automatically using the EKS worker node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access to ECR.

When the worker node IAM role has access to ECR, source-controller running on it will also have access to ECR.

When using IRSA to enable access to ECR, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 annotations:
 eks.amazonaws.com/role-arn: <role arn>
 target:
 kind: ServiceAccount
 name: source-controller

Note that you can attach the AWS managed policy arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly to the IAM role when using IRSA.

Azure

The azure provider can be used to authenticate automatically using Workload Identity and Kubelet Managed Identity to gain access to ACR.

Kubelet Managed Identity

When the kubelet managed identity has access to ACR, source-controller running on it will also have access to ACR.

Note: If you have more than one identity configured on the cluster, you have to specify which one to use by setting the AZURE_CLIENT_ID environment variable in the source-controller deployment.

If you are running into further issues, please look at the troubleshooting guide.

Workload Identity

When using Workload Identity to enable access to ACR, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |-
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 namespace: flux-system
 annotations:
 azure.workload.identity/client-id: <AZURE_CLIENT_ID>
 labels:
 azure.workload.identity/use: "true"
 - patch: |-
 apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: source-controller
 namespace: flux-system
 labels:
 azure.workload.identity/use: "true"
 spec:
 template:
 metadata:
 labels:
 azure.workload.identity/use: "true"

Ensure Workload Identity is properly set up on your cluster and the mutating webhook is installed. Create an identity that has access to ACR. Next, establish a federated identity between the source-controller ServiceAccount and the identity. Patch the source-controller Deployment and ServiceAccount as shown in the patch above. Please take a look at this guide.

GCP

The gcp provider can be used to authenticate automatically using OAuth scopes or Workload Identity, and by extension gain access to GCR or Artifact Registry.

When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry, source-controller running on it will also have access to them.

When using Workload Identity to enable access to GCR or Artifact Registry, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 annotations:
 iam.gke.io/gcp-service-account: <identity-name>
 target:
 kind: ServiceAccount
 name: source-controller

The Artifact Registry service uses the permission artifactregistry.repositories.downloadArtifacts that is located under the Artifact Registry Reader role. If you are using Google Container Registry service, the needed permission is instead storage.objects.list which can be bound as part of the Container Registry Service Agent role. Take a look at this guide for more information about setting up GKE Workload Identity.

Secret reference

.spec.secretRef.name is an optional field to specify a name reference to a Secret in the same namespace as the OCIRepository, containing authentication credentials for the OCI repository.

This secret is expected to be in the same format as imagePullSecrets. The usual way to create such a secret is with:

kubectl create secret docker-registry ...

Service Account reference

.spec.serviceAccountName is an optional field to specify a Service Account in the same namespace as OCIRepository with purpose depending on the value of the .spec.provider field:

When .spec.provider is set to generic, the controller will fetch the image pull secrets attached to the Service Account and use them for authentication.
When .spec.provider is set to aws, azure, or gcp, the Service Account will be used for Workload Identity authentication. In this case, the controller feature gate ObjectLevelWorkloadIdentity must be enabled, otherwise the controller will error out.

Note: that for a publicly accessible image repository, you don’t need to provide a secretRef nor serviceAccountName.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

Mutual TLS Authentication

.spec.certSecretRef.name is an optional field to specify a secret containing TLS certificate data for mutual TLS authentication.

To authenticate towards an OCI repository using mutual TLS, the referenced Secret’s .data should contain the following keys:

tls.crt and tls.key, to specify the client certificate and private key used for TLS client authentication. These must be used in conjunction, i.e. specifying one without the other will lead to an error.
ca.crt, to specify the CA certificate used to verify the server, which is required if the server is using a self-signed certificate.

The Secret should be of type Opaque or kubernetes.io/tls. All the files in the Secret are expected to be PEM-encoded. Assuming you have three files; client.key, client.crt and ca.crt for the client private key, client certificate and the CA certificate respectively, you can generate the required Secret using the flux create secret tls command:

flux create secret tls --tls-key-file=client.key --tls-crt-file=client.crt --ca-crt-file=ca.crt

Example usage:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: example
 namespace: default
spec:
 interval: 5m0s
 url: oci://example.com
 certSecretRef:
 name: example-tls
---
apiVersion: v1
kind: Secret
metadata:
 name: example-tls
 namespace: default
type: kubernetes.io/tls # or Opaque
data:
 tls.crt: <BASE64>
 tls.key: <BASE64>
 # NOTE: Can be supplied without the above values
 ca.crt: <BASE64>

Proxy secret reference

.spec.proxySecretRef.name is an optional field used to specify the name of a Secret that contains the proxy settings for the object. These settings are used for all the remote operations related to the OCIRepository. The Secret can contain three keys:

address, to specify the address of the proxy server. This is a required key.
username, to specify the username to use if the proxy server is protected by basic authentication. This is an optional key.
password, to specify the password to use if the proxy server is protected by basic authentication. This is an optional key.

Example:

---
apiVersion: v1
kind: Secret
metadata:
 name: http-proxy
type: Opaque
stringData:
 address: http://proxy.com
 username: mandalorian
 password: grogu

Proxying can also be configured in the source-controller Deployment directly by using the standard environment variables such as HTTPS_PROXY, ALL_PROXY, etc.

.spec.proxySecretRef.name takes precedence over all environment variables.

Warning: Cosign keyless verification is not supported for this API. If you require cosign keyless verification to use a proxy you must use the standard environment variables mentioned above. If you specify a proxySecretRef the controller will simply send out the requests needed for keyless verification without the associated object-level proxy settings.

Insecure

.spec.insecure is an optional field to allow connecting to an insecure (HTTP) container registry server, if set to true. The default value is false, denying insecure (HTTP) connections.

Interval

.spec.interval is a required field that specifies the interval at which the OCI repository must be fetched.

If the .metadata.generation of a resource changes (due to e.g. a change to the spec), this is handled instantly outside the interval window.

Note: The controller can be configured to apply a jitter to the interval in order to distribute the load more evenly when multiple OCIRepository objects are set up with the same interval. For more information, please refer to the source-controller configuration options.

Timeout

.spec.timeout is an optional field to specify a timeout for OCI operations like pulling. The value must be in a Go recognized duration string format, e.g. 1m30s for a timeout of one minute and thirty seconds. The default value is 60s.

Reference

.spec.ref is an optional field to specify the OCI reference to resolve and watch for changes. References are specified in one or more subfields (.tag, .semver, .digest), with latter listed fields taking precedence over earlier ones. If not specified, it defaults to the latest tag.

Tag example

To pull a specific tag, use .spec.ref.tag:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 ref:
 tag: "<tag-name>"

SemVer example

To pull a tag based on a SemVer range, use .spec.ref.semver:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 ref:
 # SemVer range reference: https://github.com/Masterminds/semver#checking-version-constraints
 semver: "<semver-range>"

This field takes precedence over .tag.

SemverFilter example

.spec.ref.semverFilter is an optional field to specify a SemVer filter to apply when fetching tags from the OCI repository. The filter is a regular expression that is applied to the tags fetched from the repository. Only tags that match the filter are considered for the semver range resolution.

Note: The filter is only taken into account when the .spec.ref.semver field is set.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 url: oci://ghcr.io/stefanprodan/manifests/podinfo
 ref:
 # SemVer comparisons using constraints without a prerelease comparator will skip prerelease versions.
 # Adding a `-0` suffix to the semver range will include prerelease versions.
 semver: ">= 6.1.x-0"
 semverFilter: ".*-rc.*"

In the above example, the controller fetches tags from the ghcr.io/stefanprodan/manifests/podinfo repository and filters them using the regular expression .*-rc.*. Only tags that contain the -rc suffix are considered for the semver range resolution.

Digest example

To pull a specific digest, use .spec.ref.digest:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 ref:
 digest: "sha256:<SHA-value>"

This field takes precedence over all other fields.

Layer selector

spec.layerSelector is an optional field to specify which layer should be extracted from the OCI Artifact. If not specified, the controller will extract the first layer found in the artifact.

To extract a layer matching a specific OCI media type:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 layerSelector:
 mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
 operation: extract # can be 'extract' or 'copy', defaults to 'extract'

If the layer selector matches more than one layer, the first layer matching the specified media type will be used. Note that the selected OCI layer must be compressed in the tar+gzip format.

When .spec.layerSelector.operation is set to copy, instead of extracting the compressed layer, the controller copies the tarball as-is to storage, thus keeping the original content unaltered.

Ignore

.spec.ignore is an optional field to specify rules in the .gitignore pattern format. Paths matching the defined rules are excluded while archiving.

When specified, .spec.ignore overrides the default exclusion list, and may overrule the .sourceignore file exclusions. See excluding files for more information.

Verification

.spec.verify is an optional field to enable the verification of Cosign or Notation signatures. The field offers three subfields:

.provider, to specify the verification provider. The supported options are cosign and notation at present.
.secretRef.name, to specify a reference to a Secret in the same namespace as the OCIRepository, containing the Cosign public keys of trusted authors. For Notation this Secret should also include the trust policy in addition to the CA certificate.
.matchOIDCIdentity, to specify a list of OIDC identity matchers (only supported when using cosign as the verification provider). Please see Keyless verification for more details.

Cosign

The cosign provider can be used to verify the signature of an OCI artifact using either a known public key or via the Cosign Keyless procedure.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 verify:
 provider: cosign
 secretRef:
 name: cosign-public-keys

When the verification succeeds, the controller adds a Condition with the following attributes to the OCIRepository’s .status.conditions:

type: SourceVerified
status: "True"
reason: Succeeded

Public keys verification

To verify the authenticity of an OCI artifact, create a Kubernetes secret with the Cosign public keys:

---
apiVersion: v1
kind: Secret
metadata:
 name: cosign-public-keys
type: Opaque
data:
 key1.pub: <BASE64>
 key2.pub: <BASE64>

Note that the keys must have the .pub extension for Flux to make use of them.

Flux will loop over the public keys and use them to verify an artifact’s signature. This allows for older artifacts to be valid as long as the right key is in the secret.

Keyless verification

For publicly available OCI artifacts, which are signed using the Cosign Keyless procedure, you can enable the verification by omitting the .verify.secretRef field.

To verify the identity’s subject and the OIDC issuer present in the Fulcio certificate, you can specify a list of OIDC identity matchers using .spec.verify.matchOIDCIdentity. The matcher provides two required fields:

.issuer, to specify a regexp that matches against the OIDC issuer.
.subject, to specify a regexp that matches against the subject identity in the certificate. Both values should follow the Go regular expression syntax.

The matchers are evaluated in an OR fashion, i.e. the identity is deemed to be verified if any one matcher successfully matches against the identity.

Example of verifying artifacts signed by the Cosign GitHub Action with GitHub OIDC Token:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: podinfo
spec:
 interval: 5m
 url: oci://ghcr.io/stefanprodan/manifests/podinfo
 verify:
 provider: cosign
 matchOIDCIdentity:
 - issuer: "^https://token.actions.githubusercontent.com$"
 subject: "^https://github.com/stefanprodan/podinfo.*$"

The controller verifies the signatures using the Fulcio root CA and the Rekor instance hosted at rekor.sigstore.dev.

Note that keyless verification is an experimental feature, using custom root CAs or self-hosted Rekor instances are not currently supported.

Notation

The notation provider can be used to verify the signature of an OCI artifact using known trust policy and CA certificate.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 verify:
 provider: notation
 secretRef:
 name: notation-config

When the verification succeeds, the controller adds a Condition with the following attributes to the OCIRepository’s .status.conditions:

type: SourceVerified
status: "True"
reason: Succeeded

To verify the authenticity of an OCI artifact, create a Kubernetes secret containing Certificate Authority (CA) root certificates and the a trust policy

---
apiVersion: v1
kind: Secret
metadata:
 name: notation-config
type: Opaque
data:
 certificate1.pem: <BASE64>
 certificate2.crt: <BASE64>
 trustpolicy.json: <BASE64>

Note that the CA certificates must have either .pem or .crt extension and your trust policy must be named trustpolicy.json for Flux to make use of them.

For more information on the signing and verification process see Signing and Verification Workflow.

Flux will loop over the certificates and use them to verify an artifact’s signature. This allows for older artifacts to be valid as long as the right certificate is in the secret.

Suspend

.spec.suspend is an optional field to suspend the reconciliation of a OCIRepository. When set to true, the controller will stop reconciling the OCIRepository, and changes to the resource or in the OCI repository will not result in a new Artifact. When the field is set to false or removed, it will resume.

Working with OCIRepositories

Excluding files

By default, files which match the default exclusion rules are excluded while archiving the OCI repository contents as an Artifact. It is possible to overwrite and/or overrule the default exclusions using the .spec.ignore field.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 ignore: |
 # exclude all
 /*
 # include deploy dir
 !/deploy
 # exclude file extensions from deploy dir
 /deploy/**/*.md
 /deploy/**/*.txt

`.sourceignore` file

Excluding files is possible by adding a .sourceignore file in the artifact. The .sourceignore file follows the .gitignore pattern format, and pattern entries may overrule default exclusions.

The controller recursively loads ignore files so a .sourceignore can be placed in the artifact root or in subdirectories.

Triggering a reconcile

To manually tell the source-controller to reconcile a OCIRepository outside the specified interval window, an OCIRepository can be annotated with reconcile.fluxcd.io/requestedAt: <arbitrary value>. Annotating the resource queues the OCIRepository for reconciliation if the <arbitrary-value> differs from the last value the controller acted on, as reported in .status.lastHandledReconcileAt.

Using kubectl:

kubectl annotate --field-manager=flux-client-side-apply --overwrite ocirepository/<repository-name> reconcile.fluxcd.io/requestedAt="$(date +%s)"

Using flux:

flux reconcile source oci <repository-name>

Waiting for `Ready`

When a change is applied, it is possible to wait for the OCIRepository to reach a ready state using kubectl:

kubectl wait gitrepository/<repository-name> --for=condition=ready --timeout=1m

Suspending and resuming

When you find yourself in a situation where you temporarily want to pause the reconciliation of an OCIRepository, you can suspend it using the .spec.suspend field.

Suspend an OCIRepository

In your YAML declaration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 suspend: true

Using kubectl:

kubectl patch ocirepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\": {\"suspend\" : true }}'

Using flux:

flux suspend source oci <repository-name>

Note: When an OCIRepository has an Artifact and it is suspended, and this Artifact later disappears from the storage due to e.g. the source-controller Pod being evicted from a Node, this will not be reflected in the OCIRepository’s Status until it is resumed.

Resume an OCIRepository

In your YAML declaration, comment out (or remove) the field:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
spec:
 # suspend: true

Using kubectl:

kubectl patch ocirepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\" : {\"suspend\" : false }}'

Using flux:

flux resume source oci <repository-name>

Debugging an OCIRepository

There are several ways to gather information about a OCIRepository for debugging purposes.

Describe the OCIRepository

Describing an OCIRepository using kubectl describe ocirepository <repository-name> displays the latest recorded information for the resource in the Status and Events sections:

...
Status:
...
 Conditions:
 Last Transition Time: 2025-02-14T09:40:27Z
 Message: processing object: new generation 1 -> 2
 Observed Generation: 2
 Reason: ProgressingWithRetry
 Status: True
 Type: Reconciling
 Last Transition Time: 2025-02-14T09:40:27Z
 Message: failed to pull artifact from 'oci://ghcr.io/stefanprodan/manifests/podinfo': couldn't find tag "0.0.1"
 Observed Generation: 2
 Reason: OCIOperationFailed
 Status: False
 Type: Ready
 Last Transition Time: 2025-02-14T09:40:27Z
 Message: failed to pull artifact from 'oci://ghcr.io/stefanprodan/manifests/podinfo': couldn't find tag "0.0.1"
 Observed Generation: 2
 Reason: OCIOperationFailed
 Status: True
 Type: FetchFailed
 Observed Generation: 1
 URL: http://source-controller.source-system.svc.cluster.local./ocirepository/default/podinfo/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Warning OCIOperationFailed 2s (x9 over 4s) source-controller failed to pull artifact from 'oci://ghcr.io/stefanprodan/manifests/podinfo': couldn't find tag "0.0.1"

Trace emitted Events

To view events for specific OCIRepository(s), kubectl events can be used in combination with --for to list the Events for specific objects. For example, running

kubectl events --for OCIRepository/<repository-name>

lists

LAST SEEN TYPE REASON OBJECT MESSAGE
2m14s Normal NewArtifact ocirepository/<repository-name> stored artifact for revision 'latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de'
36s Normal ArtifactUpToDate ocirepository/<repository-name> artifact up-to-date with remote revision: 'latest@sha256:3b6cdcc7adcc9a84d3214ee1c029543789d90b5ae69debe9efa3f66e982875de'
94s Warning OCIOperationFailed ocirepository/<repository-name> failed to pull artifact from 'oci://ghcr.io/stefanprodan/manifests/podinfo': couldn't find tag "0.0.1"

Besides being reported in Events, the reconciliation errors are also logged by the controller. The Flux CLI offer commands for filtering the logs for a specific OCIRepository, e.g. flux logs --level=error --kind=OCIRepository --name=<repository-name>.

OCIRepository Status

Artifact

The OCIRepository reports the latest synchronized state from the OCI repository as an Artifact object in the .status.artifact of the resource.

The .status.artifact.revision holds the tag and SHA256 digest of the upstream OCI artifact.

The .status.artifact.metadata holds the upstream OCI artifact metadata such as the OpenContainers standard annotations. If the OCI artifact was created with flux push artifact, then the metadata will contain the following annotations:

org.opencontainers.image.created the date and time on which the artifact was built
org.opencontainers.image.source the URL of the Git repository containing the source files
org.opencontainers.image.revision the Git branch and commit SHA1 of the source files

The Artifact file is a gzip compressed TAR archive (<commit sha>.tar.gz), and can be retrieved in-cluster from the .status.artifact.url HTTP address.

Artifact example

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
 name: <repository-name>
status:
 artifact:
 digest: sha256:9f3bc0f341d4ecf2bab460cc59320a2a9ea292f01d7b96e32740a9abfd341088
 lastUpdateTime: "2025-08-08T09:35:45Z"
 metadata:
 org.opencontainers.image.created: "2025-08-08T12:31:41+03:00"
 org.opencontainers.image.revision: 6.1.8/b3b00fe35424a45d373bf4c7214178bc36fd7872
 org.opencontainers.image.source: https://github.com/stefanprodan/podinfo.git
 path: ocirepository/<namespace>/<repository-name>/<digest>.tar.gz
 revision: <tag>@<digest>
 size: 1105
 url: http://source-controller.<namespace>.svc.cluster.local./ocirepository/<namespace>/<repository-name>/<digest>.tar.gz

Default exclusions

The following files and extensions are excluded from the Artifact by default:

Git files (.git/, .gitignore, .gitmodules, .gitattributes)
File extensions (.jpg, .jpeg, .gif, .png, .wmv, .flv, .tar.gz, .zip)
CI configs (.github/, .circleci/, .travis.yml, .gitlab-ci.yml, appveyor.yml, .drone.yml, cloudbuild.yaml, codeship-services.yml, codeship-steps.yml)
CLI configs (.goreleaser.yml, .sops.yaml)
Flux v1 config (.flux.yaml)

To define your own exclusion rules, see excluding files.

Conditions

OCIRepository has various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching the remote state, it can be ready, or it can fail during reconciliation.

The OCIRepository API is compatible with the kstatus specification, and reports Reconciling and Stalled conditions where applicable to provide better (timeout) support to solutions polling the OCIRepository to become Ready.

Reconciling OCIRepository

The source-controller marks an OCIRepository as reconciling when one of the following is true:

There is no current Artifact for the OCIRepository, or the reported Artifact is determined to have disappeared from the storage.
The generation of the OCIRepository is newer than the Observed Generation.
The newly resolved Artifact digest differs from the current Artifact.

When the OCIRepository is “reconciling”, the Ready Condition status becomes Unknown when the controller detects drift, and the controller adds a Condition with the following attributes to the OCIRepository’s .status.conditions:

type: Reconciling
status: "True"
reason: Progressing | reason: ProgressingWithRetry

If the reconciling state is due to a new revision, an additional Condition is added with the following attributes:

type: ArtifactOutdated
status: "True"
reason: NewRevision

Both Conditions have a “negative polarity”, and are only present on the OCIRepository while their status value is "True".

Ready OCIRepository

The source-controller marks an OCIRepository as ready when it has the following characteristics:

The OCIRepository reports an Artifact.
The reported Artifact exists in the controller’s Artifact storage.
The controller was able to communicate with the remote OCI repository using the current spec.
The digest of the reported Artifact is up-to-date with the latest resolved digest of the remote OCI repository.

When the OCIRepository is “ready”, the controller sets a Condition with the following attributes in the OCIRepository’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the OCIRepository is marked as reconciling, or e.g. a transient error occurs due to a temporary network issue.

When the OCIRepository Artifact is archived in the controller’s Artifact storage, the controller sets a Condition with the following attributes in the OCIRepository’s .status.conditions:

type: ArtifactInStorage
status: "True"
reason: Succeeded

This ArtifactInStorage Condition will retain a status value of "True" until the Artifact in the storage no longer exists.

Failed OCIRepository

The source-controller may get stuck trying to produce an Artifact for a OCIRepository without completing. This can occur due to some of the following factors:

The remote OCI repository URL is temporarily unavailable.
The OCI repository does not exist.
The Secret reference contains a reference to a non-existing Secret.
The credentials in the referenced Secret are invalid.
The OCIRepository spec contains a generic misconfiguration.
A storage related failure when storing the artifact.

When this happens, the controller sets the Ready Condition status to False, and adds a Condition with the following attributes to the OCIRepository’s .status.conditions:

type: FetchFailed | type: IncludeUnavailable | type: StorageOperationFailed
status: "True"
reason: AuthenticationFailed | reason: OCIArtifactPullFailed | reason: OCIArtifactLayerOperationFailed

This condition has a “negative polarity”, and is only present on the OCIRepository while the status value is "True". There may be more arbitrary values for the reason field to provide accurate reason for a condition.

In addition to the above Condition types, when the signature verification fails. A condition with the following attributes is added to the GitRepository’s .status.conditions:

type: SourceVerified
status: "False"
reason: VerificationError

While the OCIRepository has one or more of these Conditions, the controller will continue to attempt to produce an Artifact for the resource with an exponential backoff, until it succeeds and the OCIRepository is marked as ready.

Note that a OCIRepository can be reconciling while failing at the same time, for example due to a newly introduced configuration issue in the OCIRepository spec. When a reconciliation fails, the Reconciling Condition reason would be ProgressingWithRetry. When the reconciliation is performed again after the failure, the reason is updated to Progressing.

Observed Ignore

The source-controller reports an observed ignore in the OCIRepository’s .status.observedIgnore. The observed ignore is the latest .spec.ignore value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the ignore in spec. It indicates the ignore rules used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedIgnore: |
 hpa.yaml
 build
 ...

Observed Layer Selector

The source-controller reports an observed layer selector in the OCIRepository’s .status.observedLayerSelector. The observed layer selector is the latest .spec.layerSelector value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the layer selector in spec. It indicates the layer selection configuration used in building the current artifact in storage. It is also used by the controller to determine if an artifact needs to be rebuilt.

Example:

status:
 ...
 observedLayerSelector:
 mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip
 operation: copy
 ...

Observed Generation

The source-controller reports an observed generation in the OCIRepository’s .status.observedGeneration. The observed generation is the latest .metadata.generation which resulted in either a ready state, or stalled due to error it can not recover from without human intervention.

Last Handled Reconcile At

The source-controller reports the last reconcile.fluxcd.io/requestedAt annotation value it acted on in the .status.lastHandledReconcileAt field.

For practical information about this field, see triggering a reconcile.

Flux: Buckets

Mon, 01 Jan 0001 00:00:00 +0000

The Bucket API defines a Source to produce an Artifact for objects from storage solutions like Amazon S3, Google Cloud Storage buckets, or any other solution with a S3 compatible API such as Minio, Alibaba Cloud OSS and others.

Example

The following is an example of a Bucket. It creates a tarball (.tar.gz) Artifact with the fetched objects from an object storage with an S3 compatible API (e.g. Minio):

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: minio-bucket
 namespace: default
spec:
 interval: 5m0s
 endpoint: minio.example.com
 insecure: true
 secretRef:
 name: minio-bucket-secret
 bucketName: example
---
apiVersion: v1
kind: Secret
metadata:
 name: minio-bucket-secret
 namespace: default
type: Opaque
stringData:
 accesskey: <access key>
 secretkey: <secret key>

In the above example:

A Bucket named minio-bucket is created, indicated by the .metadata.name field.
The source-controller checks the object storage bucket every five minutes, indicated by the .spec.interval field.
It authenticates to the minio.example.com endpoint with the static credentials from the minio-secret Secret data, indicated by the .spec.endpoint and .spec.secretRef.name fields.
A list of object keys and their etags in the .spec.bucketName bucket is compiled, while filtering the keys using default ignore rules.
The digest (algorithm defaults to SHA256) of the list is used as Artifact revision, reported in-cluster in the .status.artifact.revision field.
When the current Bucket revision differs from the latest calculated revision, all objects are fetched and archived.
The new Artifact is reported in the .status.artifact field.

You can run this example by saving the manifest into bucket.yaml, and changing the Bucket and Secret values to target a Minio instance you have control over.

Note: For more advanced examples targeting e.g. Amazon S3 or GCP, see Provider.

Apply the resource on the cluster:
```
kubectl apply -f bucket.yaml
```

Run kubectl get buckets to see the Bucket:

NAME ENDPOINT AGE READY STATUS
minio-bucket minio.example.com 34s True stored artifact for revision 'sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

Run kubectl describe bucket minio-bucket to see the Artifact and Conditions in the Bucket’s Status:

...
Status:
 Artifact:
 Digest: sha256:72aa638abb455ca5f9ef4825b949fd2de4d4be0a74895bf7ed2338622cd12686
 Last Update Time: 2024-02-01T23:43:38Z
 Path: bucket/default/minio-bucket/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.tar.gz
 Revision: sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 Size: 38099
 URL: http://source-controller.source-system.svc.cluster.local./bucket/default/minio-bucket/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.tar.gz
 Conditions:
 Last Transition Time: 2024-02-01T23:43:38Z
 Message: stored artifact for revision 'sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: Ready
 Last Transition Time: 2024-02-01T23:43:38Z
 Message: stored artifact for revision 'sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: ArtifactInStorage
 Observed Generation: 1
 URL: http://source-controller.source-system.svc.cluster.local./bucket/default/minio-bucket/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal NewArtifact 82s source-controller stored artifact with 16 fetched files from 'example' bucket

Writing a Bucket spec

As with all other Kubernetes config, a Bucket needs apiVersion, kind, and metadata fields. The name of a Bucket object must be a valid DNS subdomain name.

A Bucket also needs a .spec section.

Provider

The .spec.provider field allows for specifying a Provider to enable provider specific configurations, for example to communicate with a non-S3 compatible API endpoint, or to change the authentication method.

Supported options are:

Generic
AWS
Azure
GCP

If you do not specify .spec.provider, it defaults to generic.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

Generic

When a Bucket’s spec.provider is set to generic, the controller will attempt to communicate with the specified Endpoint using the Minio Client SDK, which can communicate with any Amazon S3 compatible object storage (including GCS, Wasabi, and many others).

The generic Provider requires a Secret reference to a Secret with .data.accesskey and .data.secretkey values, used to authenticate with static credentials.

The Provider allows for specifying a region the bucket is in using the .spec.region field, if required by the Endpoint.

Generic example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: generic-insecure
 namespace: default
spec:
 provider: generic
 interval: 5m0s
 bucketName: podinfo
 endpoint: minio.minio.svc.cluster.local:9000
 timeout: 60s
 insecure: true
 secretRef:
 name: minio-credentials
---
apiVersion: v1
kind: Secret
metadata:
 name: minio-credentials
 namespace: default
type: Opaque
data:
 accesskey: <BASE64>
 secretkey: <BASE64>

AWS

When a Bucket’s .spec.provider field is set to aws, the source-controller will attempt to communicate with the specified Endpoint using the Minio Client SDK.

Without a Secret reference, authorization using credentials retrieved from the AWS EC2 service is attempted by default. When a reference is specified, it expects a Secret with .data.accesskey and .data.secretkey values, used to authenticate with static credentials.

The Provider allows for specifying the Amazon AWS Region using the .spec.region field.

For detailed setup instructions, see: https://fluxcd.io/flux/integrations/aws/#for-amazon-simple-storage-service

AWS EC2 example

Note: On EKS you have to create an IAM role for the source-controller service account that grants access to the bucket.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: aws
 namespace: default
spec:
 interval: 5m0s
 provider: aws
 bucketName: podinfo
 endpoint: s3.amazonaws.com
 region: us-east-1
 timeout: 30s

AWS IAM role example

Replace <bucket-name> with the specified .spec.bucketName.

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "",
 "Effect": "Allow",
 "Action": "s3:GetObject",
 "Resource": "arn:aws:s3:::<bucket-name>/*"
 },
 {
 "Sid": "",
 "Effect": "Allow",
 "Action": "s3:ListBucket",
 "Resource": "arn:aws:s3:::<bucket-name>"
 }
 ]
}

AWS static auth example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: aws
 namespace: default
spec:
 interval: 5m0s
 provider: aws
 bucketName: podinfo
 endpoint: s3.amazonaws.com
 region: us-east-1
 secretRef:
 name: aws-credentials
---
apiVersion: v1
kind: Secret
metadata:
 name: aws-credentials
 namespace: default
type: Opaque
data:
 accesskey: <BASE64>
 secretkey: <BASE64>

AWS Controller-Level Workload Identity example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: aws-controller-level-workload-identity
 namespace: default
spec:
 interval: 5m0s
 provider: aws
 bucketName: podinfo
 endpoint: s3.amazonaws.com
 region: us-east-1
 timeout: 30s

AWS Object-Level Workload Identity example

Note: To use Object-Level Workload Identity (.spec.serviceAccountName with cloud providers), the controller feature gate ObjectLevelWorkloadIdentity must be enabled.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: aws-object-level-workload-identity
 namespace: default
spec:
 interval: 5m0s
 provider: aws
 bucketName: podinfo
 endpoint: s3.amazonaws.com
 region: us-east-1
 serviceAccountName: aws-workload-identity-sa
 timeout: 30s
---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: aws-workload-identity-sa
 namespace: default
 annotations:
 eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/flux-bucket-role

Azure

When a Bucket’s .spec.provider is set to azure, the source-controller will attempt to communicate with the specified Endpoint using the Azure Blob Storage SDK for Go.

Without a Secret reference, authentication using a chain with:

Environment credentials
Workload Identity
Managed Identity with the AZURE_CLIENT_ID
Managed Identity with a system-assigned identity

is attempted by default. If no chain can be established, the bucket is assumed to be publicly reachable.

When a reference is specified, it expects a Secret with one of the following sets of .data fields:

tenantId, clientId and clientSecret for authenticating a Service Principal with a secret.
tenantId, clientId and clientCertificate (plus optionally clientCertificatePassword and/or clientCertificateSendChain) for authenticating a Service Principal with a certificate.
clientId for authenticating using a Managed Identity.
accountKey for authenticating using a Shared Key.
sasKey for authenticating using a SAS Token

For any Managed Identity and/or Microsoft Entra ID (Formerly Azure Active Directory) authentication method, the base URL can be configured using .data.authorityHost. If not supplied, AzurePublicCloud is assumed.

Azure example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-public
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: podinfo
 endpoint: https://podinfoaccount.blob.core.windows.net
 timeout: 30s

Azure Service Principal Secret example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-service-principal-secret
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: <bucket-name>
 endpoint: https://<account-name>.blob.core.windows.net
 secretRef:
 name: azure-sp-auth
---
apiVersion: v1
kind: Secret
metadata:
 name: azure-sp-auth
 namespace: default
type: Opaque
data:
 tenantId: <BASE64>
 clientId: <BASE64>
 clientSecret: <BASE64>

Azure Service Principal Certificate example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-service-principal-cert
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: <bucket-name>
 endpoint: https://<account-name>.blob.core.windows.net
 secretRef:
 name: azure-sp-auth
---
apiVersion: v1
kind: Secret
metadata:
 name: azure-sp-auth
 namespace: default
type: Opaque
data:
 tenantId: <BASE64>
 clientId: <BASE64>
 clientCertificate: <BASE64>
 # Plus optionally
 clientCertificatePassword: <BASE64>
 clientCertificateSendChain: <BASE64> # either "1" or "true"

Azure Managed Identity with Client ID example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-managed-identity
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: <bucket-name>
 endpoint: https://<account-name>.blob.core.windows.net
 secretRef:
 name: azure-smi-auth
---
apiVersion: v1
kind: Secret
metadata:
 name: azure-smi-auth
 namespace: default
type: Opaque
data:
 clientId: <BASE64>

Azure Blob Shared Key example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-shared-key
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: <bucket-name>
 endpoint: https://<account-name>.blob.core.windows.net
 secretRef:
 name: azure-key
---
apiVersion: v1
kind: Secret
metadata:
 name: azure-key
 namespace: default
type: Opaque
data:
 accountKey: <BASE64>

Workload Identity

If you have Workload Identity set up on your cluster, you need to create an Azure Identity and give it access to Azure Blob Storage.

export IDENTITY_NAME="blob-access"

az role assignment create --role "Storage Blob Data Reader" \
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"

Establish a federated identity between the Identity and the source-controller ServiceAccount.

export SERVICE_ACCOUNT_ISSUER="$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER-NAME> --query "oidcIssuerProfile.issuerUrl" -otsv)"

az identity federated-credential create \
 --name "kubernetes-federated-credential" \
 --identity-name "${IDENTITY_NAME}" \
 --resource-group "${RESOURCE_GROUP}" \
 --issuer "${SERVICE_ACCOUNT_ISSUER}" \
 --subject "system:serviceaccount:flux-system:source-controller"

Add a patch to label and annotate the source-controller Deployment and ServiceAccount correctly so that it can match an identity binding:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |-
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 namespace: flux-system
 annotations:
 azure.workload.identity/client-id: <AZURE_CLIENT_ID>
 labels:
 azure.workload.identity/use: "true"
 - patch: |-
 apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: source-controller
 namespace: flux-system
 labels:
 azure.workload.identity/use: "true"
 spec:
 template:
 metadata:
 labels:
 azure.workload.identity/use: "true"

If you have set up Workload Identity correctly and labeled the source-controller Deployment and ServiceAccount, then you don’t need to reference a Secret. For more information, please see documentation.

apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-bucket
 namespace: flux-system
spec:
 interval: 5m0s
 provider: azure
 bucketName: testwi
 endpoint: https://testfluxwi.blob.core.windows.net

Azure Object-Level Workload Identity example

Note: To use Object-Level Workload Identity (.spec.serviceAccountName with cloud providers), the controller feature gate ObjectLevelWorkloadIdentity must be enabled.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-object-level-workload-identity
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: testwi
 endpoint: https://testfluxwi.blob.core.windows.net
 serviceAccountName: azure-workload-identity-sa
 timeout: 30s
---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: azure-workload-identity-sa
 namespace: default
 annotations:
 azure.workload.identity/client-id: <AZURE_CLIENT_ID>
 azure.workload.identity/tenant-id: <AZURE_TENANT_ID>

Azure Blob SAS Token example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: azure-sas-token
 namespace: default
spec:
 interval: 5m0s
 provider: azure
 bucketName: <bucket-name>
 endpoint: https://<account-name>.blob.core.windows.net
 secretRef:
 name: azure-key
---
apiVersion: v1
kind: Secret
metadata:
 name: azure-key
 namespace: default
type: Opaque
data:
 sasKey: <base64>

The sasKey only contains the SAS token e.g ?sv=2020-08-0&ss=bfqt&srt=co&sp=rwdlacupitfx&se=2022-05-26T21:55:35Z&st=2022-05.... The leading question mark (?) is optional. The query values from the sasKey data field in the Secrets gets merged with the ones in the .spec.endpoint of the Bucket. If the same key is present in the both of them, the value in the sasKey takes precedence.

Note: The SAS token has an expiry date, and it must be updated before it expires to allow Flux to continue to access Azure Storage. It is allowed to use an account-level or container-level SAS token.

The minimum permissions for an account-level SAS token are:

Allowed services: Blob
Allowed resource types: Container, Object
Allowed permissions: Read, List

The minimum permissions for a container-level SAS token are:

Allowed permissions: Read, List

Refer to the Azure documentation for a full overview on permissions.

GCP

For detailed setup instructions, see: https://fluxcd.io/flux/integrations/gcp/#for-google-cloud-storage

GCP Controller-Level Workload Identity example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: gcp-controller-level-workload-identity
 namespace: default
spec:
 interval: 5m0s
 provider: gcp
 bucketName: podinfo
 endpoint: storage.googleapis.com
 region: us-east-1
 timeout: 30s

GCP Object-Level Workload Identity example

Note: To use Object-Level Workload Identity (.spec.serviceAccountName with cloud providers), the controller feature gate ObjectLevelWorkloadIdentity must be enabled.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: gcp-object-level-workload-identity
 namespace: default
spec:
 interval: 5m0s
 provider: gcp
 bucketName: podinfo
 endpoint: storage.googleapis.com
 region: us-east-1
 serviceAccountName: gcp-workload-identity-sa
 timeout: 30s
---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: gcp-workload-identity-sa
 namespace: default
 annotations:
 iam.gke.io/gcp-service-account: <identity-name>

GCP static auth example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: gcp-secret
 namespace: default
spec:
 interval: 5m0s
 provider: gcp
 bucketName: <bucket-name>
 endpoint: storage.googleapis.com
 region: <bucket-region>
 secretRef:
 name: gcp-service-account
---
apiVersion: v1
kind: Secret
metadata:
 name: gcp-service-account
 namespace: default
type: Opaque
data:
 serviceaccount: <BASE64>

Where the (base64 decoded) value of .data.serviceaccount looks like this:

{
 "type": "service_account",
 "project_id": "example",
 "private_key_id": "28qwgh3gdf5hj3gb5fj3gsu5yfgh34f45324568hy2",
 "private_key": "-----BEGIN PRIVATE KEY-----\nHwethgy123hugghhhbdcu6356dgyjhsvgvGFDHYgcdjbvcdhbsx63c\n76tgycfehuhVGTFYfw6t7ydgyVgydheyhuggycuhejwy6t35fthyuhegvcetf\nTFUHGTygghubhxe65ygt6tgyedgy326hucyvsuhbhcvcsjhcsjhcsvgdtHFCGi\nHcye6tyyg3gfyuhchcsbhygcijdbhyyTF66tuhcevuhdcbhuhhvftcuhbh3uh7t6y\nggvftUHbh6t5rfthhuGVRtfjhbfcrd5r67yuhuvgFTYjgvtfyghbfcdrhyjhbfctfdfyhvfg\ntgvggtfyghvft6tugvTF5r66tujhgvfrtyhhgfct6y7ytfr5ctvghbhhvtghhjvcttfycf\nffxfghjbvgcgyt67ujbgvctfyhVC7uhvgcyjvhhjvyujc\ncgghgvgcfhgg765454tcfthhgftyhhvvyvvffgfryyu77reredswfthhgfcftycfdrttfhf/\n-----END PRIVATE KEY-----\n",
 "client_email": "test@example.iam.gserviceaccount.com",
 "client_id": "32657634678762536746",
 "auth_uri": "https://accounts.google.com/o/oauth2/auth",
 "token_uri": "https://oauth2.googleapis.com/token",
 "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
 "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40podinfo.iam.gserviceaccount.com"
}

Interval

.spec.interval is a required field that specifies the interval which the object storage bucket must be consulted at.

After successfully reconciling a Bucket object, the source-controller requeues the object for inspection after the specified interval. The value must be in a Go recognized duration string format, e.g. 10m0s to look at the object storage bucket every 10 minutes.

If the .metadata.generation of a resource changes (due to e.g. the apply of a change to the spec), this is handled instantly outside the interval window.

Note: The controller can be configured to apply a jitter to the interval in order to distribute the load more evenly when multiple Bucket objects are set up with the same interval. For more information, please refer to the source-controller configuration options.

Endpoint

.spec.endpoint is a required field that specifies the HTTP/S object storage endpoint to connect to and fetch objects from. Connecting to an (insecure) HTTP endpoint requires enabling .spec.insecure.

Some endpoints require the specification of a .spec.region, see Provider for more (provider specific) examples.

STS

.spec.sts is an optional field for specifying the Security Token Service configuration. A Security Token Service (STS) is a web service that issues temporary security credentials. By adding this field, one may specify the STS endpoint from where temporary credentials will be fetched.

This field is only supported for the aws and generic bucket providers.

If using .spec.sts, the following fields are required:

.spec.sts.provider, the Security Token Service provider. The only supported option for the generic bucket provider is ldap. The only supported option for the aws bucket provider is aws.
.spec.sts.endpoint, the HTTP/S endpoint of the Security Token Service. In the case of aws this can be https://sts.amazonaws.com, or a Regional STS Endpoint, or an Interface Endpoint created inside a VPC. In the case of ldap this must be the LDAP server endpoint.

When using the ldap provider, the following fields may also be specified:

.spec.sts.secretRef.name, the name of the Secret containing the LDAP credentials. The Secret must contain the following keys:
- username, the username to authenticate with.
- password, the password to authenticate with.
.spec.sts.certSecretRef.name, the name of the Secret containing the TLS configuration for communicating with the STS endpoint. The contents of this Secret must follow the same structure of .spec.certSecretRef.name.

If .spec.proxySecretRef.name is specified, the proxy configuration will be used for commucating with the STS endpoint.

Example for the ldap provider:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: example
 namespace: example
spec:
 interval: 5m
 bucketName: example
 provider: generic
 endpoint: minio.example.com
 sts:
 provider: ldap
 endpoint: https://ldap.example.com
 secretRef:
 name: ldap-credentials
 certSecretRef:
 name: ldap-tls
---
apiVersion: v1
kind: Secret
metadata:
 name: ldap-credentials
 namespace: example
type: Opaque
stringData:
 username: <username>
 password: <password>
---
apiVersion: v1
kind: Secret
metadata:
 name: ldap-tls
 namespace: example
type: kubernetes.io/tls # or Opaque
stringData:
 tls.crt: <PEM-encoded cert>
 tls.key: <PEM-encoded key>
 ca.crt: <PEM-encoded cert>

Bucket name

.spec.bucketName is a required field that specifies which object storage bucket on the Endpoint objects should be fetched from.

See Provider for more (provider specific) examples.

Region

.spec.region is an optional field to specify the region a .spec.bucketName is located in.

See Provider for more (provider specific) examples.

Mutual TLS Authentication

.spec.certSecretRef.name is an optional field to specify a secret containing TLS certificate data for mutual TLS authentication.

To authenticate towards a bucket using mutual TLS, the referenced Secret’s .data should contain the following keys:

tls.crt and tls.key, to specify the client certificate and private key used for TLS client authentication. These must be used in conjunction, i.e. specifying one without the other will lead to an error.
ca.crt, to specify the CA certificate used to verify the server, which is required if the server is using a self-signed certificate.

The Secret should be of type Opaque or kubernetes.io/tls. All the files in the Secret are expected to be [PEM-encoded][pem-encoding]. Assuming you have three files; client.key, client.crt and ca.crt for the client private key, client certificate and the CA certificate respectively, you can generate the required Secret using the flux create secret tls command:

flux create secret tls minio-tls --tls-key-file=client.key --tls-crt-file=client.crt --ca-crt-file=ca.crt

If TLS client authentication is not required, you can generate the secret with:

flux create secret tls minio-tls --ca-crt-file=ca.crt

This API is only supported for the generic provider.

Example usage:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: example
 namespace: example
spec:
 interval: 5m
 bucketName: example
 provider: generic
 endpoint: minio.example.com
 certSecretRef:
 name: minio-tls
---
apiVersion: v1
kind: Secret
metadata:
 name: minio-tls
 namespace: example
type: kubernetes.io/tls # or Opaque
stringData:
 tls.crt: <PEM-encoded cert>
 tls.key: <PEM-encoded key>
 ca.crt: <PEM-encoded cert>

Proxy secret reference

address, to specify the address of the proxy server. This is a required key.
username, to specify the username to use if the proxy server is protected by basic authentication. This is an optional key.
password, to specify the password to use if the proxy server is protected by basic authentication. This is an optional key.

Example:

---
apiVersion: v1
kind: Secret
metadata:
 name: http-proxy
type: Opaque
stringData:
 address: http://proxy.com
 username: mandalorian
 password: grogu

Proxying can also be configured in the source-controller Deployment directly by using the standard environment variables such as HTTPS_PROXY, ALL_PROXY, etc.

.spec.proxySecretRef.name takes precedence over all environment variables.

Insecure

.spec.insecure is an optional field to allow connecting to an insecure (HTTP) endpoint, if set to true. The default value is false, denying insecure (HTTP) connections.

Timeout

.spec.timeout is an optional field to specify a timeout for object storage fetch operations. The value must be in a Go recognized duration string format, e.g. 1m30s for a timeout of one minute and thirty seconds. The default value is 60s.

Secret reference

.spec.secretRef.name is an optional field to specify a name reference to a Secret in the same namespace as the Bucket, containing authentication credentials for the object storage. For some .spec.provider implementations the presence of the field is required, see Provider for more details and examples.

Service Account reference

.spec.serviceAccountName is an optional field to specify a Service Account in the same namespace as Bucket with purpose depending on the value of the .spec.provider field:

When .spec.provider is set to generic, the controller will fetch the image pull secrets attached to the Service Account and use them for authentication.
When .spec.provider is set to aws, azure, or gcp, the Service Account will be used for Workload Identity authentication. In this case, the controller feature gate ObjectLevelWorkloadIdentity must be enabled, otherwise the controller will error out.

Note: that for a publicly accessible object storage, you don’t need to provide a secretRef nor serviceAccountName.

Important: .spec.secretRef and .spec.serviceAccountName are mutually exclusive and cannot be set at the same time. This constraint is enforced at the CRD level.

For a complete guide on how to set up authentication for cloud providers, see the integration docs.

Prefix

.spec.prefix is an optional field to enable server-side filtering of files in the Bucket.

Note: The server-side filtering works only with the generic, aws and gcp provider and is preferred over .spec.ignore as a more efficient way of excluding files.

Ignore

.spec.ignore is an optional field to specify rules in the .gitignore pattern format. Storage objects which keys match the defined rules are excluded while fetching.

When specified, .spec.ignore overrides the default exclusion list, and may overrule the .sourceignore file exclusions. See excluding files for more information.

Suspend

.spec.suspend is an optional field to suspend the reconciliation of a Bucket. When set to true, the controller will stop reconciling the Bucket, and changes to the resource or in the object storage bucket will not result in a new Artifact. When the field is set to false or removed, it will resume.

For practical information, see suspending and resuming.

Working with Buckets

Excluding files

By default, storage bucket objects which match the default exclusion rules are excluded while fetching. It is possible to overwrite and/or overrule the default exclusions using a file in the bucket and/or an in-spec set of rules.

`.sourceignore` file

Excluding files is possible by adding a .sourceignore file in the root of the object storage bucket. The .sourceignore file follows the .gitignore pattern format, and pattern entries may overrule default exclusions.

Ignore spec

Another option is to define the exclusions within the Bucket spec, using the .spec.ignore field. Specified rules override the default exclusion list, and may overrule .sourceignore file exclusions.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: <bucket-name>
spec:
 ignore: |
 # exclude all
 /*
 # include deploy dir
 !/deploy
 # exclude file extensions from deploy dir
 /deploy/**/*.md
 /deploy/**/*.txt

Triggering a reconcile

To manually tell the source-controller to reconcile a Bucket outside the specified interval window, a Bucket can be annotated with reconcile.fluxcd.io/requestedAt: <arbitrary value>. Annotating the resource queues the Bucket for reconciliation if the <arbitrary-value> differs from the last value the controller acted on, as reported in .status.lastHandledReconcileAt.

Using kubectl:

kubectl annotate --field-manager=flux-client-side-apply --overwrite bucket/<bucket-name> reconcile.fluxcd.io/requestedAt="$(date +%s)"

Using flux:

flux reconcile source bucket <bucket-name>

Waiting for `Ready`

When a change is applied, it is possible to wait for the Bucket to reach a ready state using kubectl:

kubectl wait bucket/<bucket-name> --for=condition=ready --timeout=1m

Suspending and resuming

When you find yourself in a situation where you temporarily want to pause the reconciliation of a Bucket, you can suspend it using the .spec.suspend field.

Suspend a Bucket

In your YAML declaration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: <bucket-name>
spec:
 suspend: true

Using kubectl:

kubectl patch bucket <bucket-name> --field-manager=flux-client-side-apply -p '{\"spec\": {\"suspend\" : true }}'

Using flux:

flux suspend source bucket <bucket-name>

Note: When a Bucket has an Artifact and is suspended, and this Artifact later disappears from the storage due to e.g. the source-controller Pod being evicted from a Node, this will not be reflected in the Bucket’s Status until it is resumed.

Resume a Bucket

In your YAML declaration, comment out (or remove) the field:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: <bucket-name>
spec:
 # suspend: true

Using kubectl:

kubectl patch bucket <bucket-name> --field-manager=flux-client-side-apply -p '{\"spec\" : {\"suspend\" : false }}'

Using flux:

flux resume source bucket <bucket-name>

Debugging a Bucket

There are several ways to gather information about a Bucket for debugging purposes.

Describe the Bucket

Describing a Bucket using kubectl describe bucket <bucket-name> displays the latest recorded information for the resource in the Status and Events sections:

...
Status:
...
 Conditions:
 Last Transition Time: 2024-02-02T13:26:55Z
 Message: processing object: new generation 1 -> 2
 Observed Generation: 2
 Reason: ProgressingWithRetry
 Status: True
 Type: Reconciling
 Last Transition Time: 2024-02-02T13:26:55Z
 Message: bucket 'my-new-bucket' does not exist
 Observed Generation: 2
 Reason: BucketOperationFailed
 Status: False
 Type: Ready
 Last Transition Time: 2024-02-02T13:26:55Z
 Message: bucket 'my-new-bucket' does not exist
 Observed Generation: 2
 Reason: BucketOperationFailed
 Status: True
 Type: FetchFailed
 Observed Generation: 1
 URL: http://source-controller.source-system.svc.cluster.local./bucket/default/minio-bucket/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Warning BucketOperationFailed 37s (x11 over 42s) source-controller bucket 'my-new-bucket' does not exist

Trace emitted Events

To view events for specific Bucket(s), kubectl events can be used in combination with --for to list the Events for specific objects. For example, running

kubectl events --for Bucket/<bucket-name>

lists

LAST SEEN TYPE REASON OBJECT MESSAGE
2m30s Normal NewArtifact bucket/<bucket-name> fetched 16 files with revision from 'my-new-bucket'
36s Normal ArtifactUpToDate bucket/<bucket-name> artifact up-to-date with remote revision: 'sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
18s Warning BucketOperationFailed bucket/<bucket-name> bucket 'my-new-bucket' does not exist

Besides being reported in Events, the reconciliation errors are also logged by the controller. The Flux CLI offer commands for filtering the logs for a specific Bucket, e.g. flux logs --level=error --kind=Bucket --name=<bucket-name>.

Bucket Status

Artifact

The Bucket reports the latest synchronized state from the object storage bucket as an Artifact object in the .status.artifact of the resource.

The Artifact file is a gzip compressed TAR archive (<calculated revision>.tar.gz), and can be retrieved in-cluster from the .status.artifact.url HTTP address.

Artifact example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: Bucket
metadata:
 name: <bucket-name>
status:
 artifact:
 digest: sha256:cbec34947cc2f36dee8adcdd12ee62ca6a8a36699fc6e56f6220385ad5bd421a
 lastUpdateTime: "2024-01-28T10:30:30Z"
 path: bucket/<namespace>/<bucket-name>/c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2.tar.gz
 revision: sha256:c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2
 size: 38099
 url: http://source-controller.<namespace>.svc.cluster.local./bucket/<namespace>/<bucket-name>/c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2.tar.gz

Default exclusions

The following files and extensions are excluded from the Artifact by default:

Git files (.git/, .gitignore, .gitmodules, .gitattributes)
File extensions (.jpg, .jpeg, .gif, .png, .wmv, .flv, .tar.gz, .zip)
CI configs (.github/, .circleci/, .travis.yml, .gitlab-ci.yml, appveyor.yml, .drone.yml, cloudbuild.yaml, codeship-services.yml, codeship-steps.yml)
CLI configs (.goreleaser.yml, .sops.yaml)
Flux v1 config (.flux.yaml)

To define your own exclusion rules, see excluding files.

Conditions

A Bucket enters various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching storage objects, it can be ready, or it can fail during reconciliation.

The Bucket API is compatible with the kstatus specification, and reports Reconciling and Stalled conditions where applicable to provide better (timeout) support to solutions polling the Bucket to become Ready.

Reconciling Bucket

The source-controller marks a Bucket as reconciling when one of the following is true:

There is no current Artifact for the Bucket, or the reported Artifact is determined to have disappeared from the storage.
The generation of the Bucket is newer than the Observed Generation.
The newly calculated Artifact revision differs from the current Artifact.

When the Bucket is “reconciling”, the Ready Condition status becomes Unknown when the controller detects drift, and the controller adds a Condition with the following attributes to the Bucket’s .status.conditions:

type: Reconciling
status: "True"
reason: Progressing | reason: ProgressingWithRetry

If the reconciling state is due to a new revision, an additional Condition is added with the following attributes:

type: ArtifactOutdated
status: "True"
reason: NewRevision

Both Conditions have a “negative polarity”, and are only present on the Bucket while their status value is "True".

Ready Bucket

The source-controller marks a Bucket as ready when it has the following characteristics:

The Bucket reports an Artifact.
The reported Artifact exists in the controller’s Artifact storage.
The Bucket was able to communicate with the Bucket’s object storage endpoint using the current spec.
The revision of the reported Artifact is up-to-date with the latest calculated revision of the object storage bucket.

When the Bucket is “ready”, the controller sets a Condition with the following attributes in the Bucket’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the Bucket is marked as reconciling, or e.g. a transient error occurs due to a temporary network issue.

When the Bucket Artifact is archived in the controller’s Artifact storage, the controller sets a Condition with the following attributes in the Bucket’s .status.conditions:

type: ArtifactInStorage
status: "True"
reason: Succeeded

This ArtifactInStorage Condition will retain a status value of "True" until the Artifact in the storage no longer exists.

Failed Bucket

The source-controller may get stuck trying to produce an Artifact for a Bucket without completing. This can occur due to some of the following factors:

The object storage Endpoint is temporarily unavailable.
The specified object storage bucket does not exist.
The Secret reference contains a reference to a non-existing Secret.
The credentials in the referenced Secret are invalid.
The Bucket spec contains a generic misconfiguration.
A storage related failure when storing the artifact.

When this happens, the controller sets the Ready Condition status to False, and adds a Condition with the following attributes to the Bucket’s .status.conditions:

type: FetchFailed | type: StorageOperationFailed
status: "True"
reason: AuthenticationFailed | reason: BucketOperationFailed

This condition has a “negative polarity”, and is only present on the Bucket while the status value is "True". There may be more arbitrary values for the reason field to provide accurate reason for a condition.

While the Bucket has this Condition, the controller will continue to attempt to produce an Artifact for the resource with an exponential backoff, until it succeeds and the Bucket is marked as ready.

Note that a Bucket can be reconciling while failing at the same time, for example due to a newly introduced configuration issue in the Bucket spec. When a reconciliation fails, the Reconciling Condition reason would be ProgressingWithRetry. When the reconciliation is performed again after the failure, the reason is updated to Progressing.

Observed Ignore

The source-controller reports an observed ignore in the Bucket’s .status.observedIgnore. The observed ignore is the latest .spec.ignore value which resulted in a ready state, or stalled due to error it can not recover from without human intervention. The value is the same as the ignore in spec. It indicates the ignore rules used in building the current artifact in storage.

Example:

status:
 ...
 observedIgnore: |
 hpa.yaml
 build
 ...

Observed Generation

The source-controller reports an observed generation in the Bucket’s .status.observedGeneration. The observed generation is the latest .metadata.generation which resulted in either a ready state, or stalled due to error it can not recover from without human intervention.

Last Handled Reconcile At

The source-controller reports the last reconcile.fluxcd.io/requestedAt annotation value it acted on in the .status.lastHandledReconcileAt field.

For practical information about this field, see triggering a reconcile.

Flux: Helm Repositories

Mon, 01 Jan 0001 00:00:00 +0000

There are 2 Helm repository types defined by the HelmRepository API:

Helm HTTP/S repository, which defines a Source to produce an Artifact for a Helm repository index YAML (index.yaml).
OCI Helm repository, which defines a source that does not produce an Artifact. It’s a data container to store the information about the OCI repository that can be used by HelmChart to access OCI Helm charts.

Examples

Helm HTTP/S repository

The following is an example of a HelmRepository. It creates a YAML (.yaml) Artifact from the fetched Helm repository index (in this example the podinfo repository):

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 url: https://stefanprodan.github.io/podinfo

In the above example:

A HelmRepository named podinfo is created, indicated by the .metadata.name field.
The source-controller fetches the Helm repository index YAML every five minutes from https://stefanprodan.github.io/podinfo, indicated by the .spec.interval and .spec.url fields.
The digest (algorithm defaults to SHA256) of the Helm repository index after stable sorting the entries is used as Artifact revision, reported in-cluster in the .status.artifact.revision field.
When the current HelmRepository revision differs from the latest fetched revision, it is stored as a new Artifact.
The new Artifact is reported in the .status.artifact field.

You can run this example by saving the manifest into helmrepository.yaml.

Apply the resource on the cluster:
```
kubectl apply -f helmrepository.yaml
```

Run kubectl get helmrepository to see the HelmRepository:

NAME URL AGE READY STATUS
podinfo https://stefanprodan.github.io/podinfo 4s True stored artifact for revision 'sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111'

Run kubectl describe helmrepository podinfo to see the Artifact and Conditions in the HelmRepository’s Status:

...
Status:
 Artifact:
 Digest: sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111
 Last Update Time: 2022-02-04T09:55:58Z
 Path: helmrepository/default/podinfo/index-83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111.yaml
 Revision: sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111
 Size: 40898
 URL: http://source-controller.flux-system.svc.cluster.local./helmrepository/default/podinfo/index-83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111.yaml
 Conditions:
 Last Transition Time: 2022-02-04T09:55:58Z
 Message: stored artifact for revision 'sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: Ready
 Last Transition Time: 2022-02-04T09:55:58Z
 Message: stored artifact for revision 'sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111'
 Observed Generation: 1
 Reason: Succeeded
 Status: True
 Type: ArtifactInStorage
 Observed Generation: 1
 URL: http://source-controller.flux-system.svc.cluster.local./helmrepository/default/podinfo/index.yaml
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal NewArtifact 1m source-controller fetched index of size 30.88kB from 'https://stefanprodan.github.io/podinfo'

Helm OCI repository

The following is an example of an OCI HelmRepository.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: podinfo
 namespace: default
spec:
 type: "oci"
 interval: 5m0s
 url: oci://ghcr.io/stefanprodan/charts

In the above example:

A HelmRepository named podinfo is created, indicated by the .metadata.name field.
A HelmChart that refers to this HelmRepository uses the URL in the .spec.url field to access the OCI Helm chart.

NOTE: The .spec.interval field is only used by the default Helm repository and is ignored for any value in oci Helm repository.

You can run this example by saving the manifest into helmrepository.yaml.

Apply the resource on the cluster:
```
kubectl apply -f helmrepository.yaml
```

Run kubectl get helmrepository to see the HelmRepository:

NAME URL AGE READY STATUS
podinfo oci://ghcr.io/stefanprodan/charts 3m22s

Because the OCI Helm repository is a data container, there’s nothing to report for READY and STATUS columns above. The existence of the object can be considered to be ready for use.

Writing a HelmRepository spec

As with all other Kubernetes config, a HelmRepository needs apiVersion, kind, and metadata fields. The name of a HelmRepository object must be a valid DNS subdomain name.

A HelmRepository also needs a .spec section.

Type

.spec.type is an optional field that specifies the Helm repository type.

Possible values are default for a Helm HTTP/S repository, or oci for an OCI Helm repository.

Note:: For improved support for OCI Helm charts, please use the OCIRepository API.

Provider

.spec.provider is an optional field that allows specifying an OIDC provider used for authentication purposes.

Supported options are:

generic
aws
azure
gcp

The generic provider can be used for public repositories or when static credentials are used for authentication. If you do not specify .spec.provider, it defaults to generic.

Note: The provider field is supported only for Helm OCI repositories. The spec.type field must be set to oci.

AWS

The aws provider can be used to authenticate automatically using the EKS worker node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access to ECR.

EKS Worker Node IAM Role

When the worker node IAM role has access to ECR, source-controller running on it will also have access to ECR.

IAM Role for Service Accounts (IRSA)

When using IRSA to enable access to ECR, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 annotations:
 eks.amazonaws.com/role-arn: <role arn>
 target:
 kind: ServiceAccount
 name: source-controller

Note that you can attach the AWS managed policy arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly to the IAM role when using IRSA.

Azure

The azure provider can be used to authenticate automatically using Workload Identity and Kubelet Managed Identity to gain access to ACR.

Kubelet Managed Identity

When the kubelet managed identity has access to ACR, source-controller running on it will also have access to ACR.

Note: If you have more than one identity configured on the cluster, you have to specify which one to use by setting the AZURE_CLIENT_ID environment variable in the source-controller deployment.

If you are running into further issues, please look at the troubleshooting guide.

Azure Workload Identity

When using Workload Identity to enable access to ACR, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |-
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 namespace: flux-system
 annotations:
 azure.workload.identity/client-id: <AZURE_CLIENT_ID>
 labels:
 azure.workload.identity/use: "true"
 - patch: |-
 apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: source-controller
 namespace: flux-system
 labels:
 azure.workload.identity/use: "true"
 spec:
 template:
 metadata:
 labels:
 azure.workload.identity/use: "true"

GCP

The gcp provider can be used to authenticate automatically using OAuth scopes or Workload Identity, and by extension gain access to GCR or Artifact Registry.

Access Scopes

When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry, source-controller running on it will also have access to them.

GKE Workload Identity

When using Workload Identity to enable access to GCR or Artifact Registry, add the following patch to your bootstrap repository, in the flux-system/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
 - gotk-components.yaml
 - gotk-sync.yaml
patches:
 - patch: |
 apiVersion: v1
 kind: ServiceAccount
 metadata:
 name: source-controller
 annotations:
 iam.gke.io/gcp-service-account: <identity-name>
 target:
 kind: ServiceAccount
 name: source-controller

Insecure

.spec.insecure is an optional field to allow connecting to an insecure (HTTP) container registry server, if set to true. The default value is false, denying insecure non-TLS connections when fetching Helm chart OCI artifacts.

Note: The insecure field is supported only for Helm OCI repositories. The spec.type field must be set to oci.

Interval

Note: This field is ineffectual for OCI Helm Repositories.

.spec.interval is a an optional field that specifies the interval which the Helm repository index must be consulted at. When not set, the default value is 1m.

After successfully reconciling a HelmRepository object, the source-controller requeues the object for inspection after the specified interval. The value must be in a Go recognized duration string format, e.g. 10m0s to fetch the HelmRepository index YAML every 10 minutes.

If the .metadata.generation of a resource changes (due to e.g. applying a change to the spec), this is handled instantly outside the interval window.

Note: The controller can be configured to apply a jitter to the interval in order to distribute the load more evenly when multiple HelmRepository objects are set up with the same interval. For more information, please refer to the source-controller configuration options.

URL

.spec.url is a required field that depending on the type of the HelmRepository object specifies the HTTP/S or OCI address of a Helm repository.

For OCI, the URL is expected to point to a registry repository, e.g. oci://ghcr.io/fluxcd/source-controller.

For Helm repositories which require authentication, see Secret reference.

Timeout

Note: This field is not applicable to OCI Helm Repositories.

.spec.timeout is an optional field to specify a timeout for the fetch operation. The value must be in a Go recognized duration string format, e.g. 1m30s for a timeout of one minute and thirty seconds. When not set, the default value is 1m.

Secret reference

.spec.secretRef.name is an optional field to specify a name reference to a Secret in the same namespace as the HelmRepository, containing authentication credentials for the repository.

Basic access authentication

To authenticate towards a Helm repository using basic access authentication (in other words: using a username and password), the referenced Secret is expected to contain .data.username and .data.password values.

For example:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: example
 namespace: default
spec:
 interval: 5m0s
 url: https://example.com
 secretRef:
 name: example-user
---
apiVersion: v1
kind: Secret
metadata:
 name: example-user
 namespace: default
stringData:
 username: "user-123456"
 password: "pass-123456"

OCI Helm repository example:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 url: oci://ghcr.io/my-user/my-private-repo
 type: "oci"
 secretRef:
 name: oci-creds
---
apiVersion: v1
kind: Secret
metadata:
 name: oci-creds
 namespace: default
stringData:
 username: "user-123456"
 password: "pass-123456"

For OCI Helm repositories, Kubernetes secrets of type kubernetes.io/dockerconfigjson are also supported. It is possible to create one such secret with kubectl create secret docker-registry or using the Flux CLI:

flux create secret oci ghcr-auth \
 --url=ghcr.io \
 --username=flux \
 --password=${GITHUB_PAT}

Warning: Support for specifying TLS authentication data using this API has been deprecated. Please use .spec.certSecretRef instead. If the controller uses the secret specified by this field to configure TLS, then a deprecation warning will be logged.

Mutual TLS Authentication

.spec.certSecretRef.name is an optional field to specify a secret containing TLS certificate data for mutual TLS authentication.

To authenticate towards a Helm repository using mutual TLS, the referenced Secret’s .data should contain the following keys:

tls.crt and tls.key, to specify the client certificate and private key used for TLS client authentication. These must be used in conjunction, i.e. specifying one without the other will lead to an error.
ca.crt, to specify the CA certificate used to verify the server, which is required if the server is using a self-signed certificate.

flux create secret tls --tls-key-file=client.key --tls-crt-file=client.crt --ca-crt-file=ca.crt

Example usage:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: example
 namespace: default
spec:
 interval: 5m0s
 url: https://example.com
 certSecretRef:
 name: example-tls
---
apiVersion: v1
kind: Secret
metadata:
 name: example-tls
 namespace: default
type: kubernetes.io/tls # or Opaque
data:
 tls.crt: <BASE64>
 tls.key: <BASE64>
 # NOTE: Can be supplied without the above values
 ca.crt: <BASE64>

Pass credentials

.spec.passCredentials is an optional field to allow the credentials from the Secret reference to be passed on to a host that does not match the host as defined in URL. This may for example be required if the host advertised chart URLs in the index differ from the specified URL.

Enabling this should be done with caution, as it can potentially result in credentials getting stolen in a man-in-the-middle attack. This feature only applies to HTTP/S Helm repositories.

Suspend

Note: This field is not applicable to OCI Helm Repositories.

.spec.suspend is an optional field to suspend the reconciliation of a HelmRepository. When set to true, the controller will stop reconciling the HelmRepository, and changes to the resource or the Helm repository index will not result in a new Artifact. When the field is set to false or removed, it will resume.

For practical information, see suspending and resuming.

Working with HelmRepositories

Note: This section does not apply to OCI Helm Repositories, being a data container, once created, they are ready to used by HelmCharts.

Triggering a reconcile

To manually tell the source-controller to reconcile a HelmRepository outside the specified interval window, a HelmRepository can be annotated with reconcile.fluxcd.io/requestedAt: <arbitrary value>. Annotating the resource queues the object for reconciliation if the <arbitrary-value> differs from the last value the controller acted on, as reported in .status.lastHandledReconcileAt.

Using kubectl:

kubectl annotate --field-manager=flux-client-side-apply --overwrite helmrepository/<repository-name> reconcile.fluxcd.io/requestedAt="$(date +%s)"

Using flux:

flux reconcile source helm <repository-name>

Waiting for `Ready`

When a change is applied, it is possible to wait for the HelmRepository to reach a ready state using kubectl:

kubectl wait helmrepository/<repository-name> --for=condition=ready --timeout=1m

Suspending and resuming

When you find yourself in a situation where you temporarily want to pause the reconciliation of a HelmRepository, you can suspend it using the .spec.suspend field.

Suspend a HelmRepository

In your YAML declaration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: <repository-name>
spec:
 suspend: true

Using kubectl:

kubectl patch helmrepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\": {\"suspend\" : true }}'

Using flux:

flux suspend source helm <repository-name>

Note: When a HelmRepository has an Artifact and is suspended, and this Artifact later disappears from the storage due to e.g. the source-controller Pod being evicted from a Node, this will not be reflected in the HelmRepository’s Status until it is resumed.

Resume a HelmRepository

In your YAML declaration, comment out (or remove) the field:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: <repository-name>
spec:
 # suspend: true

Using kubectl:

kubectl patch helmrepository <repository-name> --field-manager=flux-client-side-apply -p '{\"spec\" : {\"suspend\" : false }}'

Using flux:

flux resume source helm <repository-name>

Debugging a HelmRepository

Note: This section does not apply to OCI Helm Repositories, being a data container, they are static objects that don’t require debugging if valid.

There are several ways to gather information about a HelmRepository for debugging purposes.

Describe the HelmRepository

Describing a HelmRepository using kubectl describe helmrepository <repository-name> displays the latest recorded information for the resource in the Status and Events sections:

...
Status:
...
 Conditions:
 Last Transition Time: 2022-02-04T13:41:56Z
 Message: failed to construct Helm client: scheme "invalid" not supported
 Observed Generation: 2
 Reason: Failed
 Status: True
 Type: Stalled
 Last Transition Time: 2022-02-04T13:41:56Z
 Message: failed to construct Helm client: scheme "invalid" not supported
 Observed Generation: 2
 Reason: Failed
 Status: False
 Type: Ready
 Last Transition Time: 2022-02-04T13:41:56Z
 Message: failed to construct Helm client: scheme "invalid" not supported
 Observed Generation: 2
 Reason: Failed
 Status: True
 Type: FetchFailed
 Observed Generation: 2
 URL: http://source-controller.source-system.svc.cluster.local./helmrepository/default/podinfo/index.yaml
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Warning Failed 6s source-controller failed to construct Helm client: scheme "invalid" not supported

Trace emitted Events

To view events for specific HelmRepository(s), kubectl events can be used in combination with --for to list the Events for specific objects. For example, running

kubectl events --for HelmRepository/<repository-name>

lists

LAST SEEN TYPE REASON OBJECT MESSAGE
107s Warning Failed helmrepository/<repository-name> failed to construct Helm client: scheme "invalid" not supported
7s Normal NewArtifact helmrepository/<repository-name> fetched index of size 30.88kB from 'https://stefanprodan.github.io/podinfo'
3s Normal ArtifactUpToDate helmrepository/<repository-name> artifact up-to-date with remote revision: 'sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111'

Besides being reported in Events, the reconciliation errors are also logged by the controller. The Flux CLI offer commands for filtering the logs for a specific HelmRepository, e.g. flux logs --level=error --kind=HelmRepository --name=<chart-name>.

HelmRepository Status

Note: This section does not apply to OCI Helm Repositories, they do not contain any information in the status.

Artifact

The HelmRepository reports the last fetched repository index as an Artifact object in the .status.artifact of the resource.

The Artifact file is an exact copy of the Helm repository index YAML (index-<revision>.yaml) as fetched, and can be retrieved in-cluster from the .status.artifact.url HTTP address.

Artifact example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: <repository-name>
status:
 artifact:
 digest: sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111
 lastUpdateTime: "2022-02-04T09:55:58Z"
 path: helmrepository/<namespace>/<repository-name>/index-83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111.yaml
 revision: sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111
 size: 40898
 url: http://source-controller.flux-system.svc.cluster.local./helmrepository/<namespace>/<repository-name>/index-83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111.yaml

Conditions

A HelmRepository enters various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching the repository index, it can be ready, it can fail during reconciliation, or it can stall.

The HelmRepository API is compatible with the kstatus specification, and reports Reconciling and Stalled conditions where applicable to provide better (timeout) support to solutions polling the HelmRepository to become Ready.

Reconciling HelmRepository

The source-controller marks a HelmRepository as reconciling when one of the following is true:

There is no current Artifact for the HelmRepository, or the reported Artifact is determined to have disappeared from the storage.
The generation of the HelmRepository is newer than the Observed Generation.
The newly fetched Artifact revision differs from the current Artifact.

When the HelmRepository is “reconciling”, the Ready Condition status becomes Unknown when the controller detects drift, and the controller adds a Condition with the following attributes to the HelmRepository’s .status.conditions:

type: Reconciling
status: "True"
reason: Progressing | reason: ProgressingWithRetry

If the reconciling state is due to a new revision, it adds an additional Condition with the following attributes:

type: ArtifactOutdated
status: "True"
reason: NewRevision

Both Conditions have a “negative polarity”, and are only present on the HelmRepository while their status value is "True".

Ready HelmRepository

The source-controller marks a HelmRepository as ready when it has the following characteristics:

The HelmRepository reports an Artifact.
The reported Artifact exists in the controller’s Artifact storage.
The controller was able to fetch the Helm repository index using the current spec.
The revision of the reported Artifact is up-to-date with the latest revision of the Helm repository.

When the HelmRepository is “ready”, the controller sets a Condition with the following attributes in the HelmRepository’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the HelmRepository is marked as reconciling, or e.g. a transient error occurs due to a temporary network issue.

When the HelmRepository Artifact is archived in the controller’s Artifact storage, the controller sets a Condition with the following attributes in the HelmRepository’s .status.conditions:

type: ArtifactInStorage
status: "True"
reason: Succeeded

This ArtifactInStorage Condition will retain a status value of "True" until the Artifact in the storage no longer exists.

Failed HelmRepository

The source-controller may get stuck trying to produce an Artifact for a HelmRepository without completing. This can occur due to some of the following factors:

The Helm repository URL is temporarily unavailable.
The Secret reference contains a reference to a non-existing Secret.
The credentials in the referenced Secret are invalid.
The HelmRepository spec contains a generic misconfiguration.
A storage related failure when storing the artifact.

When this happens, the controller sets the Ready Condition status to False, and adds a Condition with the following attributes to the HelmRepository’s .status.conditions:

type: FetchFailed | type: StorageOperationFailed
status: "True"
reason: AuthenticationFailed | reason: IndexationFailed | reason: Failed

This condition has a “negative polarity”, and is only present on the HelmRepository while the status value is "True". There may be more arbitrary values for the reason field to provide accurate reason for a condition.

While the HelmRepository has this Condition, the controller will continue to attempt to produce an Artifact for the resource with an exponential backoff, until it succeeds and the HelmRepository is marked as ready.

Note that a HelmRepository can be reconciling while failing at the same time, for example due to a newly introduced configuration issue in the HelmRepository spec. When a reconciliation fails, the Reconciling Condition reason would be ProgressingWithRetry. When the reconciliation is performed again after the failure, the reason is updated to Progressing.

Stalled HelmRepository

The source-controller can mark a HelmRepository as stalled when it determines that without changes to the spec, the reconciliation can not succeed. For example because a Helm repository URL with an unsupported protocol is specified.

When this happens, the controller sets the same Conditions as when it fails, but adds another Condition with the following attributes to the HelmRepository’s .status.conditions:

type: Stalled
status: "True"
reason: URLInvalid

While the HelmRepository has this Condition, the controller will not requeue the resource any further, and will stop reconciling the resource until a change to the spec is made.

Observed Generation

The source-controller reports an observed generation in the HelmRepository’s .status.observedGeneration. The observed generation is the latest .metadata.generation which resulted in either a ready state, or stalled due to error it can not recover from without human intervention.

Last Handled Reconcile At

The source-controller reports the last reconcile.fluxcd.io/requestedAt annotation value it acted on in the .status.lastHandledReconcileAt field.

For practical information about this field, see triggering a reconcile.

Flux: Helm Charts

Mon, 01 Jan 0001 00:00:00 +0000

The HelmChart API defines a Source to produce an Artifact for a Helm chart archive with a set of specific configurations.

Example

The following is an example of a HelmChart. It fetches and/or packages a Helm chart and exposes it as a tarball (.tgz) Artifact for the specified configuration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: podinfo
 namespace: default
spec:
 interval: 5m0s
 chart: podinfo
 reconcileStrategy: ChartVersion
 sourceRef:
 kind: HelmRepository
 name: podinfo
 version: '5.*'

In the above example:

A HelmChart named podinfo is created, indicated by the .metadata.name field.
The source-controller fetches the Helm chart every five minutes from the podinfo HelmRepository source reference, indicated by the .spec.sourceRef.kind and .spec.sourceRef.name fields.
The fetched Helm chart version is the latest available chart version in the range specified in spec.version. This version is also used as Artifact revision, reported in-cluster in the .status.artifact.revision field.
When the current Helm Chart version differs from the latest available chart in the version range, it is fetched and/or packaged as a new Artifact.
The new Artifact is reported in the .status.artifact field.

You can run this example by saving the manifest into helmchart.yaml.

Note: HelmChart is usually used by the helm-controller. Based on the HelmRelease configuration, an associated HelmChart is created by the helm-controller.

Apply the resource on the cluster:
```
kubectl apply -f helmchart.yaml
```

Run kubectl get helmchart to see the HelmChart:

NAME CHART VERSION SOURCE KIND SOURCE NAME AGE READY STATUS
podinfo podinfo 5.* HelmRepository podinfo 53s True pulled 'podinfo' chart with version '5.2.1'

Run kubectl describe helmchart podinfo to see the Artifact and Conditions in the HelmChart’s Status:

Status:
 Observed Source Artifact Revision: sha256:83a3c595163a6ff0333e0154c790383b5be441b9db632cb36da11db1c4ece111
 Artifact:
 Digest: sha256:6c3cc3b955bce1686036ae6822ee2ca0ef6ecb994e3f2d19eaf3ec03dcba84b3
 Last Update Time: 2022-02-13T11:24:10Z
 Path: helmchart/default/podinfo/podinfo-5.2.1.tgz
 Revision: 5.2.1
 Size: 14166
 URL: http://source-controller.flux-system.svc.cluster.local./helmchart/default/podinfo/podinfo-5.2.1.tgz
 Conditions:
 Last Transition Time: 2022-02-13T11:24:10Z
 Message: pulled 'podinfo' chart with version '5.2.1'
 Observed Generation: 1
 Reason: ChartPullSucceeded
 Status: True
 Type: Ready
 Last Transition Time: 2022-02-13T11:24:10Z
 Message: pulled 'podinfo' chart with version '5.2.1'
 Observed Generation: 1
 Reason: ChartPullSucceeded
 Status: True
 Type: ArtifactInStorage
 Observed Chart Name: podinfo
 Observed Generation: 1
 URL: http://source-controller.flux-system.svc.cluster.local./helmchart/default/podinfo/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal ChartPullSucceeded 2m51s source-controller pulled 'podinfo' chart with version '5.2.1'

Writing a HelmChart spec

As with all other Kubernetes config, a HelmChart needs apiVersion, kind, and metadata fields. The name of a HelmChart object must be a valid DNS subdomain name.

A HelmChart also needs a .spec section.

Source reference

.spec.sourceRef is a required field that specifies a reference to the Source the chart is available at.

Supported references are:

Although there are three kinds of source references, there are only two underlying implementations. The artifact building process for GitRepository and Bucket are the same as they are already built source artifacts. In case of HelmRepository, a chart is fetched and/or packaged based on the configuration of the Helm chart.

For a HelmChart to be reconciled, the associated artifact in the source reference must be ready. If the source artifact is not ready, the HelmChart reconciliation is retried.

When the metadata.generation of the HelmChart don’t match with the status.observedGeneration, the chart is fetched from source and/or packaged. If there’s no .spec.valuesFiles specified, the chart is only fetched from the source, and not packaged. If .spec.valuesFiles are specified, the chart is fetched and packaged with the values files. When the metadata.generation matches the status.observedGeneration, the chart is only fetched from source or from the cache if available, and not packaged.

When using a HelmRepository source reference, the secret reference defined in the Helm repository is used to fetch the chart.

The HelmChart reconciliation behavior varies depending on the source reference kind, see reconcile strategy.

The attributes of the generated artifact also varies depending on the source reference kind, see artifact.

Chart

.spec.chart is a required field that specifies the name or path the Helm chart is available at in the Source reference.

For HelmRepository Source reference, it’ll be just the name of the chart.

spec:
 chart: podinfo
 sourceRef:
 name: podinfo
 kind: HelmRepository

For GitRepository and Bucket Source reference, it’ll be the path to the Helm chart directory.

spec:
 chart: ./charts/podinfo
 sourceRef:
 name: podinfo
 kind: <GitRepository|Bucket>

Version

.spec.version is an optional field to specify the version of the chart in semver. It is applicable only when the Source reference is a HelmRepository. It is ignored for GitRepository and Bucket Source reference. It defaults to the latest version of the chart with value *.

Version can be a fixed semver, minor or patch semver range of a specific version (i.e. 4.0.x) or any semver range (i.e. >=4.0.0 <5.0.0).

Values files

.spec.valuesFiles is an optional field to specify an alternative list of values files to use as the chart values (values.yaml). The file paths are expected to be relative to the Source reference. Values files are merged in the order of the list with the last file overriding the first. It is ignored when omitted. When values files are specified, the chart is fetched and packaged with the provided values.

spec:
 chart:
 spec:
 chart: podinfo
 ...
 valuesFiles:
 - values.yaml
 - values-production.yaml

Values files also affect the generated artifact revision, see artifact.

Ignore missing values files

.spec.ignoreMissingValuesFiles is an optional field to specify whether missing values files should be ignored rather than be considered errors. It defaults to false.

When .spec.valuesFiles and .spec.ignoreMissingValuesFiles are specified, the .status.observedValuesFiles field is populated with the list of values files that were found and actually contributed to the packaged chart.

Reconcile strategy

.spec.reconcileStrategy is an optional field to specify what enables the creation of a new Artifact. Valid values are ChartVersion and Revision. ChartVersion is used for creating a new artifact when the chart version changes in a HelmRepository. Revision is used for creating a new artifact when the source revision changes in a GitRepository or a Bucket Source. It defaults to ChartVersion.

Note: If the reconcile strategy is ChartVersion and the source reference is a GitRepository or a Bucket, no new chart artifact is produced on updates to the source unless the version in Chart.yaml is incremented. To produce new chart artifact on change in source revision, set the reconcile strategy to Revision.

Reconcile strategy also affects the artifact version, see artifact for more details.

Interval

.spec.interval is a required field that specifies the interval at which the Helm Chart source must be checked for updates.

After successfully reconciling a HelmChart object, the source-controller requeues the object for inspection after the specified interval. The value must be in a Go recognized duration string format, e.g. 10m0s to look at the source for updates every 10 minutes.

If the .metadata.generation of a resource changes (due to e.g. applying a change to the spec), this is handled instantly outside the interval window.

Note: The controller can be configured to apply a jitter to the interval in order to distribute the load more evenly when multiple HelmChart objects are set up with the same interval. For more information, please refer to the source-controller configuration options.

Suspend

.spec.suspend is an optional field to suspend the reconciliation of a HelmChart. When set to true, the controller will stop reconciling the HelmChart, and changes to the resource or the Helm chart Source will not result in a new Artifact. When the field is set to false or removed, it will resume.

For practical information, see suspending and resuming.

Verification

Note: This feature is available only for Helm charts fetched from an OCI Registry.

.spec.verify is an optional field to enable the verification of Cosign or Notation signatures. The field offers three subfields:

.provider, to specify the verification provider. The supported options are cosign and notation at present.
.secretRef.name, to specify a reference to a Secret in the same namespace as the HelmChart, containing the public keys of trusted authors. For Notation this Secret should also include the trust policy in addition to the CA certificate.
.matchOIDCIdentity, to specify a list of OIDC identity matchers (only supported when using cosign as the verification provider). Please see Keyless verification for more details.

Cosign

The cosign provider can be used to verify the signature of an OCI artifact using either a known public key or via the Cosign Keyless procedure.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: podinfo
spec:
 verify:
 provider: cosign
 secretRef:
 name: cosign-public-keys

When the verification succeeds, the controller adds a Condition with the following attributes to the HelmChart’s .status.conditions:

type: SourceVerified
status: "True"
reason: Succeeded

Public keys verification

To verify the authenticity of HelmChart hosted in an OCI Registry, create a Kubernetes secret with the Cosign public keys:

---
apiVersion: v1
kind: Secret
metadata:
 name: cosign-public-keys
type: Opaque
data:
 key1.pub: <BASE64>
 key2.pub: <BASE64>

Note that the keys must have the .pub extension for Flux to make use of them.

Flux will loop over the public keys and use them to verify a HelmChart’s signature. This allows for older HelmCharts to be valid as long as the right key is in the secret.

Keyless verification

For publicly available HelmCharts, which are signed using the Cosign Keyless procedure, you can enable the verification by omitting the .verify.secretRef field.

.issuer, to specify a regexp that matches against the OIDC issuer.
.subject, to specify a regexp that matches against the subject identity in the certificate. Both values should follow the Go regular expression syntax.

The matchers are evaluated in an OR fashion, i.e. the identity is deemed to be verified if any one matcher successfully matches against the identity.

Example of verifying HelmCharts signed by the Cosign GitHub Action with GitHub OIDC Token:

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: podinfo
spec:
 interval: 5m
 chart: podinfo
 reconcileStrategy: ChartVersion
 sourceRef:
 kind: HelmRepository
 name: podinfo
 version: ">=6.1.6"
 verify:
 provider: cosign
 matchOIDCIdentity:
 - issuer: "^https://token.actions.githubusercontent.com$"
 subject: "^https://github.com/stefanprodan/podinfo.*$"

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
 name: podinfo
spec:
 interval: 1m0s
 url: oci://ghcr.io/stefanprodan/charts
 type: "oci"

The controller verifies the signatures using the Fulcio root CA and the Rekor instance hosted at rekor.sigstore.dev.

Note that keyless verification is an experimental feature, using custom root CAs or self-hosted Rekor instances are not currently supported.

Notation

The notation provider can be used to verify the signature of an OCI artifact using known trust policy and CA certificate.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: podinfo
spec:
 verify:
 provider: notation
 secretRef:
 name: notation-config

When the verification succeeds, the controller adds a Condition with the following attributes to the HelmChart’s .status.conditions:

type: SourceVerified
status: "True"
reason: Succeeded

To verify the authenticity of an OCI artifact, create a Kubernetes secret containing Certificate Authority (CA) root certificates and the a trust policy

---
apiVersion: v1
kind: Secret
metadata:
 name: notation-config
type: Opaque
data:
 certificate1.pem: <BASE64>
 certificate2.crt: <BASE64>
 trustpolicy.json: <BASE64>

Note that the CA certificates must have either .pem or .crt extension and your trust policy must be named trustpolicy.json for Flux to make use of them.

For more information on the signing and verification process see Signing and Verification Workflow.

Flux will loop over the certificates and use them to verify an artifact’s signature. This allows for older artifacts to be valid as long as the right certificate is in the secret.

Working with HelmCharts

Triggering a reconcile

To manually tell the source-controller to reconcile a HelmChart outside the specified interval window, a HelmCHart can be annotated with reconcile.fluxcd.io/requestedAt: <arbitrary value>. Annotating the resource queues the object for reconciliation if the <arbitrary-value> differs from the last value the controller acted on, as reported in .status.lastHandledReconcileAt.

Using kubectl:

kubectl annotate --field-manager=flux-client-side-apply --overwrite helmchart/<chart-name> reconcile.fluxcd.io/requestedAt="$(date +%s)"

Waiting for `Ready`

When a change is applied, it is possible to wait for the HelmChart to reach a ready state using kubectl:

kubectl wait helmchart/<chart-name> --for=condition=ready --timeout=1m

Suspending and resuming

When you find yourself in a situation where you temporarily want to pause the reconciliation of a HelmChart, you can suspend it using the .spec.suspend field.

Suspend a HelmChart

In your YAML declaration:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: <chart-name>
spec:
 suspend: true

Using kubectl:

kubectl patch helmchart <chart-name> --field-manager=flux-client-side-apply -p '{\"spec\": {\"suspend\" : true }}'

Note: When a HelmChart has an Artifact and is suspended, and this Artifact later disappears from the storage due to e.g. the source-controller Pod being evicted from a Node, this will not be reflected in the HelmChart’s Status until it is resumed.

Resume a HelmChart

In your YAML declaration, comment out (or remove) the field:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: <chart-name>
spec:
 # suspend: true

Using kubectl:

kubectl patch helmchart <chart-name> --field-manager=flux-client-side-apply -p '{\"spec\" : {\"suspend\" : false }}'

Debugging a HelmChart

There are several ways to gather information about a HelmChart for debugging purposes.

Describe the HelmChart

Describing a HelmChart using kubectl describe helmchart <chart-name> displays the latest recorded information for the resource in the Status and Events sections:

...
Status:
...
 Conditions:
 Last Transition Time: 2022-02-13T14:06:27Z
 Message: invalid chart reference: failed to get chart version for remote reference: no 'podinfo' chart with version matching '9.*' found
 Observed Generation: 3
 Reason: InvalidChartReference
 Status: True
 Type: Stalled
 Last Transition Time: 2022-02-13T14:06:27Z
 Message: invalid chart reference: failed to get chart version for remote reference: no 'podinfo' chart with version matching '9.*' found
 Observed Generation: 3
 Reason: InvalidChartReference
 Status: False
 Type: Ready
 Last Transition Time: 2022-02-13T14:06:27Z
 Message: invalid chart reference: failed to get chart version for remote reference: no 'podinfo' chart with version matching '9.*' found
 Observed Generation: 3
 Reason: InvalidChartReference
 Status: True
 Type: FetchFailed
 Last Handled Reconcile At: 1644759954
 Observed Chart Name: podinfo
 Observed Generation: 3
 URL: http://source-controller.flux-system.svc.cluster.local./helmchart/default/podinfo/latest.tar.gz
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Warning InvalidChartReference 11s source-controller invalid chart reference: failed to get chart version for remote reference: no 'podinfo' chart with ver
sion matching '9.*' found

Trace emitted Events

To view events for specific HelmChart(s), kubectl events can be used in combination with --for to list the Events for specific objects. For example, running

kubectl events --for HelmChart/<chart-name>

lists

LAST SEEN TYPE REASON OBJECT MESSAGE
22s Warning InvalidChartReference helmchart/<chart-name> invalid chart reference: failed to get chart version for remote reference: no 'podinfo' chart with version matching '9.*' found
2s Normal ChartPullSucceeded helmchart/<chart-name> pulled 'podinfo' chart with version '6.0.3'
2s Normal ArtifactUpToDate helmchart/<chart-name> artifact up-to-date with remote revision: '6.0.3'

Besides being reported in Events, the reconciliation errors are also logged by the controller. The Flux CLI offer commands for filtering the logs for a specific HelmChart, e.g. flux logs --level=error --kind=HelmChart --name=<chart-name>.

Improving resource consumption by enabling the cache

When using a HelmRepository as Source for a HelmChart, the controller loads the repository index in memory to find the latest version of the chart.

The controller can be configured to cache Helm repository indexes in memory. The cache is used to avoid loading repository indexes for every HelmChart reconciliation.

The following flags are provided to enable and configure the cache:

helm-cache-max-size: The maximum size of the cache in number of indexes. If 0, then the cache is disabled.
helm-cache-ttl: The TTL of an index in the cache.
helm-cache-purge-interval: The interval at which the cache is purged of expired items.

The caching strategy is to pull a repository index from the cache if it is available, otherwise to load the index, retrieve and build the chart, then cache the index. The cached index TTL is refreshed every time the Helm repository index is loaded with the helm-cache-ttl value.

The cache is purged of expired items every helm-cache-purge-interval.

When the cache is full, no more items can be added to the cache, and the source-controller will report a warning event instead.

In order to use the cache, set the related flags in the source-controller Deployment config:

 spec:
 containers:
 - args:
 - --watch-all-namespaces
 - --log-level=info
 - --log-encoding=json
 - --enable-leader-election
 - --storage-path=/data
 - --storage-adv-addr=source-controller.$(RUNTIME_NAMESPACE).svc.cluster.local.
 ## Helm cache with up to 10 items, i.e. 10 indexes.
 - --helm-cache-max-size=10
 ## TTL of an index is 1 hour.
 - --helm-cache-ttl=1h
 ## Purge expired index every 10 minutes.
 - --helm-cache-purge-interval=10m

HelmChart Status

Artifact

The HelmChart reports the last built chart as an Artifact object in the .status.artifact of the resource.

The Artifact file is a gzip compressed TAR archive (<chart-name>-<chart-version>.tgz), and can be retrieved in-cluster from the .status.artifact.url HTTP address.

Artifact example

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: <chart-name>
status:
 artifact:
 digest: sha256:e30b95a08787de69ffdad3c232d65cfb131b5b50c6fd44295f48a078fceaa44e
 lastUpdateTime: "2022-02-10T18:53:47Z"
 path: helmchart/<source-namespace>/<chart-name>/<chart-name>-<chart-version>.tgz
 revision: 6.0.3
 size: 14166
 url: http://source-controller.flux-system.svc.cluster.local./helmchart/<source-namespace>/<chart-name>/<chart-name>-<chart-version>.tgz

When using a HelmRepository as the source reference and values files are provided, the value of status.artifact.revision is the chart version combined with the HelmChart object generation. For example, if the chart version is 6.0.3 and the HelmChart object generation is 1, the status.artifact.revision value will be 6.0.3+1.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: <chart-name>
status:
 artifact:
 digest: sha256:ee68224ded207ebb18a8e9730cf3313fa6bc1f31e6d8d3943ab541113559bb52
 lastUpdateTime: "2022-02-28T08:07:12Z"
 path: helmchart/<source-namespace>/<chart-name>/<chart-name>-6.0.3+1.tgz
 revision: 6.0.3+1
 size: 14166
 url: http://source-controller.flux-system.svc.cluster.local./helmchart/<source-namespace>/<chart-name>/<chart-name>-6.0.3+1.tgz
 observedGeneration: 1
 ...

When using a GitRepository or a Bucket as the source reference and Revision as the reconcile strategy, the value of status.artifact.revision is the chart version combined with the first 12 characters of the revision of the GitRepository or Bucket. For example if the chart version is 6.0.3 and the revision of the Bucket is 4e5cbb7b97d00a8039b8810b90b922f4256fd3bd8f78b934b4892dae13f7ca87, the status.artifact.revision value will be 6.0.3+4e5cbb7b97d0.

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmChart
metadata:
 name: <chart-name>
status:
 artifact:
 digest: sha256:8d1f0ac3f4b0e8759a32180086f17ac87ca04e5d46c356e67f97e97616ef4718
 lastUpdateTime: "2022-02-28T08:07:12Z"
 path: helmchart/<source-namespace>/<chart-name>/<chart-name>-6.0.3+4e5cbb7b97d0.tgz
 revision: 6.0.3+4e5cbb7b97d0
 size: 14166
 url: http://source-controller.flux-system.svc.cluster.local./helmchart/<source-namespace>/<chart-name>/<chart-name>-6.0.3+4e5cbb7b97d0.tgz

Conditions

A HelmChart enters various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching or building the chart, it can be ready, it can fail during reconciliation, or it can stall.

The HelmChart API is compatible with the kstatus specification, and reports Reconciling and Stalled conditions where applicable to provide better (timeout) support to solutions polling the HelmChart to become Ready.

Reconciling HelmChart

The source-controller marks a HelmChart as reconciling when one of the following is true:

There is no current Artifact for the HelmChart, or the reported Artifact is determined to have disappeared from the storage.
The generation of the HelmChart is newer than the Observed Generation.
The newly fetched Artifact revision differs from the current Artifact.

When the HelmChart is “reconciling”, the Ready Condition status becomes Unknown when the controller detects drift, and the controller adds a Condition with the following attributes to the HelmChart’s .status.conditions:

type: Reconciling
status: "True"
reason: Progressing | reason: ProgressingWithRetry

If the reconciling state is due to a new version, it adds an additional Condition with the following attributes:

type: ArtifactOutdated
status: "True"
reason: NewChart

Both Conditions have a “negative polarity”, and are only present on the HelmChart while their status value is "True".

Ready HelmChart

The source-controller marks a HelmChart as ready when it has the following characteristics:

The HelmChart reports an Artifact.
The reported Artifact exists in the controller’s Artifact storage.
The controller was able to fetch and build the Helm chart using the current spec.
The version/revision of the reported Artifact is up-to-date with the latest version/revision of the Helm chart.

When the HelmChart is “ready”, the controller sets a Condition with the following attributes in the HelmChart’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the HelmChart is marked as reconciling, or e.g. a transient error occurs due to a temporary network issue.

When the HelmChart Artifact is archived in the controller’s Artifact storage, the controller sets a Condition with the following attributes in the HelmChart’s .status.conditions:

type: ArtifactInStorage
status: "True"
reason: Succeeded

This ArtifactInStorage Condition will retain a status value of "True" until the Artifact in the storage no longer exists.

Failed HelmChart

The source-controller may get stuck trying to produce an Artifact for a HelmChart without completing. This can occur due to some of the following factors:

The Helm chart Source is temporarily unavailable.
The credentials in the Source reference Secret are invalid.
The HelmChart spec contains a generic misconfiguration.
A storage related failure when storing the artifact.

When this happens, the controller sets the Ready Condition status to False, and adds a Condition with the following attributes to the HelmChart’s .status.conditions:

type: FetchFailed | type: StorageOperationFailed
status: "True"
reason: AuthenticationFailed | reason: StorageOperationFailed | reason: URLInvalid | reason: IllegalPath | reason: Failed

This condition has a “negative polarity”, and is only present on the HelmChart while the status value is "True". There may be more arbitrary values for the reason field to provide accurate reason for a condition.

While the HelmChart has this Condition, the controller will continue to attempt to produce an Artifact for the resource with an exponential backoff, until it succeeds and the HelmChart is marked as ready.

Note that a HelmChart can be reconciling while failing at the same time, for example due to a newly introduced configuration issue in the HelmChart spec. When a reconciliation fails, the Reconciling Condition reason would be ProgressingWithRetry. When the reconciliation is performed again after the failure, the reason is updated to Progressing.

Stalled HelmChart

The source-controller can mark a HelmChart as stalled when it determines that without changes to the spec, the reconciliation can not succeed. For example because a HelmChart Version is set to a non-existing version.

When this happens, the controller sets the same Conditions as when it fails, but adds another Condition with the following attributes to the HelmChart’s .status.conditions:

type: Stalled
status: "True"
reason: InvalidChartReference

While the HelmChart has this Condition, the controller will not requeue the resource any further, and will stop reconciling the resource until a change to the spec is made.

Observed Source Artifact Revision

The source-controller reports the revision of the last Source reference’s Artifact the current chart was fetched from in the HelmChart’s .status.observedSourceArtifactRevision. It is used to keep track of the source artifact revision and detect when a new source artifact is available.

Observed Chart Name

The source-controller reports the last resolved chart name of the Artifact for the .spec.chart field in the HelmChart’s .status.observedChartName. It is used to keep track of the chart and detect when a new chart is found.

Observed Generation

The source-controller reports an observed generation in the HelmChart’s .status.observedGeneration. The observed generation is the latest .metadata.generation which resulted in either a ready state, or stalled due to error it can not recover from without human intervention.

Last Handled Reconcile At

The source-controller reports the last reconcile.fluxcd.io/requestedAt annotation value it acted on in the .status.lastHandledReconcileAt field.

For practical information about this field, see triggering a reconcile.

Flux: External Artifacts

Mon, 01 Jan 0001 00:00:00 +0000

The ExternalArtifact is a generic API designed for interoperability with Flux. It allows 3rd party controllers to produce and store Artifact objects in the same way as Flux’s own source-controller. For more details on the design and motivation behind this API, see RFC-0012.

Example

The following is an example of a ExternalArtifact produced by a 3rd party source controller:

apiVersion: source.toolkit.fluxcd.io/v1
kind: ExternalArtifact
metadata:
 name: my-artifact
 namespace: flux-system
spec:
 sourceRef:
 apiVersion: example.com/v1
 kind: Source
 name: my-source
status:
 artifact:
 digest: sha256:35d47c9db0eee6ffe08a404dfb416bee31b2b79eabc3f2eb26749163ce487f52
 lastUpdateTime: "2025-08-21T13:37:31Z"
 path: source/flux-system/my-source/35d47c9d.tar.gz
 revision: v1.0.0@sha256:35d47c9db0eee6ffe08a404dfb416bee31b2b79eabc3f2eb26749163ce487f52
 size: 20914
 url: http://example-controller.flux-system.svc.cluster.local./source/flux-system/my-source/35d47c9d.tar.gz
 conditions:
 - lastTransitionTime: "2025-08-21T13:37:31Z"
 message: stored artifact for revision v1.0.0
 observedGeneration: 1
 reason: Succeeded
 status: "True"
 type: Ready

ExternalArtifact spec

Source reference

The spec.sourceRef field is optional and contains a reference to the custom resource that the ExternalArtifact is based on.

The spec.sourceRef contains the following fields:

apiVersion: the API version of the custom resource.
kind: the kind of the custom resource.
name: the name of the custom resource.
namespace: the namespace of the custom resource. If omitted, it defaults to the namespace of the ExternalArtifact.

ExternalArtifact status

Artifact

The ExternalArtifact reports the latest synchronized state as an Artifact object in the .status.artifact.

The .status.artifact contains the following fields:

digest: The checksum of the tar.gz file in the format <algorithm>:<checksum>.
lastUpdateTime: Timestamp of the last artifact update.
path: Relative file path of the artifact in storage.
revision: Human-readable identifier with version and checksum in the format <human-readable-identifier>@<algorithm>:<checksum>.
size: Number of bytes in the tar.gz file.
url: In-cluster HTTP address for artifact retrieval.

Conditions

The ExternalArtifact reports its status using Kubernetes standard conditions.

Ready ExternalArtifact

When the 3rd party controller has successfully produced and stored an Artifact in storage, it sets a Condition with the following attributes in the ExternalArtifact’s .status.conditions:

type: Ready
status: "True"
reason: Succeeded

The message field should contain a human-readable message indicating the successful storage of the artifact and the associated revision.

If the 3rd party controller performs a signature verification of the artifact, and the verification is successful, a Condition with the following attributes is added to the ExternalArtifact’s .status.conditions:

type: SourceVerified
status: "True"
reason: Succeeded

The message field should contain a human-readable message indicating the successful verification of the artifact and the associated verification method.

Failed ExternalArtifact

If the 3rd party controller fails to produce and store an Artifact, it sets the Ready Condition status to False, and adds a Condition with the following attributes to the ExternalArtifact’s .status.conditions:

type: Ready
status: "False"
reason: FetchFailed | reason: StorageOperationFailed | reason: VerificationFailed

The message field should contain a human-readable message indicating the reason for the failure.

Flux: Artifact Generators

Mon, 01 Jan 0001 00:00:00 +0000

The ArtifactGenerator is an extension of Flux APIs that allows source composition and decomposition. It enables the generation of ExternalArtifacts from multiple sources ( GitRepositories, OCIRepositories and Buckets) or the splitting of a single source into multiple artifacts.

Source Composition Example

The following example shows how to compose an artifact from multiple sources:

apiVersion: source.extensions.fluxcd.io/v1beta1
kind: ArtifactGenerator
metadata:
 name: my-app
 namespace: apps
spec:
 sources:
 - alias: backend
 kind: GitRepository
 name: my-backend
 - alias: frontend
 kind: OCIRepository
 name: my-frontend
 - alias: config
 kind: Bucket
 name: my-configs
 artifacts:
 - name: my-app-composite
 copy:
 - from: "@backend/deploy/**"
 to: "@artifact/my-app/backend/"
 - from: "@frontend/deploy/*.yaml"
 to: "@artifact/my-app/frontend/"
 - from: "@config/envs/prod/configmap.yaml"
 to: "@artifact/my-app/env.yaml"

The above generator will create an ExternalArtifact named my-app-composite in the apps namespace, which contains the deployment manifests from both the my-backend Git repository and the my-frontend OCI repository, as well as a ConfigMap from the my-configs Bucket.

The ExternalArtifact revision is computed based on the final content of the artifact, in the format latest@sha256:<hash>, where <hash> is a SHA256 checksum of the combined files.

The generated ExternalArtifact can be deployed using a Flux Kustomization, for example:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
 name: my-app
 namespace: apps
spec:
 interval: 30m
 targetNamespace: apps
 sourceRef:
 kind: ExternalArtifact
 name: my-app-composite
 path: "./my-app"
 prune: true

Every time one of the sources is updated, a new artifact revision will be generated with the latest content and the Flux Kustomization will automatically reconcile it.

Helm Chart Composition Example

The following example shows how to compose a Helm chart from multiple sources:

apiVersion: source.extensions.fluxcd.io/v1beta1
kind: ArtifactGenerator
metadata:
 name: podinfo
 namespace: apps
spec:
 sources:
 - alias: chart
 kind: OCIRepository
 name: podinfo-chart
 namespace: apps
 - alias: repo
 kind: GitRepository
 name: podinfo-values
 namespace: apps
 artifacts:
 - name: podinfo-composite
 originRevision: "@chart"
 copy:
 - from: "@chart/"
 to: "@artifact/"
 - from: "@repo/charts/podinfo/values-prod.yaml"
 to: "@artifact/podinfo/values.yaml"
 strategy: Merge # Or `Overwrite` to replace the values.yaml

The above generator will create an ExternalArtifact named podinfo-composite in the apps namespace, which contains the Helm chart from the podinfo-chart OCI repository with the values.yaml merged with values-prod.yaml from the Git repository.

The generated ExternalArtifact can be deployed using a Flux HelmRelease, for example:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
 name: podinfo
 namespace: apps
spec:
 interval: 10m
 releaseName: podinfo
 chartRef:
 kind: ExternalArtifact
 name: podinfo-composite

Source Decomposition Example

The following example shows how to decompose a source into multiple artifacts:

apiVersion: source.extensions.fluxcd.io/v1beta1
kind: ArtifactGenerator
metadata:
 name: my-app
 namespace: apps
spec:
 sources:
 - alias: repo
 kind: GitRepository
 name: my-monorepo
 artifacts:
 - name: frontend
 originRevision: "@repo"
 copy:
 - from: "@repo/deploy/frontend/**"
 to: "@artifact/"
 - name: backend
 originRevision: "@repo"
 copy:
 - from: "@repo/deploy/backend/**"
 to: "@artifact/"

The above generator will create two ExternalArtifacts named frontend and backend in the apps namespace, each containing the respective deployment manifests from the my-monorepo Git repository.

The generated ExternalArtifacts can be deployed using Flux Kustomizations, for example:

---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
 name: backend
 namespace: apps
spec:
 interval: 30m
 sourceRef:
 kind: ExternalArtifact
 name: backend
 path: "./"
 prune: true
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
 name: frontend
 namespace: apps
spec:
 interval: 30m
 sourceRef:
 kind: ExternalArtifact
 name: frontend
 path: "./"
 prune: true

Every time the monorepo is updated, new revisions will be generated only for the affected artifacts. If the manifests in deploy/frontend/ directory are modified, only the frontend artifact will receive a new revision, triggering the Flux Kustomization that applies it. While the backend artifact remains unchanged and its Kustomization will not reconcile.

Writing an ArtifactGenerator

As with all other Kubernetes config, an ArtifactGenerator needs apiVersion,kind, metadata.name and metadata.namespace fields.

The spec field defines the desired state of the ArtifactGenerator, while the status field reports the latest observed state.

Sources

The .spec.sources field defines the Flux source-controller resources that will be used as inputs for artifact generation. Each source must specify:

alias: A unique identifier used to reference the source in copy operations. Alias names must be unique within the same ArtifactGenerator and can only contain alphanumeric characters, dashes and underscores.
kind: The type of Flux source resource (GitRepository, OCIRepository, or Bucket)
name: The name of the source resource
namespace (optional): The namespace of the source resource if different from the ArtifactGenerator namespace

Note that on multi-tenant clusters, platform admins can disable cross-namespace references by starting the controller with the --no-cross-namespace-refs=true flag.

spec:
 sources:
 - alias: backend
 kind: GitRepository
 name: my-backend
 - alias: frontend
 kind: OCIRepository
 name: my-frontend
 - alias: config
 kind: Bucket
 name: my-configs

Sources are watched for changes, and when any source is updated, the controller will regenerate the affected artifacts automatically.

Artifacts

The .spec.artifacts field defines the list of ExternalArtifacts to be generated from the sources. Each artifact must specify:

name (required): The name of the generated ExternalArtifact resource. It must be unique in the context of the ArtifactGenerator and must conform to Kubernetes resource naming conventions.
copy (required): A list of copy operations to perform from sources to the artifact.
revision (optional): A specific source revision to use in the format @alias. If not specified, the revision is automatically computed as latest@<digest> based on the artifact content.
originRevision (optional): A specific source origin revision to include in the artifact metadata in the format @alias. This is useful for the decomposition use case, where you want to track the original source revision of the artifact (e.g. the monorepo commit SHA) without affecting the artifact revision itself.

spec:
 artifacts:
 - name: my-app
 revision: "@backend"
 originRevision: "@frontend"
 copy:
 - from: "@backend/deploy/**"
 to: "@artifact/backend/"
 exclude: ["**/charts/**"]
 - from: "@frontend/manifests/*.yaml"
 to: "@artifact/frontend/"
 - from: "@config/envs/prod/configmap.yaml"
 to: "@artifact/env.yaml"

Copy Operations

Each copy operation specifies how to copy files from sources into the generated artifact:

from: Source path in the format @alias/pattern where alias references a source and pattern is a glob pattern or a specific file/directory path within that source.
to: Destination path in the format @artifact/path where artifact is the root of the generated artifact and path is the relative path to a file or directory.
exclude (optional): A list of glob patterns to filter out from the source selection. Any file matched by from that also matches an exclude pattern will be ignored.
strategy (optional): Defines how to handle existing files at the destination, either Overwrite (default) or Merge (for YAML files only).

Copy operations use cp-like semantics:

Operations are executed in order; later operations can overwrite files from earlier ones
Trailing slash in destination (@artifact/dest/) indicates copying into a directory
@source/dir/ copies as subdirectory, @source/dir/** strips directory prefix and copies contents recursively

Examples of copy operations:

# Copy file to specific path - (like `cp source/config.yaml artifact/apps/app.yaml`)
- from: "@source/config.yaml"
 to: "@artifact/apps/app.yaml" # Creates apps/app.yaml file

# Copy file to directory - (like `cp source/config.yaml artifact/apps/`)
- from: "@source/config.yaml"
 to: "@artifact/apps/" # Creates apps/config.yaml

# Copy files to directory - (like `cp source/configs/*.yaml artifact/apps/`)
- from: "@source/configs/*.yaml" # All .yaml files in configs/
 to: "@artifact/apps/" # Creates apps/file1.yaml, apps/file2.yaml

# Copy dir and files recursively - (like `cp -r source/configs/ artifact/apps/`)
- from: "@source/configs/" # All files and sub-dirs under configs/ 
 to: "@artifact/apps/" # Creates apps/configs/ with contents

# Copy files and dirs recursively - (like `cp -r source/configs/** artifact/apps/`)
- from: "@source/configs/**" # All files and sub-dirs under configs/ 
 to: "@artifact/apps/" # Creates apps/file1.yaml, apps/subdir/file2.yaml
 exclude:
 - "*.md" # Excludes all .md files
 - "**/testdata/**" # Excludes all files under any testdata/ dir

Copy Strategies

By default, copy operations use the Overwrite strategy, where later copies overwrite files from earlier ones.

When copying YAML files, the Merge strategy can be used to merge the contents from the source file into the destination file.

Example of copy with Merge strategy:

# Copy the chart contents (this includes chart-name/values.yaml)
- from: "@chart/"
 to: "@artifact/"
# Merge values.yaml files - (like `helm --values values-prod.yaml`)
- from: "@git/values-prod.yaml"
 to: "@artifact/chart-name/values.yaml"
 strategy: Merge

Note that the merge strategy will replace arrays entirely, the behavior is identical to how Helm merges values.yaml files when using multiple --values flags.

Working with ArtifactGenerators

Suspend and Resume Reconciliation

You can temporarily suspend the reconciliation of an ArtifactGenerator by setting the following annotation on the resource:

metadata:
 annotations:
 source.extensions.fluxcd.io/reconcile: "Disabled"

To resume reconciliation, remove the annotation or set its value to Enabled.

Trigger Reconciliation

You can manually trigger a reconciliation of an ArtifactGenerator by adding the following annotation to the resource:

metadata:
 annotations:
 reconcile.fluxcd.io/requestedAt: "<timestamp>"

The controller will pick up the annotation and start a reconciliation as soon as possible. After the reconciliation is complete, the controller sets the timestamp from the annotation in the .status.lastHandledReconcileAt field.

ArtifactGenerator Status

The controller reports the latest synchronized state of an ArtifactGenerator in the .status field.

Conditions

ArtifactGenerator has various states during its lifecycle, reflected as Kubernetes Conditions. It can be reconciling while fetching the remote state, it can be ready, or it can fail during reconciliation.

All conditions have a message field that provides additional context about the current state.

Reconciling ArtifactGenerator

The controller marks an ArtifactGenerator as reconciling when it is actively working to produce artifacts from source changes.

When the ArtifactGenerator is reconciling, the controller sets the Reconciling Condition with the following attributes:

type: Reconciling
status: "True"
reason: Progressing

In addition, the controller sets the Ready Condition to Unknown.

Ready ArtifactGenerator

The controller marks an ArtifactGenerator as ready when it has successfully produced and stored artifacts in the controller’s storage.

When the ArtifactGenerator is “ready”, the controller sets the Ready Condition with the following attributes:

type: Ready
status: "True"
reason: Succeeded

This Ready Condition will retain a status value of "True" until the ArtifactGenerator is marked as reconciling, or an error occurs.

Failed ArtifactGenerator

The controller may encounter errors while attempting to produce and store artifacts. These errors can be transient or terminal, such as:

The Flux source-controller is unreachable (e.g. network issues).
One of the referenced sources is not found or access is denied.
The copy operation fails due to duplicate aliases, invalid glob patterns or missing files.
Encounters a storage related failure when storing the artifacts.

When an error occurs, the controller sets the Ready Condition status to False, with one of the following reasons:

type: Ready
status: "False"
reason: BuildFaild | SourceFetchFailed | ReconciliationFailed

Transient errors (e.g. network issues) will cause the controller to retry the reconciliation after a backoff period, while terminal errors (e.g. access denied, invalid spec) will cause the controller to mark the ArtifactGenerator as stalled.

Stalled ArtifactGenerator

The controller marks an ArtifactGenerator as stalled when it encounters a terminal failure that prevents it from making progress.

When the ArtifactGenerator is stalled, the controller sets the following condition:

type: Stalled
status: "True"
reason: AccessDenied | ValidationFailed

Inventory

The controller reports the list of generated ExternalArtifacts in the.status.inventory field of the ArtifactGenerator. The inventory is used by the controller to keep track of the artifacts in storage and to perform garbage collection of orphaned artifacts.

ArtifactGenerator Events

The controller emits Kubernetes events to provide insights into the lifecycle of an ArtifactGenerator. These events can be viewed using kubectl describe or with kubectl events.

Events are emitted for the following scenarios:

ArtifactGenerator reconciliation completion (success or failure).
ExternalArtifacts creation, update, or deletion.
Source fetch failures or access issues.
Build failures (e.g. invalid glob patterns, missing files).
Storage operations (e.g. garbage collection, integrity validation failures).
Drift detection (e.g. manual changes to generated ExternalArtifacts).

All events are also logged to the controller’s standard output and contain the ArtifactGenerator name and namespace.

Flux: API Reference

Mon, 01 Jan 0001 00:00:00 +0000

Flux – Source Controllers

Flux: Controller Options

Source controller flags

Feature Gates

Source watcher flags

Feature Gates

Flux: Git Repositories

Example

Writing a GitRepository spec

URL

Secret reference

Basic access authentication

Bearer token authentication

HTTPS Certificate Authority

HTTPS Mutual TLS authentication

SSH authentication

Provider

Azure

Pre-requisites

Configure Flux controller

GitHub

Pre-requisites

Configure GitHub App secret

Service Account reference

Interval

Timeout

Reference

Branch example

Tag example

SemVer example

Name example

Commit example

Verification

Verification Secret example

Ignore

Sparse checkout

Suspend

Proxy secret reference

Recurse submodules

Include

Working with GitRepositories

Excluding files

.sourceignore file

Ignore spec

Triggering a reconcile

Waiting for Ready

Suspending and resuming

Suspend a GitRepository

Resume a GitRepository

Debugging a GitRepository

Describe the GitRepository

Trace emitted Events

GitRepository Status

Artifact

Artifact example

Default exclusions

Conditions

Reconciling GitRepository

Ready GitRepository

Failed GitRepository

Observed Ignore

Observed Recurse Submodules

Observed Include

Observed Sparse Checkout

Source Verification Mode

Observed Generation

Last Handled Reconcile At

Flux: OCI Repositories

Example

Writing an OCIRepository spec

URL

Provider

AWS

Azure

Kubelet Managed Identity

Workload Identity

GCP

Secret reference

Service Account reference

Mutual TLS Authentication

`.sourceignore` file

Waiting for `Ready`

`.sourceignore` file

Waiting for `Ready`