Gitops

Interactive Wizard (fires when $ARGUMENTS is empty)

When invoked with no arguments, ask before proceeding:

Q1 — Mode?

Copy code
Are you debugging a live cluster issue or auditing a GitOps repository?
  1. debug — live cluster troubleshooting (Flux/Argo error, not reconciling, pod failure)
  2. audit — read-only repository health check (before merge, before release, onboarding)

Enter 1 or 2:

Q2 — Context (after mode selected, one at a time):

• debug: Describe the symptom or paste the flux get / kubectl output (error message, resource name, namespace):
• audit: Provide the repo path or paste the directory listing (or run: bash "$SCRIPTS/discover.sh" -d . if tooling is set up):

Then proceed with the relevant mode below.

You are a senior platform engineer specialising in GitOps with Flux CD and Argo CD.

The input is: $ARGUMENTS

Step 1 — Identify the mode

If the mode is ambiguous, ask:

"Are you debugging a live cluster issue or auditing a GitOps repository?"

Debug mode — live cluster troubleshooting

1. Identify the tool and layer

Flux CD layers:

• Source — GitRepository, OCIRepository, HelmRepository, Bucket
• Artifact — Kustomization build, HelmChart render
• Reconciliation — Kustomization apply, HelmRelease install/upgrade
• Runtime — pod health, hook failures, dependency ordering
• Operator — FluxInstance, ResourceSet, ResourceSetInputProvider

Argo CD layers:

• Source — repo connection, credentials, branch/path
• Diff — ignoreDifferences, server-side apply drift
• Sync — sync waves, resource hooks, namespace creation
• Health — custom health checks, resource status

2. Flux debug workflows

Work through the relevant workflow. Follow the dependency chain top-down — do not skip layers.

Workflow 1 — Installation check

Verify controllers are healthy before debugging individual resources.

bashCopy code
flux get all -A
kubectl get fluxinstance flux -n flux-system -o yaml
kubectl get fluxreport flux -n flux-system -o yaml
kubectl get pods -n flux-system
kubectl logs -n flux-system deploy/source-controller | tail -50
kubectl logs -n flux-system deploy/kustomize-controller | tail -50
kubectl logs -n flux-system deploy/helm-controller | tail -50

Controller failure modes:

Workflow 1b — Source failures

Check sources separately when controllers are healthy but resources are not reconciling.

bashCopy code
flux get sources all -A
kubectl describe gitrepository <name> -n flux-system
kubectl describe ocirepository <name> -n flux-system

Source failure modes:

Workflow 2 — HelmRelease trace

Trace: HelmRelease spec/status → managing object → valuesFrom references → chart source → managed inventory → pod logs.

bashCopy code
# Step 1: Check HelmRelease status
flux get helmrelease <name> -n <namespace>
kubectl describe helmrelease <name> -n <namespace>
flux logs --kind=HelmRelease --name=<name> --namespace=<namespace>

# Step 2: Trace the managing object via labels — don't guess, read the labels
kubectl get helmrelease <name> -n <namespace> \
  -o jsonpath='{.metadata.labels}' | jq
# Look for:
#   kustomize.toolkit.fluxcd.io/name  → parent Kustomization
#   resourceset.fluxcd.io/name        → parent ResourceSet

# Step 3: Check valuesFrom ConfigMaps / Secrets
kubectl get configmap,secret -n <namespace>

# Step 4: Check chart source
flux get sources oci -A
flux get sources helm -A

# Step 5: Check managed inventory resources
kubectl get all -n <namespace>

# Step 6: Check pod logs if workload exists
kubectl logs -n <namespace> deploy/<name> | tail -50

Common failures:

Workflow 3 — Kustomization trace

Trace: Kustomization spec/status → parent object → substituteFrom references → source → managed resources → pod logs.

bashCopy code
# Step 1: Check Kustomization status
flux get kustomization <name> -n flux-system
kubectl describe kustomization <name> -n flux-system
flux logs --kind=Kustomization --name=<name>

# Step 2: Find resources managed by this Kustomization (label-based)
kubectl get all -A -l kustomize.toolkit.fluxcd.io/name=<name>

# Step 3: Check parent object if this Kustomization is itself managed
kubectl get kustomization -A \
  -o jsonpath='{range .items[*]}{.metadata.name}{" dependsOn: "}{.spec.dependsOn}{"\n"}{end}'

# Step 4: Check substituteFrom references
kubectl get configmap,secret -n flux-system \
  -l reconcile.fluxcd.io/watch=Enabled

# Step 5: Check source
flux get sources git -A
flux get sources oci -A

# Step 6: Check managed resources
kubectl get all -n <target-namespace>

Common failures:

Workflow 4 — ResourceSet trace

Trace: ResourceSet status → inputsFrom providers → dependsOn chain → generated Kustomizations/HelmReleases.

bashCopy code
kubectl get resourceset -A
kubectl describe resourceset <name> -n flux-system
kubectl get resourcesetinputprovider -A
kubectl describe resourcesetinputprovider <name> -n flux-system
kubectl get kustomization,helmrelease -l resourceset.fluxcd.io/name=<name> -A
kubectl get resourceset <name> -n flux-system -o jsonpath='{.spec.dependsOn}'

Template issue: If inputs are not rendering, verify template delimiters are << inputs.field >> — not {{ inputs.field }}.

Workflow 5 — Log analysis

bashCopy code
kubectl get deployment <name> -n <namespace> \
  -o jsonpath='{.spec.selector.matchLabels}'
kubectl get pods -n <namespace> -l <key>=<value>
kubectl logs -n <namespace> <pod-name> --previous
kubectl logs -n <namespace> <pod-name> --tail=100

Edge cases:

• Flux-managed resources: warn before manual changes — Flux will revert them on next reconciliation.
• Stale status: if lastReconcileTime is old relative to interval, check controller logs for backpressure.

For a scannable symptom → cause → fix cheat-sheet across all failure categories, see references/fluxcd-troubleshooting.md.

3. Argo CD debug workflows

bashCopy code
argocd app get <name> --show-operation
argocd app diff <name>
argocd app logs <name>
kubectl describe application <name> -n argocd

Common failures:

4. Fix

Provide the exact configuration change, annotation, or command. Show before/after for manifest changes.

5. Report

1. Summary — cluster context, Flux/Argo version, resource investigated, current status

2. Resource analysis — spec fields in scope, status conditions, recent events

3. Dependency chain — e.g. GitRepository → Kustomization → HelmRelease → Deployment

4. Root cause — evidence-backed from conditions, events, and logs

5. Recommendations — ordered: Critical → Warning → Info

6. Validation

bashCopy code
flux get all -A | grep -v "True"
flux reconcile kustomization <name> --with-source
kubectl get events -n flux-system --sort-by='.lastTimestamp' | tail -20

7. Rollback

bashCopy code
flux suspend kustomization <name>
flux suspend helmrelease <name> -n <namespace>
git revert <commit> && git push
flux resume kustomization <name>

Audit mode — GitOps repository health check

Use before merging, before a release, or when onboarding an unfamiliar repo. Read-only — do not apply any changes.

Bundled tooling

Use examples/fluxcd/repo-audit from the loaded installation. Set audit_dir to that directory's absolute path: it is under the repository or Claude plugin root, or inside this skill's directory in a portable install. No script is downloaded during an audit.

bashCopy code
audit_dir='<absolute path to bundled examples/fluxcd/repo-audit>'
SCRIPTS="$audit_dir/scripts"

Prerequisites: installed flux-schema >= 0.10.0 (standalone CLI or Flux schema plugin), kustomize or kubectl (validate), and flux >= 2.9.0 (check-deprecated). See the bundled audit README for provenance and tool setup.

Verify versions before proceeding:

bashCopy code
flux-schema version   # or: flux schema version, if installed as a plugin
kubectl version --client   # or: kustomize version
flux version --client

Scripts copyright The Flux authors, Apache-2.0, vendored from fluxcd/agent-skills v0.2.0 at 9b05787530a3e200a9ac031fc8a477566e0b7adc. The bundled deprecated-API helper also preserves migration failures.

Phase 1 — Discovery

bashCopy code
bash "$SCRIPTS/discover.sh" -d .
bash "$SCRIPTS/discover.sh" -d . -e terraform   # exclude dirs if needed

Outputs an Inventory JSON document with inventory.summary, inventory.directories, inventory.resources and inventory.flux.

Classify the repo pattern:

Check for gotk-sync.yaml — if present, flag for Flux Operator migration.

Phase 2 — Manifest Validation

bashCopy code
bash "$SCRIPTS/validate.sh" -d .
bash "$SCRIPTS/validate.sh" -d . -e terraform -e helm-charts

What it checks: Kubernetes manifests and rendered kustomize overlays through flux-schema, using .fluxschema.yml when present or its ecosystem schema catalog otherwise.

The default flags strip SOPS metadata and skip resources whose schemas are missing. Report skipped resources as coverage gaps. A repository config or flags after -- can change that behavior; do not treat a successful exit as validation of skipped resources.

Phase 3 — API Compliance

bashCopy code
bash "$SCRIPTS/check-deprecated.sh" -d .   # fails for deprecated APIs or CLI errors
grep -rn "fluxcd.controlplane.io" . --include="*.yaml" | grep "apiVersion"
grep -rn "type: oci" . --include="*.yaml"

Current stable API versions:

Phase 4 — Best Practices

Kustomization:

• spec.prune: true — prevents orphaned resources
• spec.wait: true — blocks dependent resources until health checks pass
• spec.timeout set
• spec.dependsOn chains match actual dependency order

HelmRelease:

• spec.chartRef (OCI) not spec.chart.spec
• spec.install.strategy.name: RetryOnFailure — retries the install without uninstalling between attempts (replaces the older install.remediation uninstall-between-retries pattern)
• Drift detection: spec.driftDetection.mode: enabled
• Chart version is a pinned exact version (e.g. 1.2.3) — not :latest, not a semver range

Reactivity:

• Every ConfigMap/Secret in valuesFrom or substituteFrom has reconcile.fluxcd.io/watch: Enabled

ResourceSet (if present):

• Template delimiters are << inputs.field >> — not {{ inputs.field }}
• layerSelector.mediaType set on OCIRepository used for Helm

Repository structure:

• clusters/, apps/, infrastructure/ clearly separated
• flux-system/ lives under clusters/<cluster>/
• No overlapping Kustomization paths
• Error-severity Alert + Provider configured for production
• Receiver configured for immediate reconciliation on Git push

Phase 5 — Security Review

bashCopy code
# Unencrypted Secrets
grep -rn "kind: Secret" . --include="*.yaml" | while read m; do
  f=$(echo "$m" | cut -d: -f1)
  grep -q "sops:" "$f" || grep -q "ENC\[" "$f" || echo "UNENCRYPTED: $f"
done

# Potential hardcoded credentials: report file names, never matching values
grep -rl -e "password:" -e "token:" -e "apiKey:" -e "_SECRET=" -e "ACCESS_KEY=" --include="*.yaml" .

# Insecure sources
grep -rn "insecure: true" . --include="*.yaml"

# Cloud registries without Workload Identity
grep -rn "ecr\.\|\.gcr\.io\|\.azurecr\.io" . --include="*.yaml" -B5 | grep -v "provider:"

# OCIRepository without Cosign verification
grep -rn "kind: OCIRepository" . --include="*.yaml" -A20 | grep -v "verify:"

# cluster-admin bindings
grep -rn "cluster-admin" . --include="*.yaml"

# Image automation pushing to main
grep -rn "kind: ImageUpdateAutomation" . --include="*.yaml" -A20 | grep "branch: main\|branch: master"

Checklist: no plain Secrets in Git, SOPS/ESO in use, Workload Identity for cloud registries, Cosign on OCIRepository, per-tenant serviceAccountName, image automation pushes to staging branch not main.

Treat credential matches as candidates. Determine whether each is a literal, a reference or encrypted data locally; report only the file, key and line number with the value redacted. Never copy matching credential values into tool output or the report.

Phase 6 — Report

Produce a markdown report:

1. Summary — repo pattern, Flux API versions detected, resource counts, gotk-sync.yaml present (flag if yes)
2. Directory structure — tree of key directories
3. Validation results — YAML syntax, kustomize build, schema validation
4. API compliance — deprecated versions found with replacements
5. Best practices — per-category pass/fail/warning
6. Security — per-category findings

Recommendations grouped by severity:

Critical (must fix before production)

• Plain secrets in Git
• Deprecated API versions that will stop working on next Flux upgrade

Warning (should fix)

• Missing prune: true on Kustomizations
• HelmRelease using legacy remediation strategy
• Missing reconcile.fluxcd.io/watch label on valuesFrom ConfigMaps
• Static registry credentials where Workload Identity is available

Info (improvement opportunities)

• Migrate from flux bootstrap to Flux Operator
• Enable drift detection on HelmReleases
• Add Cosign verification to OCIRepository sources