Flux Troubleshooting

failed to list CRDs: no Flux CRDs found

Fehlerlog von kubectl -n flux-system logs flux-operator-5f494d78b4-dwgt6:

{"level":"error","ts":"2024-12-01T15:26:58.134Z","msg":"report computed with errors","controller":"fluxreport","controllerGroup":"fluxcd.controlplane.io","controllerKind":"FluxReport","FluxReport":{"name":"flux","namespace":"flux-system"},"namespace":"flux-system","name":"flux","reconcileID":"ea3bdaab-8d14-431c-960e-22b2703a5608","error":"failed to list CRDs: no Flux CRDs found"}
{"level":"info","ts":"2024-12-01T15:26:58.145Z","msg":"Reporting finished in 4ms","controller":"fluxreport","controllerGroup":"fluxcd.controlplane.io","controllerKind":"FluxReport","FluxReport":{"name":"flux","namespace":"flux-system"},"namespace":"flux-system","name":"flux","reconcileID":"ea3bdaab-8d14-431c-960e-22b2703a5608"}
{"level":"error","ts":"2024-12-01T15:31:58.149Z","msg":"report computed with errors","controller":"fluxreport","controllerGroup":"fluxcd.controlplane.io","controllerKind":"FluxReport","FluxReport":{"name":"flux","namespace":"flux-system"},"namespace":"flux-system","name":"flux","reconcileID":"5d8256d6-3de6-4551-936e-7e795b13fdac","error":"failed to list CRDs: no Flux CRDs found"}
{"level":"info","ts":"2024-12-01T15:31:58.149Z","msg":"Reporting finished in 4ms","controller":"fluxreport","controllerGroup":"fluxcd.controlplane.io","controllerKind":"FluxReport","FluxReport":{"name":"flux","namespace":"flux-system"},"namespace":"flux-system","name":"flux","reconcileID":"5d8256d6-3de6-4551-936e-7e795b13fdac"}
{"level":"error","ts":"2024-12-01T15:33:59.925Z","msg":"Reconciler error","controller":"fluxinstance","controllerGroup":"fluxcd.controlplane.io","controllerKind":"FluxInstance","FluxInstance":{"name":"flux","namespace":"flux-system"},"namespace":"flux-system","name":"flux","reconcileID":"44f9290f-142b-4ca2-ab84-a11f2cc9b735","error":"pulling artifact oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests failed: Get \"https://ghcr.io/v2/\": context deadline exceeded"}

Das wird aktuell durch Problematik von Cilium / Talos (DNS) hervorgerufen. Talos forwarded die DNS Anfragen des K8S an den Talos Host. Im Zusammenspiel mit Cilium (wenn BPF Masquerade aktiv ist), werden die Anfragen nicht weitergeleitet.

Log aus dem CoreDNS:

[INFO] 10.244.0.254:56530 - 14386 "AAAA IN ghcr.io. udp 36 false 1232" - - 0 2.00103556s
[ERROR] plugin/errors: 2 ghcr.io. AAAA: read udp 10.244.0.109:47990->169.254.116.108:53: i/o timeout
[INFO] [2a02:8070:d82:2b00::3631]:55617 - 24993 "A IN ghcr.io. udp 48 false 1232" - - 0 2.000714022s
[ERROR] plugin/errors: 2 ghcr.io. A: read udp 10.244.0.109:55627->169.254.116.108:53: i/o timeout

Lösungsansatz aktuell ist den Forward abzuschalten. https://www.talos.dev/v1.8/talos-guides/network/host-dns/#forwarding-kube-dns-to-host-dns

machine:
  features:
    hostDNS:
      enabled: true
      forwardKubeDNSToHost: false

relevante GitHub Issues:

OCIrepository

gzip: invalid header

OCIRepository 'kube-system/traefik-crds' is not ready: failed to extract layer contents from artifact: requires gzip-compressed body: gzip: invalid header

Lösung:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: something
spec:
...
  layerSelector:
    mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip
    operation: copy

HelmRelease

data: Too long: may not be more than 1048576 bytes

Helm install failed for release kube-system/traefik-crds with chart traefik-crds@1.16.0+43c4d7b85230: create: failed to create: Secret "sh.helm.release.v1.traefik-crds.v1" is invalid: data: Too long: may not be more than 1048576 bytes

Lösung:

Bisher keine gefunden. Das generierte Secret ist zu groß. Alternativen suchen. Z.b. CRDs via kustomize statt Helm installieren.