Kubernetes

Kubernetes Skill

Three domains: debugging (pod triage, networking, resources), security (RBAC, pod hardening, network isolation, supply chain), and cobaltcore (KVM exporter, hypervisor metrics). Select by request signal, then follow the phases below.

Always specify -n <namespace> in every kubectl command. Use read-only commands to gather evidence before proposing changes.

Domain Selection

Phase 1: TRIAGE

Debugging Triage Flow

Follow this sequence for every pod or workload issue. Do not skip steps -- many failures are only visible in events and describe output, not in logs.

bashCopy code
kubectl get pods -n <namespace> -o wide
kubectl describe pod <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace> -c <container-name>
kubectl logs <pod-name> -n <namespace> -c <container-name> --previous
kubectl get events -n <namespace> --sort-by='.lastTimestamp'
kubectl exec -it <pod-name> -n <namespace> -c <container-name> -- /bin/sh

Always check --previous logs for crashed containers before current logs -- restarting destroys them permanently.

Diagnosis routing:

Network debugging:

bashCopy code
# Verify service has endpoints
kubectl get endpoints <service-name> -n <namespace>
# DNS lookup from inside cluster
kubectl run dns-debug --rm -it --restart=Never --image=busybox:1.36 -n <namespace> -- \
  nslookup <service-name>.<namespace>.svc.cluster.local
# Check CoreDNS
kubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50
# Port-forward for local testing
kubectl port-forward svc/<service-name> -n <namespace> 8080:80

Security Domain Selection

For security requests, provide concrete YAML manifests from the patterns below. Reference-backed specifics, not generic advice.

RBAC patterns: Prefer namespace-scoped Roles over ClusterRoles. Write exact verbs and resources. Create dedicated ServiceAccounts per workload. Set automountServiceAccountToken: false on pods that need no API access.

yamlCopy code
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: app-team
  name: deployment-reader
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]

Pod security: Enforce PodSecurity labels at namespace level. All containers: runAsNonRoot: true, readOnlyRootFilesystem: true, allowPrivilegeEscalation: false, capabilities: drop: ["ALL"]. Use distroless base images. Pin image digests.

yamlCopy code
labels:
  pod-security.kubernetes.io/enforce: restricted
  pod-security.kubernetes.io/warn: restricted

Network policies: Start with default-deny for ingress and egress. Add allow-list rules per service. Always allow DNS egress (UDP/TCP 53).

yamlCopy code
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]

Cobaltcore Domain

Components: KVM Exporter (cobaltcore-dev/kvm-exporter). Load references/cobalt-kvm-exporter.md for architecture, metric catalogs, configuration, and deployment. Pair with go-patterns for code, prometheus-grafana-engineer for metrics.

Phase 2: DIAGNOSE / RESPOND

Debugging: Follow the triage flow. Gather evidence with read-only commands before proposing changes.

Security: Provide copy-paste-ready YAML using the patterns in Phase 1.

Cobaltcore: Use component-specific references for architecture, metrics, concurrency patterns, and testing.

Phase 3: VERIFY

• Debugging: Confirm the fix resolves the symptom with the same triage commands.
• Security: Validate against the PodSecurity standards and RBAC least-privilege patterns above.
• Cobaltcore: Verify against component test patterns in references/cobalt-testing-patterns.md.

Deep References