Task Runner Reference¶

All cluster operations use just as the task runner. Commands are organized into three modules: bootstrap, kube (Kubernetes), and talos.

Command Structure¶

just <module> <command> [args]

Examples:

just bootstrap           # Run full bootstrap
just kube sync-all-hr    # Sync all Helm releases
just talos reboot-node k8s-1  # Reboot a node

View all commands:

just --list        # All commands
just kube          # Kubernetes module commands
just talos         # Talos module commands
just bootstrap     # Bootstrap module commands (shows help)

Bootstrap Module¶

Commands for initial cluster setup. Defined in bootstrap/mod.just.

Full Bootstrap¶

This is the main command for rebuilding the cluster from scratch. It runs all bootstrap stages in order.

Individual Stages¶

Run specific bootstrap stages:

Kubernetes Module¶

Commands for managing Kubernetes resources and Flux. Defined in kubernetes/mod.just.

Resource Management¶

Flux Operations¶

Sync Commands (force immediate reconciliation):

Reconcile Commands (reconcile from source, rebuilds from Git):

Restart Commands (suspend/resume failed resources):

Local Development¶

Use apply-ks to test Flux manifests locally before pushing to Git. This uses flux-local to render and validate without affecting the cluster.

VolSync Backups¶

# List available snapshots
just kube volsync-list default immich

# Restore from 2nd most recent snapshot
just kube volsync-restore default immich 2

KEDA Auto-scaling¶

Suspending KEDA pauses auto-scaling, effectively pinning the app at its current replica count.

Network Stack¶

Use this when networking is broken or after major configuration changes. Components restart sequentially to maintain dependencies.

Talos Module¶

Commands for managing Talos OS nodes. Defined in talos/mod.just.

Node Management¶

Upgrades¶

Image Management¶

Utility Commands¶

These are internal commands used by other recipes. Generally not called directly.

Bootstrap Utilities¶

Located in root .justfile:

Kubernetes Utilities¶

Internal commands in kubernetes/mod.just:

Talos Utilities¶

Internal commands in talos/mod.just:

Common Workflows¶

Updating an App¶

# 1. Edit Helm values
nano kubernetes/apps/media/plex/app/helmrelease.yaml

# 2. Test locally (optional)
just kube apply-ks media plex

# 3. Commit and push
git add kubernetes/apps/media/plex/
git commit -m "feat(plex): update config"
git push

# 4. Force reconcile (or wait for automatic sync)
just kube ks-reconcile media plex

Rebuilding a Failed App¶

# 1. Delete HelmRelease to clear stuck state
kubectl delete helmrelease <app> -n <namespace>

# 2. Force reconcile from Git
just kube ks-reconcile <namespace> <app>

# 3. Watch rollout
kubectl rollout status deployment/<app> -n <namespace> -w

Node Reboot¶

# Reboot one node at a time
just talos reboot-node k8s-1
kubectl wait --for=condition=ready node/k8s-1 --timeout=10m

just talos reboot-node k8s-2
kubectl wait --for=condition=ready node/k8s-2 --timeout=10m

just talos reboot-node k8s-3
kubectl wait --for=condition=ready node/k8s-3 --timeout=10m

Restore from Backup¶

# 1. List snapshots
just kube volsync-list default immich

# 2. Restore from snapshot (1=latest)
just kube volsync-restore default immich 1

# 3. Verify app is running
kubectl get pods -n default -l app.kubernetes.io/name=immich

Tool Dependencies¶

All commands check for required tools before running. Install via mise (.mise.toml):

mise install

Required tools: - kubectl - Kubernetes CLI - flux - Flux CD CLI - talosctl - Talos OS CLI - helm - Helm package manager - helmfile - Helmfile deployment tool - kustomize - Kustomize CLI - flux-local - Local Flux testing - akeyless - aKeyless CLI - jq - JSON processor - yq - YAML processor - gum - CLI UI tools - minijinja-cli - Jinja2 template renderer

Next Steps¶

• Operations Overview: Day-to-day workflows
• Troubleshooting Guide: Common issues and fixes