Skip to main content
Version: Next 🚧

Cluster Cleanup

Clean up TFRS Operator and its managed infrastructure.

Determine Deployment Method​

# Check for Helm releases
helm list -A | grep -E "tfrs-operator|tfrcluster|tfrtenant"

# Has output → Helm deployment
# No output → Kustomize deployment

Helm Deployment Cleanup​

Starting from v0.2.0-alpha1, the Operator supports cascade deletion: deleting a CR automatically cleans up all managed infrastructure components, Helm releases, and namespaces. No need to manually uninstall Istio, cert-manager, or other infrastructure.

1. Uninstall Helm Releases in Order​

Important

Must uninstall in order: TFRServer → TFRTenant → TFRCluster → Operator, ensuring the Operator is running to handle cascade deletion for each CR.

# List all related releases
helm list -A | grep -E "tfrserver|tfrtenant|tfrcluster|tfrs-operator"

# Step 1: Uninstall TFRServer
helm uninstall tfrserver -n <tenant-namespace>

# Step 2: Uninstall TFRTenant (after TFRServer fully cleaned up)
helm uninstall tfrtenant

# Step 3: Uninstall TFRCluster (after TFRTenant fully cleaned up)
helm uninstall tfrcluster -n tfrs-infra

# Step 4: Uninstall Operator
helm uninstall tfrs-operator -n tfrs-operator-system
Verify Cleanup Progress

After each uninstall step, verify the CR is fully deleted:

# Check if CRs still exist
kubectl get tfrservers,tfrtenants,tfrclusters -A

# Check if infrastructure is cleaned up
helm list -A | grep -E "istio|cert-manager|cnpg|minio|rabbitmq"

2. Clean Up Residual Resources​

Cascade deletion automatically cleans up infrastructure namespaces, but the following need manual cleanup:

# Delete residual empty namespaces (Helm does not auto-delete the namespace where its release resides)
kubectl delete ns tfrs-infra tfrs-operator-system 2>/dev/null

3. Delete CRDs (Optional)​

CRDs are preserved by standard Helm behavior (to prevent data loss). For full cleanup:

# TFRS CRDs
kubectl delete crd tfrclusters.infra.tfrobotserver.cn \
    tfrtenants.infra.tfrobotserver.cn \
    tfrservers.tfrobot.tfrobotserver.cn

# Infrastructure CRDs (not cleaned by cascade deletion)
kubectl get crd | grep -E "istio|cert-manager|cnpg|rabbitmq|minio" | awk '{print $1}' | xargs kubectl delete crd

Kustomize Deployment Cleanup​

Note

Do not use make undeploy - it deletes CRDs, causing subsequent Helm uninstall failures.

1. Delete CR Resources​

kubectl get tfrservers,tfrtenants,tfrclusters -A
kubectl delete tfrserver <name> -n <namespace>
kubectl delete tfrtenant <name>
kubectl delete tfrcluster <name>

2. Uninstall Infrastructure Helm Releases​

TFRCluster installs infrastructure via Helm SDK, must uninstall before deleting CRDs:

# List releases
helm list -A | grep -E "istio|cert-manager|cnpg|minio|rabbitmq"

# Istio (in order)
helm uninstall istio-ingress -n istio-ingress
helm uninstall istiod -n istio-system
helm uninstall istio-cni -n istio-system
helm uninstall istio-base -n istio-system

# Other Operators
helm uninstall cert-manager -n cert-manager
helm uninstall cnpg -n cnpg-system
helm uninstall operator -n minio-operator
helm uninstall rabbitmq-cluster-operator -n rabbitmq-system

3. Delete Operator (Manual)​

kubectl delete deployment tfrs-operator-controller-manager -n tfrs-operator-system

4. Delete Namespaces​

for ns in cert-manager cnpg-system istio-ingress istio-system \
    minio-operator rabbitmq-system tfrs-infra tfr-image-credentials \
    tfrs-operator-system; do
  kubectl delete deployments,statefulsets,daemonsets --all -n $ns 2>/dev/null
done

kubectl delete ns cert-manager cnpg-system istio-ingress istio-system \
    minio-operator rabbitmq-system tfrs-infra tfr-image-credentials \
    tfrs-operator-system

5. Delete Cluster-Scoped Resources​

Kustomize deployment creates RBAC and Webhook cluster-scoped resources, which need manual cleanup:

kubectl get clusterrole,clusterrolebinding,validatingwebhookconfiguration,mutatingwebhookconfiguration \
  -o name | grep tfrs-operator | xargs kubectl delete

6. Delete CRDs​

kubectl delete crd tfrclusters.infra.tfrobotserver.cn \
    tfrtenants.infra.tfrobotserver.cn \
    tfrservers.tfrobot.tfrobotserver.cn

FAQ​

CR Stuck in Terminating State​

Operator deleted before finalizer processed:

kubectl patch <resource> <name> -p '{"metadata":{"finalizers":null}}' --type=merge

Helm Uninstall Failed: CRD Already Deleted​

Premature CRD deletion (e.g., using make undeploy) prevents Helm from recognizing resources:

# Delete Helm release records directly
kubectl delete secret -n <namespace> -l owner=helm

# Example: clean all Helm records in tfrs-infra namespace
kubectl delete secret -n tfrs-infra -l owner=helm

Namespace Deletion Denied​

Some cloud platforms (e.g., Tencent Cloud TKE) forbid deleting namespaces with Pods:

# Delete all workloads first
kubectl delete deployments,statefulsets,daemonsets,replicasets --all -n <namespace>

# Wait for Pods to terminate
kubectl get pods -n <namespace> -w

# Then delete namespace
kubectl delete namespace <namespace>