Version: Next 🚧

TFRCluster Deployment

Deploy TFRCluster cluster-level infrastructure.

Overview

TFRCluster is a cluster-scoped resource that installs and manages the following infrastructure:

Istio - Service mesh and Ingress Gateway
CloudNativePG - PostgreSQL database Operator
MinIO - Object storage Operator
RabbitMQ - Message queue Operator
Cert-Manager - TLS certificate management

Important

Only one TFRCluster is needed per Kubernetes cluster.

Configuration

Core Fields

Field	Description	Example
`name`	TFRCluster resource name	`staging-cluster`
`clusterType`	Cluster type	`kind`, `docker-desktop`, `tke`
`operators`	Infrastructure Operators	See below
`clusterConfig`	Network and cloud provider config	See below
`imageRegistries`	Private registry credentials	See below

Operators Configuration

operators:
  cnpg:
    enabled: true      # CloudNativePG (PostgreSQL)
  minio:
    enabled: true      # MinIO (Object storage)
  rabbitmq:
    enabled: true      # RabbitMQ (Message queue)

Current Limitation

All three Operators must be enabled. Future versions will support external managed services (e.g., cloud provider RDS, object storage).

Network Configuration

Choose network mode based on cluster type:

Local Development (Kind / Docker Desktop): Use NodePort

clusterType: kind
clusterConfig:
  nodePort:
    http: 30080
    https: 30443

Tencent Cloud TKE: Use existing CLB

clusterType: tke
clusterConfig:
  tke:
    clbId: "lb-xxxxxxxx"
    annotationsKey: "service.kubernetes.io/tke-existed-lbid"

CLB Configuration

For TKE, configure listeners in Tencent Cloud CLB console:

HTTPS 443 → Backend 80 (TLS termination at CLB)
HTTP 80 → Backend 80 (optional)
Health check → Backend 15021 (auto-configured)

CLB Cluster Binding Issue

When reusing an existing CLB after recreating a TKE cluster, you may encounter:

ErrorCode: E4013
Details: The loadbalancer is used by other cluster.

Cause: Tencent Cloud CLB is tagged with tke-bindedLBs and similar tags recording the bound cluster ID. When deleting the old cluster, TKE does not automatically clean up these tags on the CLB, preventing the new cluster from binding.

Solutions:

In CLB console → Basic Info → Tags, delete tags pointing to the old cluster (e.g., tke-bindedLBs, tke-bindedService)
Or in CLB console → Listener Management, delete backend service configurations left by the old cluster
After cleanup, delete the istio-ingress Service to trigger re-binding, or redeploy TFRCluster

Private Registry Credentials (Optional)

imageRegistries:
  - secretName: "tfrs-private-registry"
    url: "docker.cnb.cool"
    username: "cnb"
    # password passed via --set, not stored in config file

Note

imageRegistries is not for TFRCluster middleware (they use public images), but prepares image pull credentials for TFRTenant/TFRServer application components. Credentials are created in tfr-image-credentials namespace, shared by all tenants.

Example Configurations

Project provides Tencent Cloud TKE example configurations:

chart/tfrcluster/values-staging.yaml - Staging environment
chart/tfrcluster/values-prod.yaml - Production environment

Other Cloud Providers

Example configurations target Tencent Cloud TKE. Deploying to other platforms (AWS EKS, Alibaba Cloud ACK, etc.) requires network configuration adjustments.

Kustomize Deployment

Create CR Manifest

# tfrcluster-staging.yaml
apiVersion: infra.tfrobotserver.cn/v1
kind: TFRCluster
metadata:
  name: staging-cluster
spec:
  clusterType: tke

  ingress:
    hosts:
      - "*"
    gatewayRef:
      namespace: istio-ingress
      name: default-gateway

  operators:
    cnpg:
      enabled: true
    minio:
      enabled: true
    rabbitmq:
      enabled: true

  clusterConfig:
    tke:
      clbId: "lb-xxxxxxxx"
      annotationsKey: "service.kubernetes.io/tke-existed-lbid"
    istio:
      accessLog:
        enabled: true
        format: "TEXT"
        providers:
          - "envoy"

Deploy

kubectl apply -f tfrcluster-staging.yaml
# tfrcluster.infra.tfrobotserver.cn/staging-cluster created

Verify

kubectl get tfrcluster
# NAME              AGE
# staging-cluster   2m

# Check component status
kubectl get tfrcluster staging-cluster -o jsonpath='{.status.components}' | jq

Check Deployment Completion

TFRCluster uses the Available condition in status.conditions to indicate deployment status:

Real-time deployment monitoring

while true; do clear; echo "--- $(date +%H:%M:%S) ---"; \
  echo "=== Conditions ==="; \
  kubectl get tfrcluster staging-cluster -o json 2>/dev/null \
    | jq -r '.status.conditions[]? | "\(.type): \(.status) - \(.message)"'; \
  echo; echo "=== Components ==="; \
  kubectl get tfrcluster staging-cluster -o json 2>/dev/null \
    | jq -r '.status.components // {} | to_entries[] | "\(.key): \(.value.state)"'; \
  sleep 5; done

Press Ctrl+C to exit.

# Check Available condition
kubectl get tfrcluster staging-cluster -o jsonpath='{.status.conditions[?(@.type=="Available")].status}'
# True = deployment complete, False = deploying or failed

# Automation script: wait for deployment (10 min timeout)
kubectl wait tfrcluster staging-cluster --for=condition=Available --timeout=600s

Uninstall

kubectl delete tfrcluster staging-cluster

Helm Deployment

Deploy

Install from OCI Registry

helm install tfrcluster oci://helm.cnb.cool/turingfocus/tfrs-operator/tfrcluster \
  --version 0.1.0 \
  --namespace tfrs-infra \
  --create-namespace \
  --set name=staging-cluster \
  --set clusterType=tke \
  --set 'clusterConfig.tke.clbId=lb-xxxxxxxx' \
  --set 'clusterConfig.istio.accessLog.enabled=true'

Install with Values File

helm install tfrcluster oci://helm.cnb.cool/turingfocus/tfrs-operator/tfrcluster \
  --version 0.1.0 \
  --namespace tfrs-infra \
  --create-namespace \
  -f values-staging.yaml \
  --set 'imageRegistries[0].password=<PASSWORD>'

Install from Local Source

helm install tfrcluster ./chart/tfrcluster \
  --namespace tfrs-infra \
  --create-namespace \
  -f chart/tfrcluster/values-staging.yaml \
  --set 'imageRegistries[0].password=<PASSWORD>'

Verify

# Check TFRCluster status
kubectl get tfrcluster

# Wait for deployment (10 min timeout)
kubectl wait tfrcluster staging-cluster --for=condition=Available --timeout=600s

# Check installed Operators
kubectl get pods -n cnpg-system
kubectl get pods -n minio-operator
kubectl get pods -n rabbitmq-system
kubectl get pods -n istio-system
kubectl get pods -n cert-manager

Check Deployment Completion

TFRCluster uses the Available condition in status.conditions to indicate deployment status:

Real-time deployment monitoring

while true; do clear; echo "--- $(date +%H:%M:%S) ---"; \
  echo "=== Conditions ==="; \
  kubectl get tfrcluster staging-cluster -o json 2>/dev/null \
    | jq -r '.status.conditions[]? | "\(.type): \(.status) - \(.message)"'; \
  echo; echo "=== Components ==="; \
  kubectl get tfrcluster staging-cluster -o json 2>/dev/null \
    | jq -r '.status.components // {} | to_entries[] | "\(.key): \(.value.state)"'; \
  sleep 5; done

Press Ctrl+C to exit.

# Check Available condition
kubectl get tfrcluster staging-cluster -o jsonpath='{.status.conditions[?(@.type=="Available")].status}'
# True = deployment complete, False = deploying or failed

# Automation script: wait for deployment (10 min timeout)
kubectl wait tfrcluster staging-cluster --for=condition=Available --timeout=600s

Uninstall

helm uninstall tfrcluster -n tfrs-infra

Note

Uninstalling TFRCluster removes all infrastructure components. Ensure all TFRTenant and TFRServer resources are deleted first.

Overview​

Configuration​

Core Fields​

Operators Configuration​

Network Configuration​

Private Registry Credentials (Optional)​

Example Configurations​

Kustomize Deployment​

Create CR Manifest​

Deploy​

Verify​

Check Deployment Completion​

Uninstall​

Helm Deployment​

Deploy​

Verify​

Check Deployment Completion​

Uninstall​

Overview

Configuration

Core Fields

Operators Configuration

Network Configuration

Private Registry Credentials (Optional)

Example Configurations

Kustomize Deployment

Create CR Manifest

Deploy

Verify

Check Deployment Completion

Uninstall

Helm Deployment

Deploy

Verify

Check Deployment Completion

Uninstall