Version: Next 🚧

Storage Architecture

This document describes the data storage solutions managed by TFRS Operator, including middleware deployment patterns, storage configuration priority, and isolation models.

Storage Configuration Priority

TFRS Operator uses a three-level priority chain to resolve storage configuration:

Component-level config > Tenant-level config > System default

Priority	Source	Example
1 - Component	`spec.postgresql.storage`, `spec.redis.storage`, etc.	Specify `100Gi` with `cbs-ssd` StorageClass for PostgreSQL only
2 - Tenant	`spec.storage`	Tenant-wide default StorageClass and size
3 - System Default	Hardcoded	`10Gi`, cluster default StorageClass

Configuration example in TFRTenant:

apiVersion: infra.tfrobot.io/v1
kind: TFRTenant
spec:
  # Tenant-level default (priority 2)
  storage:
    storageClass: "cbs-ssd"
    size: "50Gi"
  # Component-level override (priority 1)
  postgresql:
    storage:
      storageClass: "cbs-ssd-enhanced"
      size: "200Gi"
  redis:
    storage:
      size: "20Gi"   # Uses tenant-level storageClass: cbs-ssd

PostgreSQL

Deployment Method

Managed through CloudNativePG (CNPG). TFRTenant creates an independent CNPG Cluster for each tenant:

Configuration	Value
Cluster Name	`{namespace}-pg`
Image	`ghcr.io/cloudnative-pg/postgresql:16-minimal-bookworm`
Credentials Secret	`{namespace}-pg-credentials`
Default Storage	10Gi (overridable via priority chain)

Credential Management

username: User-specified (cannot be postgres, reserved by CNPG)
password: User-specified or auto-generated by the Operator

PostgreSQL Isolation Level Warning

TFRS Operator only provides database-level isolation, not schema-level isolation.

Each TFRTenant gets an independent CNPG Cluster (separate Pod and PVC), achieving database instance-level isolation. However, when multiple TFRServers share the same database, schema isolation must be managed at the application layer.

Responsibility	Owner	Automation Level
Database creation	Operator (automatic)	Fully automated
Schema creation	Python application (db-init script)	Semi-automated (triggered by TFRServer Job)
Schema naming & isolation	Operations team (via `TFS_ROBOT__DB_SCHEMA` config)	Manual configuration
Schema conflict detection	None	No validation mechanism

Risk: If two TFRServers are configured with the same TFS_ROBOT__DB_SCHEMA, they will share the same schema, potentially causing data conflicts. Operations personnel must ensure schema name uniqueness manually.

Redis

Deployment Method

Deployed as a Kubernetes Deployment + PVC, not managed by an Operator:

Configuration	Value
Image	`redis/redis-stack-server:7.2.0-v17`
PVC Name	`redis-data`
Port	6379
UI	RedisInsight (`redislabs/redisinsight:latest`, port 5540)
Default Storage	10Gi (overridable via priority chain)

Redis uses a single-instance deployment model with one Deployment per tenant. Redis Sentinel or Cluster mode is not used.

RabbitMQ

Deployment Method

Managed through RabbitMQ Cluster Operator and Messaging Topology Operator:

Operators installed by TFRCluster:

rabbitmq-cluster-operator: Manages RabbitMQ cluster lifecycle
rabbitmq-messaging-topology-operator: Manages Vhost, User, Permission and other topology resources

Instances created by TFRTenant:

Resource Type	Description
RabbitmqCluster	Tenant-level RabbitMQ instance
Vhost	Virtual host isolation
User	Application access credentials
Permission	Vhost-level access permissions

Each tenant owns an independent RabbitMQ cluster instance, with Vhost providing an additional logical isolation layer.

MinIO

Deployment Method

Managed through MinIO Operator. TFRTenant creates a MinIO Tenant CR for each tenant:

Configuration	Value
Image	`minio/minio:RELEASE.2025-07-23T15-54-02Z`
Default Credentials	`minioadmin` / `minioadmin`
TLS	Disabled by default (relies on Istio mTLS for inter-service encryption)
Replicas	Configurable (default: 1)
Port	9000 (API)
Default Storage	10Gi (overridable via priority chain)

Per-Robot Credentials

TFRServer creates independent Buckets and access credentials for each robot instance through the MinIO Init Job, achieving logical isolation of object storage.

Shared Cache (SharedCache)

Used to share image extraction cache between multiple TFRServer instances and TFRSUtils.

Configuration

Enable in TFRTenant:

spec:
  sharedCache:
    enabled: true
    pvcName: "image-extracted-dir"     # Default value
    storageClass: "cfs"                # Must support ReadWriteMany
    size: "100Gi"
    cacheDir: "~/.tfrobot/documents/extracted"  # Default value

Requirements

Requirement	Description
AccessMode	ReadWriteMany (RWX)
StorageClass	Must support RWX (e.g., NFS, CephFS, Tencent Cloud CFS)
Mount Path	Configured via `cacheDir`

Automatic Environment Variable Injection

When SharedCache is enabled, the Operator automatically injects the following environment variables into TFRServer Pods:

Variable	Value Source	Description
`TFC_IMAGE__SHARED_CACHE`	`sharedCache.cacheDir`	Shared cache root directory
`TFS_ROBOT__DOC_EXTRACTED_IMAGES_DIR`	VolumeMount.MountPath	Image extraction directory (highest priority, overrides ConfigMap)

Other Storage Components

Component	Deployment Method	Storage Type	Description
TimescaleDB	CNPG + Extension	PVC	Time-series data, independent from primary PostgreSQL
Jaeger	Helm Chart	PVC	Distributed tracing data
Grafana	Helm Chart	PVC	Dashboard configuration and data

TimescaleDB uses an independent CNPG Cluster with credentials separate from the primary PostgreSQL. The username similarly cannot be postgres.

Isolation Model Summary

Component	Isolation Level	Isolation Method	Managed By
PostgreSQL	Instance-level	Independent CNPG Cluster per tenant (Pod + PVC)	Operator
PostgreSQL Schema	Application-level	`TFS_ROBOT__DB_SCHEMA` environment variable	Operations (manual)
Redis	Instance-level	Independent Deployment + PVC per tenant	Operator
RabbitMQ	Instance + Logical	Independent Cluster + Vhost isolation	Operator
MinIO	Instance + Logical	Independent Tenant + per-robot Bucket/credentials	Operator
SharedCache	Tenant-level	RWX PVC within tenant namespace	Operator
Network	Namespace-level	mTLS + AuthorizationPolicy	Operator