Skip to main content

Overview

ArgoCD renders Helm templates client-side before applying them to the cluster. Helm’s lookup function — which the chart uses to persist auto-generated secrets across upgrades — always returns an empty result in client-side rendering. This means every ArgoCD sync generates new random values for auto-generated secrets, breaking Rails database encryption and session continuity.

Affected Secrets

Secret KeyPurposeGenerationConditionalFormat
SECRET_KEY_BASERails session signing and encryptionrandAlphaNum 64AlwaysSingle-line alphanumeric string
ENCRYPTION_KEYApplication-level data encryptionHex (64 chars via sha256sum)AlwaysSingle-line alphanumeric string
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEYRails Active Record encryptionrandAlphaNum 64AlwaysSingle-line alphanumeric string
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEYRails deterministic encryptionrandAlphaNum 64AlwaysSingle-line alphanumeric string
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALTRails encryption key derivationrandAlphaNum 64AlwaysSingle-line alphanumeric string
REGISTRY_HTTP_SECRETInternal container registry authrandAlphaNum 32AlwaysSingle-line alphanumeric string
CREWAI_PLUS_INTERNAL_API_KEYService-to-service authenticationrandAlphaNum 64AlwaysSingle-line alphanumeric string
WHARF_JWT_SECRETWharf trace collector authenticationrandAlphaNum 64Always (wharf.enabled defaults to true)Single-line alphanumeric string
DEPLOYMENT_INSTANCE_JWT_SECRETSigns JWTs for deployed crew instancesrandAlphaNum 64AlwaysSingle-line alphanumeric string
CUBE_JWT_SECRETCube analytics service authenticationrandAlphaNum 64Only when cube.enabled: trueSingle-line alphanumeric string
FACTORY_DEBUG_TOKENAuthorizes GET /health/debug requestsrandAlphaNum 64AlwaysSingle-line alphanumeric string
OIDC_PRIVATE_KEYRSA signing key for the OIDC IdPgenPrivateKey "rsa" (PEM)Always — multi-line, requires block scalarMulti-line RSA PEM — requires YAML block scalar (pipe character |)
OIDC_KEY_IDJWKS kid for the OIDC IdPcrewai-oidc-<randAlphaNum 16>Always — must rotate with OIDC_PRIVATE_KEYSingle-line alphanumeric string
UV_DEFAULT_INDEXPyPI registry URL for enterprise crew buildsNot auto-generated — must be set manually from customer portal license credentialsAlways for ArgoCD/direct HelmURL string
OIDC_PRIVATE_KEY is the only secret requiring block scalar YAML syntax because it is a multi-line RSA PEM string. All other secrets are single-line alphanumeric values. See the Configure in Values example — the | character on the OIDC_PRIVATE_KEY line is required.OIDC_KEY_ID and OIDC_PRIVATE_KEY must be rotated together. Changing one without the other breaks Workload Identity trust relationships (AWS IAM OIDC, GCP Workload Identity, Azure Federated Credentials) that pin to the key ID.
If any Active Record encryption key changes, all previously encrypted data becomes unreadable. This includes encrypted database columns that Rails cannot decrypt with new keys. Always pre-set these values before your first deployment.
UV_DEFAULT_INDEX is not auto-generated — it is auto-populated from your Replicated license in KOTS deployments only. When deploying via ArgoCD (direct Helm/OCI install), UV_DEFAULT_INDEX is never set by the chart. You must add it manually under secrets: in your values file. See the Secrets Reference for how to obtain the value.
Generate stable values and set them explicitly in your Helm values file. This bypasses lookup and randAlphaNum entirely.

Generate Secret Values

The easiest approach is to let the chart generate the values for you. Run helm template with your values file — since it renders client-side (just like ArgoCD), the chart’s auto-generation logic produces random secrets that you can extract and pin:
helm template crewai-platform <chart> --values my-values.yaml \
  | yq 'select(.kind == "Secret" and (.metadata.name | contains("-secrets"))) .data
        | to_entries | .[]
        | .key + ": " + (.value | @base64d)' \
  | grep -E '^(SECRET_KEY_BASE|ENCRYPTION_KEY|ACTIVE_RECORD_ENCRYPTION_|REGISTRY_HTTP_SECRET|CREWAI_PLUS_INTERNAL_API_KEY|WHARF_JWT_SECRET|DEPLOYMENT_INSTANCE_JWT_SECRET|CUBE_JWT_SECRET|FACTORY_DEBUG_TOKEN|OIDC_PRIVATE_KEY|OIDC_KEY_ID)'
Replace <chart> with the path to the chart (e.g., ./helm or an OCI reference). This requires yq — install with brew install yq, snap install yq, or see the yq docs. This prints the auto-generated values in plain text. Copy them into your values file:

Configure in Values

secrets:
  SECRET_KEY_BASE: "<from output above>"
  ENCRYPTION_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY: "<from output above>"
  ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT: "<from output above>"
  REGISTRY_HTTP_SECRET: "<from output above>"
  CREWAI_PLUS_INTERNAL_API_KEY: "<from output above>"
  WHARF_JWT_SECRET: "<from output above>"
  DEPLOYMENT_INSTANCE_JWT_SECRET: "<from output above>"
  CUBE_JWT_SECRET: "<from output above>"
  FACTORY_DEBUG_TOKEN: "<from output above>"
  OIDC_PRIVATE_KEY: |
    <from output above — multi-line PEM>
  OIDC_KEY_ID: "<from output above>"
Store these values in a sealed secret, SOPS-encrypted file, or your CI/CD platform’s secret management — never commit them to version control in plaintext.
If you prefer to generate values independently of the chart:
# Rails secret key base
openssl rand -base64 48 | tr -d '\n'

# Encryption key (hex format)
openssl rand -hex 32

# Active Record encryption keys (run 3 times, one for each key)
openssl rand -base64 48 | tr -d '\n'

# Registry HTTP secret
openssl rand -base64 24 | tr -d '\n'

# Internal API key
openssl rand -base64 48 | tr -d '\n'

# Deployment instance JWT secret
openssl rand -base64 48 | tr -d '\n'

# OIDC signing key (PEM)
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048

# OIDC key id
echo "crewai-oidc-$(openssl rand -hex 8)"

Alternative: ArgoCD ignoreDifferences

You can configure ArgoCD to ignore changes to the Secret resource so that auto-generated values from the initial install are preserved:
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  ignoreDifferences:
    - group: ""
      kind: Secret
      name: crewai-platform-secrets
      jsonPointers:
        - /data
This only works if the Secret already exists from a prior install. On a fresh deployment through ArgoCD, the first sync will still generate random values — and those values will persist. However, any intentional secret changes in your values file will also be ignored. Pre-setting secrets is the more reliable approach.

Self-Signed TLS Certificates

The chart can auto-generate self-signed TLS certificates (web.tls.autoGenerate: true). These also use lookup for persistence and will regenerate on every ArgoCD sync. If you use application-level TLS with ArgoCD, provide your own certificates:
secrets:
  SSL_PRIVATE_KEY: |
    -----BEGIN PRIVATE KEY-----
    ...
    -----END PRIVATE KEY-----
  SSL_CERTIFICATE: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
Or use cert-manager or your cloud provider’s certificate management instead of web.tls.autoGenerate.

Cloud Provider Workload Identity: crewai-crews Namespace

Both GCP and Azure require the default ServiceAccount in the crewai-crews namespace to be annotated for Workload Identity (GKE) or Azure Workload Identity (AKS). The chart creates this namespace via a Helm Job, so the namespace does not exist until after the first sync completes. Two-Application pattern (recommended): Create a separate ArgoCD Application that runs after the main CrewAI Application and applies only the SA annotation:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crewai-crews-sa-annotation
  annotations:
    argocd.argoproj.io/sync-wave: "2"  # Run after main CrewAI app (wave 1)
spec:
  source:
    repoURL: <your-repo>
    path: manifests/crewai-crews-sa  # Contains only the annotated ServiceAccount
  syncPolicy:
    automated:
      prune: false  # Never delete the SA
Create manifests/crewai-crews-sa/sa.yaml: GCP (GKE Workload Identity):
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: crewai-crews
  annotations:
    iam.gke.io/gcp-service-account: crewai-platform@your-project.iam.gserviceaccount.com
Azure (Workload Identity):
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: crewai-crews
  annotations:
    azure.workload.identity/client-id: "<IDENTITY_CLIENT_ID>"
The first sync will fail to apply the SA annotation (namespace does not yet exist). After the main CrewAI Application syncs successfully and the namespace exists, re-sync the annotation Application.

Dependency Ordering with Sync Waves

Do NOT add argocd.argoproj.io/sync-wave annotations to your Helm values.yaml. Helm has no podAnnotations top-level key — this is a silently-ignored unknown value. Sync wave annotations belong on ArgoCD Application manifests in your Git repository, not in chart values.
For deployments that require infrastructure dependencies (NGINX Ingress Controller, External Secrets Operator) to be ready before CrewAI starts, use multiple ArgoCD Applications with sync waves:
# Application 1: Infrastructure (wave 0 — default, runs first)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ingress-nginx
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  source:
    repoURL: https://kubernetes.github.io/ingress-nginx
    chart: ingress-nginx
    targetRevision: "4.x.x"
  ...

# Application 2: CrewAI Platform (wave 1 — runs after infrastructure)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crewai-platform
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  source:
    repoURL: oci://registry.crewai.com/crewai/stable
    chart: crewai-platform
    targetRevision: "0.x.x"
  ...
The chart’s built-in Helm hooks (pre-install, post-install) already sequence the DB migration job before the main web/worker deployments — no additional sync wave configuration within the chart is needed for internal ordering.
OIDC_PRIVATE_KEY in the secrets table requires block scalar YAML syntax because it is a multi-line RSA PEM string, unlike all other secrets in the table which are single-line alphanumeric strings. See the Configure in Values example above for the correct | block scalar syntax.