Overview
ArgoCD renders Helm templates client-side before applying them to the cluster. Helm’slookup function — which the chart uses to persist auto-generated secrets across upgrades — always returns an empty result in client-side rendering. This means every ArgoCD sync generates new random values for auto-generated secrets, breaking Rails database encryption and session continuity.
Affected Secrets
| Secret Key | Purpose | Generation | Conditional | Format |
|---|---|---|---|---|
SECRET_KEY_BASE | Rails session signing and encryption | randAlphaNum 64 | Always | Single-line alphanumeric string |
ENCRYPTION_KEY | Application-level data encryption | Hex (64 chars via sha256sum) | Always | Single-line alphanumeric string |
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY | Rails Active Record encryption | randAlphaNum 64 | Always | Single-line alphanumeric string |
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY | Rails deterministic encryption | randAlphaNum 64 | Always | Single-line alphanumeric string |
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT | Rails encryption key derivation | randAlphaNum 64 | Always | Single-line alphanumeric string |
REGISTRY_HTTP_SECRET | Internal container registry auth | randAlphaNum 32 | Always | Single-line alphanumeric string |
CREWAI_PLUS_INTERNAL_API_KEY | Service-to-service authentication | randAlphaNum 64 | Always | Single-line alphanumeric string |
WHARF_JWT_SECRET | Wharf trace collector authentication | randAlphaNum 64 | Always (wharf.enabled defaults to true) | Single-line alphanumeric string |
DEPLOYMENT_INSTANCE_JWT_SECRET | Signs JWTs for deployed crew instances | randAlphaNum 64 | Always | Single-line alphanumeric string |
CUBE_JWT_SECRET | Cube analytics service authentication | randAlphaNum 64 | Only when cube.enabled: true | Single-line alphanumeric string |
FACTORY_DEBUG_TOKEN | Authorizes GET /health/debug requests | randAlphaNum 64 | Always | Single-line alphanumeric string |
OIDC_PRIVATE_KEY | RSA signing key for the OIDC IdP | genPrivateKey "rsa" (PEM) | Always — multi-line, requires block scalar | Multi-line RSA PEM — requires YAML block scalar (pipe character |) |
OIDC_KEY_ID | JWKS kid for the OIDC IdP | crewai-oidc-<randAlphaNum 16> | Always — must rotate with OIDC_PRIVATE_KEY | Single-line alphanumeric string |
UV_DEFAULT_INDEX | PyPI registry URL for enterprise crew builds | Not auto-generated — must be set manually from customer portal license credentials | Always for ArgoCD/direct Helm | URL string |
OIDC_PRIVATE_KEY is the only secret requiring block scalar YAML syntax because it is a multi-line RSA PEM string. All other secrets are single-line alphanumeric values. See the Configure in Values example — the
| character on the OIDC_PRIVATE_KEY line is required.OIDC_KEY_ID and OIDC_PRIVATE_KEY must be rotated together. Changing one without the other breaks Workload Identity trust relationships (AWS IAM OIDC, GCP Workload Identity, Azure Federated Credentials) that pin to the key ID.Recommended: Pre-Set All Secrets
Generate stable values and set them explicitly in your Helm values file. This bypasseslookup and randAlphaNum entirely.
Generate Secret Values
The easiest approach is to let the chart generate the values for you. Runhelm template with your values file — since it renders client-side (just like ArgoCD), the chart’s auto-generation logic produces random secrets that you can extract and pin:
<chart> with the path to the chart (e.g., ./helm or an OCI reference). This requires yq — install with brew install yq, snap install yq, or see the yq docs.
This prints the auto-generated values in plain text. Copy them into your values file:
Configure in Values
Alternative: generate with openssl
Alternative: generate with openssl
If you prefer to generate values independently of the chart:
Alternative: ArgoCD ignoreDifferences
You can configure ArgoCD to ignore changes to the Secret resource so that auto-generated values from the initial install are preserved:
Self-Signed TLS Certificates
The chart can auto-generate self-signed TLS certificates (web.tls.autoGenerate: true). These also use lookup for persistence and will regenerate on every ArgoCD sync.
If you use application-level TLS with ArgoCD, provide your own certificates:
web.tls.autoGenerate.
Cloud Provider Workload Identity: crewai-crews Namespace
Both GCP and Azure require thedefault ServiceAccount in the crewai-crews namespace to be annotated for Workload Identity (GKE) or Azure Workload Identity (AKS). The chart creates this namespace via a Helm Job, so the namespace does not exist until after the first sync completes.
Two-Application pattern (recommended):
Create a separate ArgoCD Application that runs after the main CrewAI Application and applies only the SA annotation:
manifests/crewai-crews-sa/sa.yaml:
GCP (GKE Workload Identity):
Dependency Ordering with Sync Waves
For deployments that require infrastructure dependencies (NGINX Ingress Controller, External Secrets Operator) to be ready before CrewAI starts, use multiple ArgoCD Applications with sync waves:pre-install, post-install) already sequence the DB migration job before the main web/worker deployments — no additional sync wave configuration within the chart is needed for internal ordering.
OIDC_PRIVATE_KEY in the secrets table requires block scalar YAML syntax because it is a multi-line RSA PEM string, unlike all other secrets in the table which are single-line alphanumeric strings. See the Configure in Values example above for the correct
| block scalar syntax.