Upgrade Guide

Nexa is deployed as a set of container images rolled out through GitOps (ArgoCD + Kustomize). An “upgrade” means moving the running services to a newer set of image tags and applying any Kustomize or configuration changes that ship with that version. This page is for the platform team that operates a customer-managed deployment; it assumes you already have the nexa-deployments repository, cluster access, and the argocd and kubectl CLIs.

Because DataReadyAI has no access to your environment, you drive every upgrade yourself. The safe path is: read the release notes, back up state, promote the new image tags in the correct environment order, watch the rollout, validate, and roll back if a health check fails.

Before you upgrade

Do all of the following before touching production.

1. Read the release notes for every version between your current one and the target. Breaking changes and required manual actions are listed there. See Release notes.

2. Confirm the version-skip rules. Upgrade one minor version at a time and never skip a major version — required migrations (database schema, config-key renames) are written to run against the immediately preceding version. If you are several versions behind, step through each release in order.

3. Take a backup. Snapshot the application Postgres database (RDS on AWS / the equivalent managed Postgres on Azure) and export the current image tags so you have a known-good point to roll back to. Backup and restore procedures live in the Operations Guide.

4. Capture the current state. Record the image tag each service is running so rollback is a known target:

   kubectl get deploy -A -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.spec.template.spec.containers[0].image}{"\n"}{end}'

5. Upgrade a non-production environment first. Always land the new version in dev and test and validate there before promoting to prod.

Database migrations are not reversible

Some releases run forward-only schema migrations on first start of the new image. Once a migrated service is live, rolling the image back does not undo the schema change. Take the database backup in step 3 before promoting, and restore from it if you must go back.

Supported upgrade paths

From	To	Supported?	Notes
x.y.z	x.y.(z+n)	Yes	Patch releases are drop-in; no manual action unless the notes say so.
x.y.*	x.(y+1).*	Yes	Standard minor upgrade. Read the notes for required actions.
x.y.*	x.(y+2).*	Only via each intermediate minor	Land x.(y+1) first, then x.(y+2).
x.*.*	(x+1).*.*	Only from the latest minor of x	Move to the last x.y release, then to (x+1).0.
any	older version	Downgrade only via rollback (see below), not a forward upgrade	Not supported once migrations have run.

How rollout works

Environments track different image-tag patterns, and only production is promoted by hand:

Environment	Branch / tag pattern	Sync	Who triggers the upgrade
dev	develop-<sha>	Automated (ArgoCD self-heal)	Image Updater writes the new tag to Git on each build
test	main-<sha>	Automated	Image Updater on merge to the release branch
prod	v<x.y.z> (semver)	Manual	You, deliberately, via the steps below

In dev and test, ArgoCD Image Updater detects a new tag in the registry, commits it to the environment’s kustomization.yaml, and ArgoCD syncs it. Production has Image Updater disabled and automated sync off (prune: false, selfHeal: false) — nothing changes until you edit the tag and sync.

Upgrading production

1. Set the target tag in the production overlay. Edit the image tag in the production overlay for each service you are upgrading, on the main branch of nexa-deployments:

   overlays/prod/nexa-databricks-api/kustomization.yaml

   images:
     - name: nexa-databricks-api
       newName: <your-registry>/aws-iac-nexa-databricks-api
       newTag: v1.4.0   # was v1.3.2

   Repeat for every service in the release (for example nexa-web, nexa-backend, nexa-databricks-api or nexa-snowflake-api, nexa-agents-api). Upgrade all services listed in the release notes as a set — mismatched versions across the API and web tiers are not supported.

2. Preview the rendered manifests before committing, to confirm only the image tag (and any expected config) changed:

   kustomize build overlays/prod/nexa-databricks-api | grep image:

   Expected output shows the new tag:

   image: <your-registry>/aws-iac-nexa-databricks-api:v1.4.0

3. Commit and push to main. This is the audit record of the upgrade:

   git add overlays/prod/ && git commit -m "chore: upgrade prod to v1.4.0" && git push origin main

4. Sync the applications in ArgoCD, one at a time. Production sync is manual:

   argocd app sync nexa-databricks-api-prod

   Watch it reach Synced / Healthy:

   argocd app get nexa-databricks-api-prod

   Expected (truncated):

   Health Status:   Healthy
   Sync Status:     Synced to main (<commit>)

5. Wait for the rolling update to finish for each Deployment before syncing the next service:

   kubectl rollout status deployment/prod-nexa-databricks-api -n databricks-prod --timeout=300s

   Expected:

   deployment "prod-nexa-databricks-api" successfully rolled out

Tip

Sync back-end/API services before nexa-web so the UI never talks to an API that is still mid-rollout. Kubernetes does a rolling update by default (old pods stay up until new ones are Ready), so a healthy upgrade has no downtime.

Post-upgrade validation

After the rollout completes, confirm the new version is actually serving traffic:

1. Check pod health and image tags across the namespace:

   kubectl get pods -n databricks-prod
   kubectl get deploy -n databricks-prod -o jsonpath='{.items[*].spec.template.spec.containers[0].image}{"\n"}'

   Every pod should be Running and READY 1/1, on the new tag.

2. Hit the service health endpoints. Each API exposes a health/readiness probe path (used by the Kubernetes livenessProbe / readinessProbe); confirm they return healthy through the gateway.

3. Run a smoke test through the web UI — sign in, load a pipeline or agent, and confirm data-platform connectivity (a query against Snowflake or Databricks succeeds).

4. Watch logs for the first few minutes for crash loops or migration errors. If something looks wrong, collect a diagnostic bundle before rolling back — see Troubleshooting & Diagnostics.

Rollback

If validation fails, roll back promptly. Choose the method that fits how far the change has propagated.

Revert the Git change (preferred)

Reverting the tag in Git keeps ArgoCD as the source of truth:

git revert <upgrade-commit-sha> && git push origin main
argocd app sync nexa-databricks-api-prod
kubectl rollout status deployment/prod-nexa-databricks-api -n databricks-prod

ArgoCD app rollback

Roll the application to a previous synced revision (history is retained per revisionHistoryLimit, default 10):

argocd app history nexa-databricks-api-prod
argocd app rollback nexa-databricks-api-prod <history-id>

kubectl rollout undo (emergency)

For an immediate revert of a single Deployment when you cannot reach Git/ArgoCD. Re-sync from Git afterward so cluster state matches the repo:

kubectl rollout undo deployment/prod-nexa-databricks-api -n databricks-prod

Rollback does not undo data migrations

An image rollback returns the code to the prior version, but any forward-only database migration that already ran stays applied. If the release ran a schema migration, restore the database from the pre-upgrade backup (Operations Guide) instead of relying on an image rollback alone.

Related

Release notesPer-version breaking changes and required actions — read before every upgrade.

Operations GuideBackup and restore procedures referenced above.

Uninstall / TeardownFull removal when you are decommissioning, not upgrading.