Pipelines
A pipeline is the executable data-transformation code Nexa generates from your Raw → Curated → Consumption mappings, then deploys and runs on your data platform. On Databricks these are DLT (Delta Live Tables) pipelines; on Snowflake they are the equivalent scheduled transformation objects. Use the Pipelines screen (Jobs & Pipeline in the sidebar → Pipelines) to monitor the pipelines Nexa manages, and use the mapping screens to (re)generate their code.
Where pipeline code comes from
Section titled “Where pipeline code comes from”You don’t hand-write pipeline code in Nexa. It is generated from the mappings you define across the three layers:
| Layer | What you define | What it generates |
|---|---|---|
| Raw (bronze) | Sources and ingested tables | Landing/ingestion into raw tables |
| Curated (silver) | Raw → curated field mappings and cleaning rules | Transform code into typed, standardized curated tables |
| Consumption (gold) | Curated → consumption entity mappings and ETL logic | Transform code into business entities and 360° views |
The generation itself happens on the mapping and canvas screens, not on the Pipelines screen:
- Define and refine mappings on the Curated Layer and Consumption Layer screens.
- Author per-column ETL (extract, transform, load) and review generated SQL and PySpark on the Canvas.
- Generate the full set of artifacts from the Curated Landing Zone using Generate all Schemas, Entities, Mappings, Code Pipelines and Deployment Artefacts.
Once generated and deployed, the pipeline appears in the Pipelines list as a Nexa-managed pipeline. The Pipelines screen is scoped to those pipelines (it queries with nexa_managed=true), so it shows what Nexa created and deploys — not every pipeline in your workspace.
The Pipelines list
Section titled “The Pipelines list”The Pipelines list shows every Nexa-managed pipeline with its latest run state. Each row has:
| Column | Meaning |
|---|---|
| ID | Platform pipeline identifier |
| Name | Pipeline name (select it to open the detail panel) |
| Mode | Pipeline mode, for example triggered or continuous |
| Status | Latest update state, such as running, succeeded, failed, or no updates |
| Last Updated | Relative time of the last update, for example 2h ago |
| Actions | For a failed pipeline, a Fix with Nexa button that starts AI-assisted diagnosis |
Filter the list with the Filter dropdown:
- Status Filter — All Statuses, Running, Failed, Canceled, Succeeded, No Updates.
- Sort By — Name, Updated At, Created At, or Last Update Time, each ascending or descending.
Use the search box to find a pipeline by ID or name. A pipeline whose last update failed is shown with its name in red.
Pipeline detail
Section titled “Pipeline detail”Select a pipeline name to open its detail panel, which has two tabs.
- Pipeline Details — Pipeline ID, Creator, Run as, Tags, Health, and the target definition: Catalog, Schema, Channel, and whether it runs Serverless. An external-link icon opens the pipeline in your data platform.
- Updates — the update (run) history. Each entry shows the update ID, its state (a failed update is flagged), and its creation time. Select Load More to page through older updates.
Jobs vs. pipelines
Section titled “Jobs vs. pipelines”Pipelines transform data; jobs orchestrate and schedule work, including running pipelines. The two are managed side by side under Jobs & Pipeline.
Pipelines are DLT pipelines. The detail panel exposes DLT-specific fields such as Channel and Serverless, and the external-link icon opens the pipeline in the Databricks workspace.
The section is labeled Tasks & Pipeline, and the generated transformation objects are surfaced through the equivalent Snowflake constructs. The list, filters, and detail panel behave the same way.