Data Integration
Data Integration is the core of Nexa: it takes your raw data sources and turns them into clean, governed, business-ready datasets. Use it when you’ve connected a source and need to shape it into curated tables and business entities before agents, automations, or analysts consume it. Everything is organized into three layers.
| Layer | Purpose | Example |
|---|---|---|
| Raw (bronze) | Landing zone for source data exactly as ingested | raw.landing.crm_customers |
| Curated (silver) | Cleaned, typed, and standardized | curated.crm.customer |
| Consumption (gold) | Business entities and 360-degree views | consumption.entity.customer_360 |
The Data Flow screen
Section titled “The Data Flow screen”The Data Flow page is your starting point. It lists all your data sources across three tabs.
Sources that have been added but not yet mapped. Each row shows:
- Source Name — click to see the tables and entities within that source
- Description — what this source contains
- Type — User Defined, System Defined, or Unmapped Entities
- Total Entities — count of tables in this source
Unmapped entity rows are highlighted to draw your attention.
Sources where schema changes have been detected since the last mapping run. Shows the source name, the affected entity names, and when the change was last seen.
Active, mapped sources already flowing through your pipelines. Shows the source name and the last update timestamp.
AI-powered mapping and tagging
Section titled “AI-powered mapping and tagging”At the top of the Data Flow screen, two AI processes each show a status card:
- Mapper Run — analyzes your raw schemas and generates curated and consumption layer mappings automatically. It tracks progress through phases: Fetching Raw Context, Mapping Curated Layer, Mapping Consumption Layer, Detecting Conflicts, and Persisting Context.
- Tagger Run — scans your data and applies business classifications, PII (personally identifiable information) tags, and metadata labels.
Both processes show real-time progress with percentage completion. Select View mapper run or View tagger run to watch the live execution console.
Launching a run
Section titled “Launching a run”Select Generate Mapping or Generate Tagging from the New Dataflows tab header. The button is disabled while any run is already in progress. You’re taken to the AI process screen to monitor execution in real time.
Managing sources
Section titled “Managing sources”Adding a source
Section titled “Adding a source”Select Add Source to open the source creation form.
| Field | Description |
|---|---|
| Source Name | Select an existing source or type a new name |
| Description | Free-text description of what this source contains |
Viewing tables in a source
Section titled “Viewing tables in a source”Select any source name to open the Tables & Entities panel. For each table you can:
- Toggle “Send for Curation” — include or exclude a table from the curated mapping process. Excluded tables show an “Excluded from Curation” badge.
- Change Source — reassign a table to a different source if it was miscategorized.
Pattern-based source creation
Section titled “Pattern-based source creation”From the Add Data Flow screen, you can create sources using pattern matching:
- Enter a Source Name and optional Description.
- Choose a pattern type:
- Table pattern — match tables by name (for example,
crm_*or*_bronze_*) - Schema pattern — match entire schemas by name
- Table pattern — match tables by name (for example,
- The preview panel shows matching entities highlighted as you type.
- Submit to create the source and associate all matched entities.
For the full walkthrough, see Adding a Data Source.
AI process screen
Section titled “AI process screen”When you launch a mapping or tagging run, the AI process screen shows:
- Live status with a connection indicator (connected, reconnecting, and so on)
- Progress bar with percentage and current phase
- Execution log — timestamped entries for each processing step, with phase badges (Context, Curated Mapping, Consumption Mapping, Conflict Detection)
- Cancel button to stop a run in progress
When the run completes:
- If conflicts are detected, you see a count and a button to continue to the Conflict Resolution screen.
- If no conflicts, you continue directly to the Curated Layer to review mappings.
If a run fails, an expandable error-details panel shows the traceback for debugging.