Data Lineage

Data lineage is the complete traceability of a piece of data's journey from its origin to its final consumption, recording every system, transformation and process it passed through. It answers three questions every organization needs to be able to answer about its information: where a number came from, how it was calculated and which sources feed it.

In practical terms, lineage maps the flow of data across its entire life: the source table in a transactional system, the steps of an ETL or ELT pipeline that clean and combine it, the intermediate view or model where it is aggregated and, at the end, the cell on a dashboard or the row in a report the user sees. That map can be described at the table level (which tables feed which tables) or at the finer detail of the column level (which specific field derives from which other).

The concrete value is that it turns an opaque report into something auditable and explainable. When a number does not add up, instead of manually reviewing dozens of queries, you follow the trail backward until you find the transformation or source that broke it. Keeping that map alive and searchable, instead of rebuilding it by hand every time, is part of what an analytics layer like Metrix solves.

Data lineage answers a question that seems simple but that nobody at most companies can answer with certainty: where exactly did that number on the dashboard come from? As data travels through systems, pipelines and transformations, the trail gets lost, and confidence goes with it. Documenting that journey end to end is what lineage does.

What exactly it records

Good lineage does not just say that two tables are connected: it documents the complete journey and the detail of each hop. In its richest form it includes:

• Origin (source): the system or table where the data was born, for example the CRM, the ERP or a web form.
• Transformations: every step that modified the data along the way, such as a join, a filter, an aggregation, a currency conversion or a business rule applied in a pipeline.
• Destination (target): where it is ultimately consumed, such as a dashboard, a regulatory report, a machine learning model or an export to another system.
• Hop metadata: who defined the logic, when it last ran and with what rules, so the reasoning behind each number can be reconstructed.

Table level and column level

Lineage is described at different levels of granularity. Table-level lineage shows which tables feed which tables: it is fast to generate and useful for understanding the overall flow. Column-level lineage goes down to the individual field: it shows that the "net revenue" column on a dashboard derives from "gross amount" minus "taxes," taken from two different tables. The column level is the most useful for auditing and for impact analysis, but also the most expensive to maintain.

How it is captured

Rebuilding lineage by hand does not scale, so several automatic and manual techniques are combined:

• SQL query parsing: tools that read the pipeline code and infer what feeds what.
• Metadata from integration tools: ETL or ELT engines and orchestration tools already know the dependency map and expose it.
• Manual tagging: documentation added by data teams, essential for capturing business rules.
• Execution and log analysis: observing what actually ran, not just what was defined in the design.

Technical lineage and business lineage

Two views of the same journey coexist. Technical lineage speaks the language of infrastructure: tables, columns, jobs and queries, and it serves data and engineering teams. Business lineage translates that journey into concepts a manager understands: which source system the "monthly sales" metric comes from, which area owns that data and what its official definition is. Complete lineage links both layers, so technical staff and business users look at the same truth from their own angle.

Why it matters to a company in Latin America

Lineage stops being a luxury when data has consequences. In financial services, healthcare, pharmaceuticals or manufacturing, a regulatory report or a board dashboard has to be defensible: if an auditor or a regulator asks how a figure was calculated, you have to be able to show the journey, not improvise it. Lineage also enables impact analysis: before changing a column in a source system, you can see which downstream reports and models will break. And conversely, it enables root cause analysis: when a number shows up wrong, you trace it back to the source that corrupted it. Ultimately, all of this builds the most valuable and most fragile asset of a data team: trust in the reports.

How it differs from related terms

Lineage is often confused with nearby data management concepts, but each one answers a different question:

They are complementary pieces: the catalog inventories, governance sets the rules, quality measures the health of the data and lineage shows the journey that connects everything. In practice, good lineage is also a quality tool, because the trail is exactly what lets you find where a piece of data degraded.

In practice

Lineage becomes tangible the day a number does not add up. A revenue dashboard shows a 12% drop nobody can explain; with column-level lineage, the team follows the trail to a schema change in the billing system that stopped mapping a product category. Without lineage, that means days of reviewing queries blindly; with lineage, it takes minutes. That backward tracing, and its inverse for anticipating what will break before touching something, is what sustains analytics the business truly trusts.