Lineage

New Lineage experience is available from 5.41 (August 2024). Go to your Profile Settings and enable New Lineage Experience

K provides context and trust to your data items by helping you understand the lineage and relationships between data items.

You can see where your data item sources data from, where it flows to, how it joins to other data items.

Lineage crosses different item types, from tables to reports, across different sources. K automatically generates lineage across these item. Where it can’t, lineage can be manually added via the UI or via the API.

The different object types can be easily identified by their colour on the map.

Tables/Columns are coloured Blue

Reports and sheets are coloured Orange

Datasets, Dataset Tables and Dataset Fields are coloured Green

Pipelines, ML Models, and Files are coloured Red.

Lineage View

You can access Lineage by going to any profile page and clicking on the Lineage tab

Navigating the Lineage chart

The lineage chart should be read Left to Right. Items to the left are upstream (they provide data to) of the assets on the right.

1. Some items have lineage that can be displayed. They will appear with a + sign that you can click to expand

2. You can click automated links to see the details to join the two items. This might be code or a pipeline that connects the two

3. The current target you are looking at will be highlighted in Blue

There are more display options in the Map view tool bar.

1. Display options include the ability to

   1. Show reference: Reference objects can be toggled to display on or off from the map

   2. Show active only: You can toggle deleted items on or off from the map. Lineage for deleted items will have this toggle turned on.

Accessing more detail

When you click on a item in the map or an automated link (full line) a side panel will appear to provide additional details and quickly update any information in the side panel, including:

• Details about the data item including any tags, collections, and descriptions

• Properties about the data item include its type, trust score, classification etc

When you click on a specific link in a lineage map, a side panel will pop up showing:

• The data item (code, pipelines) that join the two data items

• Details of the data item like the actual code used

Quickly adding data items to lists, glossaries and collections

As you navigate through the lineage of a data item, you may want to quickly add this item to a group e.g. a list or collection. You can easily do this by right clicking on any data item (excluding reference tables) in the lineage view or directly updating information in the pop-up side panels.

Advanced Lineage Filter

When navigating a lineage map, there can sometimes be lots of data assets.

K Platform has the ability for you to filter and selectively show the assets that you are interested in seeing. You can toggle the filter to act either as ‘AND' / ‘OR'.

Updating Lineage Data

Sometimes lineage may be missing on a table (or other item). To update the Lineage information, first go to the Data Item’s Profile Page

1. Click on the Lineage tab

2. Click on the asset you want to edit the lineage for. The sidebar will appear

3. In the sidebar click Edit Lineage and a pop-up box will appear

4. Use the form to add/remove manual linage.

   1. You can add assets Upstream or Downstream