Lineage
New Lineage experience is available from 5.41 (August 2024). Go to your Profile Settings and enable New Lineage Experience
K provides context and trust to your data items by helping you understand the lineage and relationships between data items.
You can see where your data item sources data from, where it flows to, how it joins to other data items.
Lineage crosses different item types, from tables to reports, across different sources. K automatically generates lineage across these item. Where it can’t, lineage can be manually added via the UI or via the API.
Lineage View
You can access Lineage by going to any profile page and clicking on the Lineage tab
Lineage View Types
There are several types of views for lineage
View | Description | Best for |
|---|---|---|
Map | The map view provides a visual view of lineage between tables, reports, pipelines etc. | To check where a specific item sources its data from, and if those sources are trusted. |
List | The list view provides the list of immediate up and downstream items linked to the object that you are currently viewing. | When too many objects are linked and the map view is not easy to use. |
Navigating the Lineage chart
The lineage chart should be read Left to Right. Items to the left are upstream (they provide data to) of the assets on the right.
Lineage example
Some items have lineage that can be displayed. They will appear with a + sign that you can click to expand
You can click automated links to see the details to join the two items. This might be code or a pipeline that connects the two
The current target you are looking at will be highlighted in Blue
There are more display options in the Map view tool bar.
Display options include the ability to
Show reference: Reference objects can be toggled to display on or off from the map
Show active only: You can toggle deleted items on or off from the map. Lineage for deleted items will have this toggle turned on.
Show columns/fields: You can toggle columns/fields on or off from the map to simplify complex views.
Show full name: You can toggle full names if many of the map items have long names.
Accessing more detail
When you click on a item in the map or an automated link (full line) a side panel will appear to provide additional details and quickly update any information in the side panel, including:
Details about the data item including any tags, collections, and descriptions
Properties about the data item include its type, trust score, classification etc
When you click on a specific link in a lineage map, a side panel will pop up showing:
The data item (code, pipelines) that join the two data items
Details of the data item like the actual code used
Quickly adding data items to lists, glossaries and collections
As you navigate through the lineage of a data item, you may want to quickly add this item to a group e.g. a list or collection. You can easily do this by right clicking on any data item (excluding reference tables) in the lineage view or directly updating information in the pop-up side panels.
Advanced Lineage Filter
When navigating a lineage map, there can sometimes be lots of data assets.
K Platform has the ability for you to filter and selectively show the assets that you are interested in seeing. You can toggle the filter to act either as ‘AND' / ‘OR'.
Updating Lineage Data
Sometimes lineage may be missing on a table (or other item). To update the Lineage information, first go to the Data Item’s Profile Page
|
Click on the Lineage tab
Click Edit Lineage and a pop-up box will appear
Confirm if you want to add an Upstream or Downstream item
Search for the missing data item and click Close