Skip to main content
Skip table of contents

Metadata enrichment via API

K provides an API which can be sent a JSON payload to bulk update objects in K

You can use this API to bulk update descriptions, classifications, owners etc for tables, reports etc

1. Preparing the data to upload

The following information is used by the API to update the object metadata

JSON is to be UTF-8 encoded.

Data template

ATTRIBUTE

DESCRIPTION

REQUIRED

DATA TYPE

Comments

id

Row ID for each object.

The Row ID must be unique per object

This is used to all the caller to linked rejected rows and error messages in the response to the objects in the request payload.

Y

INTEGER

Generated id so we can identify rejects by id back as in the error message, completely up to how they want to generate the id as long as its unique within the payload otherwise it may cause confusion with error messages.

object_k_id

The ID attribute for a K object.

Only populate if you need to reprocess objects because they errored because the name was not unique.

This can happen for Reports and Sheets where the object meta (below) is not unique

N

Only required when reprocessing objects where the name is not unique

UUID

The ID can be found in K by searching for the object and noting and taking the UUID from he url of the object’s profile page. Eg 4daa6fa3-3b32-3f07-b587-67bcd8133d35

from the URL

https://<yourdomain>.kada.ai/profile/content/4daa6fa3-3b32-3f07-b587-67bcd8133d35

If you know the UUID inside K, specify and we will use this over object_meta to identify the object, this is helpful is cases such as reports where multiple reports potentially can have the same name but different ids and ids for reports are generally derived from the system id and not uniquely generated by K.

object_type

valid values: SCHEMA/TABLE/COLUMN/REPORT/SHEET

Y

STRING

Note these must be UPPER CASE.

object_meta

Attributes to uniquely identify the object in K

NOTE that object_meta properties are different per object_type.

Y

JSON code blocks

Example for a TABLE. See more examples below

CODE
"object_meta": {
      "source_id": 1008,
      "database": "adventureworks",
      "schema": "sales",
      "table": "customer"
    }

All object_meta requires a source_id This value can be found in Admin > Sources.

Get reference list of sources

https://<youdomain>/api/sources

Specific information about that particular object type, this can be unique to each object type, generally the information needed to identify it

Get source ID from the UI

Go to Platform Settings. Click on Sources. Source ID is displayed next to each source you have integrated

description

The description for the object.

Any existing description will be overridden

If a blank description is provided it will be ignored.

N

STRING

From this attribute onwards at least 1 should be provided otherwise there is nothing to be enriched for the object.

business_name

This will populate the business name for the object.

Any existing business name will be overridden

If a blank business name is provided it will be ignored. 

N

STRING

will populate the alternate name

business_logic

A description to describe the business logic use to create the object

N

STRING

You should check if similar business logic for the object does not exist before creating to avoid creating duplicates

owners

An owner’s username of the onboarded user (typically email).

If platform_profile_user_lookups is set to data_roles then the user must be linked to the Data Owner instance.

Set to empty list “owners":[] of included property but there are no owners

N

LIST<STRING>

list of owners identified by either the username (if synced with AD this is the email of the user) of the onboarded user

Note: Check whether you have a set of owners configured or any user can be an owner This is configured in Platform settings - Customisation

API to get list of owners

https://<your-domain>/api/collectioninstances/16e05af2-13fa-301d-b7fa-69d48bc71d7d/related?object_name=USER&relationship=MEMBERS

stewards

A list of steward usernames of the onboarded user (typically email) .

If platform_profile_user_lookups is set to data_roles then the user must be linked to the Data Steward instance.

Set to empty list “stewards":[] of included property but there are no stewards

N

LIST<STRING>

list of stewards identified by username (if synced with AD this is the email of the user) of the onboarded user (check platform_profile_user_lookups value data_roles vs any_user)

Note: Check whether you have a set of stewards configured or any user can be an owner This is configured in Platform settings - Customisation

API to get a list of stewards

https://<your-domain>/api/collectioninstances/d13d6b10-a535-3718-854d-459f086ad057/related?object_name=USER&relationship=MEMBERS

verified_use_cases

A list of IDs for the verified use cases to be linked to this object.

Set to empty list “verified_use_cases":[] of included property but there are no verified_use_cases

N

LIST<UUID>

UUID of verified use cases, separated into these categories as the links are different and easier for the job to process

API to get a list of verified use cases

https://<your-domain>/api/collectioninstances?collection_id=0ffdae93-b46c-38c2-8511-8183f1d89c8d

classifications

A ID of the classification to be linked to this object.

This should be a single value.

Set to empty list “classifications":[] of included property but there are no classifications.

N

LIST<UUID>

UUID of classifications

API to get a list of classifications

https://<your-domain>/api/collectioninstances?collection_id=e1fd2337-1b0e-3841-8d60-7c85daa1707e

domains

A list of IDs of the domains to be linked to this object.

Set to empty list “domains":[] of included property but there are no domains.

N

LIST<UUID>

UUID of domains

API to get a list of domains

https://<your-domain>/api/collectioninstances?collection_id=73f92690-2edd-33a9-ad32-4f9c225a5c18

Example payloads

Schema

JSON
[
  {
    "id": 1,
    "object_k_id": "982c18cf-643a-356b-bdef-15d69919d1d1",
    "object_type": "SCHEMA",
    "object_meta": {
      "source_id": 1008,
      "database": "adventureworks",
      "schema": "sales"
    },
    "description": "my new schema",
    "business_name": "customer ledger",
    "business_logic": "some busines logic",
    "owners": ["xyz@kada.ai"],
    "stewards": ["abc@kada.ai"],
    "verified_use_cases": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "classifications": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "domains": ["982c18cf-643a-356b-bdef-15d69919d1d1"]
  }
]

Table

JSON
[
  {
    "id": 1,
    "object_k_id": "982c18cf-643a-356b-bdef-15d69919d1d1",
    "object_type": "TABLE",
    "object_meta": {
      "source_id": 1008,
      "database": "adventureworks",
      "schema": "sales",
      "table": "customer"
    },
    "description": "my new table",
    "business_name": "customer ledger",
    "business_logic": "some busines logic",
    "owners": ["xyz@kada.ai"],
    "stewards": ["abc@kada.ai"],
    "verified_use_cases": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "classifications": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "domains": ["982c18cf-643a-356b-bdef-15d69919d1d1"]
  }
]

Column

JSON
[
  {
    "id": 1,
    "object_k_id": "982c18cf-643a-356b-bdef-15d69919d1d1",
    "object_type": "COLUMN",
    "object_meta": {
      "source_id": 1008,
      "database": "adventureworks",
      "schema": "sales",
      "table": "customer",
      "column": "customer_id"
    },
    "description": "my new column",
    "business_name": "customer ledger",
    "business_logic": "some busines logic",
    "owners": ["xyz@kada.ai"],
    "stewards": ["abc@kada.ai"],
    "verified_use_cases": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "classifications": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "domains": ["982c18cf-643a-356b-bdef-15d69919d1d1"]
  }
]

Report

JSON
[
  {
    "id": 1,
    "object_k_id": "982c18cf-643a-356b-bdef-15d69919d1d1",
    "object_type": "REPORT",
    "object_meta": {
      "source_id": 1008,
      "name": "My brand spanking new report"
    },
    "description": "Some report",
    "business_name": "Some business name",
    "business_logic": "some busines logic",
    "owners": ["xyz@kada.ai"],
    "stewards": ["abc@kada.ai"],
    "verified_use_cases": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "classifications": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "domains": ["982c18cf-643a-356b-bdef-15d69919d1d1"]
  }
]

Sheet

JSON
[
  {
    "id": 1,
    "object_k_id": "982c18cf-643a-356b-bdef-15d69919d1d1",
    "object_type": "SHEET",
    "object_meta": {
      "source_id": 1008,
      "name": "My brand spanking new report sheet"
    },
    "description": "Report tile of sorts",
    "business_name": "Some interesting business name",
    "owners": ["xyz@kada.ai"],
    "stewards": ["abc@kada.ai"],
    "verified_use_cases": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "classifications": ["982c18cf-643a-356b-bdef-15d69919d1d1"],
    "domains": ["982c18cf-643a-356b-bdef-15d69919d1d1"]
  }
]

3. Setting up the API Key

Follow the API Guides to setup an access Token

https://kada-knowledge-library.scrollhelp.site/home/api-guides

4. Using the API

API Endpoint

There is a single end point for uploading your Update payload.

CODE
POST /api/metadata/enrichment

Example

BASH
curl --location --request POST 'https://<YOUR_DOMAIN>.kada.ai:443/api/metadata/enrichment' \
--header 'Authorization: Bearer <YOUR_AUTH_TOKEN>' \
--form 'data=@"/path/to/your/enrichment_file.json"'

API Validation

The API will read your payload and perform a set of validations over the format of your payload.

If the validation fails, the API returns a error payload with additional detail on what has failed validation

In this example the key "1" refers to the element at index 1 in the json file uploaded.

CODE
{
  "errors": {
    "1": {
      "stewards": ["Not a valid list."]
    }
  }
}

API Responses

If the file passes validation, the file data will be loaded into K. You will be provided a job id in the response so you can check the current status of the load.

CODE
GET api/jobexecutions/<job id>

Which will return either the status is

SUBMITTED
RUNNING
FAILED
KILLED
COMPLETE

In the event of FAILED, the response will include additional details about the error.

CODE
{
    "created_at": "2022-02-15T03:23:29.224114+00:00",
    "data_processed": true,
    "error": "1: Object does not exist\n2001: Invalid collection UUID <> provided"
    "id": 2,
    "job_reference": "61e1b879-5a28-404a-8350-6ffec5dc6097",
    "job_status": "FAILED",
    "platform_job_id": 106,
    "source_id": 1000,
    "updated_at": "2022-02-15T03:23:30.852532+00:00"
}

Considerations

There is no guarantee that the job will load all data provided in the payload.

You will need to ensure your updates are checked and ensure you review any errors that are generated. Any performance issues should be reviewed - e.g. the job has been running for over 2 hours. Reach out to the KADA team for any support you may need.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.