Clickhouse (via Direct Connect method)

This page will walkthrough the setup of Clickhouse in K using the direct connect method.

Integration details

Step 1: Establish Clickhouse Access

The service user/account/role will require permissions to the following:

• (TBC — refer to Clickhouse documentation for required permissions)

After this step you should have the following information:

• Tenant Account ID

• Role

• Key

• Secret

Step 2: Create the Source in K

Create a Clickhouse source in K.

• Select Platform Settings in the side bar

• In the pop-out side panel, under Integrations click on Sources

• Click Add Source and select Clickhouse

• Select Direct Connect and add your Clickhouse Source Settings

  • Name: Give the Clickhouse source a name in K.

  • Host: Enter a hostname for your Clickhouse instance

  • Port: Refer to Network ports | ClickHouse Docs for your Port details

  • Confirm if you want to:

    • Enable data masking

    • Extract meta only

• Click Save & Next

• Setup your Connection details:

  • Host

  • Username and Password

• Test your connection and click Next

Step 3: Schedule Clickhouse source load

• Select Platform Settings in the side bar

• In the pop-out side panel, under Integrations click on Sources

• Locate your new Clickhouse Source and click on the Schedule Settings (clock) icon to set the schedule

Step 4: Manually run an ad hoc load to test Clickhouse

• Next to your new Source, click on the Run manual load icon

• Confirm how you want the manual run to be completed

• After the source load is triggered, a pop up bar will appear taking you to the Monitor tab in the Batch Manager page. This is the usual page you visit to view the progress of source loads

A manual source load will also require a manual run of

• DAILY

• GATHER_METRICS_AND_STATS

To load all metrics and indexes with the manually loaded metadata. These can be found in the Batch Manager page.

Troubleshooting failed loads

• If the job failed at the extraction step

  • Check the error. Contact KADA Support if required.

  • Rerun the source job

• If the job failed at the load step, the landing folder failed directory will contain the file with issues.

  • Find the bad record and fix the file

  • Rerun the source job