MySQL (via Collector method) - v3.0.0

About Collectors

Pre-requisites

Collector Server Minimum Requirements

MySQL Requirements

• Access to MySQL. Supported MySQL versions include:

  • 5.7x

  • 8.0x

• Access to the following Information Schema tables

  1. INFORMATION_SCHEMA.VIEWS

  2. INFORMATION_SCHEMA.TABLES

  3. INFORMATION_SCHEMA.COLUMNS

  4. INFORMATION_SCHEMA.KEY_COLUMN_USAGE

Step 1: Enabling logging (if desired)

See more information about logging here - https://dev.mysql.com/doc/refman/8.0/en/log-destinations.html

Note this study Impact of General Query Log on MySQL Performance and the performance impact of enabling logging.

‌

Enable global logging

SET GLOBAL log_output = 'TABLE';
SET GLOBAL general_log = 'ON';

Change logging tables from reading CSV files to MyISAM Engine for performance

SET @old_log_state = @@GLOBAL.general_log;
SET GLOBAL general_log = 'OFF';
ALTER TABLE mysql.general_log ENGINE = MyISAM;
ALTER TABLE mysql.slow_log ENGINE = MyISAM;
ALTER TABLE mysql.general_log ADD INDEX (event_time);
ALTER TABLE mysql.slow_log ADD INDEX (start_time);
SET GLOBAL general_log = @old_log_state;

Step 2: Create the Source in K

Create an MySQL source in K

• Go to Settings, Select Sources and click Add Source

• Select "Load from File system" option

• Give the source a Name - e.g. MySQLProduction

• Add the Host name for the MySQL Server

• Click Finish Setup

Step 3: Getting Access to the Source Landing Directory

Step 4: Install the Collector

It is recommended to use a python environment such as pyenv or pipenv if you are not intending to install this package at the system level.

Some python packages also have dependencies on the OS level packages, so you may be required to install additional OS packages if the below fails to install.

You can download the latest Core Library and whl via Platform Settings → Sources → Download Collectors

Run the following command to install the collector

pip install kada_collectors_extractors_<version>-none-any.whl

You will also need to install the common library kada_collectors_lib for this collector to function properly.

pip install kada_collectors_lib-<version>-none-any.whl

Step 5: Configure the Collector

The collector requires a set of parameters to connect to and extract metadata from MySQL.

The MySQL collector only extracts metadata and does not extract or process query usage on the database.