This is an overview of the Open Data Quality Initiative for integration of data quality/data health data into Alation.

Design Guide

Consumer of the information

• Identify who the primary users of the data catalog are.
• Who would be the primary users of the information that this integration would surface?
  • Most of the time, these two audiences will be the same. Sometimes, however, there may be use cases where only a subset of the users actually need to see this information.

Keep the Content Simple

Mind the white space. It is really easy to create walls of text that users of any system will ignore. Things to consider:

• Summarize the data so that only the pertinent information shows.
• Most users want to understand what the data means and quickly understand if they can trust the data.

Points of Integration

Data Health Tab (New)

API based Data Health tab for tables in the Alation Catalog that can surface and display data quality rules from integration partners.

Benefits:

• Public API to allow customers and partners to integrate Data Quality tools with the Data Catalog.
• Establishes a framework for deeper and more robust integration with the catalog in future phases.

Data Quality Overview Using Custom Fields

Custom fields provide space for integration partners to load summary information for each object (schema, tables, columns). These fields are Rich Text, so data can be set up to display the summary with visualizations. Additionally, hyperlinks can be added to allow users to easily access the details found in the 3rd party tools.

API Details

Data Health API Definition

The latest API Definition can be found here: https://developer.alation.com/dev/reference/data_health_ingestion

Example Code

Example Code for Data Health API

Python Code

import requests
import json

url = 'https://{base-url}/integration/v1/data_quality/'

headers = {'Token': '<Access Token>', 'Content-Type': 'application/json'}

myobj = {
  "fields": [
    {
      "field_key": "my.example.rule.key",
      "name": "The Example Rule",
      "description": "Description",
      "type": "STRING"
    }
  ],
  "values": [
    {
      "field_key": "my.example.rule.key",
      "object_key": "<ds_id>.<schema>.<table>.<column>",
      "object_type": "ATTRIBUTE",
      "status": "GOOD",
      "value": 25,
      "url": "https://www.alation.com"
    }  ]
}

x = requests.post(url, data = json.dumps(myobj), headers=headers)

print(x.text)

Resulting UI

Example Code for Custom Field

Please refer to the Systems Integrations to Alation with Custom Templates and Fields guide.