ULM Overview

Upload Logical Metadata

🚧

ULM Update

If you have jobs that are using this API to update schemas, tables, file systems, and etc. this API will still work, but it is not be as performant as the new Custom Field Values Overview. This API is recommend for modifying articles.

Description

This API allows you to upload values to titles, descriptions, stewards and custom fields for objects in the Alation catalog.

Custom fields are fields created in Alation's Customize Catalog interface.

Note:

  1. This API does not support modifying a data source's technical metadata (names or types of schemas, tables, or columns).
  2. In this API, the word 'attribute' refers to a column in a database table.
  3. This API does not modify the deleted articles.
  4. This API was available only for Admin users before version 5.8 but now it is available for anyone with a valid token.
  5. This API does not trigger a version history change when updating custom fields

URL

POST /api/v1/bulk_metadata/custom_fields/**<template_name>**/**<object_type>**?**<params>**

Path Parameters

NameRequiredDescription
<template_name>YesName of the template, custom fields have been applied to.
NOTE: Template name can be default or any other name given to a custom template created by an Alation user for articles. More information on using a template name for different object types is given below.
a) Use the name given to a user created custom template for article object type and use default for all supported object types.
b) article object type can also use default if they do not have any custom template applied to them.

For Example:
The URL to update/create articles with custom template Business Glossary will look like:
https://<your_host>/api/v1/bulk_metadata/custom_fields/Business%20Glossary/article

The URL to update tables will look like:
https://<your_host>/api/v1/bulk_metadata/custom_fields/default/table
<object_type>YesThe type of object to update or create

Object Types

GroupObject TypeComments
Data
source
schema
table
attributeaka database column
attribute_valueColumn values custom metadata. Generally, not used
BI
server
folder
connection
source
report
Other
domain
dataflowlineage data flow
policydata policies
business_policybusiness policies
File System
filesystem
directory
file
API Resources
api_resource
api_resource_field
mixedthe actual type of the object is determined based on the key passed. mixed is not supported for article and filesystem. To know, what key is and how to pass a request body, please refer to sections below.
NoSQLMongoDB, Kafka, JSON, AVRO, and etc.
docstore_folder
docstore_collection
doc_schema

If you're trying to figure out which object type you should you, navigate to the resource in Alation, and take a look at the url. In the image below, the otype would be doctstore_folder.

495495

NOTE:

  1. For API request with <object_type> as mixed, the actual type of the object is determined based on the key passed. mixed is not supported for article and filesystem. To know, what key is and how to pass a request body, please refer to sections below.
  2. Use api_resource as the object type to update field values for API Resource as well as API Resource Folder. Use api_resource_field to update field values for API Resource Fields. More information about API Resources can be found API Resources
  3. Article titles - if there are articles in the same template with the identical titles, please use ULM with Article ID as this API will only update the first article that it finds. It will ignore the second article.
  4. Please see ULM: Key Formatting for more details on how to build keys for each object type

Replace <params> with your list of parameters. Multiple parameters can be combined as PARAM1&PARAM2

URL Parameters

NameRequiredDefaultDescription
create_newNotrueSpecifies if a new object should be created in case the object passed is not present in Alation catalog.
create_new can have one of the following values:
true
false
create_new=true specifies that a new object should be created for objects not existing in Alation catalog.
NOTE: This parameter works only for article and attribute_value
replace_valuesNofalseSpecifies if the custom field values passed to the API request should be replaced with the existing values or should be added to the existing list of values.
replace_values can have one of the following values:
true
false
NOTE: This parameter works only for object, people set and reference custom fields. Also, if existing title and description of an object needs to be updated, this field must be set to true.

Data Parameters

NameRequiredDescription
keyYesRepresents the object the fields belong to. The format for key is based on the type of the object.
titleNoTitle of an object
descriptionNoDescription of an object
<custom_field_name>NoName of the custom field.

NOTE: Please refer ULM: Key Formatting and ULM: Custom Field Formatting to see how to format of data parameters in the request body.

Sample Request Body

{"key": "1.emp_schema.employee", "title": "Employee Table", "description": "List of Employees", "related_tables": [{"type":"table", "key":"1.emp_schema.department"}]}
{"key": "1.emp_schema.salary", "title": "Salary Table", "description": "List of salary ranges","related_tables": [{"type":"table", "key":"1.emp_schema.employee"}], "Steward":[{"type":"user", "key":"[email protected]"}]}

NOTE:

  1. Request body above is an example for /api/v1/bulk_metadata/custom_fields/default/table API call. The request body should be a list of JSON objects separated by a new line, \n. Each JSON object on a line corresponds to a single field value upload.
  2. Due to the restriction of the above note 1, each JSON object has to be flattened out. Meaning there can not be any new lines, \n, within the JSON object.
  3. As you can recover deleted data sources and articles, if the request body contains schemas, tables, attributes, attribute values or articles that have been deleted from Alation, they will still be updated.

Headers

Get your token via APIAccessToken .

{"token": "ALATION_ACCESS_TOKEN_HERE"}

Success Response

Content-Type: application/json

Status: 200 OK

Body

{
    "new_objects": 1,
    "updated_objects": 0,
    "number_received": 1,
    "error_objects": [],
    "error": ""
}
NameDescription
new_objectsRepresents the number of new objects created.
NOTE: Valid in case of article and attribute_value
updated_objectsRepresents the number of objects updated
number_receivedRepresents the number of objects received in the request
error_objectsA List of keys for objects with errors
errorAn error message

Error Response

Invalid Token

Status: 401 UNAUTHORIZED

Body

{
   "detail": "Authentication failed"
}

Missing Token Header

Status: 401 UNAUTHORIZED

Body

{
    "detail": "Authentication credentials were not provided."
}

Code Samples

cURL

#!/bin/bash

# This is an example token. Please replace this with your token.
API_TOKEN="2abcd-4c04-4c21-8692-eda27a877f90"

BASE_URL="https://alation.yourcompany.com/api/v1/bulk_metadata/custom_fields"

# Add title and description to a table
curl -X POST "${BASE_URL}/default/table" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"1.emp_schema.employee", "title":"Employee Table", "description":"List of Employees"}'

# Replace existing title and description of a table
curl -X POST "${BASE_URL}/default/table?replace_values=true" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"1.emp_schema.employee", "title":"Employee Table", "description":"List of Employees"}'

# Add value to an object set custom field of a schema
curl -X POST "${BASE_URL}/default/schema" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"1.emp_schema", "related_objects":[{"type":"data","key":"1"}]}'

# Create an article if it does not exist
# The template applied is default article template. If the new article needs to be created under a custom template, replace 'default' with custom template name
curl -X POST "${BASE_URL}/default/article" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"database info article", "description":"This article has db info."}'

# Create an attribute value if it does not exist
curl -X POST "${BASE_URL}/default/attribute_value" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"1.emp_schema.employee.emp_name.dave", "title":"Employee Name", "description":"First name of employee"}'

# Add a value to a custom field for a file system object
curl -X POST "${BASE_URL}/default/filesystem" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"3", "description":"Our s3 file system", "related_tables":[{"type":"table","key":"2.hive_schema.test_tables"}]}'

# Add a description and steward to API Resource
curl -X POST "${BASE_URL}/default/api_resource" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" -d '{"key":"1", "description":"Data API Resource", "steward":[{"type":"user","key":"[email protected]"}]}'

Python

import requests
import json

# This is an example token. Please replace this with your token.
headers = {'Token': '2abcd-4c04-4c21-8692-eda27a877f90'}
BASE_URL = 'https://alation.yourcompany.com/api/v1/bulk_metadata/custom_fields/'

# Replace existing custom and builtin field values of a schema
data = {"key": "1.emp_schema", "title": "New Employee Table", "description": "New Employee Table Description", "related_schemas": [{"type":"schema", "key":"1.dept_schema"}]}
response = requests.post(BASE_URL + 'default/schema?replace_values=true', json=data, headers=headers)
result = json.loads(response.text)
print "Updated Objects: %s , New Objects: %s" % (result['updated_objects'], result['new_objects'])
for error in result['error_objects']:
    print error

# Upload custom fields to tables in bulk
data = '{"key": "1.emp_schema.employee", "related_tables": [{"type":"table", "key":"1.emp_schema.department"}]}\n{"key":"1.emp_schema.salary", "steward": [{"type":"user", "key":"[email protected]"}, {"type":"groupprofile", "key":"test-users"}]}'
response = requests.post(BASE_URL + 'default/table', data=data, headers=headers)
result = json.loads(response.text)
print "Updated Objects: %s , New Objects: %s" % (result['updated_objects'], result['new_objects'])
for error in result['error_objects']:
    print error

# Upload custom field to article with custom template
# The custom field uploaded should be added to the custom template
data = {"key": "Salary Range", "max_salary": "20000", "related_articles": [{"type":"article", "key": "Employee names"}]}
response = requests.post(BASE_URL + 'Custom%20Template/article', json=data, headers=headers)
result = json.loads(response.text)
print "Updated Objects: %s , New Objects: %s" % (result['updated_objects'], result['new_objects'])
for error in result['error_objects']:
    print error

# Upload a custom field value to a directory inside file system
data = {"key": "1/default", "related_tables": [{"type":"table", "key": "1.emp_schema.employee"}]}
response = requests.post(BASE_URL + 'default/filesystem', json=data, headers=headers)
result = json.loads(response.text)
print "Updated Objects: %s , New Objects: %s" % (result['updated_objects'], result['new_objects'])
for error in result['error_objects']:
    print error

# Upload a custom field value to a API Resource Field
data = {"key": "1", "related_attribute": [{"type":"attribute", "key": "1.emp_schema.employee.id"}]}
response = requests.post(BASE_URL + 'default/api_resource_field', json=data, headers=headers)
result = json.loads(response.text)
print "Updated Objects: %s , New Objects: %s" % (result['updated_objects'], result['new_objects'])
for error in result['error_objects']:
    print error