Query Log Upload API

Query Log Upload (Push QLI) API

Description

This API allows admin users to send query events for ingestion in Alation.

URL

POST /api/v1/bulk_metadata/query_log_ingestion/**<ds_id>**/

Replace <ds_id> with the id of the data source for which the query events are to be ingested.

Data Parameters

NameTypeRequiredDescription
userNamestringYesName of the database account user.
sessionIdstringYesID of the session.
defaultDatabasesstringYesDefault database used in the query. If no database is found, use an empty string ("").
textstringYesText of the query.
For Example: {"text":"select * from emp_schema.emp_table"}
startTimestringYesThe starting time of the query. The format for this parameter is yyyy-MM-DD HH:MM:SS.SSS
For Example: {"startTime":"2017-10-11 10:21:22.23445"}
appIdstringNoID of the application.
clientIdstringNoID of the client.
clientAddrstringNoIP address of the client.
For Example: {"clientAddr":"10.0.0.1:8000"}
sessionStartTimestringNoThe starting time of the session. The format for this parameter is yyyy-MM-DD HH:MM:SS.SSS
For Example: {"sessionStartTime":"2017-10-11 10:21:22.23445"}
secondsTakenlongNoTotal time elapsed (time taken to execute the query in seconds).
delayTimefloatNoNumber of seconds the query was delayed.
NOTE: This parameter is available in Teradata.
totalCpuTimefloatNoThis parameter represents the AMPCPUTIME in seconds.
numResultRowslongNoNumber of rows in the result.
totalIoCountintNoThe count of IO operation.

Sample Request Body

{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-28 05:22:24.457416", "defaultDatabases": "test_db", "text":  "SELECT * from schema_1.table_1", "sessionId": "sess1234", "userName": "test_user", "secondsTaken": 1}
{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:37.457316", "defaultDatabases": "test_db", "text": "SELECT * from schema_2.table_2", "sessionId": "sess432", "userName": "test_user", "secondsTaken": 2}
{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:43.457116", "defaultDatabases": "test_db", "text": "SELECT * from schema_3.table_3", "sessionId": "sess5678", "userName": "test_user", "secondsTaken": 5}

NOTE: The Request body can contain one or more JSON objects, each on a new line. Each line represents a query event.

Headers

HTTP HeaderValue
TOKEN<your_token>

Replace <your_token> with API Token which can be obtained from getToken API call (Get API Token).

Success Response

Content-Type: application/json

Status: 200 OK

Body

{
    "num_statements_input": 3,
    "job_id": 1
}
NameDescription
num_statements_inputRepresents the num of statements in the request body.
job_idUnique identifier of the job that is triggered after a success call to the API. This job is responsible to ingest the query events mentioned in the request body.
To know the status of a job, please refer Job Status API. The job status can also be found under the 'Query Log Ingestion' tab (https://<your_host>/data/<ds_id>/settings/import_queries/) of the data source.

Error Response

Invalid Token

Status: 401 UNAUTHORIZED

Body

{
   "detail": "Authentication failed"
}

Missing Token Header

Status: 401 UNAUTHORIZED

Body

{
    "detail": "Authentication credentials were not provided."
}

Non-admin user access

Status: 403 FORBIDDEN

Body

{
    "detail": "Permission denied"
}

Code Samples

cURL

#!/bin/bash

# This is an example token. Please replace this with your token.
API_TOKEN="2abcd-4c04-4c21-8692-eda27a877f90"

BASE_URL="https://alation.yourcompany.com/api/v1/bulk_metadata/query_log_ingestion"

# Ingest queries into a data source with unique identifier, 1
curl -X POST "${BASE_URL}/1/" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" --data-binary @body.txt

# contents of body.txt
: '
{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-28 05:22:24.457416", "defaultDatabases": "test_db","text":  "SELECT * from schema_1.table_1", "sessionId": "sess1234", "userName": "test_user", "secondsTaken": 1}
{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:37.457316", "defaultDatabases": "test_db", "text": "SELECT * from schema_2.table_2", "sessionId": "sess432", "userName": "test_user", "secondsTaken": 2}
{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:43.457116", "defaultDatabases": "test_db", "text": "SELECT * from schema_3.table_3", "sessionId": "sess5678", "userName": "test_user", "secondsTaken": 5}
'

Python

import requests
import json

# This is an example token. Please replace this with your token.
headers = {'Token': '2abcd-4c04-4c21-8692-eda27a877f90'}

data = '{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-28 05:22:24.457416", "defaultDatabases": "test_db","text":  "SELECT * from schema_1.table_1", "sessionId": "sess1234", "userName": "test_user", "secondsTaken": 1}\n{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:37.457316", "defaultDatabases": "test_db", "text": "SELECT * from schema_2.table_2", "sessionId": "sess432", "userName": "test_user", "secondsTaken": 2}\n{"clientAddr": "10.11.0.75:8000", "startTime": "2017-05-29 05:22:43.457116", "defaultDatabases": "test_db", "text": "SELECT * from schema_3.table_3", "sessionId": "sess5678", "userName": "test_user", "secondsTaken": 5}\n'

# ingest query events
response = requests.post('https://alation.yourcompany.com/api/v1/bulk_metadata/query_log_ingestion/1/', data=data, headers=headers)
job_details = json.loads(response.text)
print "Job ID: %s" % (job_details['job_id'])

# check the job status
job_id = job_details['job_id']
response = requests.get('https://alation.yourcompany.com/api/v1/bulk_metadata/job?id=' + str(job_id), headers=headers)
job = json.loads(response.text)
print "Job Status: %s" % (job['status'])