Best Practices

Alation recommends that when making Create, Read, Update, or Delete API calls to Alation that:

• Payload contains 1000 objects or less per call
• Do not recommend multi-threading or multi-processing API calls to Alation

Configure API Throttling

It is possible to rate-limit requests made by users to some of the Alation's public APIs. As of 2020.3, the API which can be throttled are:

• Lineage
• BI Source (GBMv2) Overview
• Relational Integration API Overview
• Search

API throttling is disabled by default. It can be enabled and configured from the Alation shell using the alation_conf command. To view all currently available configuration options for API throttling:

1. On the Alation host, enter the Alation shell:

   {.Bash}

   sudo /etc/init.d/alation shell
   

2. Type the following command and press Enter:

   {.Bash}

   alation_conf throttling
   

This command prints all parameters relevant to API throttling and their current values:

Enable API Throttling

To enable API Throttling for an Alation instance, set the parameter alation.api.throttling.enabled to True. No restart is required:

{.Bash}

alation_conf alation.api.throttling.enabled -s True

After enabling the feature, set the throttling scope parameters to appropriate values.

Set Throttling Scope Parameters

There are two major throttle scope types:

Global (Sustained)

• Applies to all APIs which respect throttling.
• It is recommended to set the rate limit to a larger value. For example: 1,000 requests per day.
• Set using the parameters:
  • alation.api.throttling.global.read
  • alation.api.throttling.global.write

API family/ API resource specific

• Applies to the particular API scope only (for example, Lineage V2 or GBM V2).
• It is recommended to set the rate limit to a small duration to limit expensive operations or throttle DOS attacks. For example, 20 requests per second.
• Set using the parameters:
  • alation.api.throttling.lineage.read
  • alation.api.throttling.lineage.write
  • alation.api.throttling.bi.read
  • alation.api.throttling.bi.write

Each scope type comes with two variations:

• READ: applies to GET, OPTIONS. For example:
  alation.api.throttling.lineage.read = 20/sec
• WRITE: applies to POST, PATCH, DELETE. For example:
  alation.api.throttling.lineage.write = 20/sec

Note

Throttle configurations are applied based on scope and request type. The API family scope takes precedence over the Global settings when both are set.

Value Format

Rate limit can be set in the following format:

• Number of requests per second: <number>/sec.
  • Example: 20/sec
• Number of requests per minute: <number>/min.
  • Example: 20/min
• Number of requests per hour: <number>/hour.
  • Example: 20/hour
• Number of requests per day: <number>/day.
  • Example: 20/day
• null: means unlimited, or throttling disabled. You can disable each rate limit by setting it to null. The global rate limit is disabled by default.

To set a value, use the alation_conf command:

 alation_conf alation.api.throttling.lineage.read -s <value>

Throttle Response

If a rate limit for a scope is set to <number>/<time unit>, then an API request within this scope is throttled on <number> + 1 attempt after the time = <time unit>. When a request is throttled, the API responds with status 429 and the time in seconds till the next request in this scope is allowed.

Example 1

 [
   {
     "code": "429000",
     "detail": "Rate limit for [read] requests for API scope [lineage] reached. Expected available in 29 seconds."
     }
   ]

Example 2

 [
   {
     "code": "429000",
     "detail": "Global rate limit for [read] requests reached. Expected available in 86279 seconds."
     }
   ]