Lineage & Dataflow - Frequently Asked Questions

Is this API case sensitive?

No, it is case insensitive.

If lineage already exists, how can I insert a dataflow into that flow?

Yes! This can be done via API. Let's assume that we want to insert lineage for our table ACCOUNTBALANCE between our table lineage CUSTOMERS & PURCHASES. You could do this by:

NOTE

This example does not use Dataflows but we highly recommend that any lineage you add leverages Dataflows for easier curation as when you delete a dataflow, then all of the paths that are attached to it are also deleted.

Get lineage information on the object in question and copy the paths in question. We will use these keys in the next step.

{
  "paths": [
    [
      [
        {
          "otype": "table",
          "key": "1.SCHEMA.CUSTOMERS"
        }
      ],
      [
        {
          "otype": "table",
          "key": "1.SCHEMA.PURCHASES"
        }
      ]
    ]
  ]
}

Delete the lineage path between CUSTOMERS & PURCHASES via the Delete lineage information.
Identify the key for the new object to be inserted. See Lineage & Dataflows Overview for more details
Rebuild the path with the inserted key path with Post lineage information

{
  "paths": [
    [
      [
        {
          "otype": "table",
          "key": "1.SCHEMA.CUSTOMERS"
        }
      ],
      [
        {
          "otype": "table",
          "key": "1.SCHEMA.ACCOUNTBALANCE"
        }
      ],
      [
        {
          "otype": "table",
          "key": "1.SCHEMA.PURCHASES"
        }
      ]
    ]
  ]
}

Can I use the Lineage API to delete a deleted table from the Lineage graph?

If lineage is pointing incorrectly to a deleted table that hasn’t been dropped from the graph, you can use the Delete lineage information or Delete a list of Dataflow objects APIs to delete the link from the source table to the deleted table.

Can I give my dataflow a name/title in the API call?

A dataflow doesn’t have a name attribute, but it has the Title field in the catalog that you can use to give it a title. A dataflow object is identified with the external_id property that should start with api/ and be unique: api/bi_mstr_df_013.

Can I assign Stewards, Tags, and other catalog metadata to a dataflow object?

Yes, on its catalog page in Alation.

How can I search for dataflow objects in Alation?

Dataflow objects can be discovered using search

Are there character limits (length, special characters, cap, spaces, and more) for naming the dataflow object?

You can use UTF-8 encoded special characters.

Can I create multiple dataflow objects in one API push?

Yes! Up to 1000, Alation max recommendation, dataflows per POST. See Configure API Throttling for more details.

Is there a way to update the content of an already created dataflow?

Yes. See Update a list of Dataflow objects

What does the “TMP” mark on an object mean?

TMP means the object type of the object (table, bi_report, and etc.) is supported but the object itself cannot be found in Alation records. This could be because such an object doesn’t persist (it’s a temp table) or it existed earlier but was deleted.

Are report measures/attributes to dashboard measures/attributes lineage creation supported?

Yes! You can draw this lineage either through Manual Lineage Curation or POST these paths via API. See Lineage to BI Objects for more context.

Is column level lineage data ingested automatically?

Yes, but this depends on the data source. Please consult the Support Matrix for more information. If the desired data source is not supported yet, you can add column level lineage either through Manual Lineage Curation or by API via Lineage API.

Which object types support “TMP” when using the API?

By definition, TMP is used for any object whose object type Alation natively supports, but there is no physical record of it inside Alation. You are be able to push TMP data for any supported object type. For example: {"otype": " attribute ", "key": " 1.schema.table.tmp_column "}.

One reservation, though, is that due to the way the API key is designed for BI sources, TMP doesn’t work for bi_report_column and bi_datasource_column object types.

Do I see all lineage data for all objects displayed on the Lineage diagram?

The existing upstream and downstream lineage flows are shown on the lineage diagram only if they are relevant to the data object selected in the diagram. The selected object is highlighted with the orange color. When a target data object in a lineage diagram was created from several source data objects, all of these source data objects will be present in the diagram. However, not all of the lineage data existing for these source objects may be relevant to the currently selected object. Such lineage flows - existing, but unrelated to the currently selected object - will not be displayed.