HomeGuidesRecipesAPI ReferencePython SDK
Alation Help Center
API Reference

Virtual File System APIs

Upload Virtual File System

Description

This API allows users to update the contents of a File System.

Any post to this API will replace the existing set of files and directories with the uploaded set of files and directories. There is currently no support for an incremental update.

Note:

  1. Only admin users can update the contents of a File System through this API
  2. This API is available in version 4.8.1 and later

URL

POST /api/v1/bulk_metadata/file_upload/**<filesystem\_id>**/

Path Parameters

NameRequiredDescription
<filesystem_id>YesUnique identifier of the file system inside Alation catalog.
Example: https://mycompany.alation.com/api/v1/bulk_metadata/file_upload/1/
Please refer to How to find the filesystem ID to find the unique identifier of the filesystem.

How to find the File System ID

The File System ID is a unique identifier given by Alation. You can find this ID in two ways.

  1. Click on the File System in the Alation UI. Then check the url in your browser. The ID of the File System will be in the url. For example: If your URL is https://mycompany.alation.com/filesystem/2/, the filesystem ID is 2
  2. Go to the settings page for a File System. There is an About section that will list it's ID.

Data Parameters

NameTypeRequiredDescription
pathstringYesAbsolute path to the file.
This path will be interpreted using the delimiter of the File System (default is /). There is no way to escape the delimiter character. Paths are case sensitive. Leading and trailing delimiters are ignored, so "/d1/" and "d1" would be equivalent.
namestringYesName of the file or directory.
Does not include the full path to the object, only the name. Names can not contain the delimiter. Names are case sensitive.
is_directoryboolYesIndicates if the object is a directory or file.
Valid values are true or false.
size_in_bytesintNoThe size of the file or directory.
Must be specified in bytes.
ts_last_modifiedstringNoTimestamp that the file was last modified.
Format must be YYYY-mm-dd HH:MM:SS. Timezone is UTC.
ts_last_accessedstringNoTimestamp that the file was last modified.
Format must be YYYY-mm-dd HH:MM:SS. Timezone is UTC.
ownerstringNoName of the user that owns the object
This is the name of a user in the source file system. User names are case sensitive.
groupstringNoName of the group.
This is the name of a group in the source file system. Group names are case sensitive.
permission_bitsintNoUnix style file permissions.
Must be three Octal digits. Ex. 755
storage_typeintNoIndicates the S3 storage type.
Only used for S3 file systems. See storage types below.

Storage Types

Each file can optionally indicate it's storage type. This is only relevant for S3 backed files. See S3 docs for details on what the storage types mean.

ValueStorage Type
0Standard
1Standard IA
2Reduced Redundancy
3Glacier

Sample Request Body

{"path": "/", "name": "var", "is_directory": true, "owner": "root", "group": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}
{"path": "/var/", "name": "log", "group": "root", "is_directory": true, "owner": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}
{"path": "/var/log/", "name": "mail.log", "group": "root", "is_directory": false, "owner": "root", "permission_bits": 755, "size_in_bytes": 28776, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}

NOTE:

  1. ALL the files and directories in the File System must be included in the request body. Any not included will be removed from the File System because this is a replace operation and not an incremental update. Each line should contain exactly one file or directory that is represented by a JSON encoded object. The files and directories should be grouped according to their parent directory.
  2. The request body must be utf-8 encoded. Other encodings may cause errors or the names may show up with strange looking characters.
  3. The above sample request body uploads a single file mail.log to the file system.

Headers

HTTP HeaderValue
TOKEN<your_token>

Replace <your_token> with API Token which can be obtained from getToken API call (Get API Token).

Success Response

Content-Type: application/json

Status: 200 OK

Body

{
    "job": {
        "url": "/api/job/1/",
        "errors_url": "/api/job_error/?job_id=1",
        "id": 1
    }
}
NameDescription
jobThis is a nested field which includes the details of the job triggered after a successful call to the API. job has following fields:
url - URL that can be used to check the status of the job.
errors_url - URL that can be used to check for any errors if the job fails.
id - Unique identifier of the job.
NOTE: Irrespective of the job status, the errors_url parameter is present in the response.

Error Response

Invalid Token

Status: 401 UNAUTHORIZED

Body

{
   "detail": "Authentication failed"
}

Missing Token Header

Status: 401 UNAUTHORIZED

Body

{
    "detail": "Authentication credentials were not provided."
}

Code Samples

cURL

#!/bin/bash

# This is an example token. Please replace this with your token.
API_TOKEN="2abcd-4c04-4c21-8692-eda27a877f90"

BASE_URL="https://alation.yourcompany.com/api/v1/bulk_metadata/file_upload"

# Replace contents of a file system with unique identifier, 1
curl -X POST "${BASE_URL}/1/" -H 'content-type: application/json' -H "TOKEN: ${API_TOKEN}" --data-binary @body.txt

# contents of body.txt
: '
{"path": "/", "name": "var", "is_directory": true, "owner": "root", "group": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}
{"path": "/var/", "name": "log", "group": "root", "is_directory": true, "owner": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}
{"path": "/var/log/", "name": "mail.log", "group": "root", "is_directory": false, "owner": "root", "permission_bits": 755, "size_in_bytes": 28776, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}
'

Python

import requests
import json

# This is an example token. Please replace this with your token.
headers = {'Token': '2abcd-4c04-4c21-8692-eda27a877f90'}

data = '{"path": "/", "name": "var", "is_directory": true, "owner": "root", "group": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}\n{"path": "/var/", "name": "logs", "group": "root", "is_directory": true, "owner": "root", "permission_bits": 755, "size_in_bytes": 0, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}\n{"path": "/var/logs/", "name": "mail.log", "group": "root", "is_directory": false, "owner": "root", "permission_bits": 755, "size_in_bytes": 28776, "ts_last_accessed": "2016-11-20 12:00:00", "ts_last_modified": "2016-01-01 12:00:00"}'

# replace/update contents of a file system
response = requests.post('https://alation.yourcompany.com/api/v1/bulk_metadata/file_upload/1/', data=data, headers=headers)
job_details = json.loads(response.text)
print "Job URL: %s" % (job_details['job']['url'])
print "Job error URL: %s" % (job_details['job']['errors_url'])

# check the job status
job_url = job_details['job']['url']
response = requests.get('https://alation.yourcompany.com' + job_url, headers=headers)
job = json.loads(response.text)
print "Job Status: %s" % (job['status'])

# check for any erros
errors_url = job_details['job']['errors_url']
response = requests.get('https://alation.yourcompany.com' + errors_url, headers=headers)
error_details = json.loads(response.text)
if error_details:
    print "Errors %s" % (error_details)
else:
    print "No errors"