APIs for Exporting Large Dataset from CMX Cassandra Database
GET: All Available Files
Description
This API returns the list of all history data export archives that are available for download over HTTP. Note that Cisco CMX automatically deletes exported archives eight days after the date of creation.
HTTP Method
GET
Resource URI
/api/config/v1/historydata/export
Requires OAuth
N
Parameters
None.
Content Type
The response is a JSON and contains the URL of all available files.
Sample Output
application/json. The response contains the following:
-
A timestamp that indicates when the archive was exported.
-
The total size of the file(s) in bytes.
-
A URL (or a list of URLs) which serves the file over HTTP.
The file format is '.tar.gz'. You can untar it to convert it to a raw text file. If the file is greater than 400 MB in size, the response is split into a list of files of 400MB each.
[{
"lastModified": "Fri Jan 12 01:35:20 UTC 2018",
"totalFileSizeInBytes": 22719368,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720916/CassandraDataset_1515720916.tar.gz.aa",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720916/CassandraDataset_1515720916.tar.gz.ab",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720916/CassandraDataset_1515720916.tar.gz.ac"
]
},
{
"lastModified": "Fri Jan 12 01:35:40 UTC 2018",
"totalFileSizeInBytes": 132,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720939/CassandraDataset_1515720939.tar.gz"
]
},
{
"lastModified": "Fri Jan 12 01:35:26 UTC 2018",
"totalFileSizeInBytes": 22719371,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720922/CassandraDataset_1515720922.tar.gz.ab",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720922/CassandraDataset_1515720922.tar.gz.aa",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720922/CassandraDataset_1515720922.tar.gz.ac"
]
},
{
"lastModified": "Fri Jan 12 01:35:45 UTC 2018",
"totalFileSizeInBytes": 132,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720943/CassandraDataset_1515720943.tar.gz"
]
},
{
"lastModified": "Fri Jan 12 01:35:32 UTC 2018",
"totalFileSizeInBytes": 22719368,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720928/CassandraDataset_1515720928.tar.gz.ab",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720928/CassandraDataset_1515720928.tar.gz.ac",
"https://10.30.114.115/common/data/export/CassandraDataset_1515720928/CassandraDataset_1515720928.tar.gz.aa"
]
},
{
"lastModified": "Fri Jan 12 01:35:42 UTC 2018",
"totalFileSizeInBytes": 132,
"fileUrl": [
"https://10.30.114.115/common/data/export/CassandraDataset_1515720941/CassandraDataset_1515720941.tar.gz"
]
}
]
To merge the parts of .tar.gz files, use the following UNIX commands:cat CassandraDataset_1515720916.tar.gz.aa CassandraDataset_1515720916.tar.gz.ab CassandraDataset_1515720916.tar.gz.ac >> CassandraDataset_1515720916.tar.gz
OR
cat CassandraDataset_1515720916.tar.gz.* >> CassandraDataset_1515720916.tar.gz
API History
Release |
Modification |
---|---|
Cisco CMX Release 10.5 |
This API is introduced. |
GET: Is Data Export Running
Description
This API returns if the data export utility (Cassandra data extraction process) is currently running or not. Note that CMX allows for only one extraction process to run at a time.
If the process is not running, the API returns FALSE indicating that a new extraction job can be triggered using the POST API.
If the process is running, the API returns TRUE indicating that an instance of the extraction job is already running. You can keep querying this API with an interval of a few minutes to check the status before submitting another job.
HTTP Method
GET
Resource URI
/api/config/v1/historydata/export/isrunning
Requires OAuth
N
Parameters
None.
Content Type
The response is a JSON and contains the URL of all available files.
Sample Output
application/json
{"isRunning":false}
{"isRunning":true}
API History
Release |
Modification |
---|---|
Cisco CMX Release 10.5 |
This API is introduced. |
POST: Trigger History data export
Description
This API triggers data export for a given day or date. The API enables you to extract a large amount of raw history data in JSON format from the Cisco CMX's Cassandra database.
When the API is invoked, Cisco CMX spawns a process to extract data from Cassandra. All the history records that match the query parameters are dumped into a data file. This data file can then be downloaded from Cisco CMX server over HTTP and you can perform your own analytics by processing the JSON data. Each line in the data file represents a history record.
Note that you cannot run multiple extraction jobs at a time. If the job is already running, The API returns HTTP 403 Forbidden, meaning that the job is already running.
Let us observe how the API works.
The API accepts the date for which data needs to exported. This parameter is mandatory.
If you want to export all the history records that are generated on 2017/11/29,
POST /api/config/v1/historydata/export?date=2017/11/29
Once the extraction is complete, the exported data file can be download over HTTP.
GET /api/config/v1/historydata/export
You could also mention a specific range of hours for which you want to export the data for a given date. The value of fromHour cannot be greater than the value of toHour.
If you want to export history data generated on 2017/11/29, starting Midnight to 4 AM.
POST /api/config/v1/historydata/export?date=2017/11/29&fromHour=0&toHour=4
If you want to export history data generated on 2017/11/29 starting Midnight to 9 AM.
HTTP POST /api/config/v1/historydata/export?date=2017/11/29&fromHour=0&toHour=9
If you want to export history data generated on 2017/11/29 starting Midnight to 1 PM.
POST /api/config/v1/historydata/export?date=2017/11/29&fromHour=0&toHour=13
If you want to export history data generated on 2017/11/29 starting 4 PM to 9 PM.
POST /api/config/v1/historydata/export?date=2017/11/29&fromHour=16&toHour=21
HTTP Method
POST
Resource URI
/api/config/v1/historydata/export
Requires OAuth
N
Parameters
Name |
Required |
Default |
Type |
Location |
Description |
---|---|---|---|---|---|
date |
Y |
— |
String |
body |
The date for which data export needs to be triggered. The expected date format is yyyy/MM/dd . |
fromHour |
N |
— |
String |
body |
This field expects an integer value within the range zero to 23. The field is used to export the data for a specific duration of the day. The API looks at fromHour and toHour parameters and exports data accordingly. |
toHour |
N |
— |
String |
body |
This field expects an integer value within the range one to 24. This is used to export the data for a specific duration of the day. The API looks at fromHour and toHour parameters and exports data accordingly. |
Content Type
application/json
Sample Input
http://<host-ip-address>/api/config/v1/historydata/export?date=2017/11/29&fromHour=0&toHour=1
Sample Output
Response:
"Cassandra data export utility triggered successfully."