Cisco DNA Spaces Firehose Archiver
Available Languages
Cisco DNA Spaces Firehose Archiver
First Published: November 3, 2020
Table of Contents
Writing firehose events into local disk
Obtaining Documentation and Submitting a Service Request
This Firehose Archiver application pulls the data from DNA Spaces over firehose as events and writes them into local disk as hourly files. Below are the events received on the firehose.
Events
· Profile Update: When a device profile is updated. For example, this event is sent when an end-user provides
information in a captive portal.
· Device Presence: Used to track life cycle of a device at a location. Events are generated at device entry, when a device is inactive for 10 minutes, when a device is active after being inactive or when we determine that the device has exited. These events also provide current count of devices at location (active and inactive).
Writing firehose events into local disk
· The events received over the firehose are written into local disk as hourly files. It creates a file called “firehose-data.txt” for every hour for a day.
· All the received events are written as json object into that file by separating line separator.
· Below is the path format of the local disk files:
Format: /path/to/output/utc_date=${UTC_DATE}/utc_hour=${UTC_HOUR}/firehose-data.txt
E.g.: /mnt/data/output/utc_date=2020-07-23/utc_hour=15/firehose-data.txt
Instance Details
Recommended memory:
· Memory 4 GB
· Disk 100 GB for a period of 3months
Installation Details
Please refer this below link to see the installation details:
https://hub.docker.com/r/ciscodnaspaces/cisco-dna-spaces-firehose-archiver
Profile Update Event Fields
| Field |
Data Type |
Description |
| device_id |
String |
Device ID of the device in DNASpaces |
| user_id |
String |
User ID of the user in DNASpaces, applicable for only users with valid user ids |
| tags |
List<String> |
List of Tag/Persona associated with the device |
| mobile |
String |
Mobile number of the device |
| |
String |
Email of the device |
| gender |
String |
Gender of the device |
| first_name |
String |
First name of the device |
| last_name |
String |
Last name of the device |
| postal_code |
String |
Postal code of the device |
| opt_ins |
List<String> |
OptIn contains the values of MOBILE_NUMBER, EMAIL, TERMS_AND_CONDITIONS |
| attributes |
List<Attribute> |
It has attributes fields. Please see the Attribute Fields Section. |
| mac_address |
String |
Mac address of the device |
| manufacturer |
String |
Manufacturer of the device |
| os |
String |
OS of the device |
| os_version |
String |
OS Version of the device |
| Type |
String |
Type contains either of these values NOT_AVAILABLE, MOBILE, TABLET, LAPTOP, OTHER_DEVICE |
| social_network_info |
List <SocialNetworkInfo> |
It has social info fields. Please see the Social Network Info Fields Section. |
Device Presence Event Fields
| Field |
Data Type |
Description |
| presence_event_type |
String |
Presence event type contains either of these values DEVICE_ENTRY_EVENT, DEVICE_IN_ACTIVE_EVENT, DEVICE_ACTIVE_EVENT, DEVICE_EXIT_EVENT, DEVICE_SSID_CHANGE_EVENT, DEVICE_RAW_USER_ID_CHANGE_EVENT |
| was_in_active |
Boolean |
Whether the device was in-active before, this gets significance in exit to know if device has exited from in-active or entry |
| device |
Device |
Profile data of the device. Plese see PROFILE_UPDTE Events Fields section |
| location |
Location |
Location Data of the device. Please see the Location Fields section |
| Ssid |
String |
SSID of the device connected to |
| raw_user_id |
String |
User id used for Wi-Fi authentication |
| visit_id |
String |
Unique ID for a visit starting from entry to exit, Its same across all levels during the visit |
| days_since_last_visit |
Int32 |
Number of days since the last visit of a device at the location |
| entry_timestamp |
Int64 |
Entry timestamp in epoch millisecond |
| entry_date_time |
String |
Entry time in string format after converting to local timezone |
| exit_timestamp |
Int64 |
Exit timestamp in epoch millisecond |
| exit_date_time |
String |
Exit time in string format after converting to local timezone |
| visit_duration_minutes |
Int32 |
Visit duration in minutes [by doing (exit_timestamp - entry_timestamp) / 60 * 1000], applicable only for exit, otherwise its 0 |
| time_zone |
String |
TimeZone of a location |
| device_classification |
String |
Classification of device |
| visit_classification |
String |
Classification of visit |
| active_devices_count |
Int32 |
Current Active devices count at the location, Its count after the current device presence event |
| in_active_devices_count |
Int32 |
Current InActive devices count at the location, Its count after the current |
Attribute Fields
| Field |
Data Type |
Description |
| Name |
String |
Name of the field |
| values |
List<String> |
Values associated with the field |
Social Network Info Fields
| Field |
Data Type |
Description |
| social_network |
String |
Social Network contains either os these values FACEBOOK, TWITTER, LINKEDIN |
| social_handle |
String |
Handle/ID of the Social network |
| social_info |
Object |
It contains one of these objects either Facebook, Twitter or LinkedIn based on social network. Please see Facebook Fields section, Twitter Fields section, LinkedIn Fields section. |
Location Fields
| Field |
Data Type |
Description |
| location_id |
String |
Location ID of the location in DNASpaces |
| Name |
String |
Name of the location in DNASpaces |
| inferred_location_types |
List<String> |
Types of the location inferred by DNASpaces |
| parent |
Location |
Parent of the location in DNASpaces Location Hierarchy. |
Facebook Fields
| Field |
Data Type |
Description |
| id |
String |
ID of the user in Facebook |
| first_name |
String |
First name of the user in Facebook |
| last_name |
String |
Last name of the user in Facebook |
| middle_name |
String |
Middle name of the user in Facebook |
| name |
String |
Name of the user in Facebook |
| short_name |
String |
Short name of the user in Facebook |
| name_format |
String |
Name format used in Facebook |
| picture |
String |
Picture image url in Facebook |
| |
String |
Email of the user in Facebook |
| attributes |
List<Attribute> |
Other non-standard fields of the user available in Facebook. Please see Attribute Fields section |
Twitter Fields
| Field |
Data Type |
Description |
| id |
String |
ID of the user in Twitter |
| name |
String |
Name of the user in Twitter |
| screen_name |
String |
Screen name of the user in Twitter |
| friends_count |
Int32 |
Friends count of the user in Twitter |
| followers_count |
Int32 |
Followers count of the user in Twitter |
| profile_image_url |
String |
Profile image url in Twitter |
| profile_banner_url |
String |
Profile banner url in Twitter |
| location |
String |
Location info in Twitter |
| statuses_count |
Int32 |
Statuses count of the user in Twitter |
| |
String |
Email of the user in Facebook |
| profile_verified |
Boolean |
Verified status of the user profile in Twitter |
| utc_offset |
String |
Timezone UTC offset of the user in Twitter |
| time_zone |
String |
Timezone of the user in Twitter |
| geo_enabled |
Boolean |
Geo Enabled status of the user in Twitter |
| lang |
String |
Language of the user in Twitter |
| attributes |
List<Attribute> |
Other non-standard fields of the user available in Twitter. Please see Attribute Fields section |
LinkedIn Fields
| Field |
Data Type |
Description |
| id |
String |
ID of the user in LinkedIn |
| first_name |
String |
First name of the user in LinkedIn |
| last_name |
String |
Last name of the user in LinkedIn |
| profile_picture |
String |
Profile Picture image url in LinkedIn |
| |
String |
Email of the user in LinkedIn |
| attributes |
List<Attribute> |
Other non-standard fields of the user available in LinkedIn. Please see |
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation.
To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What’s New in Cisco Product Documentation RSS feed. The RSS feeds are a free service.
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
All printed copies and duplicate soft copies are considered un-Controlled copies and the original on-line version should be referred to for latest version.
Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/c/en/us/about/legal/trademarks.html. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)
© 2020 Cisco Systems, Inc. All rights reserved.
Feedback