Key Components of a Solution Using Edge Computing

Key Components enabling Edge Computing include the IoT edge gateway, the IOx framework for creating edge software, and Kinetic GMM for managing the edge applications. Each is described in more detail below.

IoT Edge Gateway Routers

IoT edge gateways are a critical component to the success of your IoT operations because they provide the ability for your IoT network to connect to local devices, applications, and ultimately to your cloud services. Cisco provides IoT gateways that also provide a layer of security to protect your IoT devices and to help prevent your IoT devices from being hacked. Cisco's IoT gateways can also function as a standalone device, processing data directly from sensors via WiFi, wired, or serial input connections providing a battle-hardened device for the data you want to process at the edge.

There are three Cisco Industrial router models that provide Edge Compute capabilities with an ability to run IOx applications at the edge of the IoT fabric:

■The IR829 is a fleet-targeted mobile gateway that addresses most use cases for fleets. The goal is to provide an off-the-shelf solution for the widest possible number of fleet applications with the fewest number of SKUs and least amount of custom developments possible. This Fleet Gateway is a ruggedized router integrating WiFi and Cellular radios.

■The IR809 targets markets such as Distribution Automation, ATM, POS, Telemetry, Enterprise Fleet, mobile machine-to-machine (M2M), bill boards, and so forth. IR809 brings in 4G LTE capabilities to small form factor M2M routers.

■The IR1101 is like the IR829 and is a general-purpose industrial router but lacks WiFi capability. Additionally, there are essential IOx differences in the IR1101 that are important to note. This information is provided below.

These routers ship with one Cisco provided IOXVM (virtual machine which enables IOx on the platform). All IOx applications run within this virtual environment. In certain cases, a customer may decide to install a custom operating system on the platform. In this situation, IOx support ceases to exist on the platform (and for the customer).

Figure 2 IOx Framework

The Cisco IOx framework provides a powerful platform for your Ops and App Developers to easily deploy applications to your IoT gateways. Cisco IOx utilizes Docker tooling to allow your development teams to build applications in a standard format that is familiar to cloud-native application developers. The Cisco IOx application environment combines the power of the Cisco IOS and the Linux OS to provide highly secure networking. This enables you to execute IoT applications in the fog or at the edge with secure connectivity to the Cisco IOS software and get powerful services for rapid, reliable integration with IoT sensors and the cloud. Cisco IOx enables development and deployment of IoT business and control applications at the edge.

By bringing application execution capability to the source of the IoT data, customers can overcome challenges with high volumes of data and the need for automated, near-real time system responsiveness. Cisco IOx offers consistent management and hosting across network infrastructure products, including Cisco routers, switches, and compute modules. Cisco IOx allows application developers to work in the familiar Linux application environment with their choice of languages and programming models with familiar open-source development tools.

Figure 3 Cisco IOx Application Framework

Kinetic GMM for IOx Life-Cycle Management

Building on Cisco IOx is Cisco Kinetic. Kinetic takes the best of Cisco IOx application management, adds IoT gateway management and automates/manages it for you as a cloud service allowing you to scale our operations. This provides ease of use for your Ops teams by allowing for both:

■Auto-provisioning of Cisco IoT gateway devices

■Easily connecting your gateways to your Kinetic account

Fog apps (for edge computing) are Cisco IOx applications that run on a Cisco gateway and transmit data from devices to the Cisco Kinetic cloud or other data destinations. Applications can be deployed and managed with the Cisco Kinetic Gateway Management Module (GMM).

Refer to Figure 4 for the deployment steps for an edge/fog application:

Figure 4 Cisco IOx Deployment Life-cycle

Figure 4 depicts the high-level steps involved as part of the IOx deployment life-cycle.

The Edge/fog application download process is embedded in the overall management of the gateway. When a gateway is powered on, it communicates with GMM to download provisioning information and update/download Edge/Fog applications and ensure that the connected sensors are detected by the gateway. This way, a field technician can easily deploy a gateway that with minimal hands-on effort.

Beyond just the deployment of IOx applications, Cisco Kinetic GMM can be used to manage the entire life-cycle of an IOx application, including:

■Centrally deploy an IOx application to a set of edge gateways.

■Change the state of an IOx application (start/stop/restart).

■Monitor the status of IOx applications.

■Upgrade IOx application to the latest version.

■View container and application logs.

■Monitor events related to IOx application deployment.

The advantage of using Cisco Kinetic GMM is that you can now centrally deploy and manage the application on hundreds of gateways from a centralized pane of glass without having to log in or connect to each of the gateways individually. IOx applications can be deployed to a set of gateways, helping save deployment time and cost.

IOx Framework Details

The IR809/829 gateway routers ship with one Cisco provided IOXVM (virtual machine which enables IOx on the platform. All IOx applications run within this virtual environment. In certain cases, a customer may decide to install a custom operating system on the platform. In this situation, IOx support ceases to exist on the platform (and for the customer).

These routers ship with one Cisco-provided IOXVM which enables IOx on the platform. All IOx applications run within this IOXVM.

In certain cases, a customer may decide to install a custom operating system on the platform. In this situation, IOx support ceases to exist on the platform (and for the customer).

Figure 5 IOx Architecture for IR8x9

Both the IR829 and the IR809 use the Intel Rangeley Dual-Core CPU, 2GB DDR3 memory, 8MB SPI Bootflash, and 8GB (4GB usable) eMMC bulk flash.

The hypervisor is provided by LynxWorks and it runs on the bare metal hardware upon which IOS and a Guest OS (e.g., Linux) run as two separate virtual machines (VMs).

The hypervisor presents underlying hardware to virtual machines (IOS and Guest OS) as a subset of actual physical hardware. The allocation of devices to VMs is provided by a configuration to the hypervisor. The VMs access a virtual CPU, pre-configured memory regions, pre-divided flash disk storage, and other hardware devices. LynxSecure Separation Kernel v. 5.1 has been selected as a hypervisor.

Each PCI device can be owned exclusively by one VM in the hypervisor architecture. However, IOS and the Guest OS need to access some shared devices, for example eMMC flash. The solution is a Virtual Device Server (VDS), which is a separate VM that owns shared devices. IOS and Guest OS access the virtual devices emulated in hypervisor. The hypervisor and VDS then coordinate access to the shared devices. The VDS also provides communication channels between VMs using emulated Ethernet interfaces.

IOS acts as a gateway to network resources for the Linux partition, using IP and a virtual switch connection provided to the hypervisor. IOS and Linux each operate as a Guest OS of the hypervisor.

The network path from IOXVM to IOS is available via an emulated Ethernet link between them. Either IPv4 or IPv6 protocol can be run on the Ethernet link.

The IOXVM runs CAF and other IOx infrastructure elements on the host Linux and hosts all the LXC applications.

The IR1101 modular gateway also supports hosting IOx applications. However, the IOx architecture for the IR1101 differs from other Cisco platforms that use the hypervisor approach. IOx runs as a process on the IR1101 versus as a virtual machine on others. Additionally, the only type of container supported on the IR1101 is the LXC container

IOx Application Resource Profile

IOx nodes may have different CPU architectures, so it becomes very complex to characterize an application’s performance or requirements on each of them. IOx attempts to bring consistency and uniformity by using the notion of resource profiles. A resource profile encapsulates a set of resources (CPU, memory, etc.) under a unique name in a consistent fashion across all Cisco IOx platforms.

The intent of resource profiles is to allow developers to obtain some level of consistency when they test their applications on one platform and want to deploy the same applications on other platforms. Currently, only CPU and memory are characterized under a resource profile.

Resource Profile Definitions

Currently, the pre-defined resource profiles in Table 2 are provided. However, the exact set of profiles supported on a specific platform is a function of the available resources on that platform. Also, if one of the pre-defined resource profiles do not meet your requirements, you can always define your own custom profile based on the resources required for your application.

■Memory assignment is platform agnostic and can be directly assigned based on the application requirements irrespective of the platform. However CPU resources are highly platform dependent and performance may vary based on the underlying CPU architecture.

■To abstract these disparate characteristics to application developers and provide a uniform and consistent view, CPU resources are expressed in the form of “units”. This means an app with “x” CPU units would have similar performance on all Cisco supported heterogeneous platforms.

■The CPU unit values for a platform are obtained by executing standard benchmarking tools on that platform and assigning a unit value based on their relative score when compared against a standard base platform.

■To get a relative idea of what those units could mean in comparison to a standard CPU, here is a sample comparison with a generic x86 Intel platform:

–Based on the benchmarking results, an x86-based 64 bit Intel Xeon processor with one core of CPU @ 2GHz will have 10000 CPU units. Based on this value developers can extrapolate and test an application in their devnet sandbox environment based on the same CPU characteristics and check the CPU units required for an app.

A note about resource bounds:

■The CPU units allocated to an app are the minimum guaranteed. This means that at any given point in time depending upon the number of applications running, an app under consideration will get the guaranteed CPU units and may even get more than that if there is no other contention for CPU units.

■Memory, however, is a hard limit. That is, at no point in time will the application get more memory than what is defined. Going beyond this limit typically results in a KILL signal to the application.

IOx Application Package

Anatomy of an Application Package

An IOx application package consists of:

■One package descriptor file named package.yaml, which should be in the root of the package.

■Zero or one application configuration file named package_config.ini. If present, it should be in the root of the package.

■Zero or one application manifest named package.mf. If present, it should be in the root of the package.

■Zero or one certificate containing signing information named package.cert. If present, it should be in the root of the package.

■One tar.gz envelope containing application or service artifacts with the name artifacts.tar.gz. These artifacts may be binaries, application code, application libraries, virtual disks, rootfs, etc. More details are provided in the following sections.

Package Format

An IOx application package should be packaged in one of the following file formats: “tar” or “tar.gz”.

Package Descriptor File

The contents of this file capture application and service metadata, requirements, etc., in a YAML format. It should be named “package.yaml”. The specifications of this file are covered in https://developer.cisco.com/docs/iox/#!package-descriptor/iox-package-descriptor

Package Configuration File

In order to bootstrap applications or services present in the package, IOx framework supports externalizing this content into a separate.ini file and provides mechanisms to edit and update the contents of this file during the provisioning of the application. This file contains sections with key, value pairs adhering to.ini file format. The name of the file if present should be package_config.ini. The administrative tools provide mechanisms to modify this file with the correct values during the installation process so that the application can be bootstrapped with the correct values.

Application Artifacts

All app artifacts are maintained in its own tarball, which offers a cleaner separation of application artifacts. This inner envelope will always be a tar.gz and is named artifacts.tar.gz. This will typically be generated by tooling (ioxclient, SDK, etc.).

Building a Dockerized IOx NodeJS Application and Deploying it Using Kinetic GMM

This section describes how to use Docker tooling to create an IOx application. Specifically, it shows how to create a Docker image to run a simple Node.js-based HTTP server and create an IOx application from it.

Figure 6 Using Docker Tools to Generate IOx Applications

Installing Docker Tools and Images

The development machine should have the following installed:

■ioxclient (version > 1.4.0)

ioxclient download: https://developer.cisco.com/docs/iox/#!iox-resource-downloads

■Docker >= version 1.12 (version 1.26 preferred)

Install Docker for Mac: https://docs.docker.com/docker-for-mac/install/

Figure 7 Verifying the Docker and IOx Client Versions on the Development Server

The sample application code is maintained at: https://github.com/CiscoIOx/docker-nodejs.git

Clone this sample code and use branch primary.

Figure 8 Cloning the Sample Code Using git

Figure 9 Inspecting the Contents of the Cloned Directory

This is a simple Node.js-based HTTP server that performs the following:

■Sets up signal handlers to shutdown gracefully.

■Inspects CAF_APP_LOG_DIR environment variable and sets up logging to a file accordingly.

■Starts HTTP server on port 8000.

■Logs the source of request and responds with “Hello World!”.

Creating the Docker Image

This section describes how to create a Docker image with the above application in it. The process involves using alpine:3.3 as the base image, installing Node.js, and setting up the application.

The following Dockerfile accomplishes these tasks:

Next build an image from this Dockerfile and tag it with a name (samplenode:1.0).

Figure 10 Building the Docker Image

Figure 11 Viewing the Docker Image

Run the image locally and test to ensure it is functioning correcting.

Figure 12 Verifying the Docker Image Locally on the Development Server

If the server sends a response to an incoming request, then the Docker image and the application should be working correctly.

Writing Package Descriptor File

Once the Docker image has been created, the developer needs to write a package descriptor file specifying requirements for the application. Here is a sample package.yaml file:

The following are important to note for compatibility:

■Descriptor schema version is 2.2, which is the minimum version that supports Docker style apps.

■Note that the cpuarch is x86_64. Alpine-based apps can only run on x86_64 bit machines.

■App type is docker.

■The required port (8000) to be opened is specified under network->ports.

■rootfs.tar is the name of the file containing the Docker image (output of Docker save command). More details are provided in the following sections.

■Command to be run when starting up the app is [“node”, “/server.js”]. Note that server.js was copied to “/” of rootfs.

Creating an IOx Application Package

Once the required Docker image (samplenode:1.0) and package.yaml are created, create an IOx application package from these artifacts. ioxclient (>= 1.4.0) provides a convenience command that generates an IOx application package from a Docker image and package.yaml file.

Figure 13 Creating the IOx Application Package

The package.tar file is an IOx application package that can be used to deploy on an IOx platform.

Note: The package.yaml uses rootfs.tar as the name of startup->rootfs. This is essential, since ioxclient saves the Docker image with the name rootfs.tar.

Deploying the IOx Application to the Gateway using Cisco Kinetic GMM

To deploy the IOx application to the gateway, follow these steps:

1. Upload the IOX Application tar file onto Kinetic GMM.

Log in to the Kinetic GMM UI using your credentials. Click Applications -> Add Application. Select the package.tar file created above. Click Import.

Figure 14 Uploading IOx Application tar File onto Kinetic GMM

2. Verify that the upload was successful.

Once the upload is successful, the application should show in the Available state under the Applications tab.

Figure 15 Verifying IOx Application tar Successfully Uploaded onto Kinetic GMM

3. Install the application onto a gateway.

■Select the SampleNodeApp application and click Install. In the pop-up window, select the appropriate application resource profile. In this case, the c1.small profile providing 200 CPU units and 64 MB RAM should be sufficient.

■Select the gateway or gateways on which you want this application to be installed from the drop-down menu.

■Leave the interface Name as default “eth0”.

■For the network name, select IOx-nat0 since access to the Node.js application from outside is required. Internally, the Node.js application listens on port 8000. Externally map this to the desired port. In the example below, it is mapped it to an external port of 8000. Hence the Node.js application can also be accessed on port 8000 externally.

■Click Install to begin installation of the IOx application onto the selected gateway(s). In the example below, a single gateway is selected on which to install this application.

Figure 16 Installing the IOx Application onto the Gateway(s)

4. Monitor the deployment.

Once you click Install, you can navigate to the Instances tab and see the application state and status as “Deploying”.

Figure 17 Monitoring IOx Application Deployment

In a few minutes you should see your application in the “Running” state. It will also display a corresponding IP address/port combination on which the application can be externally accessed.

Figure 18 IOx Application in Running State

A similar application status can be viewed under the Gateways tab by clicking the appropriate gateway on which the application has been installed and navigating to Apps. You can also view the application state in the Summary tab.

Figure 19 Viewing IOx Application State from Gateway Tab

5. Viewing the Gateway Event Log.

In case of any deployment errors, you can view the logs on the Event Log tab.

Figure 20 Inspecting Gateway Event Log

6. Externally accessing the application.

One way to access the Node.JS IOx application is to VPN into the router using the Kinetic GMM management tunnel and access the application on the specified IP address and port.

In order to VPN to the router, navigate to the Gateways tab, select the gateway in question, and click VPN.

Figure 21 Retrieving VPN Connecting Information

A pop-up window displays the requisite information needed to connect to the gateway using the Cisco AnyConnect VPN application. You need to enter your credentials to view the decrypted password.

Figure 22 Viewing VPN Connecting Information

Open up your AnyConnect Client and connect to the gateway using a secure VPN tunnel.

Figure 23 Using Cisco AnyConnect Client to Establish a VPN Connection to the Gateway

Once connected, open a web browser and connect to the gateway on the IP address and port specified for the IOx application that was deployed above. You should see the message “Hello World!”

Figure 24 Verifying IOx Application Accessibility

7. Viewing Application and Container logs.

Navigate to the Apps tab for the gateway on which the IOx application is deployed and click the application. Expand the Application Logs tab by clicking the + (plus) icon.

Figure 25 Inspecting Application and Container Logs

8. Building an updated version of the IOx application.

You can modify the application so that it responds with “Hello World! Welcome to my IOx Application” instead of just “Hello World!”. Do this by modifying the “response.end” line in the server.js file.

Figure 26 Making Modifications to the IOx Application Code

Create a new Docker image with a version tag of, for example, “2.0”.

Figure 27 Building Docker Image with Updated Code

Change the application version number in the package.yaml file to “2.0”.

Figure 28 Updating Version in package.yaml

Build an updated version of the package.tar file with the updated version of the application. Make sure to use the updated tag of “2.0” for the Docker image.

Figure 29 Re-package Updated IOx Application

There is now an updated version of the package.tar file based on the updated Docker image of samplenode:2.0.

9. Upgrading the deployed application to version 2.0 using Kinetic GMM.

Navigate to Applications and click the application you want to upgrade.

Figure 30 Upgrading to the Newer Version of the IOx Application onto Kinetic GMM

Click the Upgrade button and you see a pop-up screen where you can upload the updated version of the package.tar file that was built in the previous step. You see a small warning at the bottom of the pop-up screen indicating that all of the IOx applications will be upgraded to the newer version on all applicable gateways. Check the box I understand the risks. Proceed with editing and click Upgrade.

Figure 31 Uploading Updated IOx Application onto Kinetic GMM

Once the upgrade is successful, the application version is updated to “2.0”.

Figure 32 Verifying Updated Version of IOx Application

It is also configuring the updated version of the IOx application on any associated gateways.

Figure 33 Gateway Being Updated with Newer Version of IOx Application

Once version 2.0 of the application is rolled out to all the associated gateways, AnyConnect VPN to any one of them and verify that the updated version of the application is running.

Figure 34 Verifying Newer Version of IOx Application Successfully Running

10. Stopping the application.

To stop the application, click the check box next to the application and click Stop App.

Figure 35 Stopping IOx Application

Figure 36 Verifying IOx Application in Stopped State

In a similar manner you can click the checkbox next to the application and click Start App to start the application.

11. Uninstalling the application.

To uninstall the application, click the check box next to the application and click Uninstall. A pop-up screen is displayed asking you to confirm that you really want to uninstall the application. You can confirm by clicking the Delete button. You see the status message “Removing” and within a short time you will see that the application has been uninstalled from your gateway.

Figure 37 Uninstalling an IOx Application on a Gateway

Figure 38 Confirming Deletion of IOx Application from a Gateway

Figure 39 Verifying that IOx Application Being Removed from the Gateway