Smart Licensing on the ASR9000 Platform

Introduction

This document describes Smart Licensing Software configuration, operation, and troubleshooting on Cisco IOS^® XR Version 5.2.0 and later. Smart Licensing was developed in order to address licensing requirements management for various features and applications that run on Cisco platforms and Operating Systems (OSs).

The Smart Licensing application runs not only on ASR9000 (ASR9K) for Cisco IOS XR, but also on various platforms that run the Cisco IOS and Cisco IOS-XE OSs. This simple application greatly reduces the effort needed to manage diverse Cisco devices, systems, and platforms and brings much needed simplicity to license management, entitlements, and operational costs.

The method used by the Smart Licensing application is a dynamic 'pull' method; the ASR9K device initiates the call and pulls the information from the Cisco backend servers. Cisco backend servers WILL NOT initiate any call or connection to any device, but always respond when the connection requests come from the devices that would like to register and receive entitlement.

The initial setup is secure and easy with very little manual intervention from the operator of the device(s) and can be automated for the larger environments with a regular Tool Command Language (TcL) or Python Expect script. The reporting facilities provided by Cisco backend servers, accessible via a regular browser, will help the customers with bookkeeping of their inventory of devices, features deployed both licensed and out of compliance (OOC) and dynamically move their resources around without the need to reprovision or call for support.

Top View

Smart Licensing uses standard HTTP Secure (HTTPS) as the transport mechanism in order to reach the Cisco backend servers. Technically speaking, there is only one line of configuration that is needed in order to enable the Smart Licensing feature on the ASR9K device:

RP/0/RSP0/CPU0:SAMDD(admin-config)#license smart enable

The device defaults to HTTPS transport and upon a successful registration request, immediately queries the backend servers for entitlement. It returns either Authorized, which means the device has the license for the feature, or OOC, which means the entitlement either is not present, missing, or expired.

Note: The license compliance state WILL NOT affect the functionality of the device in any way. The Current Smart Licensing application is based on an honor system and notifies the administrator via syslog or console logs as to the compliance or OOC state. There is no functionality impediment in any way due to licensing or lack thereof. However, Cisco encourages the compliance which gives much more visibility to the customers with regards to their inventory of devices, license consumption, features used per device and in aggregate/sum total, and so on.

Note: HTTP support to the backend servers is being deprecated in CY2019, however HTTP to a satellite server will still work.

Smart Licensing can coexist with Traditional Licensing, but only one of them can be active at any given time. You can switch between them easily with the addition or deletion of the configuration from the administration plane. The ASR9K system does NOT require any reload or restart for this 'switch' to take place. Traditional Licensing will be replaced completely with Smart Licensing in future releases.

If an ASR9K device does not use a feature that requires licensing, then automatically the system is in the Authorized State and no further action needs to be taken. Only upon 'configuration' of a feature that requires a license will the system try to acquire the license dynamically from the Cisco backend servers.

Traditional Versus Smart Licensing Operations

Here are some differences between the licensing models. Note that only one of them is active at any given time.

Traditional (Node Locked) Licensing	Smart (Dynamic) Licensing
You must procure the license and manually install it on each device via the PAK file.	No software installation is needed/necessary. The device initiates an HTTP/HTTPS call-home session and requests the licenses it uses and is configured for.
Licenses tied to the chassis, moving, or reprovisioning require backup or reinstallation. All are manual operations which consume time.	Licenses tied to your account. Unconfigure the feature that is used in the current chassis and reconfigure the feature on a new chassis that needs to use the same license. A reprovision happens dynamically when the new device initiates an HTTP/HTTPS request via the call-home process.
Node-locked license - the license is associated with a specific device/slot.	License Pool(s) created already in the Customer Account, which are company account-specific and can be used with any ASR9K device in your company.
No common install base location in order to view licenses purchased or software usage trends. License bookkeeping needs to be maintained for individual chassis/systems manually.	Licenses are stored securely on Cisco backend servers, accessible 24x7x365. License count is per customer account/pool and many devices can be part of the same pool.
Extra license requires a new PAK file and manual intervention/interaction with the device.	Extra license can be transferred via a web browser that points to the Cisco URL and the account created in Backend Servers. Basically point and click operations.
No easy means to transfer licenses from one device to another.	Licenses can be moved between product instances without ANY software installation. You can also transfer licenses from one pool to the other easily with a Web interface.

Operational View

This diagram shows the comparison between the two licensing schemes.

Smart Licensing steps are very easy and intuitive. When you purchase the gear/device, you can order the licenses you need at the same time or order them later. Upon fulfillment of the purchase and provisioning of the licenses by Cisco:

• Cisco provides you a username, password, and Uniform Resource Locator (URL) to access license information via a web browser 24x7.

• This account manages licenses, generates reports, groups devices, makes pools of licenses and any other organizational need that facilitates the operational needs of the customer/organization.

• The account allows the customer to generate an idtoken, which uniquely identifies the customer device and the licensing entitlement purchased. The token can be valid from one day to one year. The idtoken can be revoked, deleted, and recreated by the customer at any time. It is a self-help model.

• The Customer uses the idtoken generated in the Cisco provided account in order to register one device or a thousand devices, as there is no limit on how many devices can use the same token. More tips on efficient use of this feature are provided in this document.

• Device registration is persistent and survives across reloads and upgrades of the system. The ASR9K device can be forced to reregister with the old idtoken or a newer one if one wishes, in case of any loss.

• No intervention is needed after registration, the ASR9K system periodically polls the account it has registered with for compliance. If the system is OOC a syslog is generated to warn the user.

Web Interface/Portal

Here is a quick tour of the web interface where the registration process begins:

Virtual Account aka License Pool is used to logically house and organize licenses per need of an organization. It is a container of licenses, registered devices for the features that require a license. You can create one pool per site, per department, and so on.

Licenses can be easily transferred from one pool to another.

Idtoken is a key generated by this account, which is used to register the ASR9K devices. It can be valid from one day to one year. The only use for the token is to register the device and after that it is not needed. The token is a stream of text that can be copied into a TcL or Python script in order to automate remote device registration.

For example, you can create a token for one day and send it to a remote site to be used by remote hands for device registration. It expires in one day and remote hands cannot use it in order to register any other device. Even if it is used to register devices that do not belong to your company, you will easily see the device in the Product Instance Tab and can take actions in order to revoke the license.

Report dynamically generates various forms of inventory and can be exported into an Excel format for offline use, bookkeeping, or analysis.

The License Tab displays the licenses requested by various ASR9K devices, which shows the count and state of each license. The Transfer link item can be used when you click on it directly and easily transfers licenses to and from any pool in the account.

The Event Log Tab records activities of the devices against the pool with a syslog type format and logs actions each device or user of the account takes, such as registration, deregistration, and so on. The interface is easy and intuitive for navigation or debugging.

Configuration

This example takes a look at how to upgrade from Traditional Licensing to Smart Licensing. Note that in some cases Smart Licensing might be the default.

Traditional Licensing

In order to check Traditional Licensing, a few commands can be run from the admin plane. Here are a few which have different outputs when compared to Smart Licensing.

Note: Traditional licensing is the default licensing mode in Cisco IOS XR releases 5.3.0 and earlier.

RP/0/RSP1/CPU0:ROA(admin)#show license pools

Pool: Owner
Feature: A9K-24X10-OPT-LIC A9K-24X10-VID-LIC A9K-24X10G-AIP-SE A9K-24X10G-AIP-TR
 A9K-2X100-OPT-LIC A9K-2X100-VID-LIC A9K-2X100G-AIP-SE A9K-2X100G-AIP-TR
 A9K-36X10-OPT-LIC A9K-36X10-VID-LIC A9K-36X10G-AIP-SE A9K-36X10G-AIP-TR
 A9K-400G-AIP-SE A9K-400G-AIP-TR A9K-400G-OPT-LIC A9K-400G-VID-LIC
 A9K-800G-AIP-SE A9K-800G-AIP-TR A9K-800G-OPT-LIC A9K-800G-VID-LIC
 A9K-ADV-OPTIC-LIC A9K-ADV-VIDEO-LIC A9K-AIP-LIC-B A9K-AIP-LIC-E

RP/0/RSP1/CPU0:ROA(admin)#show license allocated

FeatureID: A9K-800G-AIP-SE (Slot based, Permanent)
Total licenses 1
Status: Allocated   1
   Pool: Owner
     Total licenses in pool: 1
     Status: Operational:   1
     Locations with licenses: (Active/Allocated) [SDR]
             0/0/CPU0       (0/1) [Owner]

A subset of Traditional Licensing commands can also be run from exec plane, but it is a good idea to run them from the admin plane, which has the full list.

RP/0/RSP1/CPU0:ROA#show license ?    
WORD       Feature ID
active     Currently checked-out/being used by applications.
allocated  Allocated to a slot but not used.
available  Not currently active.
evaluation Display the evaluation licenses.
expired    Display evaluation licenses already expired.
location   Show information for a specific location
log        The operational or administrative logs.
|          Output Modifiers
<cr>

Smart Licensing

Smart Licensing has not been enabled yet, but this is what the system displays.

Even though no configuration is applied, the default built-in profile of call_home uses HTTPS, which points to the Cisco backend servers via the systems management ports. See more on call_home later in this document.

RP/0/RSP1/CPU0:ROA#show run call-home  
% No such configuration item(s)

RP/0/RSP1/CPU0:ROA#show call-home detail | i https
   http proxy: Not yet set up
   HTTP address(es): https://tools.cisco.com/its/service/oddce/services/DDCEService

For a bare minimum configuration, you only need steps 1 and 4. The rest of the steps are for information, verification, and reporting.

1. In admin mode, enter these commands:

   RP/0/RSP1/CPU0:ROA(admin-config)#license smart enable
   RP/0/RSP1/CPU0:ROA(admin-config)#commit

2. In exec mode configure more knobs, such as email address, or use this default profile which is generated automatically when the admin configuration is committed.

   RP/0/RSP1/CPU0:ROA#show run call-home
   call-home
   service active
   contact-email-addr sch-smart-licensing@cisco.com
   profile CiscoTAC-1
   active
   destination transport-method http

3. In admin mode, check the Smart Licensing version:

   RP/0/RSP1/CPU0:ROA(admin)#show license version
   Cisco Smart Licensing Agent, Version 1.1.4_throttle/16

4. In admin mode, enter this command:

   RP/0/RSP1/CPU0:ROA(admin)#license smart register idtoken
    NjgyMWM2NDItMzI5My00YzQ2LThmMDItMzhhNWI2Mzk2YWUwLTE0MzUzMzM3%
   0aMDQwNDB8SWRzSGkvR0d2MWZTZEhzK2RWUmJWMmh0U1ZIa2tBVzBLZKl1ZHhs%0AZGRPbz0%3D%0A ?
   force Force Registration
   <cr>  
   
   license smart register: Registration process is in progress. Please check
    the syslog for the registration status and result

   The keyword Force overwrites and wipes out any and all information in regards to the device that was registered previously. The keyword force should be used sparingly and in special cases. Alternatively, the Web User Interface can be used in order to remove the device from the account.

5. Query for the status of the operation:

   RP/0/RSP1/CPU0:ROA(admin)#show license register-status
      Registration Status: Completed
          Registration Start Time: Wed Dec 17 2014 13:07:23 PST
          Next ID Cert Renew Time: Mon Jun 15 2015 14:07:45 PST
          Next ID Cert Expiration Time: Thu Dec 17 2015 13:01:41 PST
          Last Response Time: Wed Dec 17 2014 13:07:45 PST
          Last Response Message: OK: OK

   If the Status is not 'Completed', you will see messages on the console or syslog. Here is the successful syslog message:

   RP/0/RSP1/CPU0:Dec 17 13:07:45.285 : licmgr[310]: SMART_LIC-6-AGENT_REG_SUCCESS:
   Smart Agent for Licensing Registration with Cisco licensing cloud successful
   RP/0/RSP1/CPU0:Dec 17 13:08:18.357 : licmgr[310]: SMART_LIC-3-OUT_OF_COMPLIANCE:
   One or more entitlements are out of compliance':

6. On this system there are few features configured that require licenses and this output indicates the status of 'Out of compliance':

   RP/0/RSP1/CPU0:ROA(admin)#show license entitlement | i Tag | e Not | u sort
      Tag: regid.2014-04.com.cisco.A9K-24X10-OPT-LIC,
   1.0_66d3ccf7-a374-4409-a3f9-6bc56d645f1c, Version: 1.0, Enforce Mode:
    Out of compliance
      Tag: regid.2014-04.com.cisco.A9K-24X10-VID-LIC,1.0_9f03b94f-3c76-4a39-82f2
   -1b53cdf5cb15, Version: 1.0, Enforce Mode: Out of compliance
      Tag: regid.2014-04.com.cisco.A9K-24X10G-AIP-TR,1.0_e5d7cec3-e8e3-43c6-88c9
   -a113b76679f8, Version: 1.0, Enforce Mode: Out of compliance
      Tag: regid.2014-06.com.cisco.A9K-2X100-OPT-LIC,1.0_0f74bb00-42af-4c4d-b162
   -bcb346c7510a, Version: 1.0, Enforce Mode: Out of compliance
      Tag: regid.2014-06.com.cisco.A9K-2X100-VID-LIC,1.0_a482b964-6371-4aad-8e82
   -2083c5749205, Version: 1.0, Enforce Mode: Out of compliance
      Tag: regid.2014-06.com.cisco.A9K-2X100G-AIP-SE,1.0_ce447831-e4af-4def-a98b
   -3297fab65561, Version: 1.0, Enforce Mode: Out of compliance
      Tag: regid.2014-06.com.cisco.A9K-36X10-OPT-LIC,1.0_92a8597a-f591-4afc-adeb
   -9b212cee11be, Version: 1.0, Enforce Mode: Out of compliance

7. Look at the commands you used in Traditional Licensing, which have different output.

   Either the Smart Licensing OR the Traditional Licensing CLI is available at any given time, not both.

   The pool name is used to organize/categorize devices. You can use one pool per region/geography, or department or functional area, or financial groupings, and so on. Each company can decide how they would like to pigeonhole licenses. Also note that it is very easy to use your normal browser in order to view, change, or move licenses between pools, add or change the license counts, and do so easily without any help from Cisco, independently, around the clock.

   RP/0/RSP1/CPU0:ROA(admin)#show license pool
   Assigned Pool Info: PATRICK_NO_LIC

8. From here on, the system checks every day for compliance automatically. If there is a failure, the system tries every 20 minutes for four hours and after that once a day for 30 days. Syslog messages are printed, which indicate the connectivity, reachability, communication, and so on reasons for failures. Debugging is discussed more later in this document.
9. In order to deregister the device, enter these commands:

   RP/0/RSP1/CPU0:ROA(admin)#license smart deregister
   
   license smart deregister: Success
   
   
   License command "license smart deregister " completed successfully. 

   RP/0/RSP1/CPU0:ROA(admin)#show license register-status
      Registration Status: Not Registered

10. In order to find out what licenses are available on a given chassis, enter this command:

    RP/0/RSP1/CPU0:ROA(admin)#show license features
    
    Platform Feature ID:
    A9K-ADV-OPTIC-LIC
    A9K-ADV-VIDEO-LIC
    A9K-iVRF-LIC
    A9K-AIP-LIC-B
    A9K-AIP-LIC-E
    A9K-MOD80-AIP-TR
    A9K-MOD80-AIP-SE
    A9K-MOD160-AIP-TR
    A9K-MOD160-AIP-SE
    A9K-2X100G-AIP-TR
    . . . output snipped . . .

Application's Anatomy and Flow

In order to understand the mechanics of the application, you need to have a basic understanding of its components. For operation or deployment of the software, however, no prior knowledge is necessary other than to follow the published guidelines. This section is intended more for technical staff and engineers that would like to know the details.

Deployment, Configuration, and Options

Smart Licensing can be deployed in several scenarios dependent upon the requirements with respect to security, manageability, and operational mode of the customer.

For example:

• You might choose NOT to allow the ASR9K to connect 'directly' with Cisco Cloud/Backend servers. In this case you can use a 'proxy' server on your premises and manage the firewall, traffic flow, and how the Smart Licensing application fits in the organization's security needs. This can be easily set up via Open Source Apache software that runs on Windows or Linux OSs.
• Or you might want to have all your ASR9K devices connected to an aggregator host which can receive all the local requests from all the ASR9K devices before you forward them to Cisco Backend Servers. This is a job for Transport Gateway software that runs on Linux and Windows and is available for download at Cisco Transport-Gateway download.
• Or you might want to operate totally offline with On-Prem software that runs on Linux and Windows and allows you to have only 'this On-Prem host' to do the talking for licensing information exchange with the Cisco Cloud and in turn provide information to the end devices as to their state of compliance. This software will be available in Release 5.3.1 or later.

In addition to support for HTTPS, the software can also be configured to run in a Virtual Routing Forwarding (VRF) setting which allows greater level of control over how licensing information is transported.

Furthermore, IPv6 is supported natively and only requires a valid IP6 address on the system in order to communciate with Cisco backend servers over the Internet.

These configurations assume that the ASR9K is configured with Domain Name System (DNS) or IPv4/IPv6 domain host so that it can resolve host names in order to reach the outside network.

Configuration of Network Time Protocol (NTP) is necessary in order to keep the system in-sync with the backend certificate servers.

RP/0/RSP0/CPU0:ROA#show run domain
domain name cisco.com
domain list cisco.com
domain name-server 171.70.168.183
domain name-server 2001:420:68d:4001::a

RP/0/RSP0/CPU0:ROA#show run | i ipv6 host
Building configuration...
domain ipv6 host tools.cisco.com 2001:420:1101:5::a

Configure HTTP Proxy

Apache configuration is out of the scope of this paper, but there are many good documents on the Internet that can walk you through the steps. In order to demonstrate the functionality, Apache is configured for a simple proxy on port 80. See the debug output from Apache’s mod_proxy shown here.

For Smart Licensing, however, the configuration is very simple, just mention the proxy server’s name and the port. The configuration will simply forward the request to the proxy server instead of contacting the Cisco backend servers directly. The proxy server will contact the Servers over whatever transport is configured to forward the requests; HTTPS is recommended. Apart from http-proxy mybastion.cisco.com port 80, no any other configuration is required.

RP/0/RSP0/CPU0:ROA#show run call-home
call-home
service active
http-proxy mybastion.cisco.com port 80
contact-email-addr sch-smart-licensing@cisco.com
profile CiscoTAC-1
active
destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
destination transport-method http

Enter the registration command admin license smart register idtoken <idtoken> and observe that the output shows the request/response made by the ASR9K. Note the timestamps and Success column counters.

RP/0/RSP0/CPU0:ROA#show call-home smart-licensing statistics
Success: Successfully sent and response received.
Failed : Failed to send or response indicated error occurred.
Inqueue: In queue waiting to be sent.
Dropped: Dropped due to incorrect call-home configuration.

Msg Subtype     Success Failed Inqueue Dropped Last-sent (GMT-08:00)
----------------------------------------------------------------------
ENTITLEMENT        1       0       0       0       2015-01-12 21:06:56
DEREGISTRATION     0       0       0       0             n/a
REGISTRATION       1       0       0       0       2015-01-12 21:06:21
ACKNOWLEDGEMENT    1       0       0       0       2015-01-12 21:06:38

Here is a snippet of the Apache access logs that shows the request goes out on port 443, HTTPS protocol.

root@mybastion:/var/log/httpd #tail -f proxy-*

==> proxy-error.log <==
[Mon Jan 12 21:06:10 2015] [debug] mod_proxy_connect.c(70): proxy: CONNECT:
 canonicalising URL tools.cisco.com:443
[Mon Jan 12 21:06:10 2015] [debug] proxy_util.c(1515): [client 172.27.130.65] proxy:
 *: found forward proxy worker for tools.cisco.com:443
[Mon Jan 12 21:06:21 2015] [debug] mod_proxy_connect.c(109): [client 172.27.130.65]
 (70014)End of file found: proxy: CONNECT: error on client - ap_get_brigade
[Mon Jan 12 21:06:21 2015] [debug] mod_proxy_connect.c(425): proxy: CONNECT:
 finished with poll() - cleaning up

==> proxy-access.log <==
172.27.130.65 - - [12/Jan/2015:21:06:10 -0800] "CONNECT tools.cisco.com:443 HTTP/1.1" 200 -

Configure Transport Gateway

In this scenario the Transport Gateway application is installed on a Linux or Windows host and configured to receive the licensing requests from ASR9K devices on the customer premises and relay them to the Cisco backend servers. See the Transport Gateway Deployment and User Guide for more information.

The configuration on the ASR9K is just one line. Here is a sample; consult the documentation for the exact configurations needed for your environment.

call-home
profile CiscoTAC-1
destination address http
https://TG-IP-or-FQDN/Transportgateway/services/DeviceRequestHandler

Configure VRF

VRFs allows more control over management traffic and are almost transparent to Smart Licensing. However, one line configuration is necessary in order to make the underlying software consult the VRF table rather than the global table when Smart Licensing software tries to reach the Cisco backend servers.

The string shown here is the VRF name configured in the system.

RP/0/RSP0/CPU0:ROA(config)#http client vrf MGMT

Call Home Detailed Output

A sample output to verify if Call Home works properly is shown here.

RP/0/RSP0/CPU0:ROA#show call-home detail

Current call home settings:
   call home feature : enable
   call home message's from address: mylab-roa@cisco.com ; optional, any address
   call home message's reply-to address: pasoltan@cisco.com ; optional,
 recipient address

   vrf for call-home messages: Not yet set up ; Not supported natively yet

   contact person's email address: sch-smart-licensing@cisco.com ; default

   contact person's phone number: +1-408-526-8438 ; optional
   street address: 1550 Soltani Lane, Cisco System Drive, North Pole, NP 99709
   customer ID: Not yet set up
   contract ID: Not yet set up
   site ID: BUILDING20-125 ; optional

   source interface: Not yet set up ; can be configured to use a specific interface.
   Mail-server[1]: Address: bastion.cisco.com Priority: 1 ; optional
   Mail-server[2]: Address: 171.68.58.10 Priority: 10     ; optional
   Mail-server[3]: Address: 173.37.183.72 Priority: 20   ; optional
   http proxy: Not yet set up ; when configured will change.

   Smart licensing messages: enabled
   Profile: CiscoTAC-1 (status: ACTIVE) ; default profile supported.
Can not be renamed, deleted, but can be modified, activated, deactivated.

   aaa-authorization: disable      ; optional
   aaa-authorization username: callhome (default) ; default
   data-privacy: normal     ; can be configured to use the hostname or not.
   syslog throttling: enable

   Rate-limit: 5 message(s) per minute

   Snapshot command: Not yet set up
; Non-smart licensing configuration for alerts, data collection, defaults.
Available alert groups:
   Keyword                State   Description
   ---------------------- ------- -------------------------------
   configuration         Enable configuration info
   environment           Enable environmental info
   inventory             Enable inventory info
   snapshot              Enable snapshot info
   syslog                Enable syslog info

Profiles:

Profile Name: CiscoTAC-1
   Profile status: ACTIVE
   Profile mode: Full Reporting
   Reporting Data: Smart Call Home, Smart Licensing
   Preferred Message Format: xml
   Message Size Limit: 3145728 Bytes
   Transport Method: http
   Email address(es): callhome@cisco.com
   HTTP address(es): ; Only configuration needed if default is not desired.
http://tools.cisco.com/its/service/oddce/services/DDCEService
                       https://tools.cisco.com/its/service/oddce/services/DDCEService

   Periodic inventory info message is scheduled every 23 day of the month at 11:2

   Alert-group              Severity
   ------------------------ ------------
   environment               minor      
   inventory                 normal    

   Syslog-Pattern           Severity
   ------------------------ ------------
   .*                        critical

Call Home Non-Smart Licensing Configuration Options

You can configure Call Home to do syslog and diagnostic data collection as well as core dumps, or have it send email notifications for events and so on along with the the smart licensing chores it completes.

You can view your Call Home collected information with your Smart Licensing username and password at https://tools.cisco.com/sch/reports/deviceReport.do.

See the documents linked in the "Related Information" section for more information on how to use this feature in order to benefit your environment. Also a sample of email notification is in the "Odds and Ends" section .

Debug

There are no hard and fast rules to debug Smart Licensing software due to many components that comprise the package. However, a few common approach methods usually narrow down the issues. Here are some suggestions.

Syslogs

Look into the syslog first. You will get some clues as to what component should be checked first. In these messages you see some certificate issues and a failure to send Call Home HTTP messages; finally communication is restored.

RP/0/RSP0/CPU0:ROA#sh log | i SMART

RP/0/RSP1/CPU0:Dec 17 20:01:28.522 : licmgr[308]: SMART_LIC-3-ID_CERT_RENEW_FAILED:
ID certificate renewal failed: Response error: {"product_instance_identifier":
["ProductInstance '8baecfb5-2688-429b-8519-10a3f0dec6b5' is not valid"]}

RP/0/RSP1/CPU0:Dec 17 20:01:34.273 : licmgr[308]: SMART_LIC-3-AUTH_RENEW_FAILED:
Authorization renewal with Cisco licensing cloud failed: Response error:
 LS_UNMATCH_SIGNED_DATA: Signed data and certificate does not match

RP/0/RSP0/CPU0: Dec 17 18:26:24.009 : licmgr[314]: SMART_LIC-3-COMM_FAILED:
Communications failure with Cisco licensing cloud: Fail to send out Call Home
 HTTP message

RP/0/RSP0/CPU0:Dec 17 18:28:03.057 : licmgr[314]: SMART_LIC-3-AGENT_REG_FAILED:
Smart Agent for Licensing Registration with Cisco licensing cloud failed:
 Communication message send error

RP/0/RSP0/CPU0:Dec 17 18:30:09.247 : licmgr[314]: SMART_LIC-5-COMM_RESTORED:
Communications with Cisco licensing cloud restored

RP/0/RSP0/CPU0:Dec 17 18:30:21.923 : licmgr[314]: SMART_LIC-6-AGENT_REG_SUCCESS:
Smart Agent for Licensing Registration with Cisco licensing cloud successful

Check the show command output in order to get a handle on what state the box/component is in. Here you see mobility, Internet Protocol Security (IPsec), and optical licenses.

RP/0/RSP0/CPU0:ROA#admin show license entitlement
Entitlement:
  Tag: regid.2014-06.com.cisco.A9K-MOBILE-LIC,1.0_e447924c-0a6f-41be-9202-8ae60fcc2972,
 Version: 1.0, Not In Use
  Requested Time : NA, Requested Count: NA
  Vendor String:

  Tag: regid.2014-09.com.cisco.A9K-IPSEC-20G-LIC,1.0_a165db99-eb3f-474b-bdf0-
ce4b140d9b45, Version: 1.0, Not In Use
  Requested Time : NA, Requested Count: NA
  Vendor String:

 Tag: INSTALLMGR, Version: 1.0, Not In Use
  Requested Time : NA, Requested Count: NA
  Vendor String:

  Tag: regid.2014-04.com.cisco.A9K-24X10-OPT-LIC,1.0_66d3ccf7-a374-4409-a3f9-
6bc56d645f1c, Version: 1.0, Enforce Mode: Out of compliance
  Requested Time : Mon Jan 12 2015 20:47:07 PST, Requested Count: 1
  Vendor String:
... output snipped ...

Check the license compliance.

RP/0/RSP0/CPU0:ROA#admin show license status
Compliance Status: Out of compliance

Check which pool is active.

RP/0/RSP0/CPU0:ROA#admin show licence pool
Assigned Pool Info: PATRICK_NO_LIC

Chcek the licensing certificate.

RP/0/RSP0/CPU0:ROA#admin show license cert
 Licensing Certificates:
   ID Cert Info:
    Start Date: Mon Jan 12 2015 21:00:13 PST. Expiry Date: Tue Jan 12 2016 21:00:13 PST
    Serial Number: 24724
    Version: 3
    Subject/SN: 60fe47f8-aaaa-40fc-ae3e-fae9c7b6d0ac
    Common Name: 138091632beb1f2e38069e9eec8f9c626de471ac::1,2
   Signing Cert Info:
    Start Date: Wed Sep 11 2013 12:05:34 PST. Expiry Date: Sun May 30 2038 12:48:46 PST
    Serial Number: 3
    Version: 3

Check the licensing version.

RP/0/RSP0/CPU0:ROA#admin show license version
Cisco Smart Licensing Agent, Version 1.1.4_throttle/16

This command shows the statistics on call-home attempts, which succeeded and/or failed.

RP/0/RSP1/CPU0:ROA#show call-home smart-licensing statistics
Success: Successfully sent and response received.
Failed : Failed to send or response indicated error occurred.
Inqueue: In queue waiting to be sent.
Dropped: Dropped due to incorrect call-home configuration.

Msg Subtype     Success Failed Inqueue Dropped Last-sent (GMT-08:00)
----------------------------------------------------------------------
ENTITLEMENT     1       0       0       0       2014-12-17 21:08:35
DEREGISTRATION  1       0       0       0       2014-12-17 14:33:17
REGISTRATION    1       0       0       0       2014-12-17 21:07:53
ACKNOWLEDGEMENT 1       0       1       0       2014-12-17 21:08:09
RENEW           1       0       0       0       2014-12-17 21:08:57 

Call Home Process

Check the trace files for the call_home Process next, since transport between the ASR9K and Cisco Cloud is managed by it.

RP/0/RSP0/CPU0:ROA#show call-home trace error last 2

81 wrapping entries (576 possible, 320 allocated, 0 filtered, 81 total)!
Jan 28 10:10:29.729 call_home/error 0/RSP0/CPU0 t10 call_home_http_resp_data(),
 httpc response error, Host name resolution failed

Jan 28 10:10:39.730 call_home/error 0/RSP0/CPU0 t19 call_home_events_handler() failure status 67

Smartlic Check (Software Agent)

Check the smartlic traces. These traces reveal license interaction with Cisco Cloud servers.

RP/0/RSP0/CPU0:ROA#admin show license trace smartlic last 2
987 wrapping entries (1088 possible, 0 filtered, 987 total)
Jan 28 20:10:36.245 smartlicense/smartlic 0/RSP0/CPU0 t3 [2302054]
 Failed to bind to SysDB - 'Subsystem(2091)' detected the 'success' condition
 'Code(45)': Unknown Error(292)

Jan 28 20:10:36.245 smartlicense/smartlic 0/RSP0/CPU0 t3 [2302054]
 SMART ERROR - SASACKExpirationJob: expirySeconds=3842

Licmgr Process Check

This process is the main interface to smart licensing on the ASR9K and considered the glue between various components.

RP/0/RSP1/CPU0:ROA#admin show license trace
557 wrapping entries (576 possible, 0 filtered, 5403 total)
Dec 17 13:08:18.358 license/licmgr 0/RSP1/CPU0 t3 [3125351] SLA Debug :
 Client search success pkg/bin/rsi_agent (No error)
Dec 17 13:08:18.358 license/licmgr 0/RSP1/CPU0 t3 [3125351] SLA Debug :
 A9K-MOD160-AIP-SE regid.2014-06.com.cisco.A9K-MOD160-AIP-SE,
1.0_7f1b3d9c-a183-41d1-8d0b-d98dcc2751a8 (No error)

Platform Dependent Traces

Although the Platform Dependent (PD) part of the code is just a Dynamic Link Library, it has an important role in triggering requests for license entitlement. Hence it resolves issues with regards to license types, counts, and so on.

RP/0/RSP1/CPU0:ROA#admin show license trace platform all last 5
1849 wrapping entries (5440 possible, 3136 allocated, 0 filtered, 183450 total)
Dec 17 20:43:33.480 vkg_lic/audit 0/RSP1/CPU0 t1 Agent Client Audit Cmd Start: ver:1,
 node:0x00000041 cmd:Audit(5) req:Mobile(9) feature:A9K-MOBILE-LIC(13) grant:
 Not Pending(0)
Dec 17 20:43:33.480 vkg_lic/audit 0/RSP1/CPU0 t1 Agent Client Audit Cmd Start #2:
 client restarted:False up for a day:True
Dec 17 20:43:33.480 vkg_lic/audit 0/RSP1/CPU0 t1 AUDIT Reply License Start:
 request:Mobile(9) slot:4 grant:Not Pending(0)
Dec 17 20:43:33.480 vkg_lic/audit 0/RSP1/CPU0 t1 AUDIT Reply License End:
 request:Mobile(9) slot:4 grant:Not Pending(0) rc: 0x00000000 No error
Dec 17 20:43:33.480 vkg_lic/audit 0/RSP1/CPU0 t1 Agent Client Cmd End:Audit(5),
 slot:4 rc:0x00000000 No error

Turn on the Debug

If all else fails, then turn on the debug and enter an on-demand request for renewal of the certificates or entitlements. This debug should collect all the transactions between the ASR9K and Cisco Cloud Services.

RP/0/RSP0/CPU0:ROA#debug smartlic
RP/0/RSP1/CPU0:ROA#show debug

#### debug flags set from tty 'aux0_RSP1_CPU0' ####
smartlic debug flag is ON with value 0

No direct UI/Cisco Cloud Server debugging is available. Send an email to asr9k-smart-lic@cisco.com with any issues.

Odds and Ends

1. When multiple boxes are configured to acquire entitlement from the same LICENSE POOL, even if only ONE device is short by ONE license, then ALL your devices are OOC. This is mainly due to the design that has the view of pool as the container. The new model, hierarchical organization of pools which is in the works, addresses the behavior in future releases.
2. Email yourself any show command output directly from the console. Note the double quotes and use of semicolon after each command. Call Home does many operations which are not Smart Licensing related. This is an example of what Call Home could be used for. It is a running configuration that can be modified for any environment.

   RP/0/RSP1/CPU0:ROA#show run call-home
   call-home
   service active
   site-id BUILDING20-125
   sender reply-to pasoltan@cisco.com
   sender from roa@cisco.com
   alert-group syslog
   alert-group snapshot
   alert-group inventory
   mail-server 171.68.58.10 priority 10
   mail-server 173.37.183.72 priority 20
   mail-server 2001:420:303:2008::24 priority 2
   mail-server mybastion.cisco.com priority 1
   phone-number +1-408-526-8438
   contact-email-addr sch-smart-licensing@cisco.com
   street-address 1550 E.Tasman Drive, San Jose, CA 9513
   profile CiscoTAC-1
   active
   destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService
   reporting smart-call-home-data
   reporting smart-licensing-data
   destination transport-method http
   
   RP/0/RP1/CPU0:ROA#call-home send "show run call; admin show platform"
    email pasoltan@cisco.com msg-format long-text
   
   Sending ondemand CLI output call-home message ...
   Please wait. This may take some time ...

3. The show call-home smartlic status command uses the word 'success' which simply means from a call-home process perspective the transportation of the messages from the ASR9K to Cisco Cloud Servers was successful. However, this does NOT mean that the licensing operation end-to-end with Cisco Cloud Servers was successful. For example, if there is an issue with the account, certificate, or so on with the portal, call-home transports the message and shows success, but the total operation of vetting the licenses by backend servers might fail.

   RP/0/RSP1/CPU0:ROA#show call-home smart-licensing statistics
   Success: Successfully sent and response received.
   Failed : Failed to send or response indicated error occurred.
   Inqueue: In queue waiting to be sent.
   Dropped: Dropped due to incorrect call-home configuration.
   
   Msg Subtype     Success Failed Inqueue Dropped Last-sent (GMT-08:00)
   ----------------------------------------------------------------------
   ENTITLEMENT     1       0       0       0       2014-12-17 21:08:35
   DEREGISTRATION  1       0       0       0       2014-12-17 14:33:17
   REGISTRATION    1       0       0       0       2014-12-17 21:07:53
   ACKNOWLEDGEMENT 1       0       1       0       2014-12-17 21:08:09
   RENEW           1       0       0       0       2014-12-17 21:08:57

4. When you configure the management interfaces with both IPv4 and IPv6, the order of resolution of names to IP address or DNS resolution is IPv6 first.

   RP/0/RSP1/CPU0:ROA#show run int M*
   interface MgmtEth0/RSP0/CPU0/0
   cdp
   ipv4 address 172.27.130.64 255.255.255.128
   ipv6 address fe80::172:27:130:64 link-local
   ipv6 address 2001:420:303:2008:0:28:1:64/80
   ... snipped output ...

   RP/0/RSP1/CPU0:ROA#ping tools.cisco.com
   Type escape sequence to abort.
   Sending 5, 100-byte ICMP Echos to 2001:420:1201:5::a, timeout is 2 seconds:
   !!!!!
   Success rate is 100 percent (5/5), round-trip min/avg/max = 44/45/49 ms

   RP/0/RSP1/CPU0:ROA#ping ipv4 tools.cisco.com
   Type escape sequence to abort.
   Sending 5, 100-byte ICMP Echos to 173.37.145.8, timeout is 2 seconds:
   !!!!!
   Success rate is 100 percent (5/5), round-trip min/avg/max = 43/44/45 ms

Related Information

• Smart Call Home User Guide - HTML
• Smart Call Home User Guide - PDF
• Smart Call Home Security
• Cisco Support Community
• Video: Configure Call Home
• Smart Licensing Commands - HTML
• Smart Licensing Commands - PDF
• General Information: Smart Licensing
• Smart Licensing FAQ
• Transport Gateway Guide
• Transport Gateway FAQ
• Technical Support & Documentation - Cisco Systems