The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes how to upgrade the current Identity Services Engine (ISE) version 2.4. to 2.6. set up on the Cisco ISE appliance and Virtual Machine (VM). It also includes how to use the Upgrade Readiness Tool (URT) to detect and fix any configuration data upgrade issues before the upgrade process begins.
Cisco recommends that you have knowledge of these topics:
The information in this document is based on these software and hardware versions:
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Note: The procedure is similar or identical for other ISE versions. These steps can be used on all 2.x ISE Software Releases unless stated otherwise.
A Cisco ISE deployment upgrade is a multistep process and must be performed in the order that is specified in this document. Use the time estimates provided in this document to plan for an upgrade with minimum downtime. For a deployment with multiple Policy Service Nodes (PSNs) that are part of a PSN group, there is no downtime. If there are endpoints that are authenticated through a PSN that is upgraded, the request is processed by another PSN in the node group. The endpoint is re-authenticated and is granted network access after the authentication is successful.
Use the URT to detect and fix any configuration data upgrade issues before you start the upgrade process. Most of the upgrade failures occur because of configuration data upgrade issues. The URT validates the data before the upgrade to identify, and report or fix the issue, wherever possible. The URT is available as a separate downloadable bundle that can be run on a Secondary Policy Administration node or standalone node. There is no downtime to run this tool.
Caution: URT tool cannot be run on a Primary Administration Node.
This video link explains How to use URT.
URT runs either on Secondary Administration Node or a standalone node.
Caution: The URT tool does not simulate Monitoring node (MnT) operational data upgrades.
Here is an example that demonstrates how to run URT on a standalone node (the same process can be followed on a Secondary Administration Node).
Step 1. Download the URT bundle.
Since the plan is to upgrade to version 2.6., download the URT published on Cisco.com for ISE 2.6 as shown in the image.
URT for 2.6
Step 2. Create a repository and copy the URT bundle.
It is recommended to use File Transfer Protocol (FTP) for better performance and reliability. Do not use repositories that are located across slow WAN links. It is suggested to use a local repository that is closer to the nodes.
From ISE GUI, navigate to Administration > System > Maintenance > Repository > Add as shown in the image.
Repository
Optionally, to save time, copy the URT bundle to the local disk on the Cisco ISE node with the use of this command:
copy repository_url/path/ise-urtbundle-2.6.0.xxx-1.0.0.SPA.x86_64.tar.gz disk:/
For example, if Secure FTP (SFTP) is used to copy the upgrade bundle, follow this:
(Add the host key if it does not exist) crypto host_key add host mySftpserver
copy sftp://aaa.bbb.ccc.ddd/ise-urtbundle-2.6.0.xxx-1.0.0.SPA.x86_64.tar.gz disk:/
aaa.bbb.ccc.ddd is the IP address or hostname of the SFTP server and ise-urtbundle-2.6.0.xxx-1.0.0.SPA.x86_64.tar.gz is the name of the URT bundle.
Tip: It is recommended to have the URT bundle in the local disk to save time.
Step 3. Run the URT Bundle.
Enter the application install command to install the URT:
application install ise-urtbundle-2.6.0.x.SPA.x86_64.tar.gz reponame
The warning talks about the services that run on the node and if the user would still like to continue to run the URT on this node. Type Y to proceed as shown in the image.
You notice that sometimes the URT age is old. If the bundle is the latest one downloaded from Cisco's website, it is ok to proceed further. The latest URT bundles can also be more than 45 days old. Type Y to proceed.
1. If the URT runs successfully, the output is like this:
2. The URT provides an estimated time for each of the nodes to upgrade based on the size of the configuration and MNT data.
3. After you have successfully completed the URT run, proceed to upgrade.
4. The URT:
Checks if all the prerequisites are met.
These are the prerequisites that are checked by the URT:
Note: Verify the available disk size with Disk Requirement Size. If you are required to increase the disk size, reinstall ISE and restore a config backup.
5. Clones the configuration database.
6. Copies latest upgrade files to the upgrade bundle.
Note: If there are no patches in URT bundle, then the output returns:N/A.This is expected behavior when you install a hot patch.
7. Performs a schema and data upgrade on the cloned database.
If the upgrade on the cloned database is successful, it provides an estimate of the time it takes for the upgrade to end.
If the upgrade is successful, it removes the cloned database.
If the upgrade on cloned database fails, it collects the required logs, prompts for an encryption password, generates a log bundle, and stores it in the local disk.
1. The URT can fail due to a reason that can cause issues with the upgrade. If that happens, URT returns the cause of failure.
2. Here is an example run of URT failure:
3. The URT failed with the reason: Trust certificate with the friendly name 'VeriSign Class 3 Secure Server CA - G3' is invalid: The certificate has expired.
4. As explained in the upgrade pre-checks, if ISE system has any expired certificates, the upgrade fails. It is required that all expired certificates are either renewed or replaced.
5. URT saves the failure logs which can be shared with Cisco TAC if the user is unsure of the failure reason.
6. It prompts to enter a password to encrypt the logs. These URT logs are saved in the local disk.
7. Copy them from the local disk to a repository and share it with Cisco TAC for resolution.
Before the upgrade begins, ensure that these tasks are completed:
1. Obtain a backup of the ISE configuration and operational data.
*When Cisco ISE is run on VMware, VMware snapshots are not supported for backing up ISE data.
2. Obtain a backup of the system logs.
3. Disable scheduled backups. Reconfigure the backup schedules after deployment upgrade is complete.
4. Export the certificates and private keys.
5. Configure a repository. Download the upgrade bundle and place it in the repository.
6. Make a note of Active Directory (AD) join credentials and RSA SecurID node secret, if applicable. This information is needed to connect to Active Directory or RSA SecurID server after the upgrade.
7. Purge the operational data to improve upgrade performance.
8. Ensure that the Internet connection to the repository is good.
Note: The download of upgrade bundle from a repository to a node times out if it takes more than 35 minutes to complete.
Prepare for Upgrade:
These guidelines help address the issues in the current deployment that can occur in the upgrade process. This reduces overall upgrade downtime and increase efficiency.
Latest Patch: Upgrade to the latest patch in the current version before upgrade.
Staging Environment: It is recommended to test the upgrade in a staging environment to identify and fix any upgrade issues before the upgrade for the production networks.
Patch Level: All the nodes in the Cisco ISE deployment are in the same patch level to exchange data.
Note: If all the nodes in the deployment are not on the same Cisco ISE version and patch version, a warning message: "Upgrade cannot begin" is shown. This message indicates that the upgrade is in a blocked state. Ensure that all the nodes in the deployment are in the same version (this includies the patch version, if any) before the upgrade process starts.
Operational Data (logs): It is a best practice to archive the old logs and not transit them to the new deployments. This is because operational logs restored in the MnTs are not synchronized to different nodes in case the MnT roles are changed later. If the plan is to retain the MnT logs, perform these tasks for MnT nodes and join the new deployment as MnT nodes. However, if there is no need to retain the operational logs, this can be skipped by re-imaging the MnT nodes.
In case of a necessary re-image: Cisco ISE installation can be done in parallel if it is multi-node deployment without impact to the production deployment. When you install ISE servers in-parallel, it saves time especially when backup and restores from a previous release are used. PSNs can be added to the new deployment to download the current policies at the time of the registration process from the PAN.
Use ISE latency and bandwidth calculator in order to understand the latency and bandwidth requirement in Cisco ISE deployment.
HA Datacenters: If there are Data Centers (DC) with full distributed deployment, upgrade the backup DC and test the use cases before you upgrade primary DC.
Download a day before: Download and store the latest upgrade bundle for the upgrade in a local repository before upgrade to speed up the process.
Time Prediction: For ISE upgrade from GUI, the timeout for the process is four hours. If the process takes more than four hours, the upgrade fails. If the URT takes more than four hours, Cisco recommends to use CLI for this process.
Load Balancers: Take the backup of load balancers before you change the configuration. Remove the PSNs from the load balancers at the time of the upgrade window and add them back after the upgrade.
PAN Failover: Disable automatic PAN Failover (if configured) and disable Heartbeat between PANs at the time of the upgrade.
Review Policies: Review the current policies and rules and remove outdated, redundant, and stale policy and rules.
Unwanted Logs: Remove unwanted monitoring logs and endpoint data.
Bug Check: Use the Bug Search Tool in order to find the upgrade related defects which are open or fixed.
Post Upgrade: Test all the use cases for the new deployment with fewer users to ensure service continuity.
Cisco ISE offers a GUI-based centralized upgrade from the Admin portal. The upgrade process is much simplified, and the progress of the upgrade and the status of the nodes are displayed on the screen. The Overview page under Administration > Upgrade menu option lists all the nodes in the deployment, the personas that are enabled on them, the version of ISE installed, and the status (indicates whether a node is active or inactive) of the node. The upgrade can begin only if the nodes are in the Active state.
The GUI-based upgrade from the Admin portal is supported only if ISE is on Release 2.0 or later and needs an upgrade to Release 2.0.1 or later.
In case this warning message is seen: "The node has been reverted to its pre-upgrade state", navigate to the Upgrade window, click the Details link. Address the issues that are listed in the Upgrade Failure Details window. After all those issues are fixed, click Upgrade to reinitiate the upgrade.
Step 1. Click the Upgrade tab in the Admin portal.
Step 2. Click Proceed.
The Review Checklist window appears. Read the given instructions carefully as shown in the image.
Step 3. Check the 'I have reviewed the checklist' checkbox, and click Continue.
The Download Bundle to Nodes window appears as shown in the image.
Step 4. Download the upgrade bundle from the repository to the nodes:
1. Check the checkbox next to the nodes to which the bundle is downloaded.
2. Click Download. The Select Repository and Bundle window appears as shown in the image.
3. Select the repository.
4. Check the checkbox next to the bundle that is used for the upgrade.
5. Click Confirm. Once the bundle is downloaded to the node, the node status changes to Ready for Upgrade.
Step 5. Click Continue. Then, the Upgrade Nodes window appears. Click Upgrade in order to begin.
Ensure to read the chapter titled 'Before you begin' before you proceed. Here is an example of a standalone ISE node upgrade from CLI.
Step 1. Check if the repository to be used has the upgrade file present with the use of command show repository reponame as shown in the image.
Step 2. From the Cisco ISE Command Line Interface (CLI), enter application upgrade prepare command with the bundle file name and repository name where the file is stored.
This command copies the upgrade bundle to the local repository.
Step 3. From the Cisco ISE CLI, enter the application upgrade proceed command as shown in the image.
Once this message appears, start an SSH session after 30 minutes and run the show application status ise command in order to see the progress. This message appears % NOTICE: Identity Services Engine upgrade is in progress...
The upgrade is considered complete when all the expected services' status change to running.
Note: If the upgrade fails for some reason, before you attempt an upgrade again, use the command application upgrade cleanup to clear out the old files.
1. If the Bundle takes too long to download from the repository or times out when downloaded via GUI:
2. In a distributed deployment upgrade, the error "No Secondary Administration Node in the Deployment" can be seen when:
There is no Secondary Administration node in the deployment.
The Secondary Administration node is down.
The Secondary Administration node is upgraded and moved to the upgraded deployment. Typically, this occurs if the Refresh Deployment Details option is used after the Secondary Administration node is upgraded.
In order to resolve this issue, perform one of these tasks, as applicable:
If the deployment does not have a Secondary Administration node, configure a Secondary Administration node and retry the upgrade.
If the Secondary Administration node is down, bring up the node and retry the upgrade.
If the Secondary Administration node is upgraded and moved to the upgraded deployment, use the CLI to manually upgrade the other nodes in the deployment.
3. Upgrade status for a node has not changed:
4. Roll back to the previous version of ISO image:
Revision | Publish Date | Comments |
---|---|---|
1.0 |
15-May-2020 |
Initial Release |