Cisco Identity Services Engine Administrator Guide, Release 2.0

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Book Contents
Find Matches in This Book
Available Languages
Download Options

Book Title

Cisco Identity Services Engine Administrator Guide, Release 2.0

Chapter Title

Set Up Cisco ISE in a Distributed Environment

Results

Updated:
August 26, 2022

Chapter: Set Up Cisco ISE in a Distributed Environment

Set Up Cisco ISE in a Distributed Environment

Cisco ISE Deployment Terminology

The following terms are commonly used when discussing Cisco ISE deployment scenarios:

  • Service—A service is a specific feature that a persona provides such as network access, profiler, posture, security group access, monitoring and troubleshooting, and so on.

  • Node—A node is an individual instance that runs the Cisco ISE software. Cisco ISE is available as an appliance and also as a software that can be run on VMware. Each instance, appliance or VMware that runs the Cisco ISE software is called a node.

  • Persona—The persona or personas of a node determine the services provided by a node. A Cisco ISE node can assume any of the following personas: Administration, Policy Service, Monitoring, and pxGrid. The menu options that are available through the Admin portal are dependent on the role and personas that an Cisco ISE node assumes.

  • Deployment Model—Determines if your deployment is distributed, standalone, or high availability in standalone, which is a basic two-node deployment.

Personas in Distributed Cisco ISE Deployments

A Cisco ISE node can assume the Administration, Policy Service, or Monitoring personas.

A Cisco ISE node can provide various services based on the persona that it assumes. Each node in a deployment can assume the Administration, Policy Service, and Monitoring personas. In a distributed deployment, you can have the following combination of nodes on your network:

  • Primary and secondary Administration nodes for high availability

  • Primary and secondary MnT nodes for high-availability.

  • A pair of health check nodes or a single health check node for Primary Administration Node (PAN) automatic failover

  • One or more Policy Service Nodes (PSN) for session failover

Cisco ISE Distributed Deployment

A deployment that has more than one Cisco ISE node is called a distributed deployment. To support failover and to improve performance, you can set up your deployment with multiple Cisco ISE nodes in a distributed fashion. In Cisco ISE distributed deployment, administration and monitoring activities are centralized, and processing is distributed across the Policy Service nodes. Depending on your performance needs, you can scale your deployment. Each Cisco ISE node in a deployment can assume any of the following personas: Administration, Policy Service, and Monitoring.

Cisco ISE Deployment Setup

After you install Cisco ISE on all your nodes, as described in the Cisco Identity Services Engine Hardware Installation Guide, the nodes come up in a standalone state. You must then define one node as your Primary PAN. While defining your Primary PAN, you must enable the Administration and Monitoring personas on that node. You can optionally enable the Policy Service persona on the Primary PAN. After you complete the task of defining personas on the Primary PAN, you can then register other secondary nodes to the Primary PAN and define personas for the secondary nodes.

All Cisco ISE system and functionality-related configurations should be done only on the Primary PAN. The configuration changes that you perform on the Primary PAN are replicated to all the secondary nodes in your deployment.

There must be at least one Monitoring node in a distributed deployment. At the time of configuring your Primary PAN, you must enable the Monitoring persona. After you register a Monitoring node in your deployment, you can edit the Primary PAN and disable the Monitoring persona, if required.

Data Replication from Primary to Secondary ISE Nodes

When you register an Cisco ISE node as a secondary node, Cisco ISE immediately creates a data replication channel from the primary to the secondary node and begins the process of replication. Replication is the process of sharing Cisco ISE configuration data from the primary to the secondary nodes. Replication ensures consistency among the configuration data present in all Cisco ISE nodes that are part of your deployment.

A full replication typically occurs when you first register an ISE node as a secondary node. Incremental replication occurs after a full replication and ensures that any new changes such as additions, modifications, or deletions to the configuration data in the PAN are reflected in the secondary nodes. The process of replication ensures that all Cisco ISE nodes in a deployment are in sync. You can view the status of replication in the Node Status column from the deployment pages of the Cisco ISE Admin portal. When you register a Cisco ISE node as a secondary node or perform a manual synchronization with the PAN, the node status shows an orange icon indicating that the requested action is in progress. Once it is complete, the node status turns green indicating that the secondary node is synchronized with the PAN.

Cisco ISE Node Deregistration

To remove a node from a deployment, you must deregister it. When you deregister a secondary node from the Primary PAN, the status of the deregistered node changes to standalone and the connection between the primary and the secondary node will be lost. Replication updates are no longer sent to the deregistered standalone node.


Note

You cannot deregister a Primary PAN.

Guidelines for Setting Up a Distributed Deployment

Read the following statements carefully before you set up Cisco ISE in a distributed environment.

  • Choose a node type, ISE node. For Administration, Policy Service, and Monitoring capabilities, you must choose an ISE node.

  • Choose the same Network Time Protocol (NTP) server for all the nodes. To avoid timezone issues among the nodes, you must provide the same NTP server name during the setup of each node. This setting ensures that the reports and logs from the various nodes in your deployment are always synchronized with timestamps.

  • Configure the Cisco ISE Admin password when you install Cisco ISE. The previous Cisco ISE Admin default login credentials (admin/cisco) are no longer valid. Use the username and password that was created during the initial setup or the current password if it was changed later.

  • Configure the Domain Name System (DNS) server. Enter the IP addresses and fully qualified domain names (FQDNs) of all the Cisco ISE nodes that are part of your distributed deployment in the DNS server. Otherwise, node registration will fail.

  • Configure the forward and reverse DNS lookup for all Cisco ISE nodes in your distributed deployment in the DNS server. Otherwise, you may run into deployment related issues when registering and restarting Cisco ISE nodes. Performance might be degraded if reverse DNS lookup is not configured for all the nodes.

  • (Optional) Deregister a secondary Cisco ISE node from the Primary PAN to uninstall Cisco ISE from it.

  • Back up the primary Monitoring node, and restore the data to the new secondary Monitoring node. This ensures that the history of the primary Monitoring node is in sync with the new secondary node as new changes are replicated.

  • Ensure that the Primary PAN and the standalone node that you are about to register as a secondary node are running the same version of Cisco ISE.

  • While adding a new node to the deployment, make sure that the issuer certificate chain of wildcard certificates is part of the trusted certificates of the new node. When the new node is added to the deployment, the wildcard certificates will then be replicated to the new node.

  • When configuring your ISE deployment to support Trustsec, or when ISE is integrated with Cisco DNA Center, do not configure an ISE Policy Service node as SXP-only. SXP is an interface between Trustsec and non-Trustsec devices. It does not communicate with the Trustsec-enabled network devices.

Menu Options Available on Primary and Secondary Nodes

The menu options available in Cisco ISE nodes that are part of a distributed deployment depend on the personas that are enabled on them. You must perform all administration and monitoring activities through the Primary PAN. For other tasks, you must use the secondary nodes. Therefore, the user interface of the secondary nodes provides limited menu options based on the persona that are enabled on them.

If a node assumes more than one persona, for example, the Policy Service persona, and a Monitoring persona with an Active role, then the menu options listed for Policy Service nodes and Active Monitoring node will be available on that node.

The following table lists the menu options that are available on Cisco ISE nodes that assume different persona.

Table 1. Cisco ISE Nodes and Available Menu Options

Cisco ISE Node

Available Menu Options

All Nodes

  • View and configure system time and NTP server settings.

  • Install server certificate, manage certificate signing request. You can perform server certificate operations, for all the nodes in the deployment, via the Primary PAN that centrally manages all server certificates.

    Note 

    The private keys are not stored in the local database and are not copied from the relevant node; the private keys are stored in the local file system.

Primary PAN

All menus and sub-menus.

Active Monitoring Node

  • Provides access to monitoring data (on both Primary and Active Monitoring nodes).

Policy Service Nodes

Option to join, leave, and test Active Directory connection. Each Policy Service node must be separately joined to the Active Directory domain. You must first define the domain information and join the PAN to the Active Directory domain. Then, join the other Policy Service nodes to the Active Directory domain individually.

Secondary PAN

Option to promote the Secondary PAN to become the Primary PAN.

Note 

After you have registered the secondary nodes to the Primary PAN, while logging in to the Admin portal of any of the secondary nodes, you must use the login credentials of the Primary PAN.

Configure a Cisco ISE Node

After you install a Cisco ISE node, all the default services provided by the Administration, Policy Service, and Monitoring personas run on it. This node will be in a standalone state. You must log in to the Admin portal of the Cisco ISE node to configure it. You cannot edit the personas or services of a standalone Cisco ISE node. You can, however, edit the personas and services of the primary and secondary Cisco ISE nodes. You must first configure a primary ISE node and then register secondary ISE nodes to the primary ISE node.

If you are logging in to the node for the first time, you must change the default administrator password and install a valid license.

It is recommended not to change the host name and the domain name on Cisco ISE that have been configured or in production. If it is required, then reimage the appliance, make changes, and configure the details during the initial deployment.

Before you begin

You should have a basic understanding of how distributed deployments are set up in Cisco ISE. See Guidelines for Setting Up a Distributed Deployment.

Procedure

Step 1

Choose Administration > System > Deployment.

Step 2

Check the check box next to the Cisco ISE node that you want to configure, and click Edit.

Step 3

Enter the values as required and click Save.

Configure a Primary PAN

To set up a distributed deployment, you must first configure a Cisco ISE node as your Primary PAN.

Procedure

Step 1

Choose Administration > System > Deployment.

The Register button will be disabled initially. To enable this button, you must configure a Primary PAN.

Step 2

Check the check box next to the current node, and click Edit.

Step 3

Click Make Primary to configure your Primary PAN.

Step 4

Click Save to save the node configuration.

What to do next

  1. Add secondary nodes to your deployment.

  2. Enable the profiler service and configure the probes, if required.

Install Trusted Certificates for Cisco ISE Inter-node Communication

When you set up the deployment, before you register a secondary node, you must populate the PAN's Certificate Trust List (CTL) with appropriate CA certificates that are used to validate the Admin certificate of the secondary node. The procedure to populate the CTL of the PAN is different for different scenarios:

  • If the secondary node is using a CA-signed certificate to communicate with the Admin portal, you must import the CA-signed certificate of the secondary node, the relevant intermediate certificates(if any), and the root CA certificate (of the CA that signed the secondary node's certificate) in to the CTL of the PAN.

  • If the secondary node is using a self-signed certificate to communicate with the Admin portal, you can import the self-signed certificate of the secondary node in to the CTL of the PAN.


    Note

    • If you change the Admin certificate on a registered secondary node, you must obtain appropriate CA certificates that can be used to validate the secondary node’s Admin certificate and import it in to the CTL of the PAN.

    • If you use self-signed certificates to secure communication between a client and PSN in a deployment, when BYOD users move from one location to another, EAP-TLS user authentication fails. For such authentication requests that have to be serviced between a few PSNs, you must secure communication between the client and PSN with an externally-signed CA certificate or use wildcard certificates signed by an external CA.

Ensure that the certificate issued by the external CA has basic constraints defined and the CA flag set to true. To install CA-signed certificates for inter-node communication:

Procedure

Step 1

Create a Certificate Signing Request and Submit the CSR to a Certificate Authority

Step 2

Import the Root Certificates to the Trusted Certificate Store

Step 3

Bind the CA-Signed Certificate to the CSR

Register a Secondary Cisco ISE Node

After you register the secondary node, the configuration of the secondary node is added to the database of the primary node and the application server on the secondary node is restarted. After the restart is complete, the secondary node will be running the personas and services that you have enabled on it. You can view all the configuration changes that you make from the Deployment page of the Primary PAN. However, expect a delay of 5 minutes for your changes to take effect and appear on the Deployment page.

Before you begin

Ensure that the primary node’s trusted certificate store has the appropriate certificate authority (CA) certificates to validate the HTTPS certificate of the secondary node that you are going to register. When you import the secondary node's certificate in to the trusted certificate store, check the Trust for authentication within ISE check box for the Primary PAN to validate the secondary node's certificate.

After you register the secondary node to the primary node, if you change the HTTPS certificate on the secondary node, you must import the appropriate CA certificates into the trusted certificate store of the primary node.

The certificates that you import into the trusted certificate store of the Primary PAN are replicated to the secondary nodes.

If you plan to deploy two Administration nodes for high availability, register the Secondary PAN before you register the other secondary nodes. If you register the nodes in this sequence, you do not have to restart the secondary ISE nodes after you promote the Secondary PAN as your primary.

If you plan to deploy multiple Policy Service nodes running Session services with mutual failover among these nodes, place the Policy Service nodes in a node group. You must create the node group before you register the nodes.

Procedure

Step 1

Log in to the Primary PAN.

Step 2

Choose Administration > System > Deployment.

Step 3

Choose Register > Register an Cisco ISE Node to register a secondary node.

Step 4

Enter a DNS-resolvable hostname or IP address of the secondary node.

If you are using the hostname while registering the Cisco ISE node, the fully qualified domain name (FQDN) of the standalone node that you are going to register, for example, abc.xyz.com, must be DNS-resolvable from the Primary PAN. Otherwise, node registration fails. You must have previously defined the IP address and the FQDN of the secondary node in the DNS server.

Step 5

Enter a UI-based administrator credential for the secondary node in the Username and Password fields.

Step 6

Click Next.

Cisco ISE contacts the secondary node, obtains some basic information such as the hostname, default gateway, and so on, and displays it.

If you have chosen to register a secondary node, you can edit the configuration of the secondary node.

Step 7

Click Save.

After a secondary node is registered successfully, you will receive an alarm on your Primary PAN that confirms a successful node registration. If the secondary node fails to register with the Primary PAN, the alarm is not generated. When a node is registered, the application server on that node is restarted. After successful registration and database synchronization, enter the credentials of the Primary PAN to log in to the user interface of the secondary node.


Note

In addition to the existing Primary node in the deployment, when you successfully register a new node, no alarm corresponding to the newly registered node is displayed. The Configuration Changed alarms reflect information corresponding to the newly registered nodes. You can use this information to ascertain the successful registration of the new node.

What to do next

  • For time-sensitive tasks such as guest user access and authorization, logging, and so on, ensure that the system time on your nodes is synchronized.

  • If you registered a Secondary PAN, and will be using the internal Cisco ISE CA service, you must back up the Cisco ISE CA certificates and keys from the Primary PAN and restore them on the Secondary PAN.

    See Backup and Restore of Cisco ISE CA Certificates and Keys

Administration Node

A Cisco ISE node with the Administration persona allows you to perform all administrative operations on Cisco ISE. It handles all system-related configurations that are related to functionality such as authentication, authorization, auditing, and so on. In a distributed environment, you can have a maximum of two nodes running the administration persona. The administration persona can take on any one of the following roles: Standalone, Primary, or Secondary.

High Availability for the Administrative Node

In a high availability configuration, the Primary Administration Node (PAN) is in the active state. The Secondary PAN (backup PAN) is in the standby state, which means it receives all configuration updates from the Primary PAN, but is not active in the ISE network.

Cisco ISE supports manual and automatic failover. With automatic failover, when the Primary PAN goes down, an automatic promotion of the Secondary PAN is initiated. Automatic failover requires a non-administration secondary node, which is called a health check node. The health check node checks the health of Primary PAN. If the health detects that the Primary PAN is down or unreachable, the health check node initiates the promotion of the Secondary PAN to take over the primary role.

To deploy the auto-failover feature, you must have at least three nodes, where two of the nodes assume the Administration persona, and one node acts as the health check node. A health check node is a non-administration node and can be a Policy Service, Monitoring, or pxGrid node, or a combination of these. If the PANs are in different data centers, you must have a health check node for each PAN.

The following table lists the features that are affected when the Primary PAN goes down and the Secondary PAN is yet to take over.

Features

Available When Primary PAN is Down (Yes/No)

Existing internal user RADIUS authentication

Yes

Existing or New AD user RADIUS authentication

Yes

Existing endpoint with no profile change

Yes

Existing endpoint with profile change

No

New endpoint learned through profiling.

No

Existing guest – LWA

Yes

Existing guest – CWA

Yes (apart from flows enabled for device registration, such as Hotspot, BYOD, and CWA with automatic device registration)

Guest change password

No

Guest – AUP

No

Guest – Max Failed Login Enforcement

No

New Guest (Sponsored or Self-registered)

No

Posture

Yes

BYOD with Internal CA

No

Existing Registered Devices

Yes

MDM On-boarding

No

pxGrid Service

No

Login to GUI of secondary nodes

Yes (login process is delayed as blocking call to PAN is attempted to update the last login details. The login proceeds once above call times-out)

To support certificate provisioning with the internal certificate authority, you must to import the root certificate of the original Primary PAN and its key into the new primary node, after promotion. Certificate provisioning does not work after auto-failover for PSN nodes that are added after the promotion of the secondary node to Primary PAN.

High-Availability Health Check Nodes

The health check node for Primary PAN is called the active health check node. The health check node for Secondary PAN is called the passive health check node. The active health check node is responsible for checking status of Primary PAN, and managing the automatic failover of Administration nodes. We recommended using two non-administration ISE nodes as health check nodes, one for the Primary and one for the Secondary PAN. IF you use only one health check node, and that node goes down, automatic failover will not happen.

When both PANs are in the same data center, you can use a single non-administration ISE node as the health check node for both the Primary PAN and the Secondary PAN. When a single health check node checks the health of both the Primary PAN and the Secondary PAN, it assumes both the active and passive roles.

A health check node is a non-administration node, which means it can be a Policy Service, Monitoring, or pxGrid node, or a combination of those. We recommend that you designate PSN nodes in the same data center as the Administration nodes as health check nodes. However, in a small or a centralized deployment, where the two Administration nodes are not in the same location (LAN or data center), any node (PSN/pxGrid/MnT) not having the Administration persona can be used as health check node.

If you chose not to enable automatic failover, and rely on manually promoting the secondary node when the primary PAN fails, then you do not need any check nodes.

Health Check Node for the Secondary PAN

The health check node for the Secondary PAN is a passive monitor. It does not take any action until the Secondary PAN has been promoted as the Primary PAN. When the Secondary PAN takes over the primary role, its associated health check node takes the active role for managing automatic failover of Administration nodes. The health check node of the previous Primary PAN becomes the health check node for the Secondary PAN now and would monitor it passively.

Disabling and Restarting Health Check

When a node is removed from the health check role or auto-failover configuration is disabled, the health check service is stopped on that node. When the auto-failover configuration is enabled on the designated high-availability health check node, the node starts checking health of Administration nodes again. Designating or removing the high-availability health check role on a node does not involve any application restart on that node; only the health check activities are started or stopped.

If the high-availability health check node is restarted, it ignores the previous downtimes of Primary PAN and starts checking the health status afresh.

Health Check Nodes

The active health check node checks the health status of the Primary PAN at a configured polling interval. It sends a request to the Primary PAN, and if the response that it receives matches the configuration, then the health check node considers the Primary PAN to be in good health. Otherwise, the health check node considers the Primary PAN to be in bad health. If the health of the Primary PAN is bad continuously for more than the configured failover period, then the health check node initiates failover to the Secondary PAN.

If at any time during the health check, health status is found to be good after being reported as bad previously within the failover period, then the health check node marks the Primary PAN status as good, and resets the health check cycle.

Response from health check of the Primary PAN is validated against the configuration values available on its health check node. If the response does not match it would raise an alarm. However, a promotion request will be made to the Secondary PAN.

Changing Health Nodes

You can change the ISE node that you are using for a health check, but there are some things to consider.

For example, assume that the health check node (H1) goes out-of-sync and some other node (H2) is made the health check node of the Primary PAN. In such a case, once the Primary PAN goes down, there is no way for N1 to know that there is another node (H2) checking the same Primary PAN. Later, if H2 goes down or out of network, an actual failover is required. The Secondary PAN, however, retains the right to reject the promotion request. So, once the Secondary PAN has been promoted to the primary role, a promotion request from H2 is rejected with an error. Even if a health check node for the Primary PAN is out of sync, it continues to check the health of Primary PAN.

Automatic Failover to the Secondary PAN

You can configure ISE to automatically the promote the secondary PAN when the primary PAN becomes unavailable. The configuration is done on the primary Policy Administration Node (Primary PAN) in the Deployment window. The navigation path for this window is Administration > System > Deployment. The failover period is defined as the number of times configured in Number of Failure Polls Before Failover times the number of seconds configured in Polling Interval. With the default configuration, that time is 10 minutes. Promotion of the secondary PAN to primary takes another 10 minutes. So by default, the total time from primary PAN failure to secondary PAN working is 20 minutes.

When the Secondary PAN receives the failover call, it carries out the following validations before proceeding with the actual failover:

  • The primary PAN isn’t available in network.

  • The failover request came from a valid health check node.

  • The failover request is for this PAN.

If all the validations pass, the Secondary PAN promotes itself to the primary role.

The following are some sample (but not limited to) scenarios where automatic failover of the Secondary PAN would be attempted.

  • Health of primary PAN is consistently not good for the 'Number of failure polls before failover' value during the polling period.

  • Cisco ISE services on the primary PAN are manually stopped, and remain stopped for the failover period.

  • The primary PAN is shut down using soft halt or reboot option, and remains shut down for the configured failover period.

  • The primary PAN goes down abruptly (power down), and remains down for the failover period.

  • The network interface of primary PAN is down (network port shut or network service down), or it’s not reachable by the health check node for any other reason, and remains down for the configured failover period.

Health Check Node Restarts

Upon restart, the high-availability health check node ignores the previous downtimes of primary PAN and will check the health status afresh.

Bring Your Own Device In Case Of Automatic Failover To Secondary PAN

When the primary PAN is down, authentication isn’t interrupted for the endpoints that already have certificates issued by the primary PAN root CA chain. This is because all the nodes in the deployment have the entire certificate chain for trust and validation purposes.

However, until the secondary PAN is promoted to primary, new BYOD devices won’t be onboarded. BYOD onboarding requires an active primary PAN.

Once the original primary PAN is brought back up or the secondary PAN is promoted, new BYOD endpoints will be onboarded without any issues.

If the primary PAN that failed can’t be re-joined as the primary PAN, regenerate the root CA certificate on the newly promoted primary PAN (the original secondary PAN).

For existing certificate chains, triggering a new root CA certificate results in the automatic generation of the subordinate CA certificates. Even when new subordinate certificates are generated, endpoints certificates that were generated by the previous chain continue to be valid.

Sample Scenarios when Automatic Failover is Avoided

The following are some sample scenarios that depict cases where automatic failover by the health check node would be avoided or promotion request to the secondary node would be rejected.

  • Node receiving the promotion request is not the secondary node.

  • Promotion request does not have the correct Primary PAN information.

  • Promotion request is received from an incorrect health check node.

  • Promotion request is received but the Primary PAN is up and in good health.

  • Node receiving the promotion request goes out-of-sync.

Functionalities Affected by the PAN Auto-Failover Feature

The following table lists the functionalities that are blocked or require additional configuration changes if PAN auto-failover configuration is enabled in your deployment.

Functionality

Affect Details

Operations that are Blocked

Upgrade

Upgrade via the CLI is blocked.

The PAN auto-failover feature will be available for configuration after you upgrade from a previous version of Cisco ISE to release 1.4. By default, this feature is disabled.

To deploy the auto-failover feature, you must have at least three nodes, where two of the nodes assume the Administration persona, and one node acts as the health check node. A health check node is a non-administration node and can be a Policy Service, Monitoring, or pxGrid node, or a combination of these. If the PANs are in different data centers, you must have a health check node for each PAN.

Restore of Backup

Restore via the CLI and user interface will be blocked.

If PAN auto-failover configuration was enabled prior to restore, you must reconfigure it after a successful restore.

Change Node Persona

Change of the following node personas via the user interface will be blocked:

  • Admin persona in both the Administration nodes.

  • Persona of the PAN.

  • Deregistration of health check node after enabling the PAN auto-failover feature.

Other CLI Operations

The following admin operations via the CLI will be blocked:

  • Patch Installation and Roll back

  • DNS Server change

  • IP address change of eth1, eth2, and eth3 interfaces

  • Host alias change of eth1, eth2, and eth3 interfaces

  • Timezone change

Other Administration Portal Operations

The following admin operations via the user interface will be blocked:

  • Patch Installation and Roll back

  • Change HTTPS certificate.

  • Change admin authentication type from password-based authentication to certificate-based authentication and viceversa.

Users with maximum connected devices cannot connect.

Some session data is stored on the failed PAN, and can't be updated by the PSN.

Operations that Require PAN Auto-Failover to be Disabled

CLI Operations

The following admin operations via the CLI will display a warning message if PAN auto-failover configuration is enabled. These operations may trigger auto-failover if service/system is not restarted within failover window. Hence, while performing the below operations it is recommended to disable PAN auto-failover configuration:

  • Manual ISE service stop

  • Soft reload (reboot) using admin CLI

Configure Primary PAN for Automatic Failover

Before you begin

To deploy the auto-failover feature, you must have at least three nodes, where two of the nodes assume the Administration persona, and one node acts as the health check node. A health check node is a non-administration node and can be a Policy Service, Monitoring, or pxGrid node, or a combination of these. If the PANs are in different data centers, you must have a health check node for each PAN.

Procedure

Step 1

Log in to the user interface of the Primary PAN.

Step 2

Choose Administration > System > Deployment > PAN Failover.

Step 3

Check the Enable PAN Auto Failover check box, to enable automatic failover of the Primary PAN.

You can only promote a Secondary PAN to become the Primary PAN. Cisco ISE nodes that assume only the Policy Service, Monitoring, or pxGrid persona, or a combination of these, cannot be promoted to become the Primary PAN.

Step 4

Select the health check node for Primary PAN from the Primary Health Check Node drop down list containing all the available secondary nodes.

It is recommended to have this node in the same location or data center as the Primary PAN.

Step 5

Select the health check node for Secondary PAN, from the Secondary Health Check Node drop down list containing all the available secondary nodes.

It is recommended to have this node in the same location or data center as the Secondary PAN.

Step 6

Provide the Polling Interval time after which the Administration node status will be checked . The valid range is from 30 to 300 seconds.

Step 7

Provide the count for Number of Failure Polls before Failover.

The failover will occur if the status of the Administration node is not good for the specified number of failure polls. The valid range is from 2 to 60 counts.

Step 8

Click Save.

What to do next

After the promotion of Secondary PAN to the Primary PAN, do the following:

  • Manually sync the old Primary PAN to bring it back into the deployment.

  • Manually sync any other secondary node that is out-of sync, to bring it back into the deployment.

Manually Promote Secondary PAN To Primary

If the Primary PAN fails and you have not configured PAN auto-failover, you must manually promote the Secondary PAN to become the new Primary PAN.

Before you begin

Ensure that you have a second Cisco ISE node configured with the Administration persona to promote as your Primary PAN.

Procedure

Step 1

Log in to the user interface of the Secondary PAN.
Step 2
Step 3

In the Edit Node page, click Promote to Primary.

You can only promote a Secondary PAN to become the Primary PAN. Cisco ISE nodes that assume only the Policy Service or Monitoring persona, or both, cannot be promoted to become the Primary PAN.

Step 4

Click Save.

What to do next

If the node that was originally the Primary PAN comes back up, it will be demoted automatically and become the Secondary PAN. You must perform a manual synchronization on this node (that was originally the Primary PAN) to bring it back into the deployment.

In the Edit Node page of a secondary node, you cannot modify the personas or services because the options are disabled. You have to log in to the Admin portal to make changes.

Restoring Service to the Primary PAN

Cisco ISE does not support automatic fallback to original Primary PAN. After the automatic failover to the Secondary PAN is initiated, If you bring the original Primary PAN back into the network, you should configure it as the secondary PAN.

Monitoring Node

A Cisco ISE node with the Monitoring persona functions as the log collector and stores log messages from all the administration and Policy Service nodes in your network. This persona provides advanced monitoring and troubleshooting tools that you can use to effectively manage your network and resources. A node with this persona aggregates and correlates the data that it collects to provide you with meaningful information in the form of reports.

Cisco ISE allows you to have a maximum of two nodes with this persona that can take on primary or secondary roles for high availability. Both the primary and secondary Monitoring nodes collect log messages. If the primary monitoring node goes down, Primary PAN points to secondary node to gather monitoring data. But secondary node will not be promoted to primary automatically. This should be done by manually modifying the Monitoring and Troubleshooting (MnT) role.

At least one node in your distributed setup should assume the Monitoring persona. We recommend that you not have the Monitoring and Policy Service personas enabled on the same Cisco ISE node. We recommend that the node be dedicated solely to monitoring for optimum performance.

You can access the Monitoring menu from the PAN and the Primary Monitoring Node in your deployment.

Manually Modify MnT Role

You can manually modify MnT roles (both from primary to secondary and from secondary to primary) from Primary PAN.

Procedure

Step 1

Log in to the user interface of the Primary PAN.
Step 2

Choose Administration > System > Deployment.

Step 3

Select MnT node you want to change the role from list of nodes.

Step 4

Click Edit.

Step 5

In the Monitoring section, change the role to Primary/ Secondary.

Step 6

Click Save.


Note

You can enable the option if you want to disable all other personas and services enabled on that node. When this option is enabled, configuration data replication process is stopped on that node. This helps to improve the performance of the monitoring node. When you disable this option, manual synchronization is triggered.

Automatic Failover in Monitoring Nodes

Monitoring nodes do not offer high availablity, but do offer active standby. The Policy Service Node (PSN) copies operational audit data to both the primary and secondary Monitoring nodes.

Automatic Failover Process

When a primary Monitoring node goes down, the secondary Monitoring node takes over all monitoring and troubleshooting information.

To manually convert the secondary monitoring node to a primary node, promote the secondary node to a primary role. If the primary node comes back up after the secondary node was promoted, it takes the secondary role. If the secondary node was not promoted, the primary Monitoring node resumes the primary role, after it comes back up.


Caution

When the primary node comes back up after a failover, obtain a backup of the secondary and restore the data to update the primary node.

Guidelines for Setting Up an Active Standby Pair of Monitoring Nodes

You can specify two Monitoring nodes on an ISE network, and configure then to be an active standby pair. We recommend that you back up the primary Monitoring node, and restore the data to the new secondary Monitoring node. This ensures that the history of the primary Monitoring node is synchronized with the new secondary node as the primary replicates new data. The following rules apply to an active standby pair:

  • All changes are logged to the primary Monitoring node. The secondary node is read-only.

  • Changes made to the primary node are automatically replicated on the secondary node.

  • Both the primary and secondary nodes are listed as log collectors, to which all other nodes send logs.

  • The Cisco ISE dashboard is the main entry point for monitoring and troubleshooting. Monitoring information is displayed on the dashboard from the primary Monitoring node. If the primary node goes down, monitoring information is available on the secondary node.

  • Backing up and purging monitoring data is not part of a standard Cisco ISE node backup process. You must configure repositories for backup and data purging on both the primary and secondary Monitoring nodes, and use the same repositories for each.

Monitoring Node Failover Scenarios

The following scenarios apply to the active/standby or single node configurations corresponding to the monitoring nodes:

  • In an active/standby configuration of the monitoring nodes, the Primary Administration Node (PAN) always points to the primary monitoring node to collect the monitoring data. After the primary monitoring node fails, the PAN points to the standby monitoring node. The failover from the primary monitoring node to the standby monitoring node happens after it is down for more than 5 minutes.

    However, after the primary node fails, the standby node does not become the primary node. In case the primary node comes up, the Administration node starts collecting the monitoring data again from the resumed primary node.

  • If the primary monitoring node is down, and you want to promote the standby monitoring node to active status, you can promote the standby monitoring node to primary by manually modifying MnT role or by de-registering the existing primary monitoring node. When you de-register the existing primary monitoring node, the standby node becomes the primary monitoring node, and the PAN automatically points to the newly promoted primary node.

  • In an active/standby pair, if you de-register the secondary monitoring node or if the secondary monitoring node goes down, the existing primary monitoring node remains current the primary node.

  • If there is only one monitoring node in the ISE deployment, then that node acts as the primary monitoring node, and provides monitoring data to the PAN. However, when you register a new monitoring node, and make it the primary node in the deployment, the existing primary monitoring node automatically becomes the standby node. The PAN points to the newly registered primary monitoring node to collect monitoring data.

pxGrid Node

You can use Cisco pxGrid to share the context-sensitive information from Cisco ISE session directory with other network systems such as ISE Eco system partner systems and other Cisco platforms. The pxGrid framework can also be used to exchange policy and configuration data between nodes like sharing tags and policy objects between Cisco ISE and third party vendors, and for other information exchanges. pxGrid also allows 3rd party systems to invoke adaptive network control actions (EPS) to quarantine users/devices in response to a network or security event. The TrustSec information like tag definition, value, and description can be passed from Cisco ISE via TrustSec topic to other networks. The endpoint profiles with Fully Qualified Names (FQNs) can be passed from Cisco ISE to other networks through a endpoint profile meta topic. Cisco pxGrid also supports bulk download of tags and endpoint profiles.

You can publish and subscribe to SXP bindings (IP-SGT mappings) through pxGrid. For more information about SXP bindings, see Security Group Tag Exchange Protocol.

In a high-availability configuration, Cisco pxGrid servers replicate information between the nodes through the PAN. When the PAN goes down, pxGrid server stops handling the client registration and subscription. You need to manually promote the PAN for the pxGrid server to become active.

On the Active pxGrid 1.0 node, these processes show as 'Running'. On the Standby pxGrid 1.0 node, they show as Disabled. If the Active pxGrid 1.0 node goes down, the standby pxGrid node detects this, and starts the four pxGrid processes. Within a few minutes, these processes show as 'Running', and the Standby node becomes the Active node. You can verify that pxGrid is in standby on that node by running the CLI command show logging application pxgrid or show logging application pxgrid.state.

For XMPP (Extensible Messaging and Presence Protocol) clients, pxGrid nodes work in Active or Standby high availability mode which means that the pxGrid Service is in "running" state on the active node and in "disabled" state on the standby node.

After the automatic failover to the secondary pxGrid node is initiated, if the original primary pxGrid node is brought back into the network, the original primary pxGrid node will continue to have the secondary role and will not be promoted back to the primary role unless the current primary node goes down.


Note
At times, the original primary pxGrid node might be automatically promoted back to the primary role.

In a high availability deployment, when the primary pxGrid node goes down, it might take around 3 to 5 minutes to switchover to the secondary pxGrid node. It is recommended that the client waits for the switchover to complete, before clearing the cache data in case of primary pxGrid node failure.

The following logs are available for pxGrid node:

  • pxgrid.log—State change notifications.

  • pxgrid-cm.log—Updates on publisher/subscriber and data exchange activity between client and server.

  • pxgrid-controller.log—Displays the details of client capabilities, groups, and client authorization.

  • pxgrid-jabberd.log—All logs related to system state and authentication.

  • pxgrid-pubsub.log—Information related to publisher and subscriber events.

pxGrid Client and Capability Management

Clients connecting to Cisco ISE must register and recieve account approved before using pxGrid services. pxGrid clients use the pxGrid Client Library available from Cisco through the pxGrid SDK to become the clients. Cisco ISE supports both auto and manual approvals. A client can log in to pxGrid using a unique name and certificate-based mutual authentication. Similar to the AAA setting on a switch, clients can connect to either a configured pxGrid server hostname or an IP Address.

pxGrid "Capabilities" are information topics or channels on pxGrid for clients to publish and subscribe. In Cisco ISE, only capabilities such as Identity, adaptive network control, and SGA are supported. When a client creates a new capability, it appears in the View by Capabilities window. The navigation path for this window is Administration > pxGrid Services > View by Capabilities. You can enable or disable capabilities individually. Capability information is available from the publisher through publish, directed query, or bulk download query.

Enable pxGrid Service

Before you begin

  • Enable the pxGrid persona on at least one node to view the requests from the Cisco pxGrid clients.

  • Enable Identity Mapping. For more information, see Configure Identity Mapping.

Procedure

Step 1

Choose Administration > pxGrid Services.

Step 2

Check the checkbox next to the client and click Approve.

Step 3

Click Refresh to view the latest status.

Step 4

Select the capability you want to enable and click Enable.

Step 5

Click Refresh to view the latest status.

Enable pxGrid Capabilities

Before you begin

  • Enable the pxGrid persona on at least one node to view the requests from the Cisco pxGrid clients.

  • Enable a pxGrid client.

Procedure

Step 1

Choose Administration > pxGrid Services.

Step 2

Click View by Capabilities at the top-right.

Step 3

Select the capability you want to enable and click Enable.

Step 4

Click Refresh to view the latest status.

Deploy pxGrid Node

You can enable Cisco pxGrid persona both on a standalone node and distributed deployment node.

Before you begin

  • All nodes are configured to use the CA certificate for pxGrid usage. If default certificate is used for pxGrid before upgrade, it will be replaced by the internal CA certificate after upgrade.

  • You must have port 8910 open for Websockets (pxGrid 2.0) and port 5222 open for XMPP (pxGrid V1.0). If service is disabled on a node, port 5222 goes down, but port 8910 remains functional, and will continue to respond to the requests.

Procedure

Step 1

Choose Administration > System > Deployment.

Step 2

In the Deployment Nodes page, check the check box next to the node to which you want to enable the pxGrid services, and click Edit.

Step 3

Click the General Settings tab and check the pxGrid checkbox.

Step 4

Click Save.

When you upgrade from the previous version, the Save option might be disabled. This happens when the browser cache refers to the old files from the previous version of Cisco ISE. Clear the browser cache to enable the Save option.

Cisco pxGrid Live Logs

The Live Logs window displays all the pxGrid management events. Event info includes the client and capability names along with the event type and timestamp.

The navigation path for this window is Administration > pxGrid Services > Live Log. You can also clear the logs and resynchronize or refresh the list.

View Nodes in a Deployment

In the Deployment Nodes page, you can view all the Cisco ISE nodes, primary and secondary, that are part of your deployment.

Procedure

Step 1

Log in to the primary Cisco ISE Admin portal.

Step 2

Choose Administration > System > Deployment.

Step 3

Click Deployment from the navigation pane on the left.

All the Cisco ISE nodes that are part of your deployment are listed.

Synchronize Primary and Secondary Cisco ISE Nodes

You can make configuration changes to Cisco ISE only through the Primary PAN. The configuration changes get replicated to all the secondary nodes. If, for some reason, this replication does not occur properly, you can manually synchronize the Secondary PAN with the Primary PAN.

Before you begin

You must click the Syncup button to force a full replication if the Sync Status is set to Out of Sync or if the Replication Status is Failed or Disabled.

Procedure

Step 1

Log in to the Primary PAN.

Step 2

Choose Administration > System > Deployment.

Step 3

Check the check box next to the node that you want to synchronize with the Primary PAN, and click Syncup to force a full database replication.

Change Node Personas and Services

You can edit the Cisco ISE node configuration to change the personas and services that run on the node.

Before you begin

  • When you enable or disable any of the services that run on a Policy Service node or make any changes to a Policy Service node, you will be restarting the application server processes on which these services run. Expect a delay while these services restart.

  • Due to this delay in restart of services, auto-failover if enabled in your deployment, might get initiated. To avoid this, make sure that the auto-failover configuration is turned off.

Procedure

Step 1

Log in to the Primary PAN.

Step 2

Choose Administration > System > Deployment.

Step 3

Check the check box next to the node whose personas or services you want to change, and then click Edit.

Step 4

Choose the personas and services that you want.

Step 5

Click Save.

Step 6

Verify receipt of an alarm on your Primary PAN to confirm the persona or service change. If the persona or service change is not saved successfully, an alarm is not generated.

Effects of Modifying Nodes in Cisco ISE

When you make any of the following changes to a node in a Cisco ISE ISE, that node restarts, which causes a delay:

  • Register a node (Standalone to Secondary)

  • Deregister a node (Secondary to Standalone)

  • Change a primary node to Standalone (if no other nodes are registered with it; Primary to Standalone)

  • Promote an Administration node (Secondary to Primary)

  • Change the personas (when you assign or remove the Policy Service or Monitoring persona from a node)

  • Modify the services in the Policy Service node (enable or disable the session and profiler services)

  • Restore a backup on the primary and a sync up operation is triggered to replicate data from primary to secondary nodes

Create a Policy Service Node Group

When two or more Policy Service nodes (PSNs) are connected to the same high-speed Local Area Network (LAN), we recommend that you place them in the same node group. This design optimizes the replication of endpoint profiling data by retaining less significant attributes local to the group and reducing the information that is replicated to the remote nodes in the network. Node group members also check on the availability of peer group members. If the group detects that a member has failed, it attempts to reset and recover all URL-redirected sessions on the failed node.


Note

We recommend that you make all PSNs in the same local network part of the same node group. PSNs need not be part of a load-balanced cluster to join the same node group. However, each local PSN in a load-balanced cluster should typically be part of the same node group.

Before you can add PSNs as members to a node group, you must create the node group first. You can create, edit, and delete Policy Service node groups from the Deployment pages of the Admin portal.

Before you begin

Node group members can communicate over TCP/7800.

Procedure

Step 1

Choose Administration > System > Deployment.

Step 2

Click the action icon, and then click Create Node Group.

Step 3

Enter a unique name for your node group.

Step 4

(Optional) Enter a description for your node group.

Step 5

(Optional) Check the Enable MAR Cache Distribution check box and fill in the other options. Ensure that the MAR is enabled in the Active Directory page before enabling this option.

Step 6

Click Submit to save the node group.

After you save the node group, it should appear in the navigation pane on the left. If you do not see the node group in the left pane, it may be hidden. Click the Expand button on the navigation pane to view the hidden objects.

What to do next

Add a node to a node group. Edit the node by choosing the node group from the Member of Node Group drop-down list.

Configure Monitoring Nodes for Automatic Failover

If you have two Monitoring nodes in a deployment, you can configure a primary-secondary pair for automatic failover to avoid downtime in the Cisco ISE Monitoring service. A primary-secondary pair ensures that a secondary Monitoring node automatically provides monitoring should the primary node fail.

Before you begin

  • Before you can configure Monitoring nodes for automatic failover, they must be registered as Cisco ISE nodes.

  • Configure monitoring roles and services on both nodes and name them for their primary and secondary roles, as appropriate.

  • Configure repositories for backup and data purging on both the primary and secondary Monitoring nodes. For the backup and purging features to work properly, use the same repositories for both the nodes. Purging takes place on both the primary and secondary nodes of a redundant pair. For example, if the primary Monitoring node uses two repositories for backup and purging, you must specify the same repositories for the secondary node.

    Configure a data repository for a Monitoring node using the repository command in the system CLI.


    Caution

    For scheduled backup and purge to work properly on the nodes of a Monitoring redundant pair, configure the same repository, or repositories, on both the primary and secondary nodes using the CLI. The repositories are not automatically synced between the two nodes.

From the Cisco ISE dashboard, verify that the Monitoring nodes are ready. The System Summary dashlet shows the Monitoring nodes with a green check mark to the left when their services are ready.

Procedure

Step 1

Choose Administration > System > Deployment.

Step 2

In the Deployment Nodes page, check the check box next to the Monitoring node that you want to specify as active, and click Edit .

Step 3

Click the General Settings tab and choose Primary from the Role drop-down list.

When you choose a Monitoring node as primary, the other Monitoring node automatically becomes secondary. In the case of a standalone deployment, primary and secondary role configuration is disabled.

Step 4

Click Save . The active and standby nodes restart.

Remove a Node from Deployment

To remove a node from a deployment, you must deregister it. The deregistered node becomes a standalone Cisco ISE node.

It retains the last configuration that it received from the Primary PAN and assumes the default personas of a standalone node that are Administration, Policy Service, and Monitoring. If you deregister a Monitoring node, this node will no longer be a syslog target.

You can view these changes from the Deployment page of the Primary PAN. However, expect a delay of 5 minutes for the changes to take effect and appear on the Deployment page.

Before you begin

Before you remove any secondary node from a deployment, perform a backup of Cisco ISE configuration, which you can then restore later on, if needed.

Procedure

Step 1

Choose Administration > System > Deployment.

Step 2

Check the check box next to the secondary node that you want to remove, and then click Deregister.

Step 3

Click OK.

Step 4

Verify receipt of an alarm on your Primary PAN to confirm that the secondary node is deregistered successfully. If the secondary node fails to deregister from the Primary PAN, the alarm is not generated.

Shut Down an ISE Node

Before you issue the halt command, it is advised that you stop Cisco ISE application service and ensure that it is not performing any backup, restore, installation, upgrade, or remove operation. If you issue the halt command while the Cisco ISE is performing any of these operations, you will get one of the following warning messages:

WARNING: A backup or restore is currently in progress! Continue with halt?

WARNING: An install/upgrade/remove is currently in progress! Continue with halt?

If no processes are running when you use the halt command or if you click Yes in response to the warning message displayed, then you must respond to the following question:

Do you want to save the current configuration?

If you click Yes to save the existing Cisco ISE configuration, the following message is displayed:

Saved the running configuration to startup successfully.


Note

It is recommended that you stop the application process before rebooting the appliance.

This is also applicable to reboot ISE. For more information: see Cisco Identity Services Engine CLI Reference Guide

Change the Hostname or IP Address of a Standalone Cisco ISE Node

You can change the hostname, IP address, or domain name of standalone Cisco ISE nodes. You cannot use "localhost" as the hostname for a node.

Before you begin

If the Cisco ISE node is part of a distributed deployment, you must first remove it from the deployment and ensure that it is a standalone node.

Procedure

Step 1

Change the hostname or IP address of the Cisco ISE node using the hostname , ip address , or ip domain-name command from the Cisco ISE CLI.

Step 2

Reset the Cisco ISE application configuration using the application stop ise command from the Cisco ISE CLI to restart all the services.

Step 3

Register the Cisco ISE node to the Primary PAN if it is part of a distributed deployment.

Note 

If you are using the hostname while registering the Cisco ISE node, the fully qualified domain name (FQDN) of the standalone node that you are going to register, for example, abc.xyz.com must be DNS-resolvable from the Primary PAN. Otherwise, node registration fails. You must enter the IP addresses and FQDNs of the Cisco ISE nodes that are part of your distributed deployment in the DNS server.

After you register the Cisco ISE node as a secondary node, the Primary PAN replicates the change in the IP address, hostname, or domain name to the other Cisco ISE nodes in your deployment.

Replace the Cisco ISE Appliance Hardware

You should replace the Cisco ISE appliance hardware only if there is an issue with the hardware. For any software issues, you can reimage the appliance and reinstall the Cisco ISE software.

Procedure

Step 1

Re-image or re-install the Cisco ISE software on the new nodes.

Step 2

Obtain a license with the UDI for the Primary and Secondary PANs and install it on the Primary PAN.

Step 3

Restore the backup on the replaced Primary PAN.

The restore script will try to sync the data on the Secondary PAN, but the Secondary PAN is now a standalone node and the sync will fail. Data is set to the time the backup was taken on the Primary PAN.
Step 4

Register the new node as a secondary server with the Primary PAN.

Was this Document Helpful?

FeedbackFeedback

Contact Cisco