Seed Device

The seed device is a predeployed system in the network and is the initial point through which Cisco LAN automation discovers and onboards new switches downstream. The seed device can be automated through technologies such as Cisco Plug n Play (PnP) and zero-touch provisioning, or configured manually. The following figure shows the seed device network boundaries between the Catalyst Center connection in the IP core and the to-be-discovered underlay network using LAN automation.

The peer seed (Seed-2 in the following figure) can also be automated via LAN automation. Only one seed device is required.

Device discovery happens only on the primary seed device interfaces.

PnP Agent

The PnP agent is a Cisco Catalyst switch with factory-default settings. The switch leverages the built-in day-0 mechanism to communicate with Catalyst Center and support the integrated PnP server function. Catalyst Center dynamically builds the PnP profile and configuration sets that enable complete day-0 automation. The following figure shows the PnP agent physical connection to the seed device.

Automation Boundary

In general, we recommend building structured and hierarchical network designs in enterprise networks to provide scalability and redundancy at every network tier. While the three-tier architecture is proven in large-scale enterprise campus networks, the network design varies based on the overall network size, physical connections, and so on. As part of the initial planning, the network admin must determine the physical topology to automate with Cisco LAN automation.

LAN automation in Catalyst Center supports a maximum of two hops from the initial automation boundary point device. In other words, to build the underlay network up to the access layer, the network admin must start the automation boundary from the core or distribution layer. Any additional network devices beyond two hops might be discovered but cannot be automated.

LAN automation initiates only on directly connected neighbors. Consider two scenarios:

• Scenario 1: You have a three-tier network and you want to LAN automate distribution- and access-layer switches. Because distribution-layer switches (which are directly connected to the seed) participate in LAN automation, both distribution- and access-layer switches will be discovered and LAN automated.

• Scenario 2: You have a three-tier network and you want to LAN automate distribution- and access-layer switches. You already LAN automated the distribution layer. Later, you add access-layer switches to your network and you want to LAN automate these switches. Because the distribution switches are already LAN automated and links converted to Layer 3, Tier 1 switches cannot be used as the seed. You must choose distribution as the seed in this scenario.

The following figure shows the automation boundary that Cisco LAN automation supports.

The following figure shows a two-tier and three-tier network design.

Multistep LAN Automation for Large Topologies: First Pass

Large topologies are brought up by performing LAN automation multiple times. During the first pass, distribution switches are brought up by choosing core devices as seed devices; the distribution switches come up as new devices.

Note	N is less than or equal to 50 devices at a time. All switches in the group can be booted in parallel or in a staggered fashion.

Multistep LAN Automation for Large Topologies: Second Pass with First Group

During the second pass, two of the distribution switches act as seed devices to bring up the edge devices as new devices. All new devices in this session must connect directly to the two distribution switches that act as seed devices. Repeat this process for the remaining set of distribution switches, two at a time.

1. Repeat the second pass for each set of distribution to bring up the access/edge switches (where N is less than or equal to 50 devices at a time).

2. Connect uplinks from edges to the primary and peer distribution switches only.

3. Power down IOT/extended devices during the LAN automation session.

4. Distribution switches can be connected to other distribution switches.

5. There can be two tiers of devices below the seeds.

6. Always connect new devices to the primary seed device. Connection to the peer seed device is optional.

Multistep LAN Automation for Large Topologies: Second Pass with Second Group

Edge devices in one group cannot be connected to edge devices in another group. Newly discovered PnP devices in the LAN automation session cannot be connected to existing nonseed inventory devices.

Note	In Catalyst Center 2.3.5 and later, you can establish links between the devices after LAN automation stops using the Add Link feature. For more information, see Create Link Between Interfaces.

Link Configuration

• After all devices are added to the Catalyst Center inventory, you begin the process of Layer 3 link configuration by “stopping” the LAN automation session on the GUI.

• If you accidentally stop the LAN automation prematurely before all PnP devices are added to the Catalyst Center inventory, links are not configured on in-progress devices. You must delete the in-progress devices from the inventory, begin a new LAN automation session, bring the in-progress devices to the factory-default state, and reload the devices to rediscover them and get their links configured.

• This process starts the conversion of Layer 2 links to Layer 3 links, which is done by traversing the network graph built during new device onboarding. First, the lower device link is converted to a Layer 3 IP address. Next, the upper device link is converted to a Layer 3 IP address. Router IS-IS configuration is also performed during this step in the connecting links. During this phase, there might be a temporary loss of connectivity to the lower-tier device until the upper-tier link is configured. This phase can also result in an STP topology change when the Layer 2 links are converted to Layer 3.

• The process follows an algorithm that begins with the tier-three devices, followed by the tier-two devices, and completes with the tier-one devices.

• It is important to note that only the links between devices that participate in the current session are converted to Layer 3 links. Links between the newly discovered PnP device and the existing nonseed inventory device are not converted to Layer 3.

• Consider the following scenarios when a LAN-automated device is deleted from the inventory:

  • Scenario 1: If the edge device is single-homed (connected to only one intermediate node) and the intermediate node is deleted from the inventory, then the /31 point-to-point link IP address is deleted from Catalyst Center (IPAM) but may not be unconfigured from the edge device, which is still in the inventory. This is because the edge device can become unreachable from Catalyst Center due to the point-to-point interface between the intermediate node and the fabric border being unconfigured before the one on the edge device. In this case, log in to the edge device CLI and set the interface connected to the deleted device to default configuration. This avoids duplicate IP address assignment during LAN automation workflows later due to the released IP addresses still being present on the device. You can later use the LAN automation workflow to add a new link from the edge device to the new intermediate node or border node (as required) instead of manually configuring the IP addresses.

  • Scenario 2: If the edge device is dual-homed (connected to two intermediate nodes) and one of the intermediate nodes is deleted from the inventory, then the /31 point-to-point link IP address is deleted from Catalyst Center (IPAM) and is unconfigured from the edge device as well. There is no manual configuration required on the edge devices.

Catalyst Center 2.3.5 and later provide the support for day-n link configurations (add and delete link). For more information, see Create Link Between Interfaces.

Constraints

• LAN automation does not automate the onboarding of a StackWise Virtual (SVL) switch via PnP. SVL switch can only be used as a seed device.

• LAN automation does not support stack renumbering.

• For platform support, see Supported Switches for Each Role at Different Layers.

The following figure shows supported device families for the seed and PnP agent at different layers.

Cisco LAN Automation Product Support Matrix

Use the Catalyst Center Design application to create the required sites, buildings, and floors. Consider how the primary seed and peer seed will be connected to the new devices—for example, will they all belong to the same site or follow a hierarchy? Consider also how to share IP pools across different sites, buildings, and floors. One option is to have a pool specific to a site. Another option is to share a common LAN pool for all sites in the hierarchy. If the devices are onboarded across multiple LAN automation sessions, ensure that the required IP pools are available across the various sites in the hierarchy.

Note	After devices are provisioned, the site cannot be changed. For this reason, we recommend that you complete LAN automation before you provision devices.

IP pools for LAN automation are created by first creating a global pool in Catalyst Center, followed by a site-specific LAN IP pool, which LAN automation allocates internally, as follows:

1. One part of the IP pool is reserved for a temporary DHCP server. The size of this pool depends on the size of the parent LAN pool. For example, if the parent pool is 192.168.10.0/24, a subpool of size /26 is allocated for the DHCP server. If the pool size is larger than /24, the algorithm keeps increasing the size of the DHCP pool, up to a maximum of a /23 subpool (512 IP addresses). Therefore, a /24 pool reserves 64; a /23 pool reserves 128; a /22 pool reserves 256; and anything larger reserves 512 IP addresses for the DHCP server. The minimum pool size to start LAN automation is /25; that reserves /27 or 32 IP addresses for the DHCP pool. This IP pool is reserved temporarily for the duration of the LAN automation discovery session. After the LAN automation discovery session completes, the DHCP pool is released, and the IPs are returned to the LAN pool. Because the DHCP pool is usually the largest contiguous segment of IPs required, the pool should have at least one such segment available. If the pool is too fragmented, it cannot allocate the DHCP pool and the LAN automation session ends with an IP pool allocation error.

2. Another part of the IP pool is reserved internally with a subpool size of /27. This subpool is for allocating single IPs for Loopback0 and Loopback60000 always. Also, two consecutive IPs for point-to-point L3 links are allocated from this subpool if no separate overlapping IP pool is provided. This internally reserved subpool is used throughout the LAN automation sessions for providing IPs as long as it has IPs available. In case the IPs are exhausted, a new /27 subpool is created and IPs are allocated from that subpool. These subpools are released only when all the allocated IPs are released as part of the devices being deleted from Catalyst Center. Otherwise, the subpools remain throughout the process and are not allowed to be removed. Due to this internal subpool allocation logic, the IP pool usage in IPAM counts the subpools instead of the actual IPs allocated to the devices.

3. If a shared or link overlapping IP pool is provided for the point-to-point L3 links, then the subpool of size /27 is reserved internally from the shared pool instead of the main IP pool. The subpools are automatically deleted when all the allocated IPs from the pool are released.

When a dedicated (single) IP pool is used to build the underlay networks, each of the devices discovered via LAN automation gets a unique /31 per interface for point-to-point connection, and a unique /32 for Loopback0 and the underlay multicast.

Link Overlapping IP Pool or shared IP pool is used to optimize the IPv4 addressing in the underlay network by allowing overlapping /31 IP addresses for a multisite deployment. Hosts in different sites can get duplicate IP addresses on the /31 links. The /31s in the underlay are not advertised outside of the fabric site and hence there is no need for them to be unique. However, the /32 loopback needs to be unique to every device, and should be advertised to the global routing table to identify the device in the entire network.

There are two valid roles that a LAN IP pool can have:

• Link Overlapping IP Pool: A pool with this role is optional for a LAN automation session. If provided, the allocation of IP addresses is only on the point-to-point L3 links.

• Main IP Pool (Principal IP Address Pool in Catalyst Center 2.3.5 and later) : A pool with this role is mandatory for every LAN automation session. This is the pool that is used for all management-related IP addressing such as loopbacks, multicast, and DHCP. If the Link Overlapping IP Pool is not provided, then the Main IP Pool is the default fallback pool for the L3 links IP addressing.

Note	When the seed device for LAN automation session is in a different site than the discovered device site, then the same shared IP pool cannot be used with the same seed and different discovered device site. This is to avoid the allocation of duplicate IP to the same seed device.

IP Pool Usage Example

Imagine you want to LAN automate 10 devices using the same pool, where each device has one link to the primary seed and another link to the secondary.

Consider a 192.168.199.0/24 pool. When LAN automation starts, a /26 pool is reserved for the DHCP addresses. In this example, 192.168.199.1 to 192.168.199.63 are reserved and assigned to VLAN 1 for the 10 devices.

Next, a /27 pool is reserved for loopback addresses. If there is no shared IP pool, then this pool is used for point-to-point links as well. Because there are 10 devices with two links each, a total of 2*10*2 = 40 IP addresses are reserved for point-to-point links and 10 loopback addresses are reserved.

In total, 60 IP addresses are reserved for the 10 devices: 10 for each VLAN 1, 10 for each loopback, and 40 for the point-to-point links between devices and seeds.

After LAN automation stops, the VLAN 1 IP addresses are released back to the pool, and 90 addresses are allocated for the LAN automation session.

Note the following:

• The same IP pool can be used for multiple discovery sessions. For example, you can run one discovery session and discover the first set of devices. After discovery completes, you can provide the same IP pool for a subsequent LAN automation session. Similarly, you can choose one LAN pool for one discovery session and another LAN pool for a second discovery session.

• Every time you start LAN automation, it checks for 64 available IP addresses in the IP pool. If you decide to run LAN automation multiple times with the same pool, use at least a /24 pool. If you plan to LAN automate only once for the IP pool, a /25 pool suffices.

• Don't use an address pool that is in use elsewhere in the network, such as an address pool that belongs to the loopback or to other addresses configured on the device.

To start LAN automation, a site-specific CLI and SNMPv2 read or SNMPv3 configuration is required. Use the Catalyst Center Design application to configure the site-specific CLI and SNMP. Save the configuration for the site that is used for LAN automation. If you configure the credentials at the global level, they are visible at the site level. You must click the radio button for the specific site and then save the configuration to make it available for LAN automation.

Note	SNMPv2 write credentials are not required and if configured, it won't be pushed to the device during LAN automation.

When configuring the seed devices, follow these guidelines:

• The system maximum transmission unit (MTU) value must be at least 9100.

• Turn on IP routing on the seed devices.

• Set up routing between the seed service and Catalyst Center so that Catalyst Center has IP reachability to the LAN IP pool subnet.

• We recommend that you use the default interfaces connected to PnP agents. If the peer seed device has IP interfaces configured on the interfaces connected to PnP agents, those links don't get configured. If you want to configure the peer device interfaces connected to PnP agents, use the default interfaces and perform an inventory synchronization on the peer seed device. LAN automation works only when the ports are Layer 2. The ports on Cisco Catalyst 6000 are Layer 3 by default. Convert the ports to Layer 2 before starting LAN automation.

• Configure device credentials and SNMP credentials on the seed devices.

• If the seed devices have Layer 3 interfaces configured, ensure that there are no conflicts with any of the IP pools provided in Catalyst Center. Check the IP addresses which are configured manually.

• Ensure that the seed devices don't have any other interfaces connected to another DHCP server running in VLAN 1.

• If loopback is not configured on the seed devices, LAN automation configures loopback on the seed.

• If any configuration changes are made on the seed devices before running LAN automation, synchronize the seed devices with the Catalyst Center inventory.

• Assign the seed devices to a site. (You don't have to provision the seed devices for LAN automation.)

Additional recommended configurations on seed devices:

• Run multiple discovery sessions for devices across sites connected to the same seed: If you plan to run multiple discovery sessions to onboard devices across different buildings and floors connected to the same seed devices, we recommend that you block the ports for PnP agents that do not participate in the upcoming discovery session.

  For example, imagine that seed devices are in Building-23 and are connected to PnP agents on Floor-1 and Floor-2. Floor-1 devices are connected on interfaces Gig 1/0/10 through Gig 1/0/15. Floor-2 devices are connected on interfaces Gig 1/0/16 through Gig 1/0/20. For the discovery session on Floor-1, we recommend that you shut down ports connected to Gig 1/0/16 to Gig 1/0/20. Otherwise, the PnP agents connected to Floor-2 might also get DHCP IPs from the server running on the primary seed device. Because these interfaces aren't selected for the discovery session, they remain as stale entries in the PnP database. When you run the discovery session for Floor-2, the discovery doesn't function correctly until these devices are deleted from the PnP application and write erase/reloaded. Therefore, we recommend that you shut down other discovery interfaces.

• Endpoint/client integration: For Catalyst Center 1.2.8 and earlier, if there are clients connected to a switch that is being discovered, those clients contend for DHCP IP and might exhaust the pool, causing LAN automation to fail. Therefore, we recommend that you connect the client after LAN automation is complete.

  This endpoint/client integration restriction does not apply to Catalyst Center 1.2.10 and later. Clients can remain connected while the switch is undergoing LAN automation.

Ensure that the device that you want to LAN automate is running the Advantage license level. Otherwise, some commands are not pushed.

Note	Catalyst Center 2.3.5 and later support automatic license upgrade for C9000 and C3850 series switches.

New PnP agents have factory defaults and are ready to start LAN automation.

If you are reusing existing network devices, ensure the following:

• PnP agents must have the required license that allow you to push the LISP, IS-IS routing, and CTS-related CLI commands. Use the show license command to see the current license level. If required, upgrade the license.

• PnP agents should not have stale certificate or keys from the previous runs.

• Restore the switch configurations to factory default using the following commands:

  • For Cisco IOS XE 16.11 and earlier, use:

    [CLI config mode]
    
        no pnp profile pnp-zero-touch
        no crypto pki certificate pool
        Also remove any other crypto certs shown by "show run | inc crypto"
        crypto key zeroize
        config-register 0x2102 or 0x0102 (if not already)
        do write
        end

    [CLI exec mode]
    
        delete /force nvram:*.cer
        delete /force stby-nvram:*.cer (if a stack)
        delete /force flash:pnp-reset-config.cfg
        write erase 
        reload (enter no if asked to save)

  • For Cisco IOS XE 16.12.x or later, use:

    [CLI exec mode]
    
    pnp service reset no-prompt
    
    

You can stop LAN automation to finish discovering all required devices and to prevent inadvertent discovery of additional devices.

In the LAN Automation Status window,click Stop.

After you click Stop:

• The remainder of the configuration is pushed to network devices, which includes converting the point-to-point links from Layer 2 to Layer 3.

• The VLAN 1 configuration is removed and the VLAN 1 IP addresses are returned to the LAN automation pool.

• The device is onboarded in Catalyst Center and assigned to the site.

After the LAN automation stop process is initiated, the LAN Automation Status changes to STOP in Progress.

After LAN automation stops, the following sample configuration is pushed to the discovered device.

The network orchestration service issues a RESYNC for seed and PnP devices to retrieve the state of all links. After the initial RESYNC completes, it pushes the Layer 3 configuration on all Layer 2 links. Finally, it reissues RESYNC to resynchronize the cluster's link state.

The Layer 3 link configuration is pushed when network orchestration stops. (Each interface pair gets its configuration.)

interface GigabitEthernet1/0/13
 description Fabric Physical Link
 no switchport
 dampening
 ip address 192.168.2.97 255.255.255.252
 ip router isis
 logging event link-status
 load-interval 30
 bfd interval 500 min_rx 50 multiplier 3
 no bfd echo
 isis network point-to-point

After all the point-to-point links between the seeds and discovered devices—including links between peer seed and discovered devices—are configured, the devices are added to the site and synced to Catalyst Center.

The LAN automation process completes and the LAN Automation Status changes to Completed.

Check the LAN automation logs.

This section explains how to add an existing switch that was already present in Catalyst Center.

If the switch being added was previously LAN automated (part of another stack/standalone) or was discovered by PnP, to add it, you must first remove the switch physically and then remove its entry from the inventory and PnP application/database.

Remove the Switch from Inventory

If the switch is a standalone, from the Catalyst Center home page, click Inventory and select the switch to remove. Choose Actions > Delete Device. If the switch is part of a stack, remove the switch physically, and then resync the original stack. After the sync completes, the removed switch serial number does not appear in the inventory.

Remove the Switch from PnP

• If the switch is a standalone, first unconfigure pnp profile pnp-zero-touch from the switch and then delete the entry from the PnP database under Device.

• If the switch is part of a stack, remove the switch physically. Make sure that the removed switch does not have pnp profile pnp-zero-touch; then, delete the entry from the PnP database under Device.