IP tunnels can encapsulate a same-layer or higher-layer protocol and transport the result over IP through a tunnel created between two devices.

IP tunnels consists of the following three main components:

• Passenger protocol—The protocol that needs to be encapsulated. IPv4 is an example of a passenger protocol.

• Carrier protocol—The protocol that is used to encapsulate the passenger protocol. Cisco NX-OS supports generic routing encapsulation (GRE), and IP-in-IP encapsulation and decapsulation as carrier protocols.

• Transport protocol—The protocol that is used to carry the encapsulated protocol. IPv4 is an example of a transport protocol.

An IP tunnel takes a passenger protocol, such as IPv4, and encapsulates that protocol within a carrier protocol, such as GRE. The device then transmits this carrier protocol over a transport protocol, such as IPv4.

You configure a tunnel interface with matching characteristics on each end of the tunnel.

You must enable the tunnel feature before you can configure it.

IP tunnels have the following prerequisites:

• You must be familiar with TCP/IP fundamentals to configure IP tunnels.

• You are logged on to the switch.

• You have installed the Enterprise Services license for Cisco NX-OS.

• You must enable the tunneling feature in a device before you can configure and enable any IP tunnels.

IP tunnels have the following configuration guidelines and limitations:

• Guidelines for source-direct and ipv6ipv6-decapsulate-any options for tunnels:

  • The tunnel source direct command is supported only when an administrator uses the IP-in-IP decapsulation to source route the packets through the network. The source-direct tunnel is always operationally Up unless it is administratively shut down. The directly connected interfaces are identified using the show ip route direct command.

  • The tunnel source direct command is supported only on decapsulate-any tunnel modes, for example, tunnel mode ipip decapsulate-any and tunnel mode ipv6ipv6 decapsulate-any .

  • Auto-recovery is not supported for source-direct.

  • For ipv6ipv6 decapsulate-any , inter-VRF is not supported. The tunnel interface VRF (iVRF) and tunnel transport or forwarding VRF (fVRF) must be the same. Only one decapsulate-any tunnel (irrespective of VRF) can be present in Cisco Nexus 3000 Series switches.

  • To enable IPv6 on ipv6ipv6 decap-any tunnel interface, you must configure a valid IPv6 address or configure the ipv6 address using use-link-local-only CLI command under the interface tunnel interface.

• The hardware limitations on a source direct tunnel are as follows:

  • Source direct tunnel supports Cisco Nexus 3000 Series switches with Network Forwarding Engine (NFE), Application Spine Engine (ASE), and Leaf Spine Engine (LSE). There are limitations in cases of scaled SIP (number of total IP/IPv6 addresses on the interfaces (L3, sub-interface, PC, PC-sub interfaces, loopback, SVI, and any secondary IP/IPv6 addresses.)

    See the following sample use cases.

    • Use Case 1: Non-deterministic behavior of which SIP gets installed if the number of IP/IPv6 interface scale is more.

      Both the switches have 512 entries for tunnel SIP. With tunnel source, direct any IP or IPv6 address w.r.t ipip or ipv6ipv6 decap any with tunnel source gets installed in the above table.

      The insertion of these entries is on a first come first serve basis without any CLI command to control which interface IP addresses get installed. If the system has more number of IP/IPv6 interfaces to be installed, the behavior is non-deterministic (The behavior can change across reload with interface flaps.)

    • Use Case 2: The scale numbers are different in both switches and each has its own advantages and disadvantages.

      IPv4 individual scale can be more (up to 512) in case of switches with NFE. In the switches with ASE and LSE, the IPv4 individual scale can be 256 but it is shared with IPv6. If the user plans to configure both v4 and v6 decap any tunnel in the same system, the scale numbers for the switches with NFE for individual IPv4 and IPv6 cannot be guaranteed. However, the scale numbers for the switches with ASE and LSE for individual IPv4 and IPv6 are guaranteed. These is no CLI command to change these pre-carved scale numbers, for example, allocating X for IPv4 and Y for IPv6.

      Whenever the tunnel decap table gets full, the TABLE_FULL error is displayed. If an entry gets deleted after the table is full, the table full error is cleared.

      If the tunnel-decap-table is full, the user gets a syslog similar to as follows:

      
       2017 Apr 26 10:10:51 switch %$ VDC-1 %$ %IPFIB-2-FIB_HW_IP_TUNNEL_DECAP_TABLE_FULL: 
      IP TUNNEL decap hardware table full. IP tunnel decapsulation may not work for some GRE/IPinIP traffic
        

      If the table is full and if an entry is deleted from the table because of an interface being operationally down or removal of IP address, the clear syslog for the table is displayed. Deleting of a tunnel removes all the entries that are added as part of that tunnel.

      
       2002 Sep 26 10:11:37 switch %$ VDC-1 %$ 
      %IPFIB-2-FIB_HW_IP_TUNNEL_DECAP_TABLE_FULL_CLRD: IP TUNNEL decap hardware table 
      full exception cleared
        

    • Table 1. Scale Numbers

      Commands	Switches with NFE: Table size 512, v4 takes 1 entry, v6 takes 4 entries	Switches with ASE and LSE: Table size 512, v4 takes 1 entry, v6 takes 2 entries (paired index)
      IPIP decap any with tunnel source direct	Shared between v4 and v6, v6 takes 4 entries v4 + 4 *v6 =512 Maximum entries can be 512 with no v6 entries	Dedicated 256
      IPv6IPv6 decap any with tunnel source direct	Shared between v4 and v6, v6 takes 4 entries v4 + 4 *v6 =512 Maximum entries can be 128 with no v4 entries	Dedicated 128

    • Use Case 3: Auto-recovery is not supported.

      If any entry does not get installed in the hardware due to exhaustion of above table, removal of an already installed IP/IPv6 from interfaces does not automatically trigger the addition of the failed SIP in the table though the table has space now. You need to flap the tunnel interface or IP interface to get them installed.

      However, if an entry does not get installed in the hardware due to a duplicate entry (if there was already a decap-any with one source present and now the source direct tunnel CLI command is configured, there is a duplicate entry for the prior source configured) that was taken care of by removing the entry only when both the tunnels get deleted.

• interface loopback0
    ip address 2001:0:0:4::1/128
  !
  interface Tunnel 1
    ipv6 address use-link-local-only
    tunnel mode tunnel mode ipv6ipv6 decapsulate-any
    tunnel source loopback0
    description IPinIP Decapsulation Interface
    mtu 1476
    no shutdown

• Cisco NX-OS software supports the GRE header defined in IETF RFC 2784. Cisco NX-OS software does not support tunnel keys and other options from IETF RFC 1701.

• The Cisco Nexus device supports the following maximum number tunnels:

  • GRE and IP-in-IP regular tunnels-8 tunnels

  • Multipoint IP-in-IP tunnels-32 tunnels

• Each tunnel will consume one Equal Cost Multipath (ECMP) adjacency.

• The Cisco Nexus device does not support the following features:

  • Path maximum transmission unit (MTU) discovery

  • Tunnel interface statistics

  • Access control lists (ACLs)

  • Unicast reverse path forwarding (URPF)

  • Multicast traffic and associated multicast protocols such as Internet Group Management Protocol (IGMP) and Protocol Independent Multicast (PIM)

• Cisco NX-OS software does not support the Web Cache Control Protocol (WCCP) on tunnel interfaces.

• Cisco NX-OS software supports only Layer-3 traffic.

• Cisco NX-OS software supports ECMP across tunnels and ECMP for tunnel destination.

• IPv6-in-IPv6 tunnels is not supported.

• Limited control protocols, such as Border Gateway Protocol (BGP), Open Shortest Path First (OSPF), and Enhanced Interior Gateway Routing Protocol (EIGRP), are supported for GRE tunnels.

• Starting with Release 6.0(2)U5(1), Cisco Nexus 3000 Series switches drop all the packets when the tunnel is not configured. The packets are also dropped when the tunnel is configured but the tunnel interface is not configured or the tunnel interface is in shut down state.

  Point to Point tunnel (Source and Destination) – Cisco Nexus 3000 Series switches decapsulate all IP-in-IP packets destined to it when the command feature tunnel is configured and there is an operational tunnel interface configured with the tunnel source and the destination address that matches the incoming packets' outer source and destination addresses. If there is not a source and destination packet match or if the interface is in shutdown state, the packet is dropped.

  Decapsulate Tunnel (Source only) - Cisco Nexus 3000 Series switches decapsulate all IP-in-IP packets destined to it when the command feature tunnel is configured and there is an operational tunnel interface configured with the tunnel source address that matches the incoming packets' outer destination addresses. If there is not a source packet match or if the interface is in shutdown state, the packet is dropped.

• Starting with Release 6.0(2)U6(1), Cisco Nexus 3000 Series switches support IPv6 in IPv4 with GRE header only. The new control protocols that are supported on the tunnel are:

  • BGP with v6

  • OSPFv3

  • EIGRP over v6

• GRE v4/v6 tunnel configuration is supported only in the default routing mode. It does not support the multicast traffic or multicast protocols, for example, IGMP/PIM. It does not support ACL/QoS policies. It supports a maximum of 8 tunnels in the switch, whether they are all IPinIP or GRE; or any combination of both. The packets that are sent/received over the tunnel and that are destined for the switch, are not counted in the tunnel statistics.

• The Cisco Nexus 3000 Series switches ASIC supports the GRE encapsulation and decapsulation in the hardware.

• On the encapsulation side, the Cisco Nexus 3000 Series switches performs a single lookup in the hardware.

• Since Cisco Nexus 3000 Series switches perform a single lookup in the hardware, the software has to keep the hardware information up-to-date with any changes related to the second lookup, for example, the tunnel destination adjacency.

• On the decapsulation side, the Cisco Nexus 3000 Series switches have a separate table to perform the outer IP header lookup and it does not need an ACL for the same.

• RFC5549 is not supported over tunnels.

The following table lists the default settings for IP tunnel parameters.