CVP – Cisco ASR 1000 Routers with GETVPN and IPsec – December 2018
Available Languages
b. Hardware and feature specifications
3.2.4 System health monitoring
3.2.5 System and network resiliency and robustness
5. Convergence and throughput data
6.1 Configuration and verifications
6.1.2 Interface configurations
6.1.13 Site-to-site IPsec tunnel and verification
Cisco is transforming the network edge with Cisco® ASR 1000 Series Aggregation Services Routers and Cisco 4000 Series Integrated Services Routers (ISRs). These lines of midrange routers establish a new, price-toperformance-class offering that benefits both enterprises and service providers. These routers provide a great opportunity for simplifying the WAN edge and significantly decreasing network Operating Expenses (OpEx). By efficiently integrating a critical set of WAN edge functions such as WAN aggregation, Internet edge services, firewall services, VPN termination, etc. into a single platform, enterprises can meet their business objectives by facilitating deployment of advanced services in a secure, scalable, and reliable manner while minimizing the Total Cost of Ownership (TCO).
Cisco WAN aggregation solutions distinguish themselves from other solutions by offering multiservice routers with industry-leading performance, availability, and density for concurrent data, security, voice, and application-acceleration services with maximum headroom for growth. The solutions feature embedded security, performance, and memory enhancements. High-performance interfaces featuring the latest WAN technologies can help enterprises meet the needs of the most demanding WAN network.
Cisco provides industry-leading feature-rich and flexible VPN solutions. Cisco VPN solutions integrate advanced network intelligence and routing to deliver reliable transport for complex mission-critical traffic, such as voice and client-server applications, without compromising communications quality. These solutions are built on the following underlying VPN technologies:
● Dynamic Multipoint VPN Generic Routing Encapsulation (DMVPN–GRE, which is tunnel-based)
● Group Encrypted Transport VPN (GETVPN, which is tunnel-less)
● Easy VPN, Secure Sockets Layer (SSL) VPN, IPsec Static Virtual Tunnel Interfaces (SVTI), IPSec Dynamic Vitual Tunnel Interfaces (DVTI), which are all virtual-template-based
● Static and dynamic crypto maps
Each technology has its benefits and can be customized to meet specific enterprise WAN deployment requirements.
GETVPN is a tunnel-less VPN technology that provides end-to-end security for network traffic in a native mode and maintains a full meshed topology. It uses the core network's ability to route and replicate the packets between various sites within the enterprise. The Cisco IOS® GETVPN solution preserves the original source and destination IP address information in the header of an encrypted packet for optimal routing. Cisco IOS GET VPN uses Group Domain of Interpretation (GDOI) as the keying protocol and IPsec for encryption. GDOI is used between the Group Member (GM) and Key Server (KS) for group key and group SA management. IP-Delivery Delay Detection Protocol (IP-D3P) uses the system clock of group members to create and verify the IP-D3P datagram’s timestamp. In most cases, the system clock is set from an external protocol, such as Network Time Protocol (NTP), to synchronize the system clocks of the sender and receiver.
Site-to-site IPsec tunnels are used to allow the secure transmission of data, voice, and video between two sites.
This Cisco Validated Profile covers deployment of a Cisco ASR 1000 router as a small Customer Edge (CE) providing security with GETVPN and IPsec. It includes GETVPN and IPsec co-existence with local Access Control Lists (ACL) to differentiate traffic using GETVPN and IPsec. It uses Virtual Route Forwarding (VRF)-aware GDOI for group-member-to-key-server communication. External Border Gateway Protocol (eBGP) with Bidirectional Forwarding Detection (BFD) is used for WAN and internal Border Gateway Protocol (iBGP) with BFD in the LAN. BFD is enabled with BGP to detect network failures.
GETVPN with port channel and BFD
There are three key servers in Co-operative (COOP) mode, two GMs for encryption and decryption, and another redundant GM in the same site for routing awarness. BGP runs for VRF routing and key server reachability. The GMs are registered to the key server over a port channel subinterface. The GMs and KSs uses NTP and are in sync for IPD3P. Routing awarness is enabled to detect the failures in crypto using the track on the GMs to switch over to the redundant GM.
A site-to-site IPsec tunnel is coexisting with GETVPN.
Table 1. Deployment area and features
| Deployment areas |
Features |
| Security |
GETVPN, IPD3P, site-to-site IPsec, Pre-Shared Key (PSK) |
| Management and monitoring |
SNMP, sysLog server |
| System resiliency |
Interface flaps, routing flaps, node failure |
| Network services |
BGP, port channel, GM routing awareness |
| Network resiliency |
BFD |
Based on research, customer feedback, and configuration samples, the Cisco ASR 1000 router as small CE providing security with GETVPN and IPsec profile is designed with a deployment topology that is generic and can be easily modified to fit any specific deployment scenario.
Disclaimer: The links between the different network layers in the topology are mainly to facilitate this profile validation across different platform combinations. The actual deployment could vary based on specific requirements.
Figure 1 shows the deployment diagram for our profile.
GM1 and GM5 are in the same site and they have GM redundancy. GM1, GM5 and CE1 are in the same AS (Autonomous System).
Traffic path is TGEN Ò CE1 Ò GM1 LAN Port channel subinterface Ò GM1 WAN Port-channel subinterface Ò CE1 Ò P Ò CE2 ÒGM2 WAN Port channel subinterface Ò GM2 LAN Port-channel Subinterface Ò CE2 Ò TGEN
Each port-channel has two 10-G member links.
The LAN port channel subinterface is between GM1 and CE1 and it is a layer 3 link. GM1 and CE1 have iBGP with BFD on the LAN side.
The WAN port channel subinterface is between GM1 and CE1. GM1 and P (Provider/Core router) have an eBGP session with BFD. CE1 bundle-ethernets are going to GM1, and P and are layer 2.
There are three key servers operating in COOP mode. GM-KS communication is through a different VRF
(VRF-aware GDOI).
GETVPN is enabled on the WAN port-channel subinterfaces on GM1, GM5 and GM3
Each GM registers to 15 IPv4 GDOI groups on the key server.
Site-to-site IPsec is between GM1 and GM3 with the tunnel source as a loopback interface. It is reachable through a WAN port-channel subinteraface.
Local ACL is used to deny the site-to-site IPsec traffic to be encrypted by GETVPN.
b. Hardware and feature specifications
This section details the 3D feature matrix where the hardware platforms are listed, along with their Place In Network (PIN) and the relevant deployment.
Table 2. Hardware specifications
| VM and hardware |
Software versions |
| GM1 |
ASR 1002-HX Router |
| GM3 |
ASR 1002-HX |
| GM5 |
ASR 1002-HX |
| KS1 |
Cisco 3925E Integrated Services Router (ISR) |
| KS2 |
3925E ISR |
| KS3 |
3925E ISR |
| P |
ASR 1001-HX |
| CE2 |
ASR 1002-HX |
| CE1 |
ASR 9000 router |
Table 3. Features tested
| Features and functionalities tested |
|
●
GETVPN
|
|
●
Port-channel – Link Aggregation and Control Protocol (LACP) and load-balancing
|
|
●
BFD on port-channel subinterfaces
|
|
●
IP-D3P
|
|
●
Site-to-site IPsec and GETVPN co-existence
|
|
●
GM routing awareness
|
|
●
IP service-level agreement (IP SLA)
|
|
●
WAN port-channel + eBGP + BFD
|
|
●
LAN port-channel + iBGP + BFD
|
Table 4 defines the 3D hardware, Place In Network (PIN), and the features deployed. The scale of these configured features, the test environment, the list of endpoints, and hardware and software versions of the network topology will be defined in subsequent sections of this guide.
Table 4. Key vertical features
| Deployment layer |
Platforms |
Critical vertical features |
| ASR1k GM1 |
ASR 1002-HX Router |
●
The LAN port channel subinterface is between GM1 and CE1 and is a layer 3 link. GM1 and CE1 have iBGP with BFD on the LAN side.
●
The WAN port channel subinterface is between GM1 and CE1. GM1 and P have an
eBGP session with BFD. CE1 bundle-Ethernets are going to GM1 and P are
layer 2.
●
GETVPN on the WAN port-channel subinterface
●
IPD3P
●
Site-to-site IPsec tunnel
●
GM routing awareness
●
15 customer VRF, 1 key server VRF for key server reachability, 1 LAN VRF for the site-to-site IPsec tunnel
●
Local ACL separates traffic using GETVPN and IPsec
|
| ASR 1000 GM5 |
ASR 1002-HX Router |
●
The LAN port channel subinterface is between GM5 and CE1 and is a layer 3 link. GM5 and CE1 have iBGP with BFD on the LAN side.
●
The WAN port channel subinterface is between GM5 and CE1. GM5 and P have an
eBGP session with BFD. CE1 bundle-Ethernets are going to GM5 and P are layer 2.
●
GETVPN on WAN port-channel subinterface
●
IPD3P
●
Site-to-site IPsec tunnel
●
GM routing awareness
●
15 customer VRF, 1 key server VRF for key server reachability, 1 LAN VRF for the site-to-site IPsec tunnel
●
Local ACL separates traffic using GETVPN and IPSEC
|
| ASR 9000 CE1 |
ASR 9006 Router (RP/0/RSP0/CPU0) |
●
iBGP on LAN side with GM1/GM5 on port-channel sub interface
●
Local preference to prefer GM1 over GM5
●
Layer 2 VPN to switch traffic between GM1 and GM5 and P for WAN-side traffic
|
| ASR 1000 GM3 |
ASR 1002-HX Router |
●
The LAN port channel subinterface is between GM3 and CE2 and is a layer 3 link. GM3 and CE2 have iBGP with BFD on the LAN side.
●
The WAN port channel subinterface is between GM3 and CE2. GM3 and P have an eBGP session with BFD. The CE2 port-channel is going to GM3 and P are layer 2.
●
GETVPN on WAN port-channel subinterface
●
Site-to-site IPsec tunnel
●
IPD3P
●
15-customer VRF, 1 key server VRF for key server reachability; 1 LAN VRF for site-to-site IPsec tunnel
|
| ASR 1000 P |
ASR 1001-HX Router |
●
eBGP on WAN side with GM1, GM5, and GM3 on port-channel subinterface.
●
Open Shortest Path First (OSPF) for key server reachability
●
15-customer VRF; 1 key server VRF for key server reachability
|
| c3900e KS1 |
3925E Integrated Services Router (ISR) |
●
GETVPN
●
COOP
●
OSPF for GM reachability
●
15 GDOI groups
●
IPD3P
●
TEK, KEK
●
Traffic Encryption Key (TEK), Key Encryption Key (KEK)
●
KS ACL with 100 ACEs
|
| c3900e KS2 |
3925E ISR |
●
GETVPN
●
COOP
●
OSPF for GM reachability
●
15 GDOI groups
●
IPD3P
●
TEK, KEK
●
KS ACL with 100 ACEs
|
| c3900e KS3 |
3945E Integrated Services Router (ISR) |
●
GETVPN
●
COOP
●
OSPF for GM reachability
●
15 GDOI groups
●
IPD3P
●
TEK, KEK
●
KS ACL with 100 ACEs
|
| ASR 1000 CE2 |
ASR 1002-HX Router |
●
BGP
●
15-customer VRF, 1 key server VRF for key server reachability
|
Table 5 defines the set of relevant hardware, servers, test equipment, and endpoints that are used to complete the end-to-end deployment.
A list of hardware, along with the relevant software versions and the role of these devices, complement the actual physical topology that is defined in Figure 1 of the previous section.
Table 5. Hardware profile
| VM and hardware |
Software versions |
Description |
| Spirent |
Spirent test center (Version 4.66) |
Test tool to generate traffic |
| Windows VM clients |
Windows 7 |
Endpoints to test end-to-end traffic |
This section contains the description of the features and the relevant scales at which the features are deployed across the physical topology. Table 6 lists out the scale for each respective feature.
Disclaimer: Table 6 captures a sample set of scale values used in one of the use cases. Refer to appropriate cisco.com documentation and data sheets for comprehensive scale data.
Table 6. Test environment
| Device |
Scale |
| ASR 9000 (CE1), GM1, P, CE2, GM3 |
15-customer VRF |
| GM1, GM3 |
1 key server VRF |
| GM1, GM3 |
LAN VRF for site-to-site reachability |
| KS1, KS2, KS3 |
15 GDOI groups |
| KS1, KS2, KS3 |
100 ACEs per GDOI group |
The following use cases can be executed using the topology defined in Figure 1, along with the test environment in Table 3 that are already explained in this document.
Images are loaded on the devices under test via the TFTP server using the management interface.
To validate a new release, the network topology is upgraded with the new software image with an existing configuration that comprises the use cases and relevant traffic profiles. New use cases acquired from the field or customer deployments are added on top of the existing configuration.
During each use-case execution, syslog would be monitored closely across the devices for any relevant system events, errors, or alarms. With respect to longevity for this profile setup, CPU and memory usage or leaks would be monitored during the validation phase. Furthermore, to test the robustness of the software release and platform under test, typical network events would be triggered during the use-case execution process.
Table 7 describes the use cases that were executed on the Cisco ASR 1000 router profile. These use cases are divided into buckets of technology areas to offer the complete coverage of the deployment scenarios.
Table 7. Use cases
| Number |
Focus area |
Use cases |
| 1 |
Port channel |
●
Port-channel with 2 10-G member links
|
| 2 |
BGP |
●
iBGP + BFD between ASR9k CE1 and ASR 1002-HX GM1 on the LAN side
●
eBGP + BFD (100*3) between ASR 1002-HX P and ASR 1002-HX GM1 on the WAN side
|
| 3 |
BFD |
●
BFD over port-channel subinterface
|
| 4 |
Routing awareness |
●
GM5 is configured for routing awareness as when the crypto fails to register and the redundant GM takes over
|
| 5 |
GETVPN |
●
GETVPN is used on port channel interface subinterface
●
Key server communication is done on KS-VRF
●
GETVPN version 4 with next-generation encryption (NGE), AES 256, esp-sha256-hmac, 3
key
servers
●
Each KS has 15 GDOI groups serving 2 GMs each
●
Each KS uses the default KEK of 24 hours and TEK of 900 seconds
●
Each KS has 15 GDOI groups serving 3 GMs
|
| 6 |
Site-to-site IPsec tunnel |
●
LAN VRF, which was configured as a tunnel between GM1 and GM5
●
OSPF + BFD for site-to-site reachability
|
| 7 |
IPD3P |
●
Network Time Protocol (NTP) synchronization
●
IPD3P anti-replay timer is configured on all GDOI groups
|
| 8 |
Fail close ACL |
Fail close ACL is configured on the port channel interface |
| 9 |
Monitoring |
Exporting and monitoring logs from the syslog server |
| 3.2.4 System health monitoring |
||
| 10 |
System health |
Monitor system health for CPU usage, memory consumption, and memory leaks during longevity |
| 11 |
Simple Network Management Protocol (SNMP) Mibwalk |
Monitor system health for CPU usage, memory consumption, and memory leaks during SNMP Mibwalk |
| 3.2.5 System and network resiliency and robustness |
||
| 12 |
System resiliency |
Verify system-level resiliency during the following events:
●
WAN and LAN interface flaps
●
IPsec tunnel flaps
●
Link failures
●
Node failures
●
Reload, power, and cycle
|
| 13 |
Negative events, triggers |
Verify that the system holds good and recovers to working condition after the following negative events are triggered:
●
Configuration changes - add/remove configuration snippets, config replace
●
Routing protocol flaps
●
Interface flap
●
Port-channel events
●
BFD events
●
Key server events
●
GDOI group events
|
● IPD3P and Time-Based Anti-Relay (TBAR) PST cannot be enabled at the same time on the same group. IPD3P and TBAR PST can be enabled on different GDOI groups. [[Please spell out this acronym on its first usage - PST. I could not find an appropriate meaning on cisco.com or Google.]]
● While changing IPD3P to TBAR PST or vice versa, issue the replay command, “crypto gdoi ks rekey replace” on the KS to trigger the replacement of SAs on the GM.
● Micro BFD cannot be enabled in a port-channel subinterface.
● The default minimum BFD timer is 750 ms on the port channel. Hence, to enable the 100*3 ms timer use the hidden global command, “bfd fast-timers-on-slow-interface” and “carrier-delay msec 0” under the interfaces.
● This hidden Command-Line Interface (CLI) is introduced from the CDETS CSCsu64050. The CDETS contains more history on this command.
● To have maximum utilization of the port-channel member links, the recommendation is to use “port-channel load-balance-hash-algo dst-ip” on the Cisco ASR 1000 GM and “load-balancing flow src-dst-ip” in the Cisco ASR 9000 CE1.
5. Convergence and throughput data
Table 8. Convergence data
| Event |
Port-channel Type (LACP or flow based) |
Traffic switch from --> to |
Stream1 : CE2 (ASR1002-HX) --> CE1 (ASR9k) (seconds) |
Stream2: CE1(ASR9k) --> ASR1002-HX (seconds) |
| 1 member link shutdown on GM1 |
LACP |
GM1 --> GM1 |
0.3 |
2.12 |
| both member links shutdown on GM1 |
LACP |
GM1 --> GM5 |
0.1 |
0.1 |
| one member link shut on GM5 |
LACP |
GM5 --> GM5 |
4.64 |
4.24 |
| both member links shutdown on GM5 |
LACP |
GM5 --> GM1 |
0.28 |
0.19 |
| port-channel shut on GM1 |
LACP |
GM1 --> GM5 |
0.26 |
0.07 |
| port-channel shut on GM5 |
LACP |
GM5 --> GM1 |
0.27 |
0.2 |
| port-channel subinterface shut on GM1 (one vrf) |
LACP |
GM1 --> GM5 (one vrf) |
0.2 |
0 |
| port-channel subinterface shut on GM5(one vrf) |
LACP |
GM5 --> GM1 (one vrf) |
0.04 |
0 |
| remove 1 member link from port-channel |
LACP |
GM1 --> GM1 |
0 |
0 |
| remove both member links from port-channel |
LACP |
GM1 --> GM5 |
0.06 |
0.01 |
| Client registration interface shutdown on GM1 |
LACP |
GM1 --> GM5 |
27 |
27 |
| Client registration interface shutdown on GM5 |
LACP |
GM5 --> GM1 |
27 |
27 |
| Stop rechability to Keyserver on GM1 |
LACP |
GM1 --> GM5 |
51 |
30 |
| Stop rechability to Keyserver on GM5 |
LACP |
GM5 --> GM1 |
54 |
32 |
| GM1 reload |
LACP |
GM1 --> GM5 |
0.1 |
2.01 |
| GM5 reload |
LACP |
GM5 --> GM1 |
0.1 |
0 |
| 1 member link shutdown on GM1 |
flowbased |
GM1 --> GM1 |
0.2 |
0.1 |
| both member links shutdown on GM1 |
flowbased |
GM1 --> GM5 |
0.39 |
2.12 |
| one member link shut on GM5 |
flowbased |
GM5 --> GM5 |
0.18 |
2.08 |
| both member links shutdown on GM5 |
flowbased |
GM5 --> GM1 |
0.67 |
0.2 |
| port-channel shut on GM1 |
flowbased |
GM1 --> GM5 |
2 |
9 |
| port-channel shut on GM5 |
flowbased |
GM5 --> GM1 |
3.16 |
1.39 |
| port-channel subinterface flap on GM1 (one vrf) |
flowbased |
GM1 --> GM5 (one vrf) |
0.02 |
0 |
| port-channel subinterface flap on GM5(one vrf) |
flowbased |
GM5 --> GM1 (one vrf) |
0.02 |
0 |
| Client registration interface shutdown on GM1 |
flowbased |
GM1 --> GM5 |
29 |
29 |
| Client registration interface shutdown on GM5 |
flowbased |
GM5 --> GM1 |
30 |
29 |
| Stop rechability to Keyserver on GM1 |
flowbased |
GM1 --> GM5 |
51 |
30 |
| Stop rechability to Keyserver on GM5 |
flowbased |
GM5 --> GM1 |
54 |
29 |
| GM1 reload |
flowbased |
GM1 --> GM5 |
0 |
2.01 |
| GM5 reload |
flowbased |
GM5 --> GM1 |
0 |
0 |
Table 9. Throughput data
| ASR 1K |
Packet Size |
Throughput (in Gbps) |
Throughput (in frames/ |
Crypto utilization (%) |
RP Memory |
IOS Memory |
QFP Memory |
RP CPU |
IOS CPU |
QFP CPU |
| ASR1002-HX |
1400 |
17 |
747608 |
34 |
2986MB (18%) |
6.58% |
323808KB (7%) |
1.80% |
2% |
18% |
| ASR1002-HX |
1024 |
17 |
747608 |
38 |
2988MB (18%) |
6.57% |
323808KB (7%) |
12.70% |
2% |
24% |
| ASR1002-HX |
512 |
16.2 |
1893939 |
51 |
2986MB (18%) |
6.57% |
323808KB (7%) |
14.39% |
2% |
45% |
| ASR1002-HX |
128 |
9.2 |
3858025 |
74 |
2987MB (18%) |
6.57% |
323808KB (7%) |
5.30% |
2% |
94% |
| ASR1002-HX |
82 |
6.6 |
4006410 |
70 |
2986MB (18%) |
6.58% |
373816KB (8%) |
9.00% |
2% |
98% |
| ASR1002-HX |
IMIX (64-7, 594-4, 1518-1) |
10 |
1627604 |
80 |
2988MB (18%) |
6.58% |
373816KB (8%) |
9.60% |
2% |
97% |
6.1 Configuration and verifications
6.1.1.1 GM1
vrf definition KS-VRF1 =============Ò Keyserver VRF
rd 200:1
!
address-family ipv4
route-target export 200:1
route-target import 200:1
exit-address-family
!
vrf definition LAN1 ===============Ò S2S VRF
rd 300:1
!
address-family ipv4
route-target export 300:1
route-target import 300:1
exit-address-family
!
vrf definition cust1 ==============ÒCustomer VRF
rd 100:1
route-target import 100:1
!
address-family ipv4
route-target export 100:1
route-target import 100:1
exit-address-family
!
vrf definition cust2
rd 100:2
!
address-family ipv4
route-target export 100:2
route-target import 100:2
exit-address-family
!
vrf definition cust3
rd 100:3
!
address-family ipv4
route-target export 100:3
route-target import 100:3
exit-address-family
!
…
…
vrf definition cust15
rd 100:15
!
address-family ipv4
route-target export 100:15
route-target import 100:15
exit-address-family
!
6.1.1.2 GM5
vrf definition KS-VRF1 ====Ò Key server VRF
rd 200:1
!
address-family ipv4
route-target export 200:1
route-target import 200:1
exit-address-family
!
vrf definition LAN1 ===============Ò S2S VRF
rd 300:1
!
address-family ipv4
route-target export 300:1
route-target import 300:1
exit-address-family
!
vrf definition cust1 ========Ò Customer VRF
rd 100:1
route-target import 100:1
!
address-family ipv4
route-target export 100:1
route-target import 100:1
exit-address-family
!
vrf definition cust2
rd 100:2
!
address-family ipv4
route-target export 100:2
route-target import 100:2
exit-address-family
!
…
…
vrf definition cust15
rd 100:15
!
address-family ipv4
route-target export 100:15
route-target import 100:15
exit-address-family
6.1.1.3 CE1
vrf cust1 ===Ò Customer VRF
address-family ipv4 unicast
import route-target
100:1
!
export route-target
100:1
!
!
!
vrf cust2
address-family ipv4 unicast
import route-target
100:2
!
export route-target
100:2
!
!
!
…
…
vrf cust15
address-family ipv4 unicast
import route-target
100:15
!
export route-target
100:15
!
!
!
vrf KS-VRF1 === Key server VRF
address-family ipv4 unicast
import route-target
200:1
!
export route-target
200:1
!
!
6.1.1.4 P
vrf definition KS-VRF1
rd 200:1
!
address-family ipv4
route-target export 200:1
route-target import 200:1
exit-address-family
!
vrf definition cust1
rd 100:1
!
address-family ipv4
route-target export 100:1
route-target import 100:1
exit-address-family
!
address-family ipv6
route-target export 100:1
route-target import 100:1
exit-address-family
!
vrf definition cust2
rd 100:2
!
address-family ipv4
route-target export 100:2
route-target import 100:2
exit-address-family
!
…
…
vrf definition cust15
rd 100:15
!
address-family ipv4
route-target export 100:15
route-target import 100:15
exit-address-family
!
6.1.1.5 CE2
vrf definition KS-VRF1
rd 200:1
!
address-family ipv4
route-target export 200:1
route-target import 200:1
exit-address-family
!
vrf definition cust1
rd 100:1
!
address-family ipv4
route-target export 100:1
route-target import 100:1
exit-address-family
!
!
vrf definition cust2
rd 100:2
!
address-family ipv4
route-target export 100:2
route-target import 100:2
exit-address-family
!
…
…
vrf definition cust15
rd 100:15
!
address-family ipv4
route-target export 100:15
route-target import 100:15
exit-address-family
!
6.1.2 Interface configurations
6.1.2.1 GM1
port-channel load-balance-hash-algo dst-ip
interface Loopback1000
vrf forwarding KS-VRF1
ip address 33.1.1.1 255.255.255.255
ipv6 address 9898::1/64
ipv6 enable
end
interface TenGigabitEthernet0/1/6
description "Interface connected to ASR9k"
carrier-delay msec 0
no ip address
channel-group 1
!
interface TenGigabitEthernet0/1/7
description "Interface connected to ASR9k"
carrier-delay msec 0
no ip address
channel-group 1
!
interface Port-channel1
carrier-delay msec 0
no ip address
!
interface Port-channel1.102
description "LAN facing port channel"
encapsulation dot1Q 102
vrf forwarding cust1
ip address 3.3.2.1 255.255.255.0
!
interface Port-channel1.103
description "LAN facing port channel"
encapsulation dot1Q 103
vrf forwarding cust2
ip address 3.3.3.1 255.255.255.0
!
…
…
interface Port-channel1.116
description "LAN facing port channel"
encapsulation dot1Q 116
vrf forwarding cust15
ip address 3.3.16.1 255.255.255.0
!
interface Port-channel1.2
description "WAN facing port channel"
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.2 255.255.255.0
no ip redirects
!
interface Port-channel1.3
description "WAN facing port channel"
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.2 255.255.255.0
no ip redirects
!
…
…
interface Port-channel1.16
description "WAN facing port channel"
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.2 255.255.255.0
no ip redirects
!
interface Port-channel1.100 ==============Òfor Key server
description "WAN facing for Key server reachability"
encapsulation dot1Q 100
vrf forwarding KS-VRF1
ip address 38.1.1.1 255.255.255.0
!
6.1.2.2 GM5
interface Loopback1000
vrf forwarding KS-VRF1
ip address 33.2.1.1 255.255.255.255
!
interface TenGigabitEthernet0/1/4
description "Interface connected to ASR9k"
carrier-delay msec 0
no ip address
channel-group 1
!
interface TenGigabitEthernet0/1/7
description "Interface connected to ASR9k"
carrier-delay msec 0
no ip address
channel-group 1
!
interface Port-channel1
no ip address
carrier-delay msec 0
!
interface Port-channel1.2
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.50 255.255.255.0
no ip redirects
!
interface Port-channel1.3
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.50 255.255.255.0
no ip redirects
!
…
…
interface Port-channel1.16
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.50 255.255.255.0
no ip redirects
!
interface Port-channel1.100
description "WAN facing for Key server reachability"
encapsulation dot1Q 100
vrf forwarding KS-VRF1
ip address 38.1.1.50 255.255.255.0
!
interface Port-channel1.102
description "LAN facing port channel"
encapsulation dot1Q 102
vrf forwarding cust1
ip address 3.33.2.1 255.255.255.0
!
interface Port-channel1.103
description "LAN facing port channel"
encapsulation dot1Q 103
vrf forwarding cust2
ip address 3.33.3.1 255.255.255.0
!
…
…
interface Port-channel1.115
description "LAN facing port channel"
encapsulation dot1Q 115
vrf forwarding cust14
ip address 3.33.15.1 255.255.255.0
!
6.1.2.3 CE1
interface TenGigE0/0/1/0
description "Interface connected to GM5"
bundle id 6 mode on
!
interface TenGigE0/0/1/1
description "Interface connected to GM5"
bundle id 6 mode on
!
interface TenGigE0/0/1/2
transceiver permit pid all
!
interface TenGigE0/0/1/2.2
description "Spirent interface"
vrf cust1
ipv4 address 100.100.100.1/24
encapsulation dot1q 2
!
interface TenGigE0/0/1/2.3
description "Spirent interface"
vrf cust2
ipv4 address 100.100.101.1/24
encapsulation dot1q 3
!
…
…
interface TenGigE0/0/1/2.16
description "Spirent interface"
vrf cust15
ipv4 address 100.100.114.1/24
encapsulation dot1q 16
!
interface TenGigE0/2/1/0
description "Connected to P"
bundle id 2 mode on
!
interface TenGigE0/2/1/1
description "Connected to P"
bundle id 2 mode on
!
interface TenGigE0/2/1/2
description "Interafce Connected to GM1"
bundle id 1 mode on
carrier-delay up 50 down 0
load-interval 30
dampening
!
interface TenGigE0/2/1/3
description "Interface connected to GM1"
bundle id 1 mode on
carrier-delay up 50 down 0
load-interval 30
dampening
!
interface Bundle-Ether1
mtu 4484
bundle minimum-active links 1
load-interval 30
!
interface Bundle-Ether1.2 l2transport =====Ò GM1 WAN Facing
description "GM1 facing"
encapsulation dot1q 2
!
interface Bundle-Ether1.3 l2transport
description "GM1 facing"
encapsulation dot1q 3
!
…
…
interface Bundle-Ether1.16 l2transport
description "GM1 facing"
encapsulation dot1q 16
!
interface Bundle-Ether6.2 l2transport =====Ò GM5 WAN facing
description "GM5 facing"
encapsulation dot1q 2
!
interface Bundle-Ether6.3 l2transport
description "GM5 facing"
encapsulation dot1q 3
!
…
…
interface Bundle-Ether6.16 l2transport
description "GM5 facing"
encapsulation dot1q 16
!
interface Bundle-Ether1.102 ===========Ò LAN Facing
description "LAN facing Bundle-Ethernet"
vrf cust1
ipv4 address 3.3.2.2/24
encapsulation dot1q 102
!
interface Bundle-Ether1.103
vrf cust2
ipv4 address 3.3.3.2/24
encapsulation dot1q 103
!
…
…
interface Bundle-Ether1.116
vrf cust15
ipv4 address 3.3.16.2/24
encapsulation dot1q 116
!
6.1.2.4 P
interface TenGigabitEthernet0/1/4
description "Interface connected to CE1"
carrier-delay msec 0
no ip address
channel-group 2
!
interface TenGigabitEthernet0/1/5
description "Interface connected to CE1"
carrier-delay msec 0
no ip address
channel-group 2
!
interface TenGigabitEthernet0/1/6
description "Interface connected to CE2"
no ip address
channel-group 3
!
interface TenGigabitEthernet0/1/7
description "Interface connected to CE2"
no ip address
channel-group 3
!
interface Loopback9
vrf forwarding KS-VRF1
ip address 155.1.2.1 255.255.255.255
!
interface Port-channel2
description "WAN port channel"
no ip address
carrier-delay msec 0
no negotiation auto
!
interface Port-channel2.2
description "WAN port-channel subinterface"
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.1 255.255.255.0
no ip redirects
!
interface Port-channel2.3
description "WAN port-channel subinterface"
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.1 255.255.255.0
no ip redirects
!
…
…
interface Port-channel2.15
description "WAN port-channel subinterface"
encapsulation dot1Q 15
vrf forwarding cust14
ip address 2.2.15.1 255.255.255.0
no ip redirects
!
interface Port-channel2.100
description "WAN port-channel for KS-VRF"
encapsulation dot1Q 100
vrf forwarding KS-VRF1
ip address 38.1.1.2 255.255.255.0
!
interface Port-channel3 ==============Ò CE2 Port-channel
no ip address
no negotiation auto
!
interface Port-channel3.2
encapsulation dot1Q 2
vrf forwarding cust1
ip address 20.1.1.2 255.255.255.0
!
interface Port-channel3.3
encapsulation dot1Q 3
vrf forwarding cust2
ip address 20.1.2.2 255.255.255.0
!
…
…
interface Port-channel3.15
encapsulation dot1Q 15
vrf forwarding cust14
ip address 20.1.14.2 255.255.255.0
!
interface Port-channel3.16
encapsulation dot1Q 16
vrf forwarding cust15
ip address 20.1.15.2 255.255.255.0
!
interface Port-channel3.100
encapsulation dot1Q 100
vrf forwarding KS-VRF1
ip address 39.1.1.2 255.255.255.0
!
6.1.2.5 CE2
interface TenGigabitEthernet0/1/6
description "Interface Connected to P”
no ip address
channel-group 60
!
interface TenGigabitEthernet0/1/7
description " Interface connected to P“
no ip address
channel-group 60
!
interface TenGigabitEthernet0/1/4
description " Interface connected to GM3“
no ip address
channel-group 61
!
interface TenGigabitEthernet0/1/5
description " Interface connected to GM3“
no ip address
channel-group 4
!
interface Port-channel60 =====Ò Bridge between GM3 and P
no ip address
service instance 3 ethernet
encapsulation dot1q 3
bridge-domain 3
!
service instance 4 ethernet
encapsulation dot1q 4
bridge-domain 4
!
…
…
service instance 15 ethernet
encapsulation dot1q 15
bridge-domain 15
!
service instance 16 ethernet
encapsulation dot1q 16
bridge-domain 16
!
service instance 60 ethernet
encapsulation dot1q 2
bridge-domain 60
!
service instance 100 ethernet
encapsulation dot1q 100
bridge-domain 100
!
!
interface Port-channel61 ============Ò Bridge between GM3 and P
no ip address
service instance 3 ethernet
encapsulation dot1q 3
bridge-domain 3
!
service instance 4 ethernet
encapsulation dot1q 4
bridge-domain 4
!
…
…
service instance 60 ethernet
encapsulation dot1q 2
bridge-domain 60
!
service instance 100 ethernet
encapsulation dot1q 100
bridge-domain 100
!
!
interface Port-channel4
no ip address
!
interface Port-channel4.100
encapsulation dot1Q 100
vrf forwarding KS-VRF1
ip address 39.1.1.2 255.255.255.0
!
interface Port-channel4.102
encapsulation dot1Q 102
vrf forwarding cust1
ip address 30.1.2.2 255.255.255.0
!
interface Port-channel4.103
encapsulation dot1Q 103
vrf forwarding cust2
ip address 30.1.3.2 255.255.255.0
!
…
…
interface Port-channel4.115
encapsulation dot1Q 115
vrf forwarding cust14
ip address 30.1.15.2 255.255.255.0
!
6.1.3.1 GM1
router bgp 200
bgp router-id 100.100.100.100
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
neighbor 38.1.1.2 remote-as 100 ==Ò Keyserver neighbor to P
neighbor 38.1.1.2 ebgp-multihop 255
neighbor 38.1.1.2 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.1 remote-as 100 ===Ò WAN neighbor to P
neighbor 2.2.2.1 activate
neighbor 3.3.2.2 remote-as 200 ====ÒLAN neighbor to CE1
neighbor 3.3.2.2 activate
exit-address-family
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.1 remote-as 100
neighbor 2.2.3.1 activate
neighbor 3.3.3.2 remote-as 200
neighbor 3.3.3.2 activate
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.1 remote-as 100
neighbor 2.2.16.1 activate
neighbor 3.3.16.2 remote-as 200
neighbor 3.3.16.2 activate
exit-address-family
!
6.1.3.2 GM5
router bgp 200
bgp router-id 100.100.100.103
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
neighbor 38.1.1.2 remote-as 100
neighbor 38.1.1.2 ebgp-multihop 255
neighbor 38.1.1.2 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.1 remote-as 100
neighbor 2.2.2.1 activate
neighbor 3.33.2.2 remote-as 200
neighbor 3.33.2.2 activate
exit-address-family
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.1 remote-as 100
neighbor 2.2.3.1 activate
neighbor 3.33.3.2 remote-as 200
neighbor 3.33.3.2 activate
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.1 remote-as 100
neighbor 2.2.16.1 activate
neighbor 3.33.16.2 remote-as 200
neighbor 3.33.16.2 activate
exit-address-family
!
6.1.3.3 CE1
l2vpn ====================Ò WAN side L2
load-balancing flow src-dst-ip
bridge group DT-R4
bridge-domain VLAN2
mtu 4484
interface Bundle-Ether1.2
!
interface Bundle-Ether2.2
!
interface Bundle-Ether6.2
!
!
bridge-domain VLAN3
mtu 4484
interface Bundle-Ether1.3
!
interface Bundle-Ether2.3
!
interface Bundle-Ether6.3
!
…
…
bridge-domain VLAN15
mtu 4484
interface Bundle-Ether1.15
!
interface Bundle-Ether2.15
!
interface Bundle-Ether6.15
!
!
bridge-domain VLAN16
mtu 4484
interface Bundle-Ether1.16
!
interface Bundle-Ether2.16
!
interface Bundle-Ether6.16
!
!
bridge-domain VLAN100
mtu 4484
interface Bundle-Ether1.100
!
interface Bundle-Ether2.100
!
interface Bundle-Ether6.100
!
!
!
!
router bgp 200 ==============Ò BGP for LAN side
nsr
timers bgp 10 30
bgp router-id 10.23.90.245
bgp bestpath med missing-as-worst
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
address-family ipv6 unicast
!
address-family vpnv6 unicast
!
af-group cust1_gm_af_group address-family ipv4 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
af-group cust1_ipv6_gm_af_group address-family ipv6 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
session-group cust1_gm_session_group
remote-as 200
timers 10 30
!
session-group cust1_ipv6_gm_session_group
remote-as 200
timers 10 30
!
neighbor-group cust1_neighbor_group
use session-group cust1_gm_session_group
bfd fast-detect
bfd multiplier 3
bfd minimum-interval 100
address-family ipv4 unicast
use af-group cust1_gm_af_group
!
!
!
vrf cust1
rd 200:30
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
vrf cust2
rd 200:32
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
…
…
vrf cust15
rd 200:45
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
!
6.1.3.4 P
router bgp 100
bgp router-id 99.99.99.99
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
neighbor 38.1.1.1 remote-as 200 =====Ò GM1 KS-VRF neighbor
neighbor 38.1.1.1 ebgp-multihop 255
neighbor 38.1.1.1 activate
neighbor 38.1.1.50 remote-as 200 =====Ò GM5 KS-VRF neighbor
neighbor 38.1.1.50 ebgp-multihop 255
neighbor 38.1.1.50 activate
neighbor 39.1.1.1 remote-as 300
neighbor 39.1.1.1 ebgp-multihop 255 =====Ò GM3 KS-VRF neighbor
neighbor 39.1.1.1 activate
neighbor 39.1.1.3 remote-as 400
neighbor 39.1.1.3 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.2 remote-as 200 =====Ò GM1WAN neighbor
neighbor 2.2.2.2 activate
neighbor 2.2.2.50 remote-as 200 ==Ò GM5 WAN neighbor
neighbor 2.2.2.50 activate
neighbor 20.1.1.1 remote-as 300 ==Ò GM3 WAN neighbor
neighbor 20.1.1.1 ebgp-multihop 255
neighbor 20.1.1.1 activate
exit-address-family
!
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.2 remote-as 200
neighbor 2.2.3.2 activate
neighbor 2.2.3.50 remote-as 200
neighbor 2.2.3.50 activate
neighbor 20.1.2.1 remote-as 300
neighbor 20.1.2.1 activate
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.2 remote-as 200
neighbor 2.2.16.2 activate
neighbor 2.2.16.50 remote-as 200
neighbor 2.2.16.50 activate
neighbor 20.1.15.1 remote-as 300
neighbor 20.1.15.1 activate
exit-address-family
!
6.1.3.5 CE2
router bgp 300
bgp router-id 103.103.103.103
bgp log-neighbor-changes
!
address-family ipv4
exit-address-family
!
address-family ipv4 vrf KS-VRF1
redistribute connected
neighbor 35.1.1.2 remote-as 100
neighbor 35.1.1.2 activate
neighbor 36.1.1.1 remote-as 100
neighbor 36.1.1.1 activate
neighbor 60.1.1.2 remote-as 300
neighbor 60.1.1.2 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 30.1.2.1 remote-as 300
neighbor 30.1.2.1 activate
exit-address-family
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 30.1.3.1 remote-as 300
neighbor 30.1.3.1 activate
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 30.1.16.1 remote-as 300
neighbor 30.1.16.1 activate
exit-address-family
!
6.1.4.1 GM1
bfd fast-timers-on-slow-interface
bfd-template single-hop max
interval min-tx 100 min-rx 100 multiplier 3
interface Port-channel1.102
description "LAN facing port channel"
encapsulation dot1Q 102
vrf forwarding cust1
ip address 3.3.2.1 255.255.255.0
ipv6 enable
bfd interval 100 min_rx 100 multiplier 3
!
interface Port-channel1.103
description "LAN facing port channel"
encapsulation dot1Q 103
vrf forwarding cust2
ip address 3.3.3.1 255.255.255.0
bfd interval 100 min_rx 100 multiplier 3
!
…
…
interface Port-channel1.116
description "LAN facing port channel"
encapsulation dot1Q 116
vrf forwarding cust15
ip address 3.3.16.1 255.255.255.0
bfd interval 100 min_rx 100 multiplier 3
!
interface Port-channel1.2
description "WAN facing port channel"
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.2 255.255.255.0
no ip redirects
bfd template max
!
interface Port-channel1.3
description "WAN facing port channel"
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.2 255.255.255.0
no ip redirects
bfd template max
!
…
…
interface Port-channel1.16
description "WAN facing port channel"
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.2 255.255.255.0
no ip redirects
bfd template max
!
router bgp 200
bgp router-id 100.100.100.100
bgp log-neighbor-changes
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.1 remote-as 100
neighbor 2.2.2.1 fall-over bfd
neighbor 2.2.2.1 activate
neighbor 2.2.2.1 route-map site-med out
neighbor 3.3.2.2 remote-as 200
neighbor 3.3.2.2 fall-over bfd
neighbor 3.3.2.2 activate
neighbor 3.3.2.2 route-map LAN101_1 out
exit-address-family
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.1 remote-as 100
neighbor 2.2.3.1 fall-over bfd
neighbor 2.2.3.1 activate
neighbor 2.2.3.1 route-map site-med1 out
neighbor 3.3.3.2 remote-as 200
neighbor 3.3.3.2 fall-over bfd
neighbor 3.3.3.2 activate
neighbor 3.3.3.2 route-map LAN101_2 out
exit-address-family
!
address-family ipv4 vrf cust3
redistribute connected
neighbor 2.2.4.1 remote-as 100
neighbor 2.2.4.1 fall-over bfd
neighbor 2.2.4.1 activate
neighbor 2.2.4.1 route-map site-med2 out
neighbor 3.3.4.2 remote-as 200
neighbor 3.3.4.2 fall-over bfd
neighbor 3.3.4.2 activate
neighbor 3.3.4.2 route-map LAN101_3 out
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.1 remote-as 100
neighbor 2.2.16.1 fall-over bfd
neighbor 2.2.16.1 activate
neighbor 2.2.16.1 route-map site-med14 out
neighbor 3.3.16.2 remote-as 200
neighbor 3.3.16.2 fall-over bfd
neighbor 3.3.16.2 activate
neighbor 3.3.16.2 route-map LAN101_15 out
exit-address-family
!
6.1.4.2 GM5
bfd fast-timers-on-slow-interface
bfd-template single-hop max
interval min-tx 100 min-rx 100 multiplier 3
interface Port-channel1.2
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.50 255.255.255.0
no ip redirects
bfd template max
!
interface Port-channel1.3
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.50 255.255.255.0
no ip redirects
bfd template max
!
…
…
interface Port-channel1.16
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.50 255.255.255.0
no ip redirects
bfd template max
!
interface Port-channel1.102
description "LAN facing port channel"
encapsulation dot1Q 102
vrf forwarding cust1
ip address 3.33.2.1 255.255.255.0
bfd interval 100 min_rx 100 multiplier 3
!
interface Port-channel1.103
description "LAN facing port channel"
encapsulation dot1Q 103
vrf forwarding cust2
ip address 3.33.3.1 255.255.255.0
bfd interval 100 min_rx 100 multiplier 3
!
…
…
interface Port-channel1.115
description "LAN facing port channel"
encapsulation dot1Q 115
vrf forwarding cust14
ip address 3.33.15.1 255.255.255.0
bfd interval 100 min_rx 100 multiplier 3
!
6.1.4.3 CE1
bfd
multipath include location 0/0/CPU0
multipath include location 0/2/CPU0
!
router bgp 200
nsr
timers bgp 10 30
bgp router-id 10.23.90.245
bgp bestpath med missing-as-worst
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
address-family ipv6 unicast
!
address-family vpnv6 unicast
!
af-group cust1_gm_af_group address-family ipv4 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
af-group cust1_ipv6_gm_af_group address-family ipv6 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
session-group cust1_gm_session_group
remote-as 200
timers 10 30
!
session-group cust1_ipv6_gm_session_group
remote-as 200
timers 10 30
!
neighbor-group cust1_neighbor_group
use session-group cust1_gm_session_group
bfd fast-detect
bfd multiplier 3
bfd minimum-interval 100
address-family ipv4 unicast
use af-group cust1_gm_af_group
!
!
!
vrf cust1
rd 200:30
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
vrf cust2
rd 200:32
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
…
…
vrf cust15
rd 200:45
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
!
6.1.4.4 P
bfd fast-timers-on-slow-interface
bfd-template single-hop max
interval min-tx 100 min-rx 100 multiplier 3
interface Port-channel2.2
description "WAN port-channel subinterface"
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.1 255.255.255.0
no ip redirects
bfd template max
!
interface Port-channel2.3
description "WAN port-channel subinterface"
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.1 255.255.255.0
no ip redirects
bfd template max
!
…
…
interface Port-channel2.15
description "WAN port-channel subinterface"
encapsulation dot1Q 15
vrf forwarding cust14
ip address 2.2.15.1 255.255.255.0
no ip redirects
bfd template max
!
6.1.5.1 P
interface GigabitEthernet0/0/7.152
description "Interface connected to KS switch"
encapsulation dot1Q 152
vrf forwarding KS-VRF1
ip address 150.1.1.1 255.255.255.0
end
router ospf 100 vrf KS-VRF1
redistribute connected subnets
network 10.1.1.5 0.0.0.0 area 0
network 150.1.1.0 0.0.0.255 area 0
router bgp 100
bgp router-id 99.99.99.99
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
redistribute ospf 100
neighbor 37.1.1.1 remote-as 400
neighbor 37.1.1.1 activate
neighbor 38.1.1.1 remote-as 200
neighbor 38.1.1.1 ebgp-multihop 255
neighbor 38.1.1.1 fall-over bfd
neighbor 38.1.1.1 activate
neighbor 38.1.1.50 remote-as 200
neighbor 38.1.1.50 ebgp-multihop 255
neighbor 38.1.1.50 fall-over bfd
neighbor 38.1.1.50 activate
neighbor 39.1.1.1 remote-as 300
neighbor 39.1.1.1 ebgp-multihop 255
neighbor 39.1.1.1 activate
neighbor 39.1.1.3 remote-as 400
neighbor 39.1.1.3 fall-over bfd
neighbor 39.1.1.3 activate
exit-address-family
!
6.1.5.2 KS2 (primary)
interface Loopback0
ip address 70.0.0.2 255.255.255.0
interface GigabitEthernet0/1
description “Interface connected to P via switch”
no ip address
duplex auto
speed auto
!
interface GigabitEthernet0/1.152
encapsulation dot1Q 152
ip address 150.1.1.2 255.255.255.0
!
router ospf 100
network 70.0.0.2 0.0.0.0 area 0
network 150.1.1.0 0.0.0.255 area 0
!
6.1.5.3 KS1
interface Loopback0
ip address 70.0.0.1 255.255.255.255
!
interface GigabitEthernet0/1
no ip address
duplex auto
speed auto
!
interface GigabitEthernet0/1.152
description “Connected to P via switch”
encapsulation dot1Q 152
ip address 150.1.1.4 255.255.255.0
!
router ospf 100
network 70.0.0.1 0.0.0.0 area 0
network 150.1.1.0 0.0.0.255 area 0
!
6.1.5.4 KS3
interface Loopback0
ip address 70.0.0.3 255.255.255.0
interface GigabitEthernet0/1
no ip address
duplex auto
speed auto
!
interface GigabitEthernet0/1.152
encapsulation dot1Q 152
ip address 150.1.1.3 255.255.255.0
!
interface GigabitEthernet0/2
no ip address
duplex auto
speed auto
!
router ospf 100
network 70.0.0.3 0.0.0.0 area 0
network 150.1.1.0 0.0.0.255 area 0
!
6.1.6.1 GM1
GM1#show ip route vrf cust1
2.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 2.2.2.0/24 is directly connected, Port-channel1.2
L 2.2.2.2/32 is directly connected, Port-channel1.2
3.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C 3.3.2.0/24 is directly connected, Port-channel1.102
L 3.3.2.1/32 is directly connected, Port-channel1.102
B 3.33.2.0/24 [200/0] via 3.3.2.2, 04:47:13
10.0.0.0/24 is subnetted, 1 subnets
B 10.0.0.0 [200/0] via 3.3.2.2, 04:53:15
20.0.0.0/24 is subnetted, 1 subnets
B 20.1.1.0 [20/0] via 2.2.2.1, 04:53:15
29.0.0.0/24 is subnetted, 1 subnets
B 29.29.29.0 [20/0] via 2.2.2.1, 04:53:15
30.0.0.0/24 is subnetted, 2 subnets
B 30.1.1.0 [20/0] via 2.2.2.1, 04:53:15
B 30.1.2.0 [20/0] via 2.2.2.1, 04:53:15
33.0.0.0/32 is subnetted, 1 subnets
C 33.33.33.33 is directly connected, Loopback4000
40.0.0.0/32 is subnetted, 1 subnets
B 40.0.0.1 [200/0] via 3.3.2.2, 04:53:15
44.0.0.0/32 is subnetted, 1 subnets
B 44.44.44.44 [20/0] via 2.2.2.1, 04:53:15
55.0.0.0/32 is subnetted, 1 subnets
C 55.1.1.1 is directly connected, Loopback100
77.0.0.0/32 is subnetted, 1 subnets
B 77.1.1.1 [20/0] via 2.2.2.1, 04:53:15
100.0.0.0/24 is subnetted, 2 subnets
B 100.100.100.0 [200/0] via 3.3.2.2, 04:53:15 ===Ò CE1 LAN Network
B 100.101.100.0 [200/0] via 3.3.2.2, 04:53:15
B 200.200.200.0/24 [20/0] via 2.2.2.1, 04:53:15 Ò GM3 LAN Network
GM1#
GM1#show ip route vrf KS-VRF1
33.0.0.0/32 is subnetted, 1 subnets
C 33.1.1.1 is directly connected, Loopback1000
37.0.0.0/24 is subnetted, 1 subnets
B 37.1.1.0 [20/0] via 38.1.1.2, 04:54:23
38.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C 38.1.1.0/24 is directly connected, Port-channel1.100
L 38.1.1.1/32 is directly connected, Port-channel1.100
S 38.2.1.0/24 [1/0] via 38.1.1.2, Port-channel1.100
39.0.0.0/24 is subnetted, 1 subnets
B 39.1.1.0 [20/0] via 38.1.1.2, 04:54:23
44.0.0.0/32 is subnetted, 1 subnets
B 44.1.1.1 [20/0] via 38.1.1.2, 04:54:23
70.0.0.0/32 is subnetted, 3 subnets
B 70.0.0.1 [20/2] via 38.1.1.2, 04:54:23
B 70.0.0.2 [20/2] via 38.1.1.2, 04:54:23 =======Ò Key server IP
B 70.0.0.3 [20/2] via 38.1.1.2, 04:54:23
102.0.0.0/32 is subnetted, 1 subnets
B 102.102.102.102 [20/0] via 38.1.1.2, 04:54:23
150.1.0.0/24 is subnetted, 1 subnets
B 150.1.1.0 [20/0] via 38.1.1.2, 04:54:23
155.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
B 155.1.1.0/24 [20/0] via 38.1.1.2, 04:54:23
B 155.1.2.1/32 [20/0] via 38.1.1.2, 04:54:23
C 155.1.3.1/32 is directly connected, Loopback9
6.1.6.2 GM5
GM5#show ip route vrf cust1
Routing Table: cust1
2.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 2.2.2.0/24 is directly connected, Port-channel1.2
L 2.2.2.50/32 is directly connected, Port-channel1.2
3.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
B 3.3.2.0/24 [200/0] via 3.33.2.2, 04:49:48
C 3.33.2.0/24 is directly connected, Port-channel1.102
L 3.33.2.1/32 is directly connected, Port-channel1.102
10.0.0.0/24 is subnetted, 1 subnets
B 10.0.0.0 [200/0] via 3.33.2.2, 04:49:48
20.0.0.0/24 is subnetted, 1 subnets
B 20.1.1.0 [20/0] via 2.2.2.1, 04:49:48
29.0.0.0/24 is subnetted, 1 subnets
B 29.29.29.0 [20/0] via 2.2.2.1, 04:49:48
30.0.0.0/24 is subnetted, 2 subnets
B 30.1.1.0 [20/0] via 2.2.2.1, 04:49:48
B 30.1.2.0 [20/0] via 2.2.2.1, 04:49:48
33.0.0.0/32 is subnetted, 1 subnets
C 33.3.33.33 is directly connected, Loopback4000
40.0.0.0/32 is subnetted, 1 subnets
B 40.0.0.1 [200/0] via 3.33.2.2, 04:49:48
44.0.0.0/32 is subnetted, 1 subnets
B 44.44.44.44 [20/0] via 2.2.2.1, 04:49:48
77.0.0.0/32 is subnetted, 1 subnets
B 77.1.1.1 [20/0] via 2.2.2.1, 04:49:48
88.0.0.0/32 is subnetted, 1 subnets
C 88.1.1.1 is directly connected, Loopback100
100.0.0.0/24 is subnetted, 2 subnets
B 100.100.100.0 [200/0] via 3.33.2.2, 04:49:48 ===Ò CE1 LAN Network
B 100.101.100.0 [200/0] via 3.33.2.2, 04:49:48
B 200.200.200.0/24 [20/0] via 2.2.2.1, 04:49:48 ==Ò GM3 LAN Network
GM5#
GM5#
GM5#show ip route vrf KS-VRF1
33.0.0.0/32 is subnetted, 1 subnets
C 33.2.1.1 is directly connected, Loopback1000
37.0.0.0/24 is subnetted, 1 subnets
B 37.1.1.0 [20/0] via 38.1.1.2, 04:50:39
38.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 38.1.1.0/24 is directly connected, Port-channel1.100
L 38.1.1.50/32 is directly connected, Port-channel1.100
39.0.0.0/24 is subnetted, 1 subnets
B 39.1.1.0 [20/0] via 38.1.1.2, 04:50:39
44.0.0.0/32 is subnetted, 1 subnets
B 44.1.1.1 [20/0] via 38.1.1.2, 04:50:39
70.0.0.0/32 is subnetted, 3 subnets
B 70.0.0.1 [20/2] via 38.1.1.2, 04:50:39
B 70.0.0.2 [20/2] via 38.1.1.2, 04:50:39
B 70.0.0.3 [20/2] via 38.1.1.2, 04:50:39
102.0.0.0/32 is subnetted, 1 subnets
B 102.102.102.102 [20/0] via 38.1.1.2, 04:50:39
150.1.0.0/24 is subnetted, 1 subnets
B 150.1.1.0 [20/0] via 38.1.1.2, 04:50:39
155.1.0.0/16 is variably subnetted, 2 subnets, 2 masks
B 155.1.1.0/24 [20/0] via 38.1.1.2, 04:50:39
B 155.1.2.1/32 [20/0] via 38.1.1.2, 04:50:39
GM5#
GM5#
6.1.6.3 GM3
GM3#show ip route vrf cust1
2.0.0.0/24 is subnetted, 1 subnets
B 2.2.2.0 [20/0] via 20.1.1.2, 2d04h
3.0.0.0/24 is subnetted, 2 subnets
B 3.3.2.0 [20/0] via 20.1.1.2, 04:56:41
B 3.33.2.0 [20/0] via 20.1.1.2, 04:53:08
10.0.0.0/24 is subnetted, 1 subnets
B 10.0.0.0 [20/0] via 20.1.1.2, 04:56:41
20.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 20.1.1.0/24 is directly connected, Port-channel61.2
L 20.1.1.1/32 is directly connected, Port-channel61.2
29.0.0.0/24 is subnetted, 1 subnets
B 29.29.29.0 [200/0] via 30.1.2.2, 2d04h
30.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
B 30.1.1.0/24 [200/0] via 30.1.2.2, 2d04h
C 30.1.2.0/24 is directly connected, Port-channel4.102
L 30.1.2.1/32 is directly connected, Port-channel4.102
33.0.0.0/32 is subnetted, 2 subnets
B 33.3.33.33 [20/0] via 20.1.1.2, 04:49:47
B 33.33.33.33 [20/0] via 20.1.1.2, 04:56:41
40.0.0.0/32 is subnetted, 1 subnets
B 40.0.0.1 [20/0] via 20.1.1.2, 04:56:41
44.0.0.0/32 is subnetted, 1 subnets
C 44.44.44.44 is directly connected, Loopback4000
55.0.0.0/32 is subnetted, 1 subnets
B 55.1.1.1 [20/0] via 20.1.1.2, 04:56:41
77.0.0.0/32 is subnetted, 1 subnets
C 77.1.1.1 is directly connected, Loopback100
88.0.0.0/32 is subnetted, 1 subnets
B 88.1.1.1 [20/0] via 20.1.1.2, 04:49:47
100.0.0.0/24 is subnetted, 2 subnets
B 100.100.100.0 [20/0] via 20.1.1.2, 04:56:41 ===Ò GM1 LAN Network
B 100.101.100.0 [20/0] via 20.1.1.2, 04:56:41
B 200.200.200.0/24 [200/0] via 30.1.2.2, 2d04h ====Ò CE2 LAN Network
GM3#
GM3#show ip route vrf KS-VRF1
33.0.0.0/32 is subnetted, 2 subnets
B 33.1.1.1 [20/0] via 39.1.1.2, 05:00:05
B 33.2.1.1 [20/0] via 39.1.1.2, 04:53:02
37.0.0.0/24 is subnetted, 1 subnets
B 37.1.1.0 [20/0] via 39.1.1.2, 2d04h
38.0.0.0/24 is subnetted, 1 subnets
B 38.1.1.0 [20/0] via 39.1.1.2, 2d04h
39.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C 39.1.1.0/24 is directly connected, Port-channel61.100
L 39.1.1.1/32 is directly connected, Port-channel61.100
S 39.2.1.0/24 [1/0] via 39.1.1.2
44.0.0.0/32 is subnetted, 1 subnets
C 44.1.1.1 is directly connected, Loopback1000
70.0.0.0/32 is subnetted, 3 subnets
B 70.0.0.1 [20/2] via 39.1.1.2, 2d04h
B 70.0.0.2 [20/2] via 39.1.1.2, 2d04h
B 70.0.0.3 [20/2] via 39.1.1.2, 2d04h
102.0.0.0/32 is subnetted, 1 subnets
C 102.102.102.102 is directly connected, Loopback9
150.1.0.0/24 is subnetted, 1 subnets
B 150.1.1.0 [20/0] via 39.1.1.2, 2d04h
155.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
B 155.1.1.0/24 [20/0] via 39.1.1.2, 2d04h
B 155.1.2.1/32 [20/0] via 39.1.1.2, 2d04h
B 155.1.3.1/32 [20/0] via 39.1.1.2, 05:00:05
GM3#
GM1#show bfd neighbors
IPv4 Sessions
NeighAddr LD/RD RH/RS State Int
2.2.2.1 7/7 Up Up Po1.2
ÒP WAN neighbor
2.2.3.1 1/1 Up Up Po1.3
2.2.4.1 6/6 Up Up Po1.4
2.2.5.1 11/11 Up Up Po1.5
2.2.6.1 2/2 Up Up Po1.6
2.2.7.1 15/15 Up Up Po1.7
2.2.8.1 9/9 Up Up Po1.8
2.2.9.1 8/8 Up Up Po1.9
2.2.10.1 3/3 Up Up Po1.10
2.2.11.1 5/5 Up Up Po1.11
2.2.12.1 10/10 Up Up Po1.12
2.2.13.1 14/14 Up Up Po1.13
2.2.14.1 12/12 Up Up Po1.14
2.2.15.1 4/4 Up Up Po1.15
2.2.16.1 13/13 Up Up Po1.16
3.3.2.2 4099/131073 Up Up Po1.102
===Ò CE1 LAN neighbor
3.3.3.2 4097/65538 Up Up Po1.103
3.3.4.2 4098/65540 Up Up Po1.104
3.3.5.2 4108/65541 Up Up Po1.105
3.3.6.2 4100/65542 Up Up Po1.106
IPv4 Sessions
NeighAddr LD/RD RH/RS State Int
3.3.7.2 4101/65543 Up Up Po1.107
3.3.8.2 4111/65544 Up Up Po1.108
3.3.9.2 4102/65545 Up Up Po1.109
3.3.10.2 4104/65546 Up Up Po1.110
3.3.11.2 4107/65547 Up Up Po1.111
3.3.12.2 4110/65548 Up Up Po1.112
3.3.13.2 4106/65549 Up Up Po1.113
3.3.14.2 4103/65550 Up Up Po1.114
3.3.15.2 4109/65551 Up Up Po1.115
3.3.16.2 4105/65552 Up Up Po1.116
6.1.8.1 KS2 (primary)
ip access-list extended GETVPN_ACL
deny udp any any eq 20001
deny udp any any eq 20002
…
…
deny udp any any eq 20072
deny udp any any eq 20073
deny udp any any eq 20074
deny udp any any eq 20075
deny udp any eq 848 any eq 848
deny udp any any eq 3784
deny udp any any eq 3785
deny udp any any eq 4784
deny tcp any any eq 3784
deny tcp any any eq 3785
deny tcp any any eq 4784
deny tcp any any eq tacacs
deny tcp any eq tacacs any
deny tcp any any eq bgp
deny tcp any eq bgp any
deny ospf any any
deny eigrp any any
deny udp any any eq ntp
deny udp any eq ntp any
deny udp any any eq snmp
deny udp any eq snmp any
deny udp any any eq syslog
deny udp any eq isakmp any eq isakmp
deny esp any any
deny ahp any any
deny udp any eq syslog any
permit ip any any
!
NTP
ntp master 4
Crypto:
Crypto Configs
crypto isakmp policy 5
encr aes 256
hash sha256
authentication pre-share
group 5
lifetime 3600
!
crypto isakmp policy 10
encr aes
hash sha512
authentication pre-share
group 5
lifetime 1800
!
crypto isakmp policy 20
encr aes 256
group 2
lifetime 60
crypto isakmp key KS-cisco address 70.0.0.2
crypto isakmp key KS-cisco address 70.0.0.1
crypto isakmp key KS-cisco address 70.0.0.3
crypto isakmp key cisco123 address 61.1.1.1
crypto isakmp key cisco123 address 51.1.1.1
crypto isakmp key cisco123 address 33.1.1.1
crypto isakmp key cisco123 address 33.2.1.1
crypto isakmp key KEYSERVER address 172.168.10.0
crypto isakmp identity dn
crypto isakmp keepalive 10 periodic
!
crypto ipsec security-association lifetime seconds 900
!
crypto ipsec transform-set AES_SHA esp-aes 256 esp-sha256-hmac
mode tunnel
crypto ipsec transform-set AES_SHA_IPV6 esp-aes 256 esp-sha-hmac
mode tunnel
crypto ipsec transform-set TSET-V6-SuiteB esp-gcm 192
mode tunnel
!
crypto ipsec profile GLOBAL_SAT
set security-association lifetime seconds 1800
set transform-set AES_SHA
!
crypto gdoi group G1
identity number 1
server local
rekey algorithm aes 256
rekey lifetime seconds 300
rekey retransmit 10 periodic
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5 =========================Ò IPD3P
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.2
redundancy
local priority 200
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.3
!
crypto gdoi group G2
identity number 2
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.2
redundancy
local priority 200
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.3
!
…
…
crypto gdoi group G15
identity number 15
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.2
redundancy
local priority 200
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.3
!
6.1.8.2 KS1
ip access-list extended GETVPN_ACL
deny udp any any eq 20001
deny udp any any eq 20002
deny udp any any eq 20003
…
…
deny udp any any eq 20075
deny udp any eq 848 any eq 848
deny udp any any eq 3784
deny udp any any eq 3785
deny udp any any eq 4784
deny tcp any any eq 3784
deny tcp any any eq 3785
deny tcp any any eq 4784
deny tcp any any eq tacacs
deny tcp any eq tacacs any
deny tcp any any eq bgp
deny tcp any eq bgp any
deny ospf any any
deny eigrp any any
deny udp any any eq ntp
deny udp any eq ntp any
deny udp any any eq snmp
deny udp any eq snmp any
deny udp any any eq syslog
deny udp any eq isakmp any eq isakmp
deny esp any any
deny ahp any any
deny udp any eq syslog any
permit ip any any
!
NTP
ntp trusted-key 1
ntp source GigabitEthernet0/0
ntp server 9.44.49.11
Crypto:
crypto isakmp policy 5
encr aes 256
hash sha256
authentication pre-share
group 5
lifetime 3600
crypto isakmp key KS-cisco address 70.0.0.2
crypto isakmp key KS-cisco address 70.0.0.1
crypto isakmp key KS-cisco address 70.0.0.3
crypto isakmp key cisco123 address 38.1.1.1
crypto isakmp key cisco123 address 36.1.1.1
crypto isakmp key cisco123 address 35.1.1.2
crypto isakmp key cisco123 address 39.1.1.1
crypto isakmp key cisco123 address 61.1.1.1
crypto isakmp key cisco123 address 51.1.1.1
crypto isakmp key cisco123 address 44.1.1.1
crypto isakmp key cisco123 address 33.1.1.1
crypto isakmp key cisco123 address 33.2.1.1
crypto isakmp key cisco123 address ipv6 2003::1/64
crypto isakmp key cisco123 address ipv6 2004::1/64
crypto isakmp identity dn
crypto isakmp keepalive 10 periodic
!
crypto ipsec security-association lifetime seconds 900
!
crypto ipsec transform-set AES_SHA esp-aes 256 esp-sha256-hmac
mode tunnel
crypto ipsec transform-set AES_SHA_IPV6 esp-aes 256 esp-sha-hmac
mode tunnel
crypto ipsec transform-set TSET-V6-SuiteB esp-gcm 192
mode tunnel
!
crypto ipsec profile GLOBAL_SAT
set security-association lifetime seconds 1800
set transform-set AES_SHA
!
crypto gdoi group G1
identity number 1
server local
rekey algorithm aes 256
rekey retransmit 20 number 10
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.1
redundancy
local priority 150
peer address ipv4 70.0.0.2
peer address ipv4 70.0.0.3
!
crypto gdoi group G2
identity number 2
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.1
redundancy
local priority 150
peer address ipv4 70.0.0.2
peer address ipv4 70.0.0.3
!
…
…
crypto gdoi group G15
identity number 15
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.1
redundancy
local priority 150
peer address ipv4 70.0.0.3
peer address ipv4 70.0.0.2
!
6.1.8.3 KS3
ip access-list extended GETVPN_ACL
deny udp any any eq 20001
deny udp any any eq 20002
deny udp any any eq 20003
…
…
deny udp any any eq 20073
deny udp any any eq 20074
deny udp any any eq 20075
deny udp any eq 848 any eq 848
deny udp any any eq 3784
deny udp any any eq 3785
deny udp any any eq 4784
deny tcp any any eq 3784
deny tcp any any eq 3785
deny tcp any any eq 4784
deny tcp any any eq tacacs
deny tcp any eq tacacs any
deny tcp any any eq bgp
deny tcp any eq bgp any
deny ospf any any
deny eigrp any any
deny udp any any eq ntp
deny udp any eq ntp any
deny udp any any eq snmp
deny udp any eq snmp any
deny udp any any eq syslog
deny udp any eq isakmp any eq isakmp
deny esp any any
deny ahp any any
deny udp any eq syslog any
permit ip any any
!
!
NTP
ntp trusted-key 1
ntp source GigabitEthernet0/0
ntp server 9.44.49.11
Crypto Config
crypto isakmp policy 5
encr aes 256
hash sha256
authentication pre-share
group 5
lifetime 3600
!
crypto isakmp policy 10
encr aes
hash sha512
authentication pre-share
group 5
lifetime 1800
crypto isakmp key KS-cisco address 70.0.0.2
crypto isakmp key KS-cisco address 70.0.0.1
crypto isakmp key KS-cisco address 70.0.0.3
crypto isakmp key cisco123 address 38.1.1.1
crypto isakmp key cisco123 address 36.1.1.1
crypto isakmp key cisco123 address 35.1.1.2
crypto isakmp key cisco123 address 39.1.1.1
crypto isakmp key cisco123 address 61.1.1.1
crypto isakmp key cisco123 address 51.1.1.1
crypto isakmp key cisco123 address 91.1.1.1
crypto isakmp key cisco123 address 44.1.1.1
crypto isakmp key cisco123 address 33.1.1.1
crypto isakmp key cisco123 address 33.2.1.1
crypto isakmp key KEYSERVER address 172.168.10.0
crypto isakmp key cisco123 address ipv6 2003::1/64
crypto isakmp key cisco123 address ipv6 2004::1/64
crypto isakmp identity dn
crypto isakmp keepalive 10 periodic
crypto isakmp profile ikev1
vrf INTERNET
keyring key
match identity address 0.0.0.0
!
crypto ipsec security-association lifetime seconds 900
crypto ipsec security-association idle-time 86400
!
crypto ipsec transform-set AES_SHA esp-aes 256 esp-sha256-hmac
mode tunnel
crypto ipsec transform-set AES_SHA_IPV6 esp-aes 256 esp-sha-hmac
mode tunnel
crypto ipsec transform-set tunnel esp-3des esp-sha-hmac
mode tunnel
crypto ipsec transform-set transport esp-3des esp-sha-hmac
mode transport
crypto ipsec transform-set TSET-V6-SuiteB esp-gcm 192
mode tunnel
!
crypto ipsec profile GLOBAL_SAT
set security-association lifetime seconds 1800
set transform-set AES_SHA
!
crypto ipsec profile ipsec
set transform-set transport
!
crypto gdoi group G1
identity number 1
server local
rekey algorithm aes 256
rekey retransmit 20 number 10
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.3
redundancy
local priority 100
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.2
!
crypto gdoi group G2
identity number 2
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.3
redundancy
local priority 100
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.2
!
…
…
crypto gdoi group G15
identity number 15
server local
rekey algorithm aes 256
rekey authentication mypubkey rsa KeyServer.cisco.com
rekey transport unicast
sa d3p window sec 5
sa ipsec 1
profile GLOBAL_SAT
match address ipv4 GETVPN_ACL
no replay
no tag
address ipv4 70.0.0.3
redundancy
local priority 100
peer address ipv4 70.0.0.1
peer address ipv4 70.0.0.2
!
6.1.8.4 COOP verification
KS2:
show crypto gdoi group G1 ks coop
Crypto Gdoi Group Name :G1
Group handle: 2147483680, Local Key Server handle: 2147483760
Local Address: 70.0.0.2
Local Priority: 200
Local KS Role: Primary , Local KS Status: Alive
Local KS version: 1.0.18
Primary Timers:
Primary Refresh Policy Time: 20
Remaining Time: 14
Per-user timer remaining time: 0
Antireplay Sequence Number: 91090
Peer Sessions:
Session 1:
Server handle: 2147483761
Peer Address: 70.0.0.1
Peer Version: 1.0.18
Peer Priority: 150
Peer KS Role: Secondary , Peer KS Status: Alive
Antireplay Sequence Number: 32414
IKE status: Established
Counters:
Ann msgs sent: 91091
Ann msgs sent with reply request: 4
Ann msgs recv: 32366
Ann msgs recv with reply request: 1
Packet sent drops: 8
Packet Recv drops: 0
Total bytes sent: 68294659
Total bytes recv: 15448651
Session 2:
Server handle: 2147483730
Peer Address: 70.0.0.3
Peer Version: 1.0.18
Peer Priority: 100
Peer KS Role: Secondary , Peer KS Status: Alive
Antireplay Sequence Number: 46785
IKE status: Established
Counters:
Ann msgs sent: 91084
Ann msgs sent with reply request: 4
Ann msgs recv: 32176
Ann msgs recv with reply request: 1
Packet sent drops: 8
Packet Recv drops: 0
Total bytes sent: 68289368
Total bytes recv: 14162644
KS3:
KS3#show crypto gdoi group G1 ks coop
Crypto Gdoi Group Name :G1
Group handle: 2147483652, Local Key Server handle: 2147483656
Local Address: 70.0.0.3
Local Priority: 100
Local KS Role: Secondary , Local KS Status: Alive
Local KS version: 1.0.18
Secondary Timers:
Sec Primary Periodic Time: 30
Remaining Time: 18, Retries: 0
Invalid ANN PST recvd: 0
New GM Temporary Blocking Enforced?: No
Per-user timer remaining time: 0
Antireplay Sequence Number: 46786
Peer Sessions:
Session 1:
Server handle: 2147483657
Peer Address: 70.0.0.1
Peer Version: 1.0.18
Peer Priority: 150
Peer KS Role: Secondary , Peer KS Status: Unknown
Antireplay Sequence Number: 32414
IKE status: Established
Counters:
Ann msgs sent: 46572
Ann msgs sent with reply request: 3
Ann msgs recv: 46850
Ann msgs recv with reply request: 4
Packet sent drops: 207
Packet Recv drops: 0
Total bytes sent: 20398967
Total bytes recv: 22254388
Session 2:
Server handle: 2147483662
Peer Address: 70.0.0.2
Peer Version: 1.0.18
Peer Priority: 200
Peer KS Role: Primary , Peer KS Status: Alive
Antireplay Sequence Number: 91092
IKE status: Established
Counters:
Ann msgs sent: 46661
Ann msgs sent with reply request: 1
Ann msgs recv: 145049
Ann msgs recv with reply request: 4
Packet sent drops: 120
Packet Recv drops: 0
Total bytes sent: 20438374
Total bytes recv: 112111617
KS3#
KS1::
show crypto gdoi group G1 ks coop
Crypto Gdoi Group Name :G1
Group handle: 2147483650, Local Key Server handle: 2147483650
Local Address: 70.0.0.1
Local Priority: 150
Local KS Role: Secondary , Local KS Status: Alive
Local KS version: 1.0.18
Secondary Timers:
Sec Primary Periodic Time: 30
Remaining Time: 25, Retries: 0
Invalid ANN PST recvd: 0
New GM Temporary Blocking Enforced?: No
Per-user timer remaining time: 0
Antireplay Sequence Number: 32415
Peer Sessions:
Session 1:
Server handle: 2147483651
Peer Address: 70.0.0.2
Peer Version: 1.0.18
Peer Priority: 200
Peer KS Role: Primary , Peer KS Status: Alive
Antireplay Sequence Number: 91097
IKE status: Established
Counters:
Ann msgs sent: 32403
Ann msgs sent with reply request: 1
Ann msgs recv: 91163
Ann msgs recv with reply request: 4
Packet sent drops: 9
Packet Recv drops: 7
Total bytes sent: 15464015
Total bytes recv: 68351722
Session 2:
Server handle: 2147483652
Peer Address: 70.0.0.3
Peer Version: 1.0.18
Peer Priority: 100
Peer KS Role: Secondary , Peer KS Status: Unknown
Antireplay Sequence Number: 46785
IKE status: Established
Counters:
Ann msgs sent: 32267
Ann msgs sent with reply request: 3
Ann msgs recv: 32211
Ann msgs recv with reply request: 3
Packet sent drops: 144
Packet Recv drops: 0
Total bytes sent: 15405176
Total bytes recv: 14177552
6.1.9.1 GM1
interface Loopback1000
vrf forwarding KS-VRF1
ip address 33.1.1.1 255.255.255.255
end
NTP configs::
ntp trusted-key 1
ntp source GigabitEthernet0
ntp server vrf Mgmt-intf 9.44.49.11
track 1 stub-object ===Ò Routing awareness
!
track 2 stub-object
!
track 3 stub-object
!
…
…
track 15 stub-object
!
crypto keyring GETVPN_KS vrf KS-VRF1
pre-shared-key address 70.0.0.1 key cisco123
pre-shared-key address 70.0.0.2 key cisco123
pre-shared-key address 70.0.0.3 key cisco123
!
crypto isakmp policy 5
encr aes 256
hash sha256
authentication pre-share
group 5
lifetime 3600
!
crypto isakmp policy 10
encr aes 256
hash sha256
authentication pre-share
group 14
lifetime 3600
crypto isakmp key cisco123 address 33.1.1.1
crypto isakmp profile IKE_KS
keyring GETVPN_KS
match identity address 70.0.0.1 255.255.255.255 KS-VRF1
match identity address 70.0.0.2 255.255.255.255 KS-VRF1
match identity address 70.0.0.3 255.255.255.255 KS-VRF1
local-address Loopback1000
!
!
crypto ipsec transform-set TS1 esp-aes 256 esp-sha512-hmac
mode tunnel
!
crypto gdoi group G1
identity number 1
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 1
client registration interface Loopback1000
!
crypto gdoi group G2
identity number 2
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 2
client registration interface Loopback1000
!
crypto gdoi group G3
identity number 3
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 3
client registration interface Loopback1000
!
…
…
crypto gdoi group G15
identity number 15
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 15
client registration interface Loopback1000
crypto map G1 1 gdoi
set group G1
!
crypto map G2 2 gdoi
set group G2
match address pingdeny
!
crypto map G3 3 gdoi
set group G3
match address pingdeny
!
…
…
crypto map G15 15 gdoi
set group G15
match address pingdeny
!
ip access-list extended pingdeny
deny icmp any any
interface Port-channel1.2
description "WAN facing port channel"
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.2 255.255.255.0
no ip redirects
bfd template max
crypto map G1
!
interface Port-channel1.3
description "WAN facing port channel"
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.2 255.255.255.0
no ip redirects
bfd template max
crypto map G2
!
…
…
interface Port-channel1.16
description "WAN facing port channel"
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.2 255.255.255.0
no ip redirects
bfd template max
crypto map G15
!
6.1.9.2 GM5
interface Loopback1000
vrf forwarding KS-VRF1
ip address 33.2.1.1 255.255.255.255
end
NTP configs::
ntp trusted-key 1
ntp source GigabitEthernet0
ntp server vrf Mgmt-intf 9.44.49.11
track 1 stub-object ==============Ò Routing awarness
!
track 2 stub-object
!
track 3 stub-object
!
…
…
!
track 15 stub-object
!
track 99 stub-object
!
track 100 stub-object
!
!
!
crypto keyring GETVPN_KS vrf KS-VRF1
pre-shared-key address 70.0.0.1 key cisco123
pre-shared-key address 70.0.0.2 key cisco123
pre-shared-key address 70.0.0.3 key cisco123
!
!
!
crypto isakmp policy 5
encr aes 256
hash sha256
authentication pre-share
group 5
lifetime 3600
!
crypto isakmp policy 10
encr aes 256
hash sha256
authentication pre-share
group 14
lifetime 3600
crypto isakmp profile IKE_KS
keyring GETVPN_KS
match identity address 70.0.0.1 255.255.255.255 KS-VRF1
match identity address 70.0.0.2 255.255.255.255 KS-VRF1
match identity address 70.0.0.3 255.255.255.255 KS-VRF1
local-address Loopback1000
!
crypto ipsec transform-set TS1 esp-aes 256 esp-sha512-hmac
mode tunnel
crypto ipsec fragmentation after-encryption
!
!
crypto gdoi group G1
identity number 1
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 1
client registration interface Loopback1000
!
crypto gdoi group G2
identity number 2
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 2
client registration interface Loopback1000
!
…
…
crypto gdoi group G15
identity number 15
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 15
client registration interface Loopback1000
!
crypto map G1 1 gdoi
set group G1
!
crypto map G2 2 gdoi
set group G2
match address pingdeny
!
…
…
crypto map G15 15 gdoi
set group G15
match address pingdeny
!
ip access-list extended pingdeny
deny icmp any any
interface Port-channel1.2
encapsulation dot1Q 2
vrf forwarding cust1
ip address 2.2.2.50 255.255.255.0
no ip redirects
bfd template max
crypto map G1
!
interface Port-channel1.3
encapsulation dot1Q 3
vrf forwarding cust2
ip address 2.2.3.50 255.255.255.0
no ip redirects
bfd template max
crypto map G2
!
…
…
interface Port-channel1.16
encapsulation dot1Q 16
vrf forwarding cust15
ip address 2.2.16.50 255.255.255.0
no ip redirects
bfd template max
crypto map G15
!
6.1.10.1 GM1
GM1#show crypto gdoi group G1
Group Name : G1
Group Identity : 1
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 16
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G1:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G1_temp_acl
Group member : 33.1.1.1 vrf: KS-VRF1
Local addr/port : 33.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 1658 sec
Succeeded registration: 1
Attempted registration: 5
Last rekey from : 70.0.0.2
Last rekey seq num : 0
Unicast rekey received: 16
…
Rekey Acks sents : 16
ACL Downloaded From KS 70.0.0.2:
access-list deny udp any any port = 20001
access-list deny udp any any port = 20002
…
…
access-list deny udp any any port = 20075
access-list deny udp any port = 848 any port = 848
access-list deny udp any any port = 3784
access-list deny udp any any port = 3785
access-list deny udp any any port = 4784
access-list deny tcp any any port = 3784
access-list deny tcp any any port = 3785
access-list deny tcp any any port = 4784
access-list deny tcp any any port = 49
access-list deny tcp any port = 49 any
access-list deny tcp any any port = 179
access-list deny tcp any port = 179 any
access-list deny ospf any any
access-list deny eigrp any any
access-list deny udp any any port = 123
access-list deny udp any port = 123 any
access-list deny udp any any port = 161
access-list deny udp any port = 161 any
access-list deny udp any any port = 514
access-list deny udp any port = 500 any port = 500
access-list deny esp any any
access-list deny ahp any any
access-list deny udp any port = 514 any
access-list permit ip any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 1754
Encrypt Algorithm : AES
Key Size : 256
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1296
TEK POLICY for the current KS-Policy ACEs Downloaded:
Port-channel1.2:
IPsec SA:
spi: 0xB9760A99(3111520921)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): (5125)
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
KGS POLICY:
REG_GM: local_addr 33.1.1.1 (client_reg enabled)
P2P POLICY:
REG_GM: local_addr 33.1.1.1 (client_reg enabled)
GM1#
GM1#
GM1#
GM1#show crypto gdoi group G2
Group Name : G2
Group Identity : 2
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 15
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G2:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G2_temp_acl
Group member : 33.1.1.1 vrf: KS-VRF1
Local addr/port : 33.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 1325 sec
Succeeded registration: 1
Attempted registration: 5
Last rekey from : 70.0.0.2
…
…
GM1#
GM1#show crypto gdoi group G15
Group Name : G15
Group Identity : 15
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 15
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G15:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G15_temp_acl
Group member : 33.1.1.1 vrf: KS-VRF1
Local addr/port : 33.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 1277 sec
…
…
ACL Downloaded From KS 70.0.0.2:
access-list deny udp any any port = 20001
access-list deny udp any any port = 20002
…
access-list deny udp any any port = 20075
access-list deny udp any port = 848 any port = 848
access-list deny udp any any port = 3784
access-list deny udp any any port = 3785
access-list deny udp any any port = 4784
access-list deny tcp any any port = 3784
access-list deny tcp any any port = 3785
access-list deny tcp any any port = 4784
access-list deny tcp any any port = 49
access-list deny tcp any port = 49 any
access-list deny tcp any any port = 179
access-list deny tcp any port = 179 any
access-list deny ospf any any
access-list deny eigrp any any
access-list deny udp any any port = 123
access-list deny udp any port = 123 any
access-list deny udp any any port = 161
access-list deny udp any port = 161 any
access-list deny udp any any port = 514
access-list deny udp any port = 500 any port = 500
access-list deny esp any any
access-list deny ahp any any
access-list deny udp any port = 514 any
access-list permit ip any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 1390
Encrypt Algorithm : AES
Key Size : 256
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1296
TEK POLICY for the current KS-Policy ACEs Downloaded:
Port-channel1.16:
IPsec SA:
spi: 0x9F92FC02(2677210114)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): (6301)
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
IPsec SA:
spi: 0x62E94F35(1659457333)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): expired
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
KGS POLICY:
REG_GM: local_addr 33.1.1.1 (client_reg enabled)
P2P POLICY:
REG_GM: local_addr 33.1.1.1 (client_reg enabled)
GM1#
6.1.10.2 GM5
GM5#show crypto gdoi group G1
Group Name : G1
Group Identity : 1
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 15
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G1:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G1_temp_acl
Group member : 33.2.1.1 vrf: KS-VRF1
Local addr/port : 33.2.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 1284 sec
Succeeded registration: 1
Attempted registration: 5
Last rekey from : 70.0.0.2
ACL Downloaded From KS 70.0.0.2:
access-list deny udp any any port = 20001
access-list deny udp any any port = 20002
...
...
access-list deny udp any any port = 20075
access-list deny udp any port = 848 any port = 848
access-list deny udp any any port = 3784
access-list deny udp any any port = 3785
access-list deny udp any any port = 4784
access-list deny tcp any any port = 3784
access-list deny tcp any any port = 3785
access-list deny tcp any any port = 4784
access-list deny tcp any any port = 49
access-list deny tcp any port = 49 any
access-list deny tcp any any port = 179
access-list deny tcp any port = 179 any
access-list deny ospf any any
access-list deny eigrp any any
access-list deny udp any any port = 123
access-list deny udp any port = 123 any
access-list deny udp any any port = 161
access-list deny udp any port = 161 any
access-list deny udp any any port = 514
access-list deny udp any port = 500 any port = 500
access-list deny esp any any
access-list deny ahp any any
access-list deny udp any port = 514 any
access-list permit ip any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 1401
Encrypt Algorithm : AES
Key Size : 256
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1296
TEK POLICY for the current KS-Policy ACEs Downloaded:
Port-channel1.2:
IPsec SA:
spi: 0xB9760A99(3111520921)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): (4772)
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
KGS POLICY:
REG_GM: local_addr 33.2.1.1 (client_reg enabled)
P2P POLICY:
REG_GM: local_addr 33.2.1.1 (client_reg enabled)
GM5#show crypto gdoi group G2
Group Name : G2
Group Identity : 2
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 14
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G2:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G2_temp_acl
Group member : 33.2.1.1 vrf: KS-VRF1
Local addr/port : 33.2.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 977 sec
Succeeded registration: 1
Attempted registration: 5
...
...
...
GM5#show crypto gdoi group G15
Group Name : G15
Group Identity : 15
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 14
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G15:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G15_temp_acl
Group member : 33.2.1.1 vrf: KS-VRF1
Local addr/port : 33.2.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
ACL Downloaded From KS 70.0.0.2:
access-list deny udp any any port = 20001
access-list deny udp any any port = 20002
...
...
...
access-list deny udp any any port = 20075
access-list deny udp any port = 848 any port = 848
access-list deny udp any any port = 3784
access-list deny udp any any port = 3785
access-list deny udp any any port = 4784
access-list deny tcp any any port = 3784
access-list deny tcp any any port = 3785
access-list deny tcp any any port = 4784
access-list deny tcp any any port = 49
access-list deny tcp any port = 49 any
access-list deny tcp any any port = 179
access-list deny tcp any port = 179 any
access-list deny ospf any any
access-list deny eigrp any any
access-list deny udp any any port = 123
access-list deny udp any port = 123 any
access-list deny udp any any port = 161
access-list deny udp any port = 161 any
access-list deny udp any any port = 514
access-list deny udp any port = 500 any port = 500
access-list deny esp any any
access-list deny ahp any any
access-list deny udp any port = 514 any
access-list permit ip any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 1036
Encrypt Algorithm : AES
Key Size : 256
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1296
TEK POLICY for the current KS-Policy ACEs Downloaded:
Port-channel1.16:
IPsec SA:
spi: 0x9F92FC02(2677210114)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): (5947)
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
IPsec SA:
spi: 0x62E94F35(1659457333)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): expired
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
KGS POLICY:
REG_GM: local_addr 33.2.1.1 (client_reg enabled)
P2P POLICY:
REG_GM: local_addr 33.2.1.1 (client_reg enabled)
GM5#
6.1.10.3 GM3
GM3#show crypto gdoi group G1
Group Name : G1
Group Identity : 1
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 98
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G1:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G1_temp_acl
Group member : 44.1.1.1 vrf: KS-VRF1
Local addr/port : 44.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
Re-registers in : 1645 sec
Succeeded registration: 1
Attempted registration: 2
Last rekey from : 70.0.0.2
...
...
ACL Downloaded From KS 70.0.0.2:
access-list deny udp any any port = 20001
access-list deny udp any any port = 20002
...
...
...
access-list deny udp any any port = 20075
access-list deny udp any port = 848 any port = 848
access-list deny udp any any port = 3784
access-list deny udp any any port = 3785
access-list deny udp any any port = 4784
access-list deny tcp any any port = 3784
access-list deny tcp any any port = 3785
access-list deny tcp any any port = 4784
access-list deny tcp any any port = 49
access-list deny tcp any port = 49 any
access-list deny tcp any any port = 179
access-list deny tcp any port = 179 any
access-list deny ospf any any
access-list deny eigrp any any
access-list deny udp any any port = 123
access-list deny udp any port = 123 any
access-list deny udp any any port = 161
access-list deny udp any port = 161 any
access-list deny udp any any port = 514
access-list deny udp any port = 500 any port = 500
access-list deny esp any any
access-list deny ahp any any
access-list deny udp any port = 514 any
access-list permit ip any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 1769
Encrypt Algorithm : AES
Key Size : 256
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1296
TEK POLICY for the current KS-Policy ACEs Downloaded:
Port-channel61.2:
IPsec SA:
spi: 0xB9760A99(3111520921)
transform: esp-256-aes esp-sha256-hmac
sa timing:remaining key lifetime (sec): (5140)
Anti-Replay(Time Based - IPD3P) : 5000 msec interval
tag method : disabled
alg key size: 32 (bytes)
sig key size: 32 (bytes)
encaps: ENCAPS_TUNNEL
KGS POLICY:
REG_GM: local_addr 44.1.1.1 (client_reg enabled)
P2P POLICY:
REG_GM: local_addr 44.1.1.1 (client_reg enabled)
GM3#
GM3#show crypto gdoi group G2
Group Name : G2
Group Identity : 2
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 93
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G2:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G2_temp_acl
Group member : 44.1.1.1 vrf: KS-VRF1
Local addr/port : 44.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
GM3#show crypto gdoi group G15
Group Name : G15
Group Identity : 15
Group Type : GDOI (ISAKMP)
Crypto Path : ipv4
Key Management Path : ipv4
Rekeys received : 93
IPSec SA Direction : Both
Group Server list : 70.0.0.1
70.0.0.2
70.0.0.3
Group Member Information For Group G15:
IPSec SA Direction : Both
ACL Received From KS : gdoi_group_G15_temp_acl
Group member : 44.1.1.1 vrf: KS-VRF1
Local addr/port : 44.1.1.1/848
Remote addr/port : 70.0.0.2/848
fvrf/ivrf : KS-VRF1/KS-VRF1
Version : 1.0.19
Registration status : Registered
Registered with : 70.0.0.2
GM3#
1.1.1.1 KS2 (primary KS)
KS2#show crypto gdoi group G1 ks members sum
Group Member Information :
Group Name: G1, ID: 1, Group Members: 3
Key Server ID: 70.0.0.1, GMDB state: REDUNDANT, Group Members: 2
Member ID Version Rekey sent Rekey Ack missed
33.1.1.1 1.0.19 1 0
44.1.1.1 1.0.19 1 0
Key Server ID: 70.0.0.2, GMDB state: LOCAL, Group Members: 1
Member ID Version Rekey sent Rekey Ack missed
33.2.1.1 1.0.19 1 0
Key Server ID: 70.0.0.3, GMDB state: REDUNDANT, Group Members: 0
Member ID Version Rekey sent Rekey Ack missed
6.1.11.1 GM1
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 447471977 #pkts decrypt : 447457861
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 447689152 #pkts decrypt : 447675068
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#
GM1#show crypto gdoi group G2 gm dataplane counters
Data-plane statistics for group G2:
#pkts encrypt : 447993245 #pkts decrypt : 448012760
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#
GM1#show crypto gdoi group G2 gm dataplane counters
Data-plane statistics for group G2:
#pkts encrypt : 448258929 #pkts decrypt : 448278362
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto gdoi group G15 gm dataplane counters
Data-plane statistics for group G15:
#pkts encrypt : 448527654 #pkts decrypt : 448544920
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#show crypto gdoi group G15 gm dataplane counters
Data-plane statistics for group G15:
#pkts encrypt : 448720889 #pkts decrypt : 448738156
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto ipsec sa
interface: Port-channel1.2
Crypto map tag: G1, local addr 2.2.2.2
protected vrf: cust1
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
Group: G1
current_peer 0.0.0.0 port 848
PERMIT, flags={}
#pkts encaps: 19479584, #pkts encrypt: 19479584, #pkts digest: 19479584
#pkts decaps: 223785339, #pkts decrypt: 223785339, #pkts verify: 223785339
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 2.2.2.2, remote crypto endpt.: 0.0.0.0
plaintext mtu 1426, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1.2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
inbound ah sas:
inbound pcp sas:
outbound esp sas:
outbound ah sas:
outbound pcp sas:
6.1.11.2 GM3
GM3#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 511680523 #pkts decrypt : 1272871959
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 1300
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#
GM3#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 511868235 #pkts decrypt : 1273059668
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 1300
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#
GM3#show crypto gdoi group G2 gm dataplane counters
Data-plane statistics for group G2:
#pkts encrypt : 513326680 #pkts decrypt : 1266191750
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 1248
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#show crypto gdoi group G2 gm dataplane counters
Data-plane statistics for group G2:
#pkts encrypt : 513467453 #pkts decrypt : 1266332522
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 1248
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#
GM3#show crypto gdoi group G15 gm dataplane counters
Data-plane statistics for group G15:
#pkts encrypt : 515316987 #pkts decrypt : 1244909158
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 340
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#
GM3#show crypto gdoi group G15 gm dataplane counters
Data-plane statistics for group G15:
#pkts encrypt : 515481233 #pkts decrypt : 1245096844
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 340
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM3#
GM3#show crypto ipsec sa
interface: Port-channel61.2
Crypto map tag: G1, local addr 20.1.1.1
protected vrf: cust1
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
Group: G1
current_peer 0.0.0.0 port 848
PERMIT, flags={}
#pkts encaps: 2999351582, #pkts encrypt: 2999351582, #pkts digest: 2999351582
#pkts decaps: 343392618, #pkts decrypt: 343392618, #pkts verify: 343392618
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 20.1.1.1, remote crypto endpt.: 0.0.0.0
plaintext mtu 1426, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel61.2
current outbound spi: 0xFCA515D9(4238677465)
PFS (Y/N): N, DH group: none
inbound esp sas:
spi: 0xFCA515D9(4238677465)
transform: esp-256-aes esp-sha256-hmac ,
in use settings ={Tunnel, }
conn id: 3325, flow_id: HW:1325, sibling_flags FFFFFFFF80000048, crypto map: G1
sa timing: remaining key lifetime (sec): 1188
Kilobyte Volume Rekey has been disabled
IV size: 16 bytes
replay detection support: Y replay window size: 5000
Status: ACTIVE(ACTIVE)
inbound ah sas:
inbound pcp sas:
outbound esp sas:
spi: 0xFCA515D9(4238677465)
transform: esp-256-aes esp-sha256-hmac ,
in use settings ={Tunnel, }
conn id: 3326, flow_id: HW:1326, sibling_flags FFFFFFFF80000048, crypto map: G1
sa timing: remaining key lifetime (sec): 1188
Kilobyte Volume Rekey has been disabled
IV size: 16 bytes
replay detection support: Y replay window size: 5000
Status: ACTIVE(ACTIVE)
outbound ah sas:
outbound pcp sas:
6.1.12.1 GM1
track 1 stub-object
!
track 2 stub-object
!
track 3 stub-object
!
…
…
track 15 stub-object
!
crypto gdoi group G1
identity number 1
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 1
client registration interface Loopback1000
!
crypto gdoi group G2
identity number 2
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 2
client registration interface Loopback1000
!
crypto gdoi group G3
identity number 3
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 3
client registration interface Loopback1000
!
…
…
crypto gdoi group G15
identity number 15
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 15
client registration interface Loopback1000
ip access-list standard PERMIT-OUT
permit any
ip access-list standard LAN_PEERMIT
permit any
route-map site-med permit 20
match ip address PERMIT-OUT
match track 1
set metric 200
!
route-map site-med1 permit 21
match ip address PERMIT-OUT
match track 2
set metric 200
!
…
…
route-map site-med14 permit 34
match ip address PERMIT-OUT
match track 15
set metric 200
!
route-map LAN101_1 permit 101
match ip address LAN_PEERMIT
match track 1
!
route-map LAN101_2 permit 102
match ip address LAN_PEERMIT
match track 2
!
…
…
route-map LAN101_15 permit 115
match ip address LAN_PEERMIT
match track 15
!
router bgp 200
bgp router-id 100.100.100.100
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
neighbor 38.1.1.2 remote-as 100
neighbor 38.1.1.2 ebgp-multihop 255
neighbor 38.1.1.2 fall-over bfd
neighbor 38.1.1.2 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.1 remote-as 100
neighbor 2.2.2.1 fall-over bfd
neighbor 2.2.2.1 activate
neighbor 2.2.2.1 route-map site-med out
neighbor 3.3.2.2 remote-as 200
neighbor 3.3.2.2 fall-over bfd
neighbor 3.3.2.2 activate
neighbor 3.3.2.2 route-map LAN101_1 out
exit-address-family
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.1 remote-as 100
neighbor 2.2.3.1 fall-over bfd
neighbor 2.2.3.1 activate
neighbor 2.2.3.1 route-map site-med1 out
neighbor 3.3.3.2 remote-as 200
neighbor 3.3.3.2 fall-over bfd
neighbor 3.3.3.2 activate
neighbor 3.3.3.2 route-map LAN101_2 out
exit-address-family
!
address-family ipv4 vrf cust3
redistribute connected
neighbor 2.2.4.1 remote-as 100
neighbor 2.2.4.1 fall-over bfd
neighbor 2.2.4.1 activate
neighbor 2.2.4.1 route-map site-med2 out
neighbor 3.3.4.2 remote-as 200
neighbor 3.3.4.2 fall-over bfd
neighbor 3.3.4.2 activate
neighbor 3.3.4.2 route-map LAN101_3 out
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.1 remote-as 100
neighbor 2.2.16.1 fall-over bfd
neighbor 2.2.16.1 activate
neighbor 2.2.16.1 route-map site-med14 out
neighbor 3.3.16.2 remote-as 200
neighbor 3.3.16.2 fall-over bfd
neighbor 3.3.16.2 activate
neighbor 3.3.16.2 route-map LAN101_15 out
exit-address-family
!
6.1.12.2 GM5
track 1 stub-object
!
track 2 stub-object
!
track 3 stub-object
…
…
track 15 stub-object
!
track 99 stub-object
!
track 100 stub-object
!
!
crypto gdoi group G1
identity number 1
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 1
client registration interface Loopback1000
!
crypto gdoi group G2
identity number 2
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 2
client registration interface Loopback1000
!
…
…
crypto gdoi group G15
identity number 15
server address ipv4 70.0.0.1
server address ipv4 70.0.0.2
server address ipv4 70.0.0.3
client status active-sa track 15
client registration interface Loopback1000
!
ip access-list standard PERMIT-OUT
permit any
ip access-list standard LAN_PEERMIT
permit any
6.1.12.3 P
Route Map
route-map gm1 permit 10
set local-preference 200 =========Ò GM1 is preferred over GM5
!
route-map gm5 permit 10
set local-preference 100
!
router bgp 100
bgp router-id 99.99.99.99
bgp log-neighbor-changes
!
address-family ipv4 vrf KS-VRF1
redistribute connected
redistribute ospf 100
neighbor 38.1.1.1 remote-as 200
neighbor 38.1.1.1 ebgp-multihop 255
neighbor 38.1.1.1 fall-over bfd
neighbor 38.1.1.1 activate
neighbor 38.1.1.50 remote-as 200
neighbor 38.1.1.50 ebgp-multihop 255
neighbor 38.1.1.50 fall-over bfd
neighbor 38.1.1.50 activate
neighbor 39.1.1.1 remote-as 300
neighbor 39.1.1.1 ebgp-multihop 255
neighbor 39.1.1.1 activate
neighbor 39.1.1.3 remote-as 400
neighbor 39.1.1.3 fall-over bfd
neighbor 39.1.1.3 activate
exit-address-family
!
address-family ipv4 vrf cust1
redistribute connected
neighbor 2.2.2.2 remote-as 200
neighbor 2.2.2.2 fall-over bfd
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 route-map gm1 out
neighbor 2.2.2.50 remote-as 200
neighbor 2.2.2.50 fall-over bfd
neighbor 2.2.2.50 activate
neighbor 2.2.2.50 route-map gm5 out
neighbor 20.1.1.1 remote-as 300
neighbor 20.1.1.1 ebgp-multihop 255
neighbor 20.1.1.1 fall-over bfd multi-hop
neighbor 20.1.1.1 activate
exit-address-family
!
!
address-family ipv4 vrf cust2
redistribute connected
neighbor 2.2.3.2 remote-as 200
neighbor 2.2.3.2 fall-over bfd
neighbor 2.2.3.2 activate
neighbor 2.2.3.2 route-map gm1 out
neighbor 2.2.3.50 remote-as 200
neighbor 2.2.3.50 fall-over bfd
neighbor 2.2.3.50 activate
neighbor 2.2.3.50 route-map gm5 out
neighbor 20.1.2.1 remote-as 300
neighbor 20.1.2.1 ebgp-multihop 255
neighbor 20.1.2.1 fall-over bfd multi-hop
neighbor 20.1.2.1 activate
exit-address-family
!
…
…
address-family ipv4 vrf cust15
redistribute connected
neighbor 2.2.16.2 remote-as 200
neighbor 2.2.16.2 fall-over bfd
neighbor 2.2.16.2 activate
neighbor 2.2.16.2 route-map gm1 out
neighbor 2.2.16.50 remote-as 200
neighbor 2.2.16.50 fall-over bfd
neighbor 2.2.16.50 activate
neighbor 2.2.16.50 route-map gm5 out
neighbor 20.1.15.1 remote-as 300
neighbor 20.1.15.1 ebgp-multihop 255
neighbor 20.1.15.1 fall-over bfd multi-hop
neighbor 20.1.15.1 activate
exit-address-family
!
1.1.1.2 CE1
RP/0/RSP0/CPU0:BNG613#show run route-policy gm_route_policy
Thu Jul 19 11:16:35.621 UTC
route-policy gm_route_policy
set local-preference 200 ========Ò GM1 is preferred over GM5
pass
end-policy
!
RP/0/RSP0/CPU0:BNG613#show run route-policy pass_all
Thu Jul 19 11:16:41.164 UTC
route-policy pass_all
pass
end-policy
!
RP/0/RSP0/CPU0:BNG613#show run route-policy gm5_route_policy
Thu Jul 19 11:16:53.929 UTC
route-policy gm5_route_policy
set local-preference 100
pass
end-policy
!
RP/0/RSP0/CPU0:BNG613#
router bgp 200
nsr
timers bgp 10 30
bgp router-id 10.23.90.245
bgp bestpath med missing-as-worst
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
address-family ipv6 unicast
!
address-family vpnv6 unicast
!
af-group cust1_gm_af_group address-family ipv4 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
af-group cust1_ipv6_gm_af_group address-family ipv6 unicast
maximum-prefix 8000 75 warning-only
soft-reconfiguration inbound
!
session-group cust1_gm_session_group
remote-as 200
timers 10 30
!
session-group cust1_ipv6_gm_session_group
remote-as 200
timers 10 30
!
neighbor-group cust1_neighbor_group
use session-group cust1_gm_session_group
bfd fast-detect
bfd multiplier 3
bfd minimum-interval 100
address-family ipv4 unicast
use af-group cust1_gm_af_group
!
!
!
vrf cust1
rd 200:30
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.2.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
vrf cust2
rd 200:32
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.3.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
…
…
vrf cust15
rd 200:45
address-family ipv4 unicast
redistribute connected
!
address-family ipv6 unicast
redistribute connected
!
neighbor 3.3.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm_route_policy in
route-policy pass_all out
!
!
neighbor 3.33.16.1
use neighbor-group cust1_neighbor_group
address-family ipv4 unicast
route-policy gm5_route_policy in
route-policy pass_all out
!
!
!
!
6.1.12.4 Port channel shut on GM1 Ò Traffic switches from GM1 to GM5
Traffic will switch from GM1 to GM5 once GM1 port channel is shut
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1288399671 #pkts decrypt : 197801913
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1288440525 #pkts decrypt : 198318123
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 0 #pkts decrypt : 0
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1(config)#int port-channel1
GM1(config-if)#shut
GM1(config-if)#
Jul 17 12:31:42.802: TenGigabitEthernet0/1/6 taken out of port-channel1
Jul 17 12:31:42.802: TenGigabitEthernet0/1/7 taken out of port-channel1
Jul 17 12:31:42.807: %LINK-5-CHANGED: Interface Port-channel1, changed state to administratively down
Jul 17 12:31:42.818: %BGP-5-NBR_RESET: Neighbor 2.2.2.1 reset (Route to peer lost)
...
...
Jul 17 12:31:42.832: %BGP-5-ADJCHANGE: neighbor 2.2.2.1 vpn vrf cust1 Down Route to peer lost
Jul 17 12:31:42.832: %BGP_SESSION-5-ADJCHANGE: neighbor 2.2.2.1 IPv4 Unicast vpn vrf cust1 topology base removed from session Route to peer lost
Jul 17 12:31:42.833: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: bfd_session_destroyed,
Jul 17 12:31:43.807: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down
Jul 17 12:31:55.894: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel2, changed state to down
Jul 17 12:31:55.894: %OSPF-5-ADJCHG: Process 100, Nbr 8.8.8.1 on Tunnel2 from FULL to DOWN, Neighbor Down: Interface down or detached
GM5#
GM5#
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 308547 #pkts decrypt : 255234
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM5#
GM5#
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 333814 #pkts decrypt : 278247
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM5#
GM1#clear crypto gdoi dataplane counters
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 0 #pkts decrypt : 0
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 0 #pkts decrypt : 0
==Ò Not incrementing on GM1
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
6.1.12.5 Verify routing awareness by blocking the reachability between GM1 and KS
Initially traffic goes through GM1.
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 0 #pkts decrypt : 0
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM5#
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1297953685 #pkts decrypt : 298828014
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1297982278 #pkts decrypt : 299156505
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
Stop reachability between KS and GM
Log on GM1:
Jul 17 16:55:53.223: %CRYPTO-5-GM_REGSTER: Start registration to KS 70.0.0.3 for group G11 using address 33.1.1.1 fvrf KS-VRF1 ivrf KS-VRF1
Jul 17 16:59:52.022: %TRACK-6-STATE: 1 stub-object Up -> Down Track on GM1 goes down
Jul 17 17:20:29.222: %TRACK-6-STATE: 11 stub-object Up -> Down
Jul 17 17:20:30.226: %TRACK-6-STATE: 12 stub-object Up -> Down
Jul 17 17:20:31.233: %TRACK-6-STATE: 13 stub-object Up -> Down
Jul 17 17:20:32.236: %TRACK-6-STATE: 14 stub-object Up -> Down
Jul 17 17:20:32.236: %TRACK-6-STATE: 15 stub-object Up -> Down
Jul 17 17:20:36.096: %TRACK-6-STATE: 2 stub-object Up -> Down
Jul 17 17:20:38.096: %TRACK-6-STATE: 3 stub-object Up -> Down
Jul 17 17:20:40.096: %TRACK-6-STATE: 4 stub-object Up -> Down
Jul 17 17:20:42.120: %TRACK-6-STATE: 5 stub-object Up -> Down
Jul 17 17:20:43.131: %TRACK-6-STATE: 6 stub-object Up -> Down
Jul 17 17:20:48.160: %TRACK-6-STATE: 7 stub-object Up -> Down
Jul 17 17:20:48.190: %TRACK-6-STATE: 8 stub-object Up -> Down
Jul 17 17:20:48.190: %TRACK-6-STATE: 9 stub-object Up -> Down
Jul 17 17:20:48.224: %TRACK-6-STATE: 10 stub-object Up -> Down
Traffic switches to GM5
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 25781110 #pkts decrypt : 296119418
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM5#
GM5#
GM5#
GM5#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 25797936 #pkts decrypt : 296312664
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM5#
Counters not incrementing on GM1
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1298280499 #pkts decrypt : 302582009
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
GM1#show crypto gdoi group G1 gm dataplane counters
Data-plane statistics for group G1:
#pkts encrypt : 1298280499 #pkts decrypt : 302582009
#pkts tagged (send) : 0 #pkts untagged (rcv) : 0
#pkts no sa (send) : 0 #pkts invalid sa (rcv) : 0
#pkts encaps fail (send) : 0 #pkts decap fail (rcv) : 0
#pkts invalid prot (rcv) : 0 #pkts verify fail (rcv) : 0
#pkts not tagged (send) : 0 #pkts not untagged (rcv) : 0
#pkts internal err (send): 0 #pkts internal err (rcv) : 0
GM1#
6.1.13 Site-to-site IPsec tunnel and verification
6.1.13.1 GM1
ip access-list extended S2S_IPSEC
deny tcp any any eq bgp
deny tcp any eq bgp any
deny esp any any
deny udp any any range 3784 3785
deny udp any range 3784 3785 any
deny icmp any any
vrf definition LAN1
rd 300:1
!
address-family ipv4
route-target export 300:1
route-target import 300:1
exit-address-family
!
crypto ipsec profile PROFILE-S2S
set transform-set TS1
set pfs group2
set isakmp-profile S22_PROFILE
!
crypto keyring S2S_IPSEC vrf cust1
pre-shared-key address 77.1.1.1 key cust1
pre-shared-key address 55.1.1.1 key cust1
pre-shared-key address 88.1.1.1 key cust1
crypto map G1 1 gdoi
set group G1
match address S2S_IPSEC
!
interface Loopback100
description S2S IPSEC Tunnel Sourec
vrf forwarding cust1
ip address 55.1.1.1 255.255.255.255
!
interface Loopback10000
vrf forwarding LAN1
ip address 8.8.8.2 255.255.255.255
ip ospf network point-to-point
ip ospf 100 area 0
!
interface Tunnel2
description #### Tunnel 2 to GM3
vrf forwarding LAN1
ip address 192.168.150.2 255.255.255.0
ip ospf network point-to-point
ip ospf 100 area 0
load-interval 30
bfd interval 100 min_rx 100 multiplier 3
tunnel source 55.1.1.1
tunnel mode ipsec ipv4
tunnel destination 77.1.1.1
tunnel vrf cust1
tunnel protection ipsec profile PROFILE-S2S
!
router ospf 100 vrf LAN1
redistribute connected subnets
network 8.8.8.2 0.0.0.0 area 0
network 192.168.150.0 0.0.0.255 area 0
!
6.1.13.2 GM5
ip access-list extended S2S_IPSEC
deny tcp any any eq bgp
deny tcp any eq bgp any
deny esp any any
deny udp any any range 3784 3785
deny udp any range 3784 3785 any
deny icmp any any
vrf definition LAN1
rd 300:1
!
address-family ipv4
route-target export 300:1
route-target import 300:1
exit-address-family
!
crypto isakmp profile S22_PROFILE
keyring S2S_IPSEC
match identity address 77.1.1.1 255.255.255.255 cust1
match identity address 55.1.1.1 255.255.255.255 cust1
match identity address 88.1.1.1 255.255.255.255 cust1
!
crypto ipsec profile PROFILE-S2S
set transform-set TS1
set pfs group2
set isakmp-profile S22_PROFILE
!
!
crypto map G1 1 gdoi
set group G1
match address S2S_IPSEC
!
!
!
interface Loopback100
description S2S IPSEC Tunnel Sourec
vrf forwarding cust1
ip address 88.1.1.1 255.255.255.255
!
interface Loopback10000
vrf forwarding LAN1
ip address 8.8.8.3 255.255.255.255
ip ospf network point-to-point
ip ospf 100 area 0
!
interface Tunnel2
description #### Tunnel 2 to GM1
vrf forwarding LAN1
ip address 192.168.150.3 255.255.255.0
ip ospf network point-to-point
ip ospf 100 area 0
load-interval 30
tunnel source 88.1.1.1
tunnel mode ipsec ipv4
tunnel destination 77.1.1.1
tunnel vrf cust1
tunnel protection ipsec profile PROFILE-S2S
!
router ospf 100 vrf LAN1
redistribute connected subnets
network 8.8.8.2 0.0.0.0 area 0
network 192.168.150.0 0.0.0.255 area 0
!
6.1.13.3 GM3
vrf definition LAN1
rd 300:1
!
address-family ipv4
route-target export 300:1
route-target import 300:1
exit-address-family
!
ip access-list extended S2S_IPSEC
deny tcp any any eq bgp
deny tcp any eq bgp any
deny esp any any
deny udp any any range 3784 3785
deny udp any range 3784 3785 any
deny icmp any any
crypto keyring S2S_IPSEC vrf cust1
pre-shared-key address 77.1.1.1 key cust1
pre-shared-key address 55.1.1.1 key cust1
pre-shared-key address 88.1.1.1 key cust1
!
crypto isakmp profile S22_PROFILE
keyring S2S_IPSEC
match identity address 77.1.1.1 255.255.255.255 cust1
match identity address 55.1.1.1 255.255.255.255 cust1
match identity address 88.1.1.1 255.255.255.255 cust1
!
crypto ipsec profile PROFILE-S2S
set transform-set TS1
set pfs group2
set isakmp-profile S22_PROFILE
!
crypto map G1 1 gdoi
set group G1
match address S2S_IPSEC
!
interface Loopback100
description S2S IPSEC Tunnel Sourec
vrf forwarding cust1
ip address 77.1.1.1 255.255.255.255
!
interface Loopback10000
vrf forwarding LAN1
ip address 8.8.8.1 255.255.255.255
ip ospf network point-to-point
ip ospf 100 area 0
!
interface Tunnel2
description "Tunnel to GM1"
vrf forwarding LAN1
ip address 192.168.150.1 255.255.255.0
ip ospf network point-to-point
ip ospf 100 area 0
load-interval 30
bfd interval 100 min_rx 100 multiplier 3
tunnel source 77.1.1.1
tunnel mode ipsec ipv4
tunnel destination 55.1.1.1
tunnel vrf cust1
tunnel protection ipsec profile PROFILE-S2S
!
router ospf 100 vrf LAN1
redistribute connected subnets
network 8.8.8.1 0.0.0.0 area 0
network 192.168.150.0 0.0.0.255 area 0
!
!
6.1.13.4 Tunnel route verification
6.1.13.4.1 GM1
GM1#show ip route vrf LAN1
8.0.0.0/32 is subnetted, 2 subnets
O 8.8.8.1 [110/1001] via 192.168.150.1, 04:53:07, Tunnel2 Ò Tunnel
C 8.8.8.2 is directly connected, Loopback10000
192.168.30.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.30.0/24 is directly connected, TenGigabitEthernet0/1/0.1000
L 192.168.30.1/32 is directly connected, TenGigabitEthernet0/1/0.1000
192.168.150.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.150.0/24 is directly connected, Tunnel2
L 192.168.150.2/32 is directly connected, Tunnel2
GM1#
GM1#show crypto ipsec sa
interface: Tunnel2
Crypto map tag: Tunnel2-head-0, local addr 55.1.1.1
protected vrf: LAN1
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 77.1.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 7049, #pkts encrypt: 7049, #pkts digest: 7049
#pkts decaps: 6801, #pkts decrypt: 6801, #pkts verify: 6801
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 55.1.1.1, remote crypto endpt.: 77.1.1.1
plaintext mtu 1422, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1.2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
inbound ah sas:
inbound pcp sas:
outbound esp sas:
outbound ah sas:
outbound pcp sas:
GM1#show crypto ipsec sa
interface: Tunnel2
Crypto map tag: Tunnel2-head-0, local addr 55.1.1.1
protected vrf: LAN1
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 77.1.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 8443, #pkts encrypt: 8443, #pkts digest: 7049
#pkts decaps: 8765, #pkts decrypt: 8765, #pkts verify: 6801
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 55.1.1.1, remote crypto endpt.: 77.1.1.1
plaintext mtu 1422, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1.2
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none
inbound esp sas:
inbound ah sas:
inbound pcp sas:
outbound esp sas:
outbound ah sas:
outbound pcp sas:
6.1.13.4.2 GM3
GM3#show ip route vrf LAN1
Gateway of last resort is not set
8.0.0.0/32 is subnetted, 2 subnets
C 8.8.8.1 is directly connected, Loopback10000
O 8.8.8.2 [110/1001] via 192.168.150.2, 04:57:01, Tunnel2
O E2 192.168.30.0/24 [110/20] via 192.168.150.2, 04:57:01, Tunnel2
192.168.150.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.150.0/24 is directly connected, Tunnel2
L 192.168.150.1/32 is directly connected, Tunnel2
GM3#
ICMP ECHO
GM1:
interface Loopback4000
vrf forwarding cust1
ip address 33.33.33.33 255.255.255.255
end
ip sla 21
icmp-echo 44.44.44.44 source-ip 33.33.33.33
vrf cust1
tag cust1_tag
threshold 50
timeout 100
frequency 1
ip sla schedule 21 life forever start-time now
GM3::
interface Loopback4000
vrf forwarding cust1
ip address 44.44.44.44 255.255.255.255
end
ip sla responder
Verification
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 21
Latest RTT: 23 milliseconds
Latest operation start time: 17:27:11 IST Tue Jul 17 2018
Latest operation return code: OK
Number of successes: 208
Number of failures: 0
Operation time to live: Forever
GM1#
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 21
Latest RTT: 23 milliseconds
Latest operation start time: 17:27:11 IST Tue Jul 17 2018
Latest operation return code: OK
Number of successes: 216
Number of failures: 0
Operation time to live: Forever
GM1#
UDP JITTER
GM1:
interface Loopback4000
vrf forwarding cust1
ip address 33.33.33.33 255.255.255.255
end
ip sla 20
udp-jitter 44.44.44.44 16384 source-ip 33.33.33.33 source-port 16384
vrf cust1
tag cust1_tag
threshold 50
timeout 100
frequency 1
ip sla schedule 20 life forever start-time now
GM1#
GM3:
ip sla responder
interface Loopback4000
vrf forwarding cust1
ip address 44.44.44.44 255.255.255.255
end
Verification:
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 20
Type of operation: udp-jitter
Latest RTT: 1 milliseconds
Latest operation start time: 18:26:22 IST Thu Jul 19 2018
Latest operation return code: OK
RTT Values:
Number Of RTT: 10 RTT Min/Avg/Max: 1/1/1 milliseconds
Latency one-way time:
Number of Latency one-way Samples: 0
Source to Destination Latency one way Min/Avg/Max: 0/0/0 milliseconds
Destination to Source Latency one way Min/Avg/Max: 0/0/0 milliseconds
Jitter Time:
Number of SD Jitter Samples: 9
Number of DS Jitter Samples: 9
Source to Destination Jitter Min/Avg/Max: 0/1/1 milliseconds
Destination to Source Jitter Min/Avg/Max: 0/1/1 milliseconds
Over Threshold:
Number Of RTT Over Threshold: 0 (0%)
Packet Loss Values:
Loss Source to Destination: 0
Source to Destination Loss Periods Number: 0
Source to Destination Loss Period Length Min/Max: 0/0
Source to Destination Inter Loss Period Length Min/Max: 0/0
Loss Destination to Source: 0
Destination to Source Loss Periods Number: 0
Destination to Source Loss Period Length Min/Max: 0/0
Destination to Source Inter Loss Period Length Min/Max: 0/0
Out Of Sequence: 0 Tail Drop: 0
Packet Late Arrival: 0 Packet Skipped: 0
Voice Score Values:
Calculated Planning Impairment Factor (ICPIF): 0
Mean Opinion Score (MOS): 0
Number of successes: 194
Number of failures: 0
Operation time to live: Forever
GM1#
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 20
Type of operation: udp-jitter
Latest RTT: 1 milliseconds
Latest operation start time: 18:26:47 IST Thu Jul 19 2018
Latest operation return code: OK
RTT Values:
Number Of RTT: 10 RTT Min/Avg/Max: 1/1/1 milliseconds
Latency one-way time:
Number of Latency one-way Samples: 0
Source to Destination Latency one way Min/Avg/Max: 0/0/0 milliseconds
Destination to Source Latency one way Min/Avg/Max: 0/0/0 milliseconds
Jitter Time:
Number of SD Jitter Samples: 9
Number of DS Jitter Samples: 9
Source to Destination Jitter Min/Avg/Max: 0/1/1 milliseconds
Destination to Source Jitter Min/Avg/Max: 0/1/1 milliseconds
Over Threshold:
Number Of RTT Over Threshold: 0 (0%)
Packet Loss Values:
Loss Source to Destination: 0
Source to Destination Loss Periods Number: 0
Source to Destination Loss Period Length Min/Max: 0/0
Source to Destination Inter Loss Period Length Min/Max: 0/0
Loss Destination to Source: 0
Destination to Source Loss Periods Number: 0
Destination to Source Loss Period Length Min/Max: 0/0
Destination to Source Inter Loss Period Length Min/Max: 0/0
Out Of Sequence: 0 Tail Drop: 0
Packet Late Arrival: 0 Packet Skipped: 0
Voice Score Values:
Calculated Planning Impairment Factor (ICPIF): 0
Mean Opinion Score (MOS): 0
Number of successes: 219
Number of failures: 0
Operation time to live: Forever
GM1#
UDP ECHO
GM1
interface Loopback4000
vrf forwarding cust1
ip address 33.33.33.33 255.255.255.255
end
ip sla 30
udp-echo 44.44.44.44 2009
vrf cust1
tag cust1_tag
threshold 50
timeout 100
frequency 1
ip sla schedule 30 life forever start-time now
GM3:
ip sla responder
interface Loopback4000
vrf forwarding cust1
ip address 44.44.44.44 255.255.255.255
end
Verification:
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 30
Latest RTT: 1 milliseconds
Latest operation start time: 21:22:52 IST Thu Jul 19 2018
Latest operation return code: OK
Number of successes: 103
Number of failures: 1
Operation time to live: Forever
GM1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 30
Latest RTT: 1 milliseconds
Latest operation start time: 21:22:55 IST Thu Jul 19 2018
Latest operation return code: OK
Number of successes: 106
Number of failures: 1
Operation time to live: Forever
GM – Group member
KS – Key server
GETVPN – Group Encrypted Transport VPN
GDOI - Group Domain of Interpretation
BGP – Border Gateway Protocol
BFD - Bidirectional Forwarding Detection
IPsec – Internet Protocol Security
SNMP – Simple Network Management Protocol
IPD3P - IP-Delivery Delay Detection Protocol
PSK – Pre-shared key
NTP – Network Time Protocol
Feedback