此产品的文档集力求使用非歧视性语言。在本文档集中,非歧视性语言是指不隐含针对年龄、残障、性别、种族身份、族群身份、性取向、社会经济地位和交叉性的歧视的语言。由于产品软件的用户界面中使用的硬编码语言、基于 RFP 文档使用的语言或引用的第三方产品使用的语言,文档中可能无法确保完全使用非歧视性语言。 深入了解思科如何使用包容性语言。
思科采用人工翻译与机器翻译相结合的方式将此文档翻译成不同语言,希望全球的用户都能通过各自的语言得到支持性的内容。 请注意:即使是最好的机器翻译,其准确度也不及专业翻译人员的水平。 Cisco Systems, Inc. 对于翻译的准确性不承担任何责任,并建议您总是参考英文原始文档(已提供链接)。
本文档介绍如何在AWS环境中部署的Catalyst 8000V上启用和利用多TXQ以提高吞吐量性能。
多个队列的存在简化并加速了将传入和传出数据包映射到特定vCPU的过程。在Catalyst 8000V上使用多TXQ可实现跨已分配的可用数据平面内核的有效核心利用率,从而提高吞吐性能。本文简要概述多TXQ的工作方式、配置方式、显示自主和SD-WAN Catalyst 8000V部署的CLI配置示例,并查看故障排除命令以帮助发现性能瓶颈。
在17.18软件版本之前,进入Catalyst 8000V的数据包会分配到所有vCPU(数据包处理核心),而不管数据流如何。一旦PP完成数据包处理,流顺序将恢复为在接口上发送。
在将数据包放入传输队列(TxQ)之前,Catalyst 8000V为每个接口创建一个TxQ。因此,如果只有一个可用的出口接口,则多个流进入一个TxQ。
如果只有一个可用的接口,Catalyst 8000V将无法利用此多TxQ进程。这会导致吞吐量性能瓶颈,以及可用数据平面核心之间的负载分布不均。如果只有一个出口接口用于从C8000V实例传输数据,则只有一个TxQ可用于传输网络流量,并且可能导致数据包因单个队列填充较快而被丢弃。
如需参考,您可以在图1中找到AWS中部署的Catalyst 8000V的单一TxQ架构模型。
图 1:AWS中部署的Catalyst 8000V的单一TxQ架构模型。
AWS ENA提供多个传输队列(Multi-TxQ)以降低内部开销并提高可扩展性。多个队列的存在简化并加速了将传入和传出数据包映射到特定vCPU的过程。AWS和DPDK网络参考模型是基于流的,其中每个vCPU处理一个流并将来自该流的数据包传输到分配的传输队列(TxQ)。每个vCPU的RX/TX队列对基于流的模型是有效的。
由于Catalyst 8000V不是基于流的,因此语句“每个vCPU的RX/TX队列对”不适用于Catalyst 8000V。
在这种情况下,RX/TX队列不是每个vCPU,而是每个接口。RX/TX队列充当应用(Catalyst 8000V)和AWS基础设施/硬件之间的接口,用于发送数据/网络流量。AWS控制每个接口在每个实例上的可用速度和可用的RX/TX队列数量。
Catalyst 8000V必须具有多个接口才能创建多个TxQ。为了保持流顺序,多个流从接口流出(一旦Catalyst 8000V在此流程后启用多个TxQ),Catalyst 8000V将基于5元组的数据流散列以选择适当的TxQ。通过使用环回接口或辅助IP地址,用户可以使用连接到实例的同一物理NIC在Catalyst 8000V上创建多个接口。
在图2中,您可以找到在AWS中使用带Catalyst 8000V的Multi-TxQ架构处理数据包的方式。
图2:在AWS中部署的Catalyst 8000V的多TxQ架构模型。
如图2的步骤4所示,在TM处理结束时,在将数据包放入TxQ之前,TM查看数据包报头并提取5个元组(目标地址、源地址、协议、目标端口和源端口),然后将数据包散列到TxQ。
TxQ根据五个元组进行散列和模数以及实例支持的TxQ数选取。
相同实例系列类型的AWS EC2实例都支持不同数量的TXQ,具体取决于实例大小。从IOS® XE 17.7开始,C8000V开始支持多个TxQ。
从IOS® XE 17.7开始,C8000V在C5n.9xlarge上支持多个TxQ,最多可有8个TXQ。
从IOS® XE 17.9开始,C8000V支持C5n.18xlarge实例大小,该大小最多可以有12个TXQ(比C5n.9xlarge多50%)。
虽然IOS® XE 17.7支持Multi-TxQ,但强烈建议使用IOS® XE 17.9来提供软件生命周期和更高的吞吐量性能功能,同时支持12 TxQ。
要均匀散列所有可用TxQ之间的流量,当Catalyst 8000V终止IPsec/GRE隧道时,需要使用特殊IP地址。
有公共脚本可用于生成这些特殊的IP地址,用于配置负责终止这些隧道的Catalyst 8000V接口。本节介绍如何下载和使用脚本来设计所需的IP地址,以实现Multi-TxQ散列。
如果Catalyst 8000V处理明文流量(如TCP/UDP),则不需要特殊的IP编址方案。
可以在以下位置找到原始说明:https://github.com/CiscoDevNet/python-c8000v-aws-multitx-queues/
注意:对于运行17.18或更高版本的Catalyst 8000V,数据包的分配方式不同。因此,需要使用不同的散列算法。
您可以使用命令“python3 —version”检查计算机使用的Python版本。
user@computer ~ % python3 --version
Python 3.9.6
一旦Python版本经过验证且正在运行(版本等于或高于3.8.9),请安装最新版本的PIP。
user@computer ~ % curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2570k 100 2570k 0 0 6082k 0 --:--:-- --:--:-- --:--:-- 6135k
user@computer ~ % python3 get-pip.py
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Downloading pip-23.3.1-py3-none-any.whl.metadata (3.5 kB)
Downloading pip-23.3.1-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 7.4 MB/s eta 0:00:00
Installing collected packages: pip
WARNING: The scripts pip, pip3 and pip3.9 are installed in '/Users/name/Library/Python/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.3.1
[notice] A new release of pip is available: 21.2.4 -> 23.3.1
[notice] To update, run: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -m pip install --upgrade pip
安装先决条件后,创建虚拟环境并下载用于生成多TxQ的唯一IP地址方案的IP地址散列脚本。
命令摘要:
Python中的虚拟环境用于创建不影响其他项目或依赖项的隔离工作区。使用以下命令创建虚拟环境“c8kv-hash”:
user@computer Desktop % python3 -m venv c8kv-hash
在虚拟环境中导航到“c8kv-hash”文件夹(之前创建)。
user@computer Desktop % cd c8kv-hash
激活虚拟环境。
user@computer c8kv-hash % source bin/activate
克隆具有Multi-TxQ散列python脚本的存储库。
(c8kv-hash) user@computer c8kv-hash % git clone https://github.com/CiscoDevNet/python-c8000v-aws-multitx-queues.git
Cloning into 'c8kv-aws-pmd-hash'...
remote: Enumerating objects: 82, done.
remote: Counting objects: 100% (82/82), done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 82 (delta 34), reused 57 (delta 19), pack-reused 0
Receiving objects: 100% (82/82), 13.01 KiB | 2.60 MiB/s, done.
Resolving deltas: 100% (34/34), done.
复制存储库后,导航到“c8kv-aws-pmd-hash”文件夹。由于它位于创建的虚拟环境中,因此请安装最新版本的PIP。
(c8kv-hash) user@computer c8kv-hash % cd c8kv-aws-pmd-hash
(c8kv-hash) user@computer c8kv-aws-pmd-hash % python3 -m pip install --upgrade pip
Requirement already satisfied: pip in /Users/name/Desktop/c8kv-hash/lib/python3.9/site-packages (21.2.4)
Collecting pip
Downloading pip-23.3.1-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 2.7 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.2.4
Uninstalling pip-21.2.4:
Successfully uninstalled pip-21.2.4
Successfully installed pip-23.3.1
升级PIP后,在文件夹中安装requirements.txt文件中的依赖项。
(c8kv-hash) user@computer c8kv-aws-pmd-hash % pip install -r requirements.txt
Collecting crc32c==2.3 (from -r requirements.txt (line 1))
Downloading crc32c-2.3-cp39-cp39-macosx_11_0_arm64.whl (27 kB)
Installing collected packages: crc32c
Successfully installed crc32c-2.3
虚拟环境现已更新,可用于生成Multi-TxQ的IP地址方案。
注意:7.7和17.8散列脚本不久将弃用。强烈建议使用17.9哈希脚本
命令摘要:
'—old_crc 1'生成基于17.7和17.8版本的散列索引,采用modulo 8来匹配支持的PMD TXQ(请勿修改)
“ — dest_network”定义目标网络地址子网(根据网络IP地址方案进行修改)
“ — src_network”定义源网络地址子网(根据网络IP地址方案修改)
“ — unique_hash 1”生成一组唯一散列IP地址(8对,用于8 TXQ)。可以修改。
(c8kv-hash) user@computer c8kv-aws-pmd-hash % python3 c8kv_multitxq_hash.py --old_crc 1 --dest_network 192.168.1.0/24 --src_network 192.168.2.0/24 --unique_hash 1
Dest: Src: Prot dstport srcport Hash: Rev-hash:
192.168.1.0 192.168.2.0 2 5
192.168.1.0 192.168.2.1 2 7
192.168.1.0 192.168.2.2 2 1
192.168.1.0 192.168.2.3 2 3
192.168.1.0 192.168.2.4 2 5
192.168.1.0 192.168.2.5 2 7
192.168.1.0 192.168.2.6 2 1
192.168.1.0 192.168.2.7 2 3
192.168.1.0 192.168.2.8 2 5
192.168.1.0 192.168.2.9 2 7
192.168.1.0 192.168.2.10 2 1
.
. ### trimmed output ###
.
192.168.1.255 192.168.2.247 5 2
192.168.1.255 192.168.2.248 5 4
192.168.1.255 192.168.2.249 5 6
192.168.1.255 192.168.2.250 5 0
192.168.1.255 192.168.2.251 5 2
192.168.1.255 192.168.2.252 5 4
192.168.1.255 192.168.2.253 5 6
192.168.1.255 192.168.2.254 5 0
192.168.1.255 192.168.2.255 5 2
Unique hash:
------ Tunnels set 0 ---------
192.168.1.37<===>192.168.2.37<===>0
192.168.1.129<===>192.168.2.129<===>1
192.168.1.36<===>192.168.2.36<===>2
192.168.1.128<===>192.168.2.128<===>3
192.168.1.39<===>192.168.2.39<===>4
192.168.1.131<===>192.168.2.131<===>5
192.168.1.38<===>192.168.2.38<===>6
192.168.1.130<===>192.168.2.130<===>7
命令摘要:
请注意,在IOS® XE版本17.9及更高版本中,脚本使用不带 — old_crc选项的模12,与支持的PMD TXQ匹配。
“ — dest_network”定义目标网络地址子网(根据网络IP地址方案进行修改)
“ — src_network”定义源网络地址子网(根据网络IP地址方案修改)
“ — 端口udp”定义使用的协议。用户可以指定协议参数为“gre”、“tcp”或“udp”或任何十进制值(可选)
“ — src_port”定义使用的源端口(可选)
“ — dst_port”定义使用的目标端口(可选)
“ — unique_hash 1”生成一组唯一散列IP地址(12对12 TXQ)。可以修改。
(c8kv-hash) user@computer c8kv-aws-pmd-hash % python3 c8kv_multitxq_hash.py --dest_network 192.168.1.0/24 --src_network 192.168.2.0/24 --prot udp --src_port 12346 --dst_port 12346 --unique_hash 1
Dest: Src: Prot dstport srcport Hash: Rev-hash:
192.168.1.0 192.168.2.0 17 12346 12346 ==> 4 4 <-- Unique Hash Value
192.168.1.0 192.168.2.1 17 12346 12346 ==> 4 4
192.168.1.0 192.168.2.2 17 12346 12346 ==> 8 8 <-- Unique Hash Value
192.168.1.0 192.168.2.3 17 12346 12346 ==> 0 0 <-- Unique Hash Value
192.168.1.0 192.168.2.4 17 12346 12346 ==> 0 0
192.168.1.0 192.168.2.5 17 12346 12346 ==> 0 0
192.168.1.0 192.168.2.6 17 12346 12346 ==> 4 4
192.168.1.0 192.168.2.7 17 12346 12346 ==> 0 0
192.168.1.0 192.168.2.8 17 12346 12346 ==> 9 9 <-- Unique Hash Value
192.168.1.0 192.168.2.9 17 12346 12346 ==> 9 9
192.168.1.0 192.168.2.10 17 12346 12346 ==> 9 9
192.168.1.0 192.168.2.11 17 12346 12346 ==> 1 1 <-- Unique Hash Value
192.168.1.0 192.168.2.12 17 12346 12346 ==> 1 1
.
. ### trimmed output ###
.
192.168.1.255 192.168.2.250 17 12346 12346 ==> 1 1
192.168.1.255 192.168.2.251 17 12346 12346 ==> 1 1
192.168.1.255 192.168.2.252 17 12346 12346 ==> 9 9
192.168.1.255 192.168.2.253 17 12346 12346 ==> 1 1
192.168.1.255 192.168.2.254 17 12346 12346 ==> 5 5 <-- Unique Hash Value
192.168.1.255 192.168.2.255 17 12346 12346 ==> 9 9
Unique hash:
------ Tunnels set 0 ---------
192.168.1.38 <===> 192.168.2.38<===>0
192.168.1.37 <===> 192.168.2.37<===>1
192.168.1.53 <===> 192.168.2.53<===>2
192.168.1.39 <===> 192.168.2.39<===>3
192.168.1.48 <===> 192.168.2.48<===>4
192.168.1.58 <===> 192.168.2.58<===>5
192.168.1.42 <===> 192.168.2.42<===>6
192.168.1.46 <===> 192.168.2.46<===>7
192.168.1.40 <===> 192.168.2.40<===>8
192.168.1.43 <===> 192.168.2.43<===>9
192.168.1.36 <===> 192.168.2.36<===>10
192.168.1.56 <===> 192.168.2.56<===>11
图 3:使用环回接口的八个TxQ的示例拓扑。
这是“c8kv-ut”(图3)的CLI配置示例,使用上一节中计算出的散列IP地址(192.168.1.X)创建具有环回接口的8个IPsec隧道。
另一个路由器端点(c8kv-peer)上也将应用类似的配置,其余八个计算出的散列IP地址(192.168.2.X)。
ip cef load-sharing algorithm include-ports source destination 00ABC123
crypto keyring tunnel0
local-address Loopback0
pre-shared-key address 192.168.2.37 key cisco
crypto keyring tunnel1
local-address Loopback1
pre-shared-key address 192.168.2.129 key cisco
crypto keyring tunnel2
local-address Loopback2
pre-shared-key address 192.168.2.36 key cisco
crypto keyring tunnel3
local-address Loopback3
pre-shared-key address 192.168.2.128 key cisco
crypto keyring tunnel4
local-address Loopback4
pre-shared-key address 192.168.2.39 key cisco
crypto keyring tunnel5
local-address Loopback5
pre-shared-key address 192.168.2.131 key cisco
crypto keyring tunnel6
local-address Loopback6
pre-shared-key address 192.168.2.38 key cisco
crypto keyring tunnel7
local-address Loopback7
pre-shared-key address 192.168.2.130 key cisco
crypto isakmp policy 200
encryption aes
hash sha
authentication pre-share
group 16
lifetime 28800
crypto isakmp profile isakmp-tunnel0
keyring tunnel0
match identity address 0.0.0.0
local-address Loopback0
crypto isakmp profile isakmp-tunnel1
keyring tunnel1
match identity address 0.0.0.0
local-address Loopback1
crypto isakmp profile isakmp-tunnel2
keyring tunnel2
match identity address 0.0.0.0
local-address Loopback2
crypto isakmp profile isakmp-tunnel3
keyring tunnel3
match identity address 0.0.0.0
local-address Loopback3
crypto isakmp profile isakmp-tunnel4
keyring tunnel4
match identity address 0.0.0.0
local-address Loopback4
crypto isakmp profile isakmp-tunnel5
keyring tunnel5
match identity address 0.0.0.0
local-address Loopback5
crypto isakmp profile isakmp-tunnel6
keyring tunnel6
match identity address 0.0.0.0
local-address Loopback6
crypto isakmp profile isakmp-tunnel7
keyring tunnel7
match identity address 0.0.0.0
local-address Loopback7
crypto ipsec transform-set ipsec-prop-vpn-tunnel esp-gcm 256
mode tunnel
crypto ipsec df-bit clear
crypto ipsec profile ipsec-vpn-tunnel
set transform-set ipsec-prop-vpn-tunnel
set pfs group16
interface Loopback0
ip address 192.168.1.37 255.255.255.255
!
interface Loopback1
ip address 192.168.1.129 255.255.255.255
!
interface Loopback2
ip address 192.168.1.36 255.255.255.255
!
interface Loopback3
ip address 192.168.1.128 255.255.255.255
!
interface Loopback4
ip address 192.168.1.39 255.255.255.255
!
interface Loopback5
ip address 192.168.1.131 255.255.255.255
!
interface Loopback6
ip address 192.168.1.38 255.255.255.255
!
interface Loopback7
ip address 192.168.1.130 255.255.255.255
!
interface Tunnel0
ip address 10.101.100.101 255.255.255.0
load-interval 30
tunnel source Loopback0
tunnel mode ipsec ipv4
tunnel destination 192.168.2.37
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel1
ip address 10.101.101.101 255.255.255.0
load-interval 30
tunnel source Loopback1
tunnel mode ipsec ipv4
tunnel destination 192.168.2.129
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel2
ip address 10.101.102.101 255.255.255.0
load-interval 30
tunnel source Loopback2
tunnel mode ipsec ipv4
tunnel destination 192.168.2.36
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel3
ip address 10.101.103.101 255.255.255.0
load-interval 30
tunnel source Loopback3
tunnel mode ipsec ipv4
tunnel destination 192.168.2.128
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel4
ip address 10.101.104.101 255.255.255.0
load-interval 30
tunnel source Loopback4
tunnel mode ipsec ipv4
tunnel destination 192.168.2.39
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel5
ip address 10.101.105.101 255.255.255.0
load-interval 30
tunnel source Loopback5
tunnel mode ipsec ipv4
tunnel destination 192.168.2.131
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel6
ip address 10.101.106.101 255.255.255.0
load-interval 30
tunnel source Loopback6
tunnel mode ipsec ipv4
tunnel destination 192.168.2.38
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel7
ip address 10.101.107.101 255.255.255.0
load-interval 30
tunnel source Loopback7
tunnel mode ipsec ipv4
tunnel destination 192.168.2.130
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface GigabitEthernet2
mtu 9216
ip address dhcp
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
mtu 9216
ip address dhcp
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
! ### IP route from servers to c8kv-uut
ip route 10.1.0.0 255.255.0.0 GigabitEthernet2 10.0.1.10
! ### IP routes from c8kv-uut to clients on c8kv-peer side, routes are evenly distributed to all 8 TXQ’s
ip route 10.10.0.0 255.255.0.0 Tunnel0
ip route 10.10.0.0 255.255.0.0 Tunnel1
ip route 10.10.0.0 255.255.0.0 Tunnel2
ip route 10.10.0.0 255.255.0.0 Tunnel3
ip route 10.10.0.0 255.255.0.0 Tunnel4
ip route 10.10.0.0 255.255.0.0 Tunnel5
ip route 10.10.0.0 255.255.0.0 Tunnel6
ip route 10.10.0.0 255.255.0.0 Tunnel7
! ### IP route from c8kv-uut Loopback int tunnel endpoint to c8kv-peer Loopback int tunnel endpoints
ip route 192.168.2.0 255.255.255.0 GigabitEthernet3 10.0.2.30
图4.使用环回接口的12个TxQ的示例拓扑。
这是“c8kv-ut”(图4)的CLI配置示例,使用上一节中计算出的散列IP地址(192.168.1.X)创建具有环回接口的12个IPsec隧道。
另一个路由器端点(c8kv-peer)上也将应用类似的配置,其余八个计算出的散列IP地址(192.168.2.X)。
ip cef load-sharing algorithm include-ports source destination 00ABC123
crypto keyring tunnel0
local-address Loopback0
pre-shared-key address 192.168.2.38 key cisco
crypto keyring tunnel1
local-address Loopback1
pre-shared-key address 192.168.2.37 key cisco
crypto keyring tunnel2
local-address Loopback2
pre-shared-key address 192.168.2.53 key cisco
crypto keyring tunnel3
local-address Loopback3
pre-shared-key address 192.168.2.39 key cisco
crypto keyring tunnel4
local-address Loopback4
pre-shared-key address 192.168.2.48 key cisco
crypto keyring tunnel5
local-address Loopback5
pre-shared-key address 192.168.2.58 key cisco
crypto keyring tunnel6
local-address Loopback6
pre-shared-key address 192.168.2.42 key cisco
crypto keyring tunnel7
local-address Loopback7
pre-shared-key address 192.168.2.46 key cisco
crypto keyring tunnel8
local-address Loopback8
pre-shared-key address 192.168.2.40 key cisco
crypto keyring tunnel9
local-address Loopback9
pre-shared-key address 192.168.2.43 key cisco
crypto keyring tunnel10
local-address Loopback10
pre-shared-key address 192.168.2.36 key cisco
crypto keyring tunnel11
local-address Loopback11
pre-shared-key address 192.168.2.56 key cisco
crypto isakmp policy 200
encryption aes
hash sha
authentication pre-share
group 16
lifetime 28800
crypto isakmp profile isakmp-tunnel0
keyring tunnel0
match identity address 0.0.0.0
local-address Loopback0
crypto isakmp profile isakmp-tunnel1
keyring tunnel1
match identity address 0.0.0.0
local-address Loopback1
crypto isakmp profile isakmp-tunnel2
keyring tunnel2
match identity address 0.0.0.0
local-address Loopback2
crypto isakmp profile isakmp-tunnel3
keyring tunnel3
match identity address 0.0.0.0
local-address Loopback3
crypto isakmp profile isakmp-tunnel4
keyring tunnel4
match identity address 0.0.0.0
local-address Loopback4
crypto isakmp profile isakmp-tunnel5
keyring tunnel5
match identity address 0.0.0.0
local-address Loopback5
crypto isakmp profile isakmp-tunnel6
keyring tunnel6
match identity address 0.0.0.0
local-address Loopback6
crypto isakmp profile isakmp-tunnel7
keyring tunnel7
match identity address 0.0.0.0
local-address Loopback7
crypto isakmp profile isakmp-tunnel8
keyring tunnel8
match identity address 0.0.0.0
local-address Loopback8
crypto isakmp profile isakmp-tunnel9
keyring tunnel9
match identity address 0.0.0.0
local-address Loopback9
crypto isakmp profile isakmp-tunnel10
keyring tunnel10
match identity address 0.0.0.0
local-address Loopback10
crypto isakmp profile isakmp-tunnel11
keyring tunnel11
match identity address 0.0.0.0
local-address Loopback11
crypto ipsec transform-set ipsec-prop-vpn-tunnel esp-gcm 256
mode tunnel
crypto ipsec df-bit clear
crypto ipsec profile ipsec-vpn-tunnel
set transform-set ipsec-prop-vpn-tunnel
set pfs group16
interface Loopback0
ip address 192.168.1.38 255.255.255.255
!
interface Loopback1
ip address 192.168.1.37 255.255.255.255
!
interface Loopback2
ip address 192.168.1.53 255.255.255.255
!
interface Loopback3
ip address 192.168.1.39 255.255.255.255
!
interface Loopback4
ip address 192.168.1.48 255.255.255.255
!
interface Loopback5
ip address 192.168.1.58 255.255.255.255
!
interface Loopback6
ip address 192.168.1.42 255.255.255.255
!
interface Loopback7
ip address 192.168.1.46 255.255.255.255
!
interface Loopback8
ip address 192.168.1.40 255.255.255.255
!
interface Loopback9
ip address 192.168.1.43 255.255.255.255
!
interface Loopback10
ip address 192.168.1.36 255.255.255.255
!
interface Loopback11
ip address 192.168.1.56 255.255.255.255
interface Tunnel0
ip address 10.101.100.101 255.255.255.0
load-interval 30
tunnel source Loopback0
tunnel mode ipsec ipv4
tunnel destination 192.168.2.38
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel1
ip address 10.101.101.101 255.255.255.0
load-interval 30
tunnel source Loopback1
tunnel mode ipsec ipv4
tunnel destination 192.168.2.37
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel2
ip address 10.101.102.101 255.255.255.0
load-interval 30
tunnel source Loopback2
tunnel mode ipsec ipv4
tunnel destination 192.168.2.53
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel3
ip address 10.101.103.101 255.255.255.0
load-interval 30
tunnel source Loopback3
tunnel mode ipsec ipv4
tunnel destination 192.168.2.39
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel4
ip address 10.101.104.101 255.255.255.0
load-interval 30
tunnel source Loopback4
tunnel mode ipsec ipv4
tunnel destination 192.168.2.48
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel5
ip address 10.101.105.101 255.255.255.0
load-interval 30
tunnel source Loopback5
tunnel mode ipsec ipv4
tunnel destination 192.168.2.58
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel6
ip address 10.101.106.101 255.255.255.0
load-interval 30
tunnel source Loopback6
tunnel mode ipsec ipv4
tunnel destination 192.168.2.42
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel7
ip address 10.101.107.101 255.255.255.0
load-interval 30
tunnel source Loopback7
tunnel mode ipsec ipv4
tunnel destination 192.168.2.46
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel8
ip address 10.101.108.101 255.255.255.0
load-interval 30
tunnel source Loopback8
tunnel mode ipsec ipv4
tunnel destination 192.168.2.40
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel9
ip address 10.101.109.101 255.255.255.0
load-interval 30
tunnel source Loopback9
tunnel mode ipsec ipv4
tunnel destination 192.168.2.43
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel10
ip address 10.101.110.101 255.255.255.0
load-interval 30
tunnel source Loopback10
tunnel mode ipsec ipv4
tunnel destination 192.168.2.36
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel11
ip address 10.101.111.101 255.255.255.0
load-interval 30
tunnel source Loopback11
tunnel mode ipsec ipv4
tunnel destination 192.168.2.56
tunnel protection ipsec profile ipsec-vpn-tunnel
interface GigabitEthernet2
mtu 9216
ip address dhcp
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
mtu 9216
ip address dhcp
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
! ### IP route from c8kv-uut to local servers
ip route 10.1.0.0 255.255.0.0 GigabitEthernet2 10.0.1.10
! ### IP routes from c8kv-uut to clients on c8kv-peer side, routes are evenly distributed to all 12 TXQ’s
ip route 10.10.0.0 255.255.0.0 Tunnel0
ip route 10.10.0.0 255.255.0.0 Tunnel1
ip route 10.10.0.0 255.255.0.0 Tunnel2
ip route 10.10.0.0 255.255.0.0 Tunnel3
ip route 10.10.0.0 255.255.0.0 Tunnel4
ip route 10.10.0.0 255.255.0.0 Tunnel5
ip route 10.10.0.0 255.255.0.0 Tunnel6
ip route 10.10.0.0 255.255.0.0 Tunnel7
ip route 10.10.0.0 255.255.0.0 Tunnel8
ip route 10.10.0.0 255.255.0.0 Tunnel9
ip route 10.10.0.0 255.255.0.0 Tunnel10
ip route 10.10.0.0 255.255.0.0 Tunnel11
! ### IP route from c8kv-uut Loopback int tunnel endpoint to c8kv-peer Loopback int tunnel endpoints
ip route 192.168.2.0 255.255.255.0 GigabitEthernet3 10.0.2.30
图5.使用辅助IP地址的12个TxQ的示例拓扑。
如果环回地址不能在AWS环境中使用,则可以改为使用连接到ENI的辅助IP地址。
这是“c8kv-uut”的CLI配置示例(图5),使用计算出的散列IP地址(10.0.2.X)创建12个IPsec隧道,源为1个主IP地址+ 11个连接到GigabitEthernet3接口的辅助IP地址。另一个路由器端点(c8kv-peer)上也将应用类似的配置,其中剩余的12个计算出的散列IP地址(20.0.2.X)。
注意:在本示例中,我们使用第二个C8000V作为隧道终端,但也可以使用其他云网络终端(如TGW或DX)。
ip cef load-sharing algorithm include-ports source destination 00ABC123
crypto keyring tunnel0
local-address 10.0.2.20
pre-shared-key address 20.0.2.30 key cisco
crypto keyring tunnel1
local-address 10.0.2.21
pre-shared-key address 20.0.2.31 key cisco
crypto keyring tunnel2
local-address 10.0.2.22
pre-shared-key address 20.0.2.32 key cisco
crypto keyring tunnel3
local-address 10.0.2.23
pre-shared-key address 20.0.2.33 key cisco
crypto keyring tunnel4
local-address 10.0.2.24
pre-shared-key address 20.0.2.36 key cisco
crypto keyring tunnel5
local-address 10.0.2.25
pre-shared-key address 20.0.2.35 key cisco
crypto keyring tunnel6
local-address 10.0.2.26
pre-shared-key address 20.0.2.37 key cisco
crypto keyring tunnel7
local-address 10.0.2.27
pre-shared-key address 20.0.2.38 key cisco
crypto keyring tunnel8
local-address 10.0.2.28
pre-shared-key address 20.0.2.40 key cisco
crypto keyring tunnel9
local-address 10.0.2.29
pre-shared-key address 20.0.2.41 key cisco
crypto keyring tunnel10
local-address 10.0.2.30
pre-shared-key address 20.0.2.44 key cisco
crypto keyring tunnel11
local-address 10.0.2.31
pre-shared-key address 20.0.2.46 key cisco
crypto isakmp policy 200
encryption aes
hash sha
authentication pre-share
group 16
lifetime 28800
crypto isakmp profile isakmp-tunnel0
keyring tunnel0
match identity address 20.0.2.30 255.255.255.255
local-address 10.0.2.20
crypto isakmp profile isakmp-tunnel1
keyring tunnel1
match identity address 20.0.2.31 255.255.255.255
local-address 10.0.2.21
crypto isakmp profile isakmp-tunnel2
keyring tunnel2
match identity address 20.0.2.32 255.255.255.255
local-address 10.0.2.22
crypto isakmp profile isakmp-tunnel3
keyring tunnel3
match identity address 20.0.2.33 255.255.255.255
local-address 10.0.2.23
crypto isakmp profile isakmp-tunnel4
keyring tunnel4
match identity address 20.0.2.36 255.255.255.255
local-address 10.0.2.24
crypto isakmp profile isakmp-tunnel5
keyring tunnel5
match identity address 20.0.2.35 255.255.255.255
local-address 10.0.2.25
crypto isakmp profile isakmp-tunnel6
keyring tunnel6
match identity address 20.0.2.37 255.255.255.255
local-address 10.0.2.26
crypto isakmp profile isakmp-tunnel7
keyring tunnel7
match identity address 20.0.2.38 255.255.255.255
local-address 10.0.2.27
crypto isakmp profile isakmp-tunnel8
keyring tunnel8
match identity address 20.0.2.40 255.255.255.255
local-address 10.0.2.28
crypto isakmp profile isakmp-tunnel9
keyring tunnel9
match identity address 20.0.2.41 255.255.255.255
local-address 10.0.2.29
crypto isakmp profile isakmp-tunnel10
keyring tunnel10
match identity address 20.0.2.44 255.255.255.255
local-address 10.0.2.30
crypto isakmp profile isakmp-tunnel11
keyring tunnel11
match identity address 20.0.2.46 255.255.255.255
local-address 10.0.2.31
crypto ipsec transform-set ipsec-prop-vpn-tunnel esp-gcm 256
mode tunnel
crypto ipsec df-bit clear
crypto ipsec profile ipsec-vpn-tunnel
set transform-set ipsec-prop-vpn-tunnel
set pfs group16
interface Tunnel0
ip address 10.101.100.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.20
tunnel mode ipsec ipv4
tunnel destination 20.0.2.30
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel1
ip address 10.101.101.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.21
tunnel mode ipsec ipv4
tunnel destination 20.0.2.31
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel2
ip address 10.101.102.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.22
tunnel mode ipsec ipv4
tunnel destination 20.0.2.32
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel3
ip address 10.101.103.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.23
tunnel mode ipsec ipv4
tunnel destination 20.0.2.33
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel4
ip address 10.101.104.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.24
tunnel mode ipsec ipv4
tunnel destination 20.0.2.36
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel5
ip address 10.101.105.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.25
tunnel mode ipsec ipv4
tunnel destination 20.0.2.35
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel6
ip address 10.101.106.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.26
tunnel mode ipsec ipv4
tunnel destination 20.0.2.37
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel7
ip address 10.101.107.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.27
tunnel mode ipsec ipv4
tunnel destination 20.0.2.38
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel8
ip address 10.101.108.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.28
tunnel mode ipsec ipv4
tunnel destination 20.0.2.40
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel9
ip address 10.101.109.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.29
tunnel mode ipsec ipv4
tunnel destination 20.0.2.41
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel10
ip address 10.101.110.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.30
tunnel mode ipsec ipv4
tunnel destination 20.0.2.44
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface Tunnel11
ip address 10.101.111.101 255.255.255.0
load-interval 30
tunnel source 10.0.2.31
tunnel mode ipsec ipv4
tunnel destination 20.0.2.46
tunnel protection ipsec profile ipsec-vpn-tunnel
!
interface GigabitEthernet2
mtu 9216
ip address dhcp
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
mtu 9216
ip address 10.0.2.20 255.255.255.0
ip address 10.0.2.21 255.255.255.0 secondary
ip address 10.0.2.22 255.255.255.0 secondary
ip address 10.0.2.23 255.255.255.0 secondary
ip address 10.0.2.24 255.255.255.0 secondary
ip address 10.0.2.25 255.255.255.0 secondary
ip address 10.0.2.26 255.255.255.0 secondary
ip address 10.0.2.27 255.255.255.0 secondary
ip address 10.0.2.28 255.255.255.0 secondary
ip address 10.0.2.29 255.255.255.0 secondary
ip address 10.0.2.30 255.255.255.0 secondary
ip address 10.0.2.31 255.255.255.0 secondary
load-interval 30
speed 25000
no negotiation auto
no mop enabled
no mop sysid
!
! ### IP route from c8kv-uut to local servers
ip route 10.1.0.0 255.255.255.0 GigabitEthernet2 10.0.1.10
! ### IP routes from c8kv-uut to clients on c8kv-peer side, routes are evenly distributed to all 12 TXQ’s
ip route 10.10.0.0 255.255.0.0 Tunnel0
ip route 10.10.0.0 255.255.0.0 Tunnel1
ip route 10.10.0.0 255.255.0.0 Tunnel2
ip route 10.10.0.0 255.255.0.0 Tunnel3
ip route 10.10.0.0 255.255.0.0 Tunnel4
ip route 10.10.0.0 255.255.0.0 Tunnel5
ip route 10.10.0.0 255.255.0.0 Tunnel6
ip route 10.10.0.0 255.255.0.0 Tunnel7
ip route 10.10.0.0 255.255.0.0 Tunnel8
ip route 10.10.0.0 255.255.0.0 Tunnel9
ip route 10.10.0.0 255.255.0.0 Tunnel10
ip route 10.10.0.0 255.255.0.0 Tunnel11
! ### IP route from c8kv-uut Gi3 int tunnel endpoint to c8kv-peer Gi3
int tunnel endpoints (secondary IP addresses on c8kv-peer side)
ip route 20.0.2.30 255.255.255.255 10.0.2.1
ip route 20.0.2.31 255.255.255.255 10.0.2.1
ip route 20.0.2.32 255.255.255.255 10.0.2.1
ip route 20.0.2.33 255.255.255.255 10.0.2.1
ip route 20.0.2.36 255.255.255.255 10.0.2.1
ip route 20.0.2.35 255.255.255.255 10.0.2.1
ip route 20.0.2.37 255.255.255.255 10.0.2.1
ip route 20.0.2.38 255.255.255.255 10.0.2.1
ip route 20.0.2.40 255.255.255.255 10.0.2.1
ip route 20.0.2.41 255.255.255.255 10.0.2.1
ip route 20.0.2.44 255.255.255.255 10.0.2.1
ip route 20.0.2.46 255.255.255.255 10.0.2.1
请参阅前面的CLI配置和拓扑示例。可以根据网络编址方案和生成的散列IP地址复制和修改CLI配置。
要成功创建隧道,请务必在C8000V和AWS VPC的路由表中创建IP路由。
这是一个拓扑和SD-WAN配置示例,使用位于AWS VPC中的C8000Vs上的环回接口创建TLOC。
图6.在AWS VPC中的C8000Vs上使用带环回接口的TLOC的SD-WAN拓扑示例。
注意:在图6中,黑色连接表示SD-WAN控制平面元素和SD-WAN边缘设备之间的控制(VPN0)连接。蓝色连接表示使用TLOC的两个SD-WAN边缘设备之间的隧道。
您可以找到图6(此处)的SD-WAN CLI配置示例。
csr_uut#show sdwan run
system
system-ip 29.173.249.161
site-id 5172
admin-tech-on-failure
sp-organization-name SP_ORG_NAME
organization-name ORG_NAME
upgrade-confirm 15
vbond X.X.X.X
!
memory free low-watermark processor 68484
service timestamps debug datetime msec
service timestamps log datetime msec
no service tcp-small-servers
no service udp-small-servers
platform console virtual
platform qfp utilization monitor load 80
platform punt-keepalive disable-kernel-core
hostname csr_uut
username ec2-user privilege 15 secret 5 $1$4P16$..ag88eFsOMLIemjNcWSt0
vrf definition 11
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
!
vrf definition Mgmt-intf
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
!
no ip finger
no ip rcmd rcp-enable
no ip rcmd rsh-enable
no ip dhcp use class
ip route 0.0.0.0 0.0.0.0 X.X.X.X
ip route 0.0.0.0 0.0.0.0 X.X.X.X
ip route 0.0.0.0 0.0.0.0 X.X.X.X
ip route vrf 11 10.1.0.0 255.255.0.0 X.X.X.X
ip route vrf Mgmt-intf 0.0.0.0 0.0.0.0 X.X.X.X
no ip source-route
ip ssh pubkey-chain
username ec2-user
key-hash ssh-rsa 353158c28c7649710b3c933da02e384b ec2-user
!
!
!
no ip http server
ip http secure-server
ip nat settings central-policy
ip nat settings gatekeeper-size 1024
ipv6 unicast-routing
class-map match-any class0
match dscp 1
!
class-map match-any class1
match dscp 2
!
class-map match-any class2
match dscp 3
!
class-map match-any class3
match dscp 4
!
class-map match-any class4
match dscp 5
!
class-map match-any class5
match dscp 6
!
class-map match-any class6
match dscp 7
!
class-map match-any class7
match dscp 8
!
policy-map qos_map1
class class0
priority percent 20
!
class class1
bandwidth percent 18
random-detect
!
class class2
bandwidth percent 15
random-detect
!
class class3
bandwidth percent 12
random-detect
!
class class4
bandwidth percent 10
random-detect
!
class class5
bandwidth percent 10
random-detect
!
class class6
bandwidth percent 10
random-detect
!
class class7
bandwidth percent 5
random-detect
!
!
interface GigabitEthernet1
no shutdown
ip address dhcp
no mop enabled
no mop sysid
negotiation auto
exit
interface GigabitEthernet2
no shutdown
ip address dhcp
load-interval 30
speed 10000
no negotiation auto
service-policy output qos_map1
exit
interface GigabitEthernet3
shutdown
ip address dhcp
load-interval 30
speed 10000
no negotiation auto
exit
interface GigabitEthernet4
no shutdown
vrf forwarding 11
ip address X.X.X.X 255.255.255.0
load-interval 30
speed 10000
no negotiation auto
exit
interface Loopback1
no shutdown
ip address 192.168.1.21 255.255.255.255
exit
interface Loopback2
no shutdown
ip address 192.168.1.129 255.255.255.255
exit
interface Loopback3
no shutdown
ip address 192.168.1.20 255.255.255.255
exit
interface Loopback4
no shutdown
ip address 192.168.1.128 255.255.255.255
exit
interface Loopback5
no shutdown
ip address 192.168.1.23 255.255.255.255
exit
interface Loopback6
no shutdown
ip address 192.168.1.131 255.255.255.255
exit
interface Loopback7
no shutdown
ip address 192.168.1.22 255.255.255.255
exit
interface Loopback8
no shutdown
ip address 192.168.1.130 255.255.255.255
exit
interface Tunnel1
no shutdown
ip unnumbered GigabitEthernet1
tunnel source GigabitEthernet1
tunnel mode sdwan
exit
interface Tunnel14095001
no shutdown
ip unnumbered Loopback1
no ip redirects
ipv6 unnumbered Loopback1
no ipv6 redirects
tunnel source Loopback1
tunnel mode sdwan
exit
interface Tunnel14095002
no shutdown
ip unnumbered Loopback2
no ip redirects
ipv6 unnumbered Loopback2
no ipv6 redirects
tunnel source Loopback2
tunnel mode sdwan
exit
interface Tunnel14095003
no shutdown
ip unnumbered Loopback3
no ip redirects
ipv6 unnumbered Loopback3
no ipv6 redirects
tunnel source Loopback3
tunnel mode sdwan
exit
interface Tunnel14095004
no shutdown
ip unnumbered Loopback4
no ip redirects
ipv6 unnumbered Loopback4
no ipv6 redirects
tunnel source Loopback4
tunnel mode sdwan
exit
interface Tunnel14095005
no shutdown
ip unnumbered Loopback5
no ip redirects
ipv6 unnumbered Loopback5
no ipv6 redirects
tunnel source Loopback5
tunnel mode sdwan
exit
interface Tunnel14095006
no shutdown
ip unnumbered Loopback6
no ip redirects
ipv6 unnumbered Loopback6
no ipv6 redirects
tunnel source Loopback6
tunnel mode sdwan
exit
interface Tunnel14095007
no shutdown
ip unnumbered Loopback7
no ip redirects
ipv6 unnumbered Loopback7
no ipv6 redirects
tunnel source Loopback7
tunnel mode sdwan
exit
interface Tunnel14095008
no shutdown
ip unnumbered Loopback8
no ip redirects
ipv6 unnumbered Loopback8
no ipv6 redirects
tunnel source Loopback8
tunnel mode sdwan
exit
no logging console
aaa authentication enable default enable
aaa authentication login default local
aaa authorization console
aaa authorization exec default local none
login on-success log
license smart transport smart
license smart url https://smartreceiver.cisco.com/licservice/license
line aux 0
!
line con 0
stopbits 1
!
line vty 0 4
transport input ssh
!
line vty 5 80
transport input ssh
!
sdwan
interface GigabitEthernet1
tunnel-interface
encapsulation ipsec
color private1 restrict
allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface GigabitEthernet2
exit
interface GigabitEthernet3
exit
interface Loopback1
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color private2 restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback2
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color private3 restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback3
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color private4 restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback4
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color private5 restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback5
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color private6 restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback6
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color red restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback7
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color blue restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
interface Loopback8
tunnel-interface
encapsulation ipsec preference 150 weight 1
no border
color green restrict
no last-resort-circuit
no low-bandwidth-link
max-control-connections 0
no vbond-as-stun-server
vmanage-connection-preference 0
port-hop
carrier default
nat-refresh-interval 5
hello-interval 1000
hello-tolerance 12
bind GigabitEthernet2
no allow-service all
no allow-service bgp
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
no allow-service netconf
no allow-service ntp
no allow-service ospf
no allow-service stun
allow-service https
no allow-service snmp
no allow-service bfd
exit
exit
appqoe
no tcpopt enable
no dreopt enable
no httpopt enable
!
omp
no shutdown
send-path-limit 16
ecmp-limit 16
graceful-restart
no as-dot-notation
timers
graceful-restart-timer 43200
exit
address-family ipv4
advertise connected
advertise static
!
address-family ipv6
advertise connected
advertise static
!
!
!
security
ipsec
replay-window 8192
integrity-type ip-udp-esp esp
!
!
sslproxy
no enable
rsa-key-modulus 2048
certificate-lifetime 730
eckey-type P256
ca-tp-label PROXY-SIGNING-CA
settings expired-certificate drop
settings untrusted-certificate drop
settings unknown-status drop
settings certificate-revocation-check none
settings unsupported-protocol-versions drop
settings unsupported-cipher-suites drop
settings failure-mode close
settings minimum-tls-ver TLSv1
dual-side optimization enable
!
policy
app-visibility
flow-visibility
!
注意:在公共云环境中执行性能测试引入可能会影响吞吐量性能的新变量。在执行此类测试时,需要考虑以下几点:
进行吞吐量性能测试时,可以使用这些故障排除命令查明性能下降的瓶颈或原因。
“show platform hardware qfp active statistics drop” — 允许我们了解c8kv上是否有丢包。我们需要确保没有明显的尾部丢包或任何递增的相关计数器。
"show platform hardware qfp active statistics drop clear" — 此命令清除计数器。
“show platform hardware qfp active datapath infrastructure sw-cio” — 此命令提供数据包处理器(PP)、流量管理器(TM)在性能运行期间所占百分比的详细信息。这使我们能够从c8kv确定是否有足够的处理能力。
“show platform hardware qfp active datapath util summary” — 此命令提供从所有端口发送/接收c8kv的输入/输出的完整信息。
确保检查输入/输出速率并查看是否存在任何丢弃。此外,请确保检查处理负载百分比。如果达到100%;这意味着c8kv已达到其容量。
“show platt hardware qfp active infrastructure bqs interface GigabitEthernetX” — 此命令允许我们检查接口级别的统计信息,包括队列数、带宽和尾部丢弃。
“show controller” — 此命令提供有关rx/tx正常数据包和丢失数据包的非常精细的信息。
此命令可用于我们看不到任何尾部丢弃,但流量生成器仍显示丢弃的场景。
在数据利用率已达到100%,PP也达到100%的情形中,可能会出现这种情况。
如果rx_missed_errors计数器持续增加,则表示CSR对云基础设施进行反压,因为它无法处理任何其他流量。
“show platform hardware qfp active datapath infrastructure sw-hqf” — 可用于检查由于AWS的背压而导致的任何拥塞。
“show plat hardware qfp active datapath infrastructure sw-nic” — 确定如何在多个队列之间对流量进行负载均衡。在17.7之后,我们有8个多TXQ。
此外,它还可以确定是否有任何特定队列正在接收所有流量或正进行正确负载均衡。
"show controllers | in errors|exceeded|Giga" — 显示由于AWS端执行pps限制而丢弃的数据包,可通过pps_allowance_exceeded计数器进行观察。
尾部丢包计数器保持递增的示例输出 — 多次发出命令以查看计数器是否递增,从而允许我们确认计数器确实是尾部丢包。
csr_uut#show platform hardware qfp active statistics drop
Last clearing of QFP drops statistics : never
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
Disabled 30 3693
IpFragErr 192 290976
Ipv4NoRoute 43 3626
Ipv6NoRoute 4 224
SdwanImplicitAclDrop 31 3899
TailDrop 19099700 22213834441
UnconfiguredIpv6Fia 3816 419760
此处显示的输出示例 — 每30秒发出一次命令以获取实时数据
csr_uut#show platform hardware qfp active datapath infrastructure sw-cio
Credits Usage:
ID Port Wght Global WRKR0 WRKR1 WRKR2 WRKR3 WRKR4 WRKR5 WRKR6 WRKR7 WRKR8 WRKR9 WRKR10 WRKR11 WRKR12 WRKR13 Total
1 rcl0 16: 455 0 4 1 2 3 2 2 4 4 4 4 0 4 23 512
1 rcl0 32: 496 0 0 0 0 0 0 0 0 0 0 0 0 0 16 512
2 ipc 1: 468 4 2 4 3 0 1 1 4 0 2 0 4 0 18 511
3 vxe_punti 4: 481 0 0 0 0 0 0 0 0 0 0 0 0 0 31 512
4 Gi1 4: 446 0 0 1 1 0 2 3 0 3 2 0 1 1 52 512
5 Gi2 4: 440 4 4 4 3 2 1 1 3 2 4 4 3 2 59 504
6 Gi3 4: 428 1 1 1 0 4 4 1 0 4 4 0 0 2 43 494
7 Gi4 4: 427 1 1 0 1 4 2 0 4 3 4 1 1 7 56 512
Core Utilization over preceding 12819.5863 seconds
--------------------------------------------------
ID: 0 1 2 3 4 5 6 7 8 9 10 11 12 13
% PP: 6.11 6.23 6.09 6.09 6.04 6.05 6.06 6.07 6.05 6.03 6.04 6.06 0.00 0.00
% RX: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.23
% TM:0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 4.79 0.00
% IDLE: 93.89 93.77 93.91 93.91 93.96 93.95 93.94 93.93 93.95 93.97 93.96 93.94 95.21 97.77
此处显示的输出示例 — 确保检查输入/输出速率并查看是否存在任何丢弃。此外,请确保检查处理负载百分比。如果达到100%;这意味着节点已达到其容量。
csr_uut#show platform hardware qfp active datapath util summary
CPP 0: 5 secs 1 min 5 min 60 min
Input: Total (pps)900215 980887 903176 75623
(bps) 10276623992 11197595912 10310265440 863067008
Output: Total (pps)900216 937459 865930 72522
(bps) 10276642720 10712432752 9894215928 828417104
Processing: Load (pct)56 58 54 4
此处显示的接口级别统计信息输出示例:
csr_uut#sh plat hardware qfp active infrastructure bqs interface GigabitEthernet2
Interface: GigabitEthernet2, QFP interface: 7
Queue: QID: 111 (0x6f)
bandwidth (cfg) : 0 , bandwidth (hw) : 1050000000
shape (cfg) : 0 , shape (hw) : 0
prio level (cfg) : 0 , prio level (hw) : n/a
limit (pkts ) : 1043
Statistics:
depth (pkts ) : 0
tail drops (bytes): 0 , (packets) : 0
total enqs (bytes): 459322360227 , (packets) : 374613901
licensed throughput oversubscription drops:
(bytes): 0 , (packets) : 0
Schedule: (SID:0x8a)
Schedule FCID : n/a
bandwidth (cfg) : 10500000000 , bandwidth (hw) : 10500000000
shape (cfg) : 10500000000 , shape (hw) : 10500000000
Schedule: (SID:0x87)
Schedule FCID : n/a
bandwidth (cfg) : 200000000000 , bandwidth (hw) : 200000000000
shape (cfg) : 200000000000 , shape (hw) : 200000000000
Schedule: (SID:0x86)
Schedule FCID : n/a
bandwidth (cfg) : 500000000000 , bandwidth (hw) : 500000000000
shape (cfg) : 500000000000 , shape (hw) : 500000000000
csr_uut#sh plat hardware qfp active infrastructure bqs interface GigabitEthernet3 | inc tail
tail drops (bytes): 55815791988 , (packets) : 43177643
RX/TX正常数据包、丢失数据包统计信息的输出示例
c8kv-aws-1#show controller
GigabitEthernet1 - Gi1 is mapped to UIO on VXE
rx_good_packets 346
tx_good_packets 243
rx_good_bytes 26440
tx_good_bytes 31813
rx_missed_errors 0
rx_errors 0
tx_errors 0
rx_mbuf_allocation_errors 0
rx_q0packets 0
rx_q0bytes 0
rx_q0errors 0
tx_q0packets 0
tx_q0bytes 0
GigabitEthernet2 - Gi2 is mapped to UIO on VXE
rx_good_packets 96019317
tx_good_packets 85808651
rx_good_bytes 12483293931
tx_good_bytes 11174853219
rx_missed_errors 522036
rx_errors 0
tx_errors 0
rx_mbuf_allocation_errors 0
rx_q0packets 0
rx_q0bytes 0
rx_q0errors 0
tx_q0packets 0
tx_q0bytes 0
GigabitEthernet3 - Gi3 is mapped to UIO on VXE
rx_good_packets 171596935
tx_good_packets 191911304
rx_good_bytes 11668588022
tx_good_bytes 13049984257
rx_missed_errors 21356065
rx_errors 0
tx_errors 0
rx_mbuf_allocation_errors 0
rx_q0packets 0
rx_q0bytes 0
rx_q0errors 0
tx_q0packets 0
tx_q0bytes 0
GigabitEthernet4 - Gi4 is mapped to UIO on VXE
rx_good_packets 95922932
tx_good_packets 85831238
rx_good_bytes 12470124252
tx_good_bytes 11158486786
rx_missed_errors 520328
rx_errors 46
tx_errors 0
rx_mbuf_allocation_errors 0
rx_q0packets 0
rx_q0bytes 0
rx_q0errors 0
tx_q0packets 0
tx_q0bytes 0
用于检查由于AWS背压引起的任何拥塞的示例输出:
csr_uut#show platform hardware qfp active datapath infrastructure sw-hqf
Name : Pri1 Pri2 None / Inflight pkts
GigabitEthernet4 : XON XON XOFF / 43732
HQF[0] IPC: send 514809 fc 0 congested_cnt 0
HQF[0] recycle: send hi 0 send lo 228030112
fc hi 0 fc lo 0
cong hi 0 cong lo 0
HQF[0] pkt: send hi 433634 send lo 2996661158
fc/full hi 0 fc/full lo 34567275
cong hi 0 cong lo 4572971630**************Congestion counters keep incrementing
HQF[0] aggr send stats 3225639713 aggr send lo state 3225206079
aggr send hi stats 433634
max_tx_burst_sz_hi 0 max_tx_burst_sz_lo 0
HQF[0] gather: failed_to_alloc_b4q 0
HQF[0] ticks 662109543, max ticks accumulated 348
HQF[0] mpsc stats: count: 0
enq 3225683472 enq_spin 0 enq_post 0 enq_flush 0
sig_cnt:0 enq_cancel 0
deq 3225683472 deq_wait 0 deq_fail 0 deq_cancel 0
deq_wait_timeout
有关流量如何在多个队列之间实现负载均衡的示例输出:
um-csr-uut#sh plat hardware qfp active datapath infrastructure sw-nic
pmd b1c5a400 device Gi1
RX: pkts 50258 bytes 4477620 return 0 badlen 0
pkts/burst 1 cycl/pkt 579 ext_cycl/pkt 996
Total ring read 786244055, empty 786197491
TX: pkts 57860 bytes 6546349
pri-0: pkts 7139 bytes 709042
pkts/send 1
pri-1: pkts 3868 bytes 451352
pkts/send 1
pri-2: pkts 1875 bytes 219403
pkts/send 1
pri-3: pkts 2417 bytes 242527
pkts/send 1
pri-4: pkts 8301 bytes 984022
pkts/send 1
pri-5: pkts 10268 bytes 1114859
pkts/send 1
pri-6: pkts 1740 bytes 175353
pkts/send 1
pri-7: pkts 22252 bytes 2649791
pkts/send 1
Total: pkts/send 1 cycl/pkt 1091
send 56756 sendnow 0
forced 56756 poll 0 thd_poll 0
blocked 0 retries 0 mbuf alloc err 0
TX Queue 0: full 0 current index 0 hiwater 0
TX Queue 1: full 0 current index 0 hiwater 0
TX Queue 2: full 0 current index 0 hiwater 0
TX Queue 3: full 0 current index 0 hiwater 0
TX Queue 4: full 0 current index 0 hiwater 0
TX Queue 5: full 0 current index 0 hiwater 0
TX Queue 6: full 0 current index 0 hiwater 0
TX Queue 7: full 0 current index 0 hiwater 0
pmd b1990b00 device Gi2
RX: pkts 1254741010 bytes 511773562848 return 0 badlen 0
pkts/burst 16 cycl/pkt 792 ext_cycl/pkt 1342
Total ring read 1012256968, empty 937570790
TX: pkts 1385120320 bytes 564465308380
pri-0: pkts 168172786 bytes 68650796972
pkts/send 1
pri-1: pkts 177653235 bytes 72542203822
pkts/send 1
pri-2: pkts 225414300 bytes 91947701824
pkts/send 1
pri-3: pkts 136817435 bytes 55908224442
pkts/send 1
pri-4: pkts 256461818 bytes 104687120554
pkts/send 1
pri-5: pkts 176043289 bytes 71879529606
pkts/send 1
pri-6: pkts 83920827 bytes 34264110122
pkts/send 1
pri-7: pkts 160636635 bytes 64585622696
pkts/send 1
Total: pkts/send 1 cycl/pkt 442
send 1033104466 sendnow 41250092
forced 1776500651 poll 244223290 thd_poll 0
blocked 1060879040 retries 3499069 mbuf alloc err 0
TX Queue 0: full 0 current index 0 hiwater 31
TX Queue 1: full 718680 current index 0 hiwater 255
TX Queue 2: full 0 current index 0 hiwater 31
TX Queue 3: full 0 current index 0 hiwater 31
TX Queue 4: full 15232240 current index 0 hiwater 255
TX Queue 5: full 0 current index 0 hiwater 31
TX Queue 6: full 0 current index 0 hiwater 31
TX Queue 7: full 230668 current index 0 hiwater 224
pmd b1712d00 device Gi3
RX: pkts 1410702537 bytes 498597093510 return 0 badlen 0
pkts/burst 18 cycl/pkt 269 ext_cycl/pkt 321
Total ring read 1011915032, empty 934750846
TX: pkts 754803798 bytes 266331910366
pri-0: pkts 46992577 bytes 16616415156
pkts/send 1
pri-1: pkts 49194201 bytes 17379760716
pkts/send 1
pri-2: pkts 46991555 bytes 16616509252
pkts/send 1
pri-3: pkts 49195026 bytes 17381741474
pkts/send 1
pri-4: pkts 48875656 bytes 17283423414
pkts/send 1
pri-5: pkts 417370776 bytes 147056906106
pkts/send 6
pri-6: pkts 46992860 bytes 16617923068
pkts/send 1
pri-7: pkts 49191147 bytes 17379231180
pkts/send 1
Total: pkts/send 2 cycl/pkt 0
send 339705775 sendnow 366141927
forced 3138709511 poll 2888466204 thd_poll 0
blocked 1758644571 retries 27927046 mbuf alloc err 0
TX Queue 0: full 0 current index 0 hiwater 0
TX Queue 1: full 0 current index 0 hiwater 0
TX Queue 2: full 0 current index 0 hiwater 0
TX Queue 3: full 0 current index 0 hiwater 0
TX Queue 4: full 0 current index 1 hiwater 0
TX Queue 5: full 27077270 current index 0 hiwater 224
TX Queue 6: full 0 current index 0 hiwater 0
TX Queue 7: full 0 current index 0 hiwater 0
显示因AWS端执行的pps限制而丢包的示例输出,可通过pps_allowance_exceeded计数器进行观察:
C8k-AWS-2#show controllers | in errors|exceeded|Giga
GigabitEthernet1 - Gi1 is mapped to UIO on VXE
rx_missed_errors 1750262
rx_errors 0
tx_errors 0
rx_mbuf_allocation_errors 0
rx_q0_errors 0
rx_q1_errors 0
rx_q2_errors 0
rx_q3_errors 0
bw_in_allowance_exceeded 0
bw_out_allowance_exceeded 0
pps_allowance_exceeded 11750
conntrack_allowance_exceeded 0
linklocal_allowance_exceeded 0
版本 | 发布日期 | 备注 |
---|---|---|
1.0 |
07-Aug-2025
|
初始版本 |