为什么流量未通过SD-WAN中心路由器的ECMP路径进行负载均衡

简介

本文档介绍当来自分支路由器的流量在通告相同前缀的多个中心路由器上未进行负载均衡时，SD-WAN交换矩阵中的等价多路径(ECMP)路由的典型问题。它还说明如何解决此问题以及如何使用各种故障排除命令(包括show sdwan policy service-path)来排除17.2 Cisco IOS®-XE软件中添加的路由问题。

先决条件

要求

Cisco 建议您了解以下主题：

• 对重叠管理协议(OMP)的基本了解
• SD-WAN组件及其之间的交互

使用的组件

出于演示目的，使用了以下软件路由器：

• 4台运行17.2.1v软件版本、在控制器模式(SD-WAN)下运行的Cisco IOS-XE CSR1000v路由器
• 运行20.1.12软件版本的vSmart控制器

本文档中的信息都是基于特定实验室环境中的设备编写的。本文档中使用的所有设备最初均采用原始（默认）配置。如果您的网络处于活动状态，请确保您了解所有命令的潜在影响。

背景信息

为了本文的目的，我们使用以下实验拓扑：

您可以在此处找到分配给SD-WAN交换矩阵中每台设备的site-id和system-ip参数的摘要：

每个集线器有4个TLOC(传输位置标识符)，其颜色按照拓扑图分配，每个集线器将0.0.0.0/0默认路由与192.168.2.0/24子网一起通告给分支（分支路由器cE3）。vSmart上未配置策略以首选任何路径/设备，并且所有设备上的所有OMP设置也均设置为默认值。其余配置是基本SD-WAN重叠功能的标准最小配置，因此不提供此配置是为了简洁。您可以期望主动 — 主动冗余和流向中心路由器的出口流量在分支路由器的所有可用上行链路上实现负载均衡。

问题

分支路由器安装默认路由并仅通过cE1路由器(hub1)路由到192.168.2.0/24子网：

ce3#show ip route vrf 2 | b Gateway
Gateway of last resort is 192.168.30.214 to network 0.0.0.0

m*    0.0.0.0/0 [251/0] via 192.168.30.214, 00:08:30, sdwan_system_ip
m     192.168.2.0/24 [251/0] via 192.168.30.214, 00:10:01, sdwan_system_ip
      192.168.216.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.216.0/24 is directly connected, Loopback2
L        192.168.216.216/32 is directly connected, Loopback2

这是因为cE3仅接收默认路由0.0.0.0/0和192.168.2.0/24的4条路由。

ce3#show sdwan omp routes vpn 2 | begin PATH
                                            PATH                      ATTRIBUTE
VPN    PREFIX              FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE
--------------------------------------------------------------------------------------------------------------------------------------
2      0.0.0.0/0           192.168.30.113   61614  1003     C,I,R     installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.113   61615  1003     C,I,R     installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.113   61616  1003     C,I,R     installed  192.168.30.214   private1         ipsec  -
                           192.168.30.113   61617  1003     C,I,R     installed  192.168.30.214   private2         ipsec  -
2      192.168.2.0/24      192.168.30.113   61610  1003     C,I,R     installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.113   61611  1003     C,I,R     installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.113   61612  1003     C,I,R     installed  192.168.30.214   private1         ipsec  -
                           192.168.30.113   61613  1003     C,I,R     installed  192.168.30.214   private2         ipsec  -
2      192.168.216.0/24    0.0.0.0          68     1003     C,Red,R   installed  192.168.30.216   biz-internet     ipsec  -
                           0.0.0.0          81     1003     C,Red,R   installed  192.168.30.216   private1         ipsec  -
                           0.0.0.0          82     1003     C,Red,R   installed  192.168.30.216   private2         ipsec  -

虽然在vSmart上，您仍可以看到它接收所有8条路由（每个集线器上每个TLOC颜色对应的4条路由）：

vsmart1# show omp routes vpn 2 | b PATH
                                            PATH                      ATTRIBUTE
VPN    PREFIX              FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE
--------------------------------------------------------------------------------------------------------------------------------------
2      0.0.0.0/0           192.168.30.214   66     1003     C,R       installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.214   68     1003     C,R       installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.214   81     1003     C,R       installed  192.168.30.214   private1         ipsec  -
                           192.168.30.214   82     1003     C,R       installed  192.168.30.214   private2         ipsec  -
                           192.168.30.215   66     1003     C,R       installed  192.168.30.215   mpls             ipsec  -
                           192.168.30.215   68     1003     C,R       installed  192.168.30.215   biz-internet     ipsec  -
                           192.168.30.215   81     1003     C,R       installed  192.168.30.215   private1         ipsec  -
                           192.168.30.215   82     1003     C,R       installed  192.168.30.215   private2         ipsec  -
2      192.168.2.0/24      192.168.30.214   66     1003     C,R       installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.214   68     1003     C,R       installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.214   81     1003     C,R       installed  192.168.30.214   private1         ipsec  -
                           192.168.30.214   82     1003     C,R       installed  192.168.30.214   private2         ipsec  -
                           192.168.30.215   66     1003     C,R       installed  192.168.30.215   mpls             ipsec  -
                           192.168.30.215   68     1003     C,R       installed  192.168.30.215   biz-internet     ipsec  -
                           192.168.30.215   81     1003     C,R       installed  192.168.30.215   private1         ipsec  -
                           192.168.30.215   82     1003     C,R       installed  192.168.30.215   private2         ipsec  -

如果来自cE1(hub1)的默认路由丢失，分支路由器将安装来自cE2(hub2)的路由。 因此，没有主用 — 主用冗余，而是主用 — 备用，而cE1充当主路由器。

您还可以使用show sdwan policy service-path命令检查特定流量采用哪个出口路径，如以下示例所示：

ce3#show sdwan policy service-path vpn 2  interface Loopback2 source-ip 192.168.216.216 dest-ip 192.168.2.1 protocol 6 source-port 53453 dest-port 22 dscp 48 app ssh
Next Hop: IPsec
  Source: 192.168.109.216 12347 Destination: 192.168.110.214 12427 Local Color: biz-internet Remote Color: mpls Remote System IP: 192.168.30.214

要查看特定流量类型的所有可用路径，请使用all关键字：

ce3#show sdwan policy service-path vpn 2  interface Loopback2 source-ip 192.168.216.216 dest-ip 192.168.2.1 protocol 6 source-port 53453 dest-port 22 dscp 48 app ssh all
Number of possible next hops: 4
Next Hop: IPsec
  Source: 192.168.109.216 12347 Destination: 192.168.110.214 12427 Local Color: biz-internet Remote Color: mpls Remote System IP: 192.168.30.214
Next Hop: IPsec
  Source: 192.168.108.216 12367 Destination: 192.168.108.214 12407 Local Color: private2 Remote Color: private2 Remote System IP: 192.168.30.214
Next Hop: IPsec
  Source: 192.168.107.216 12367 Destination: 192.168.107.214 12407 Local Color: private1 Remote Color: private1 Remote System IP: 192.168.30.214
Next Hop: IPsec
  Source: 192.168.109.216 12347 Destination: 192.168.109.214 12387 Local Color: biz-internet Remote Color: biz-internet Remote System IP: 192.168.30.214

因此，这还确认了仅有4条路径可用，而不是8条路径可用于路由器cE3(spoke2)。

如果检查vSmart通告的内容，您仅看到向cE3通告的4条路由：

vsmart1# show omp routes vpn 2 0.0.0.0/0 detail | nomore | exclude not\ set | b ADVERTISED\ TO: | b peer\ \ \ \ 192.168.30.216
peer    192.168.30.216
    Attributes:
     originator       192.168.30.214
     label            1003
     path-id          61629
     tloc             192.168.30.214, private2, ipsec
     site-id          214
     overlay-id        1
     origin-proto     static
     origin-metric    0
    Attributes:
     originator       192.168.30.214
     label            1003
     path-id          61626
     tloc             192.168.30.214, mpls, ipsec
     site-id          214
     overlay-id        1
     origin-proto     static
     origin-metric    0
    Attributes:
     originator       192.168.30.214
     label            1003
     path-id          61628
     tloc             192.168.30.214, private1, ipsec
     site-id          214
     overlay-id        1
     origin-proto     static
     origin-metric    0
    Attributes:
     originator       192.168.30.214
     label            1003
     path-id          61627
     tloc             192.168.30.214, biz-internet, ipsec
     site-id          214
     overlay-id        1
     origin-proto     static
     origin-metric    0

根据此输出，您可以断定，问题是由于vSmart控制器引起的。

解决方案

此行为是由在vSmart控制器上默认配置send-path-limit引起的。send-path-limit定义从边缘路由器通告到vSmart控制器以及从vSmart控制器通告到其他边缘路由器的ECMP路由的最大数量。默认值为4，通常情况下，它对于边缘路由器（如本拓扑中每个集线器路由器上有4个上行链路）足够，但对于虚拟智能控制器而言，它不足以将所有可用路径发送到其他边缘路由器。可以为send-path-limit设置的最大值为16，但在某些极端情况下，这仍然不够，尽管打开了增强请求CSCvs89015，以将最大值增加到128。 

要解决此问题，必须重新配置vSmart设置，如以下示例所示：

vsmart1# conf t
Entering configuration mode terminal
vsmart1(config)# omp
vsmart1(config-omp)# send-path-limit 8
vsmart1(config-omp)# commit
Commit complete.
vsmart1(config-omp)# end
vsmart1# show run omp
omp
 no shutdown
 send-path-limit  8
 graceful-restart
!
vsmart1#

然后，vSmart将所有8条路由通告给分支路由器，并由它们接收：

ce3#show sdwan omp routes vpn 2 | begin PATH
                                            PATH                      ATTRIBUTE
VPN    PREFIX              FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE
--------------------------------------------------------------------------------------------------------------------------------------
2      0.0.0.0/0           192.168.30.113   61626  1003     C,I,R     installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.113   61627  1003     C,I,R     installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.113   61628  1003     C,I,R     installed  192.168.30.214   private1         ipsec  -
                           192.168.30.113   61629  1003     C,I,R     installed  192.168.30.214   private2         ipsec  -
                           192.168.30.113   61637  1003     C,R       installed  192.168.30.215   mpls             ipsec  -
                           192.168.30.113   61638  1003     C,R       installed  192.168.30.215   biz-internet     ipsec  -
                           192.168.30.113   61639  1003     C,R       installed  192.168.30.215   private1         ipsec  -
                           192.168.30.113   61640  1003     C,R       installed  192.168.30.215   private2         ipsec  -
2      192.168.2.0/24      192.168.30.113   61610  1003     C,I,R     installed  192.168.30.214   mpls             ipsec  -
                           192.168.30.113   61611  1003     C,I,R     installed  192.168.30.214   biz-internet     ipsec  -
                           192.168.30.113   61612  1003     C,I,R     installed  192.168.30.214   private1         ipsec  -
                           192.168.30.113   61613  1003     C,I,R     installed  192.168.30.214   private2         ipsec  -
                           192.168.30.113   61633  1003     C,R       installed  192.168.30.215   mpls             ipsec  -
                           192.168.30.113   61634  1003     C,R       installed  192.168.30.215   biz-internet     ipsec  -
                           192.168.30.113   61635  1003     C,R       installed  192.168.30.215   private1         ipsec  -
                           192.168.30.113   61636  1003     C,R       installed  192.168.30.215   private2         ipsec  -
2      192.168.216.0/24    0.0.0.0          68     1003     C,Red,R   installed  192.168.30.216   biz-internet     ipsec  -
                           0.0.0.0          81     1003     C,Red,R   installed  192.168.30.216   private1         ipsec  -
                           0.0.0.0          82     1003     C,Red,R   installed  192.168.30.216   private2         ipsec  -

虽然仍然为分支机构的路由器仅通过cE1(hub1)安装路由：

ce3#sh ip route vrf 2 0.0.0.0

Routing Table: 2
Routing entry for 0.0.0.0/0, supernet
  Known via "omp", distance 251, metric 0, candidate default path, type omp
  Last update from 192.168.30.214 on sdwan_system_ip, 01:11:26 ago
  Routing Descriptor Blocks:
  * 192.168.30.214 (default), from 192.168.30.214, 01:11:26 ago, via sdwan_system_ip
      Route metric is 0, traffic share count is 1

ce3#sh ip route vrf 2 192.168.2.0

Routing Table: 2
Routing entry for 192.168.2.0/24
  Known via "omp", distance 251, metric 0, type omp
  Last update from 192.168.30.214 on sdwan_system_ip, 01:33:56 ago
  Routing Descriptor Blocks:
  * 192.168.30.214 (default), from 192.168.30.214, 01:33:56 ago, via sdwan_system_ip
      Route metric is 0, traffic share count is 1
ce3#

show sdwan policy service-path将确认相同内容，因此不提供输出以简明扼要。

此原因也是另一个命令ecmp-limit值的默认配置。默认情况下，边缘路由器仅将前4个ECMP路径安装到路由表中，因此，要解决此问题，必须重新配置分支路由器，如以下示例所示：

ce3#config-t

admin connected from 127.0.0.1 using console on ce3
ce3(config)# sdwan
ce3(config-sdwan)# omp
ce3(config-omp)# ecmp-limit 8
ce3(config-omp)# commit
Commit complete.

show ip route确认两个通过集线器的路由现在都已安装：

ce3#sh ip ro vrf 2 | b Gateway
Gateway of last resort is 192.168.30.215 to network 0.0.0.0

m*    0.0.0.0/0 [251/0] via 192.168.30.215, 00:00:37, sdwan_system_ip
                [251/0] via 192.168.30.214, 00:00:37, sdwan_system_ip
m     192.168.2.0/24 [251/0] via 192.168.30.215, 00:00:37, sdwan_system_ip
                     [251/0] via 192.168.30.214, 00:00:37, sdwan_system_ip
      192.168.216.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.216.0/24 is directly connected, Loopback2
L        192.168.216.216/32 is directly connected, Loopback2
ce3#

如果使用基于功能模板的vManage设备模板，为了获得与此屏幕截图相同的OMP功能模板（路由器使用的OMP功能模板的ECMP限制，以及vSmart使用的OMP功能模板的每个前缀的通告路径数）：

相关信息

• https://www.cisco.com/c/en/us/td/docs/routers/sdwan/command/sdwan-cr-book/config-cmd.html#wp3085259372
• https://www.cisco.com/c/en/us/td/docs/routers/sdwan/command/sdwan-cr-book/config-cmd.html#wp2570227565
• https://www.cisco.com/c/en/us/td/docs/routers/sdwan/command/sdwan-cr-book/operational-cmd.html#wp5579365410
• 技术支持和文档 - Cisco Systems