本產品的文件集力求使用無偏見用語。針對本文件集的目的,無偏見係定義為未根據年齡、身心障礙、性別、種族身分、民族身分、性別傾向、社會經濟地位及交織性表示歧視的用語。由於本產品軟體使用者介面中硬式編碼的語言、根據 RFP 文件使用的語言,或引用第三方產品的語言,因此本文件中可能會出現例外狀況。深入瞭解思科如何使用包容性用語。
思科已使用電腦和人工技術翻譯本文件,讓全世界的使用者能夠以自己的語言理解支援內容。請注意,即使是最佳機器翻譯,也不如專業譯者翻譯的內容準確。Cisco Systems, Inc. 對這些翻譯的準確度概不負責,並建議一律查看原始英文文件(提供連結)。
本文檔介紹如何辨識和解決ACI中vPC可能出現的問題。
虛擬埠通道(vPC)允許物理連線到兩個不同ACI枝葉節點的鏈路顯示為連線到第三台裝置(即網路交換機、伺服器和支援鏈路聚合技術的任何其他網路裝置)的單個埠通道。
vPC包括兩個指定為vPC對等交換機的ACI枝葉交換機。在vPC對等裝置中,一個為主要,另一個為輔助。交換機形成的系統稱為vPC域。
vPC對等體之間沒有專用的對等鏈路;交換矩陣本身充當MCT。
· 對等連通性協定-使用ZMQ代替CFS。
· ZMQ是使用TCP作為傳輸協定的開源高效能消息庫。
· 此庫在交換機上打包為libzmq,並連結到需要與vPC對等裝置通訊的每個應用程式。
對等連通性不是透過物理對等鏈路處理的;而是使用路由觸發器來檢測對等連通性。
· vPC Manager向URIB註冊對等路由通知。
· 當ISIS發現到對等體的路由時,URIB會通知vPC管理器,然後嘗試打開對等體的ZMQ套接字。
· 當對等路由被ISIS撤銷時,vPC管理器將再次被URIB通知,並使MCT鏈路斷開。
作為升級最佳實踐的一部分,建議升級每個Pod中的交換機(至少分為兩個組),以使每個Pod中一半的枝葉和主幹節點在任何給定時間都處於運行狀態。例如,一個組具有偶數編號的枝葉和主幹節點,另一個組在每個枝葉和主幹中具有奇數編號的枝葉和主幹。透過vPC配置裝置,我們可以確保至少有一個裝置在升級過程中處於運行狀態,方法是將它們放入不同的組中。這可以防止升級期間出現任何中斷,因為至少有一個裝置在另一個裝置升級期間保持運行。
ACI:以應用為中心的基礎設施
vPC:虛擬埠通道
MCT:多機箱EtherChannel中繼
CFS:思科交換矩陣服務
ZMQ:零訊息佇列
LACP:鏈路聚合控制協定
PDU:協定資料單元
LAG:鏈路聚合
FAB3-L1# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : Disabled
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Operational Layer3 Peer : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 up -
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
686 Po3 up success success 86
FAB3-L2# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : Disabled
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Operational Layer3 Peer : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 up -
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
686 Po2 up success success 86
輸出顯示,使用vPC域ID 101形成對等鄰接關係,注意vPC保持連線狀態在ACI中停用,因為不需要專用鏈路。Po3在使用活動VLAN 86的vPC中處於運行狀態。請注意,vPC對交換機上的埠通道編號可以不同。
FAB3-L1# show vpc role
vPC Role status
----------------------------------------------------
vPC role : primary, operational secondary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:65
vPC system-priority : 32667
vPC local system-mac : 00:81:c4:b1:25:4f
vPC local role-priority : 101
FAB3-L2# show vpc role
vPC Role status
----------------------------------------------------
vPC role : secondary, operational primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:65
vPC system-priority : 32667
vPC local system-mac : 00:5d:73:57:c4:2c
vPC local role-priority : 102
此命令顯示L1為主要,L2為次要。
由於終端裝置連線到兩台不同的vPC交換機,因此必須有一種機制使它們將vPC對等裝置標識為一個邏輯裝置。這是透過在對等體之間共用的LAG ID中使用vPC系統Mac實現的。這使得終端裝置將vPC對等體視為一個邏輯單元。
N3K# show lacp interface ethernet 1/24
Interface Ethernet1/24 is up
Channel group is 1 port channel is Po1
PDUs sent: 31726
PDUs rcvd: 31634
Markers sent: 0
Markers rcvd: 0
Marker response sent: 0
Marker response rcvd: 0
Unknown packets rcvd: 0
Illegal packets rcvd: 0
Lag Id: [ [(7f9b, 0-23-4-ee-be-65, 82ae, 8000, 4121), (8000, 0-a6-ca-75-6f-c1, 8000, 8000, 15d)] ]
Operational as aggregated link since Fri Sep 2 08:05:52 2022
Local Port: Eth1/24 MAC Address= 0-a6-ca-75-6f-c1
System Identifier=0x8000, Port Identifier=0x8000,0x15d
Operational key=32768
LACP_Activity=active
LACP_Timeout=Long Timeout (30s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner information refresh timeout=Long Timeout (90s)
Actor Admin State=61
Actor Oper State=61
Neighbor: 0x4121
MAC Address= 0-23-4-ee-be-65
System Identifier=0x7f9b, Port Identifier=0x8000,0x4121
Operational key=33454
LACP_Activity=active
LACP_Timeout=Long Timeout (30s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner Admin State=61
Partner Oper State=61
Aggregate or Individual(True=1)= 1
N3K# show lacp interface ethernet 1/25
Interface Ethernet1/25 is up
Channel group is 1 port channel is Po1
PDUs sent: 31666
PDUs rcvd: 31651
Markers sent: 0
Markers rcvd: 0
Marker response sent: 0
Marker response rcvd: 0
Unknown packets rcvd: 0
Illegal packets rcvd: 0
Lag Id: [ [(7f9b, 0-23-4-ee-be-65, 82ae, 8000, 111), (8000, 0-a6-ca-75-6f-c1, 8000, 8000, 161)] ]
Operational as aggregated link since Fri Sep 2 08:00:34 2022
Local Port: Eth1/25 MAC Address= 0-a6-ca-75-6f-c1
System Identifier=0x8000, Port Identifier=0x8000,0x161
Operational key=32768
LACP_Activity=active
LACP_Timeout=Long Timeout (30s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner information refresh timeout=Long Timeout (90s)
Actor Admin State=61
Actor Oper State=61
Neighbor: 0x111
MAC Address= 0-23-4-ee-be-65
System Identifier=0x7f9b, Port Identifier=0x8000,0x111
Operational key=33454
LACP_Activity=active
LACP_Timeout=Long Timeout (30s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner Admin State=61
Partner Oper State=61
Aggregate or Individual(True=1)= 1
輸出顯示LAG ID (7f9b, 0-23-4-ee-be-65, 82ae, 8000, 4121),它是Priority as System ID (32667 in Hex)、vPC系統mac (00:23:04:ee:be:65)、操作金鑰(33454 in Hex)和埠識別符號的組合。
FAB3-L1# show port-channel extended
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-----------------------------------------------------------------------------
Group Port- BundleGrp Protocol Member Ports
Channel
-----------------------------------------------------------------------------
3 Po3(SU) 101-102 LACP Eth1/33(P)
Show port-channel extended顯示作為埠通道捆綁一部分的物理鏈路狀態的詳細資訊。
FAB3-L1# show system internal epm vpc
Local TEP IP : 10.3.208.64
Peer TEP IP : 10.3.208.67
vPC configured : Yes
vPC VIP : 10.3.16.67
MCT link status : Up
Local vPC version bitmap : 0x7
Peer vPC version bitmap : 0x7
Negotiated vPC version : 3
Peer advertisement received : Yes
Tunnel to vPC peer : Up
vPC# 686
if : port-channel3, if index : 0x16000002
local vPC state : MCEC_STATE_UP, peer vPC state : MCEC_STATE_UP
current link state : LOCAL_UP_PEER_UP
vPC fast conv : Off
FAB3-L1# show system internal vpcm zmq statistics
--------------------------------------------
MCECM ZMQ counters
----------------------------------------------
ZMQ server : 1
ZmQ: Registered ZmQ print callback
ZmQ: ====== Start ZMQ statistics printing ======
ZmQ: ZMQ socket type: 5, local ID: 40d0030a
ZmQ: Socket base 0x1109c3b4, #endpoints 1
ZmQ: Total 1 I/O pipes, CONNECT CNT: 0, DISCONNECT CNT: 0
ZmQ: RX CNT: 66, BYTES: 124132, ERRORS: 0
ZmQ: TX CNT: 66, BYTES: 125096, ERRORS: 0
ZmQ: Pipe tcp://10.3.208.64:5001 (ID: FD 54 flag 1 state 0): read 66 (124132 bytes) write 66 (125096 bytes) Peer I/O pipe: read 66 (125096 bytes) write 66 (124132 bytes)
ZmQ: Stream engine 0xae90049c ZMQ SOCKET 0x1109c3b4 TCP FD: 54 @ 10.3.208.67:58740
ZmQ: RX CNT: 72 BYTES: 124494 ERRORS: 0 TX CNT: 73 BYTES: 125458 ERRORS: 0
ZmQ: CONNECT CNT: 0 DISCONNECT CNT: 0
ZmQ: ====== End ZMQ statistics printing ======
ZMQ統計資訊顯示ZMQ會話的狀態、連線次數、斷開出現次數以及發生的任何錯誤。
FAB3-L1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : Disabled
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Operational Layer3 Peer : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 up -
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
686 Po3 down* success success
輸出顯示Po3關閉。
FAB3-L1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
3 Po3(SD) Eth LACP Eth1/33(D)
我們進一步瞭解作為port-channel一部分的介面的狀態。此處,Eth1/33處於Down狀態。LACP被配置為捆綁協定。
FAB3-L1# show int e1/33
Ethernet1/33 is down (notconnect)
admin state is up, Dedicated Interface
Belongs to po3
Hardware: 100/1000/10000/auto Ethernet, address: 0081.c4b1.2521 (bia 0081.c4b1.2521)
MTU 9000 bytes, BW 0 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk
full-duplex, 10 Gb/s
FEC (forward-error-correction) : disable-fec
Beacon is turned off
Auto-Negotiation is turned on
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Switchport monitor is off
EtherType is 0x8100
EEE (efficient-ethernet) : n/a
Last link flapped 00:08:15
Last clearing of "show interface" counters never
9 interface resets
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
show interface output提供了有關介面e1/33的詳細資訊。可以看到E1/33發生故障,顯示notconnect狀態。
建議的動作:
確保埠連線正確並且配置正確。
FAB3-L1# show port-channel extended
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-----------------------------------------------------------------------------
Group Port- BundleGrp Protocol Member Ports
Channel
-----------------------------------------------------------------------------
3 Po3(SD) 101-102 LACP Eth1/33(s)
輸出顯示Eth1/33處於掛起狀態。接下來,我們將檢視show interface Eth1/33以瞭解更多詳細資訊。
FAB3-L1# show int e1/33
Ethernet1/33 is down (suspended-due-to-no-lacp-pdus)
admin state is up, Dedicated Interface
Belongs to po3
Hardware: 100/1000/10000/auto Ethernet, address: 0081.c4b1.2521 (bia 0081.c4b1.2521)
MTU 9000 bytes, BW 0 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk
full-duplex, 10 Gb/s
FEC (forward-error-correction) : disable-fec
Beacon is turned off
Auto-Negotiation is turned on
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Switchport monitor is off
EtherType is 0x8100
EEE (efficient-ethernet) : n/a
Last link flapped 00:00:13
Last clearing of "show interface" counters never
12 interface resets
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 1640 bits/sec, 0 packets/sec
show interface建議埠掛起,因為沒有LACP PDU。我們可以進一步檢視LACP計數器,並確定是否正在傳送和接收LACP PDU。
FAB3-L1# show lacp counters interface port-channel 3
LACPDUs Marker Marker Response LACPDUs
Port Sent Recv Sent Recv Sent Recv Pkts Err
---------------------------------------------------------------------
port-channel3
Ethernet1/33 314 264 0 0 0 0 0
FAB3-L1#
FAB3-L1#
FAB3-L1# show lacp counters interface port-channel 3
LACPDUs Marker Marker Response LACPDUs
Port Sent Recv Sent Recv Sent Recv Pkts Err
---------------------------------------------------------------------
port-channel3
Ethernet1/33 315 264 0 0 0 0 0
輸出顯示,計數器只會對Sent LACPDU遞增,而Recv計數器會保持不變。這表明我們沒有收到來自遠端的LACP PDU。
我們還可以檢視特定介面的LACP協商引數、計數器等,使用「show lacp interface e1/33」。
FAB3-L1# show lacp interface e1/33
Interface Ethernet1/33 is suspended
Channel group is 3 port channel is Po3
PDUs sent: 317
PDUs rcvd: 264 received
Markers sent: 0
Markers rcvd: 0
Marker response sent: 0
Marker response rcvd: 0
Unknown packets rcvd: 0
Illegal packets rcvd: 0
Lag Id: [ [(7f9b, 00-23-04-ee-be-65, 82ae, 8000, 121), (0, 0-0-0-0-0-0, 0, 0, 0)] ]
Operational as aggregated link since Mon Aug 22 09:29:53 2022
Local Port: Eth1/33 MAC Address= 00-81-c4-b1-25-4f
System Identifier=0x8000,00-81-c4-b1-25-4f
Port Identifier=0x8000,0x121
Operational key=33454
LACP_Activity=active
LACP_Timeout=Long Timeout (30s)
Synchronization=NOT_IN_SYNC
Collecting=false
Distributing=false
Partner information refresh timeout=Long Timeout (90s)
Actor Admin State=(Ac-1:To-0:Ag-1:Sy-0:Co-0:Di-0:De-1:Ex-0)
Actor Oper State=Ac-1:To-0:Ag-1:Sy-0:Co-0:Di-0:De-1:Ex-0
Neighbor: 0x0
MAC Address= 0-0-0-0-0-0
System Identifier=0x0,0x0
Port Identifier=0x0,0x0
Operational key=0
LACP_Activity=unknown
LACP_Timeout=Long Timeout (30s)
Synchronization=NOT_IN_SYNC
Collecting=false
Distributing=false
Partner Admin State=(Ac-0:To-0:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0)
Partner Oper State=(Ac-0:To-0:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0)
Aggregate or Individual(True=1)= 2
此外,還可以在枝葉上為LACP資料包執行資料包捕獲。您可以使用特定過濾器過濾掉有問題的介面。
tcpdump -vvvi kpm_inb ether proto 0x8809
建議的動作:
確保在遠端端正確配置了LACP,並且裝置在正確的介面上傳送了LACP PDU。
FAB3-L1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : Disabled
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Operational Layer3 Peer : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 up -
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
686 Po3 down* failed vpc port channel
mis-config due to
vpc links in the 2
switches connected
to different
partners
此輸出顯示vPC埠通道已關閉,原因是vPC配置錯誤。觀察埠通道狀態。
FAB3-L1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
3 Po3(SD) Eth LACP Eth1/33(D)
此處,Eth1/33處於Down狀態。有關詳細資訊,請觀察show interface e1/33。
FAB3-L1# show int e1/33
Ethernet1/33 is down (suspend-by-vpc)
admin state is up, Dedicated Interface
Belongs to po3
Hardware: 100/1000/10000/auto Ethernet, address: 0081.c4b1.2521 (bia 0081.c4b1.2521)
MTU 9000 bytes, BW 0 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk
full-duplex, 10 Gb/s
FEC (forward-error-correction) : disable-fec
Beacon is turned off
Auto-Negotiation is turned on
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Switchport monitor is off
EtherType is 0x8100
vPC使用LAG ID確定vPC對等體是否連線到同一主機。如果LAG ID不匹配,則介面由vPC掛起。
Show vpc brief顯示出vPC對等體上port-channel中的物理鏈路未連線到同一遠端裝置。
可以使用「show vpc consistency-parameters interface port-channel 3」檢查LAG ID比較。
FAB3-L1# show vpc consistency-parameters interface port-channel 3
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-65, 82ae, 0-23-4-ee-be-68, 82ae,
0, 0), (8000, 0, 0), (8000,
0-a6-ca-75-6f-c1, 0-a6-ca-75-6f-c1,
8000, 0, 0)] 8000, 0, 0)]
mode 1 active active
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
Port Mode 1 trunk trunk
Native Vlan 1 0 0
MTU 1 9000 9000
vPC card type 1 Empty Empty
Allowed VLANs - 86 86
Local suspended VLANs - - -
如果LAG-ID不匹配,埠將掛起。
建議的動作:
確保埠通道中的物理鏈路連線到同一遠端裝置。
如果埠沒有從對等體接收LACP PDU,LACP會將埠設定為掛起狀態。這可能會導致某些伺服器無法啟動,因為它們需要LACP以邏輯方式啟動埠。可以透過停用LACP suspend individual將行為調整為單獨使用。
為此,請在vPC策略組中建立一個埠通道策略,並將模式設定為LACP active後,刪除Suspend Individual Port。現在,vPC中的埠保持活動狀態,並繼續傳送LACP資料包。
FAB3-L1# show port-channel extended
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-----------------------------------------------------------------------------
Group Port- BundleGrp Protocol Member Ports
Channel
-----------------------------------------------------------------------------
1 Po1(SD) 101-102 LACP Eth1/33(I)
輸出顯示,即使刪除LACP Suspend-Individual(LACP掛起-獨立)標誌後,我們未在Eth1/33上收到LACP PDU,但埠作為獨立埠處於UP狀態。請注意,我們仍會使用此配置從ACI枝葉傳送LACP PDU。收到LACP PDU後,埠將返回捆綁模式。
存在其他並非特定於vPC但仍適用於vPC介面的介面錯誤。詳細資訊請參閱連結。
修訂 | 發佈日期 | 意見 |
---|---|---|
1.0 |
31-Oct-2022 |
初始版本 |