Troubleshoot DHCP Relay in ACI with Local Endpoints

Introduction

This document describes the troubleshoot of DHCP Relay in ACI Fabrics.

Abbreviations

• BD: Bridge domain
• EPG: Endpoint group
• ExEPG: External Endpoint Group
• VRF: Virtual Routing and Forwarding
• Class ID or pcTag: Tag that identifies an EPG
• DHCP: Dynamic Host Configuration Protocol
• SVI: Switched Virtual Interface

Requirements

For this article, it is recommended that you have general knowledge of these topics:

• DHCP concepts and workflow (DORA process)
• ACI concepts: Access Policies, Endpoint Learning, Contracts and L3out
• DHCP Relay Policies must be already created

Components Used

This troubleshooting exercise was made on ACI version 6.0(8f) using second generation Nexus switches N9K-C93180YC-EX and N9K-C93240YC-FX2.

All commands on this article were run on a lab environment and using RFC1819 for IP addressing. If your network is live, ensure that you understand the potential impact of any command making sure you adequate the command in question with your specific needs.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Troubleshooting Recommendations

Endpoint Learning

If endpoints are not learned, validate static port policies such as switch, interface, and VLAN configurations. For virtual servers, confirm the port-group is properly deployed and assigned to the VM.

Policy deployment

Ensure the DHCP Relay Policy (dhcpRelayP) and DHCP Relay Label (dhcpLbl) are correctly configured and consumed by the appropriate Bridge Domain (BD). Ownership rules for label policies are:

• User tenant ownership: Only that tenant can use the policy
• Common tenant ownership: All tenants can use the policy, but the DHCP server must be learned on a common EPG
• Infra tenant ownership: All tenants can use the policy, and the DHCP server can be learned anywhere in the fabric

If the dhcpRtLblDefToRelayP child class is missing under the dhcpRelayP parent policy, no BD is consuming the Relay Policy, and corrective action is required.

Host reachability

The DHCP client must be reachable from the SVI of its BD. If unreachable, verify contracts and routing configurations to ensure connectivity.

Endpoint Learning and Routing

Ensure that the DHCP server is reachable from the SVI of the Bridge Domain where the client resides. 

iping -V [ tenant : VRF ] -S [ SVI IP of the Client ] [ DHCP server IP]

Leaf101# iping -V tz:VRF1 -S 172.16.19.1 172.16.18.100
PING 172.16.18.100 (172.16.18.100) from 172.16.19.1: 56 data bytes
64 bytes from 172.16.18.100: icmp_seq=0 ttl=64 time=0.912 ms
64 bytes from 172.16.18.100: icmp_seq=1 ttl=64 time=0.706 ms
64 bytes from 172.16.18.100: icmp_seq=2 ttl=64 time=0.643 ms
64 bytes from 172.16.18.100: icmp_seq=3 ttl=64 time=0.689 ms
64 bytes from 172.16.18.100: icmp_seq=4 ttl=64 time=0.717 ms

--- 172.16.18.100 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.643/0.733/0.912 ms

Both DHCP clients and DHCP servers must be learned as endpoints when configured under an EPG. To validate that these endpoints are correctly learned, you can check the endpoint manager table on the leaf. 

show system internal epm endpoint [ip | mac] [ DHCP server IP | DHCP client MAC]

Leaf101# show system internal epm endpoint ip 172.16.18.100

MAC : 0050.56b7.80cf ::: Num IPs : 1
IP# 0 : 172.16.18.100 ::: IP# 0 flags : ::: l3-sw-hit: No
Vlan id : 12 ::: Vlan vnid : 8535 ::: VRF name : tz:VRF1
BD vnid : 15400880 ::: VRF vnid : 2981888
Phy If : 0x1a02c000 ::: Tunnel If : 0
Interface : Ethernet1/45
Flags : 0x80004c04 ::: sclass : 16402 ::: Ref count : 5
EP Create Timestamp : 09/11/2025 17:37:15.158380
EP Update Timestamp : 09/11/2025 19:17:41.261985
EP Flags : local|IP|MAC|sclass|timer|

::::

•••

Leaf101# show system internal epm endpoint mac 0050.56b7.33ee

MAC : 0050.56b7.33ee ::: Num IPs : 0
Vlan id : 25 ::: Vlan vnid : 8494 ::: VRF name : tz:VRF1
BD vnid : 15630228 ::: VRF vnid : 2981888
Phy If : 0x1a02c000 ::: Tunnel If : 0
Interface : Ethernet1/45
Flags : 0x80004804 ::: sclass : 32780 ::: Ref count : 4
EP Create Timestamp : 09/11/2025 17:33:36.158122
EP Update Timestamp : 09/11/2025 19:17:41.258478
EP Flags : local|MAC|sclass|timer|

::::

Policy Validaiton

When validating a DHCP Relay policy, these key attributes can be confirmed:

• Name: The identifier of the DHCP Relay policy.
• Owner: The tenant under which the policy is configured, indicating its visibility scope limited to that tenant.
• Address: The IP address of the DHCP server to which DHCP requests are relayed.
• Location of the DHCP Server: Validated through tags such as epgDn, bdDefDn, and ctxDefDn to ensure correct association with the EPG, bridge domain, and VRF.
• State of the Policy: The policy must be in a formed state, indicating it is correctly deployed and active in the fabric.

This validation is performed by running moqueries on the APIC to confirm that the DHCP relay policies are correctly created, associated with the appropriate tenants, and properly linked to the relevant bridge domains. This step helps identify misconfigurations early, preventing DHCP relay failures caused by missing or incorrect policy deployment.

moquery -c dhcpRelayP -f 'dhcp.RelayP.dn*"[ tenant name ].*[ DHCP Relay Policy name ]"' -x rsp-subtree=children

APIC# moquery -c dhcpRelayP -f 'dhcp.RelayP.dn*"tz.*Relay"' -x rsp-subtree=children
Total Objects shown: 1

# dhcp.RelayP
name : tz-DHCP_Relay
<-- cut for brevity-->
dn : uni/tn-tz/relayp-tz-DHCP_Relay
<-- cut for brevity-->
owner : tenant
<-- cut for brevity-->
rn : relayp-tz-DHCP_Relay

# dhcp.ProvDhcp
epgDn : uni/tn-tz/ap-AP1/epg-EPG1
addr : 172.16.18.100
bdDefDn : uni/bd-[uni/tn-tz/BD-BD1]-isSvc-no
<-- cut for brevity-->
ctxDefDn : uni/ctx-[uni/tn-tz/ctx-VRF1]
ctxDefStQual : none
ctxSeg : 2981888
descr : 
dn : uni/tn-tz/relayp-tz-DHCP_Relay/provdhcp-[uni/tn-tz/ap-AP1/epg-EPG1]
l3CtxEncap : vxlan-2981888
<-- cut for brevity-->
name : EPG1
<-- cut for brevity-->
pcTag : 16402
<-- cut for brevity-->

# dhcp.RsProv
tDn : uni/tn-tz/ap-AP1/epg-EPG1
addr : 172.16.18.1
<-- cut for brevity-->
state : formed

Once a DHCP Relay Policy is associated with the Bridge Domain (BD) of the client, a corresponding DHCP Label policy is automatically created. This DHCP Label policy acts as a linkage between the DHCP Relay Policy and the BD, enabling the relay functionality.

You can verify the DHCP Label policy using the APIC CLI with a command such as:

moquery -c dhcpLbl -f 'dhcp.Lbl.dn*"[ tenant ].*[ DHCP Relay Policy name]"'

APIC# moquery -c dhcpLbl -f 'dhcp.Lbl.dn*"tz.*Relay"'
Total Objects shown: 1

# dhcp.Lbl
name : tz-DHCP_Relay
annotation : 
childAction : 
descr : 
dn : uni/tn-tz/BD-BD2/dhcplbl-tz-DHCP_Relay
extMngdBy : 
lcOwn : local
modTs : 2025-09-11T16:30:03.016+00:00
monPolDn : uni/tn-common/monepg-default
nameAlias : 
owner : tenant
ownerKey : 
ownerTag : 
rn : dhcplbl-tz-DHCP_Relay
status : modified
tag : yellow-green
uid : 15374
userdom : :all:

This shows the DHCP Label object associated with the BD.

If the DHCP Relay Policy is correctly configured, it is going to have a child object where the DHCP Label is consumed, which can be verified with:

APIC# moquery -c dhcpRelayP -f 'dhcp.RelayP.dn*"tz.*Relay"' -x rsp-subtree=children rsp-subtree-class=dhcpRtLblDefToRelayP
Total Objects shown: 1

# dhcp.RelayP
name : tz-DHCP_Relay
annotation : 
childAction : 
descr : 
dn : uni/tn-tz/relayp-tz-DHCP_Relay
extMngdBy : 
lcOwn : local
modTs : 2025-09-11T16:10:56.421+00:00
mode : visible
monPolDn : uni/tn-common/monepg-default
nameAlias : 
owner : tenant
ownerKey : 
ownerTag : 
rn : relayp-tz-DHCP_Relay
status : modified
uid : 15374
userdom : :all:

# dhcp.RtLblDefToRelayP
tDn : uni/bd-[uni/tn-tz/BD-BD2]-isSvc-no/dhcplbldef-tz-DHCP_Relay
childAction : deleteNonPresent
dn : uni/tn-tz/relayp-tz-DHCP_Relay/rtlblDefToRelayP-[uni/bd-[uni/tn-tz/BD-BD2]-isSvc-no/dhcplbldef-tz-DHCP_Relay]
lcOwn : local
modTs : 2025-09-11T16:30:03.106+00:00
rn : rtlblDefToRelayP-[uni/bd-[uni/tn-tz/BD-BD2]-isSvc-no/dhcplbldef-tz-DHCP_Relay]
status : 
tCl : dhcpLblDef

DHCP Packet Trace

ACI logs all DHCP packets sent to the CPU in trace files, which can be analyzed to troubleshoot the DHCP Discover, Offer, Request, and Acknowledge (DORA) process. Use this command to view DHCP packet traces:

show dhcp internal event-history traces

Tip: A single DHCP packet generates over 100 trace entries. It is highly recommended to use grep with regular expressions to filter relevant output for efficient analysis.

During DHCP trace analysis in Cisco ACI, several key attributes can be confirmed to ensure proper DHCP relay operation:

• Add circuit id suboption:

Indicates that DHCP Option 82 is being added to the packet, which is essential for relay agent information.

• gi address:

Represents the DHCP server IP address configured in the DHCP Relay Policy.

• Helper address:

The SVI IP address used by the relay to reach the DHCP server.

• Client and Server are in the same VRF:

Confirms that both DHCP client and server belong to the same VRF context.

• msg:

The type of DHCP message observed, such as Discover, Offer, Request, or Ack.

• ctx name:

The VRF name where the DHCP Label policy is configured.

• Smac:

The MAC address of the DHCP client.

• UDP src/dst port:

The DHCP ports used, typically 68 for clients and 67 for servers.

Leaf101# show dhcp internal event-history traces | grep -A34 -B70 "00 50 56 b7 33 ee" | egrep "(Rec.*pkt.*intf|ip add|UDP|packet vlan|IfIndex|interface:|[DS]mac|ctx.*is.*:|Pkt.*ID|relay_handle.*(ifindex|msg|from.*ctx)|relayback|relay_send.*(ifindex|Client.*Server)|Adding option82|Mac addr|dhcp_get_vlan|Add.*suboption.*epg_vnid|Helper|Outgoing|gi.*is|Cross-vrf|Sending.*Server|Relaying.*DHCP)" | head -29

2) 2025 Sep 11 04:14:46.660433 _relay_handle_packet_from_pkt_mgr: 480 : Relaying the DHCP pkt on intf: Vlan24
28) 2025 Sep 11 04:14:46.659985 _relay_add_circuitid_rmtid_msiteinfo: 3354 : Add circuit id suboption: if_index: Ethernet1/45 (1a02c000) , svlan: 24, option def id: 0 epg_vnid 8529.
••
31) 2025 Sep 11 04:14:46.659934 _relay_add_option82: 3151 : Mac addr is 28:6f:7f:eb:54:9f
32) 2025 Sep 11 04:14:46.659930 _relay_add_option82: 3147 : Adding option82 suboptions
35) 2025 Sep 11 04:14:46.659924 _relay_send_packet: 1975 : gi address is 172.16.18.1
••
37) 2025 Sep 11 04:14:46.659921 _relay_send_packet: 1965 : Helper address is 172.16.18.100
38) 2025 Sep 11 04:14:46.659918 _relay_send_packet: 1956 : Client and Server are in the same VRF
39) 2025 Sep 11 04:14:46.659793 _relay_send_packet: 1898 : ifindex is Vlan24
40) 2025 Sep 11 04:14:46.659786 _relay_send_packet: 1833 : dhcp_relay_send_packet: relayback_ifindex is Ethernet1/45
••
42) 2025 Sep 11 04:14:46.659730 _relay_handle_packet_from_pkt_mgr: 447 : DHCPDISCOVER msg
43) 2025 Sep 11 04:14:46.659728 _relay_handle_packet_from_pkt_mgr: 438 : ifindex is Vlan24
••
61) 2025 Sep 11 04:14:46.657274 _snoop_handle_istack_packet: 1763 : ctx name is tz:VRF1
64) 2025 Sep 11 04:14:46.657062 _snoop_handle_istack_packet: 1751 :  Smac = [00 50 56 b7 33 ee ]
65) 2025 Sep 11 04:14:46.657057 _snoop_handle_istack_packet: 1749 : Dmac = [ff ff ff ff ff ff ];
68) 2025 Sep 11 04:14:46.657050 _snoop_handle_istack_packet: 1737 : Logical interface: Vlan24
72) 2025 Sep 11 04:14:46.657044 _snoop_handle_istack_packet: 1721 : Physical interface: Ethernet1/45
••
86) 2025 Sep 11 04:14:46.657024 _snoop_handle_istack_packet: 1669 : UDP src port 68 UDP dst port 67
88) 2025 Sep 11 04:14:46.657021 _snoop_handle_istack_packet: 1577 : destination ip address 255.255.255.255
89) 2025 Sep 11 04:14:46.657018 _snoop_handle_istack_packet: 1574 : source ip address 0.0.0.0
95) 2025 Sep 11 04:14:46.656991 _snoop_handle_istack_packet: 1533 : Received pkt on Vlan 25  intf Ethernet1/45

Common Issues

Issue 1: L3out silent drop

• Problem

When an L3Out uses a vPC interface to form a neighbor relationship with an adjacent router, each switch sends packets using its own vTEP. If the vPC peer receives a DHCP offer with a vTEP address that is not its own, it forwards the packet through the fabric, causing the originator vTEP to drop it silently without fault logs.

To check for such drops, use this command:

show dhcp internal event-history traces | egrep “(failed|Drop).*packet" | head

Leaf101# show dhcp internal event-history traces | egrep "(failed|Drop).*packet" | head

53) 2025 Sep 10 04:14:26.685020 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
172) 2025 Sep 10 04:14:23.792669 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
239) 2025 Sep 10 04:14:22.516679 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
444) 2025 Sep 10 04:14:17.055216 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
563) 2025 Sep 10 04:14:14.450437 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
736) 2025 Sep 10 04:14:09.056993 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
803) 2025 Sep 10 04:14:07.344467 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
906) 2025 Sep 10 04:14:06.290135 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
1025) 2025 Sep 10 04:14:03.770388 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.
1094) 2025 Sep 10 04:14:03.234017 _snoop_handle_istack_packet: 1881 : Drop DHCP DISCOVER/REQUEST packet because it is for a BD SVI, and recvd from fabric facing intf.

• Solution

Navigate to Tenants > [ tenant name ] > Networking > L3outs > [ L3out name ] > Logical Node Profile > [ LNP name ] > Logical Interface Profile [ LIP name ] > SVI > [ SVI policy]

Once there, create or open Secondary IP address and enable Enable for DHCP Relay chekbox.

This is going to force the messages to be sent using the vPC vTEP address instead of the local vTEP and packets get to be forwarded as expected.

Issue 2: DHCP server does not support Option 82

Option 82 is critical in VXLAN environments like ACI, creating a circuit between the source leaf and destination based on vTEP addresses. It includes:

• Circuit ID: Incoming interface, VLAN, and EPG VNID where the DHCP discover is seen.
• Remote ID: TEP address of the switch that received the discover.

If Option 82 is missing, DHCP relay packets are dropped. Validate Option 82 presence on the leaf connected to the DHCP server by checking the DHCP trace logs for errors indicating missing Option 82.

This command validates the leaf switch where the DCHP server is connected receives the offers with a valid DHCP option 82

Leaf101# show dhcp internal event-history traces | egrep “(failed|Drop).*packet" | head

67) 2025 Sep 10 05:16:52.336785 _relay_handle_packet: 1478 : dhcp_relay_handle_packet: DHCP UDP failed to relay packet back to client - Unknown error -1
68) 2025 Sep 10 05:16:52.336772 _relayback_response: 929 : dhcp_relayback_response : option 82 not present. Drop the packet
479) 2025 Sep 10 05:11:07.308085 _relay_handle_packet: 1478 : dhcp_relay_handle_packet: DHCP UDP failed to relay packet back to client - Unknown error -1
480) 2025 Sep 10 05:11:07.308073 _relayback_response: 929 : dhcp_relayback_response : option 82 not present. Drop the packet
891) 2025 Sep 10 05:10:22.312386 _relay_handle_packet: 1478 : dhcp_relay_handle_packet: DHCP UDP failed to relay packet back to client - Unknown error -1
892) 2025 Sep 10 05:10:22.312374 _relayback_response: 929 : dhcp_relayback_response : option 82 not present. Drop the packet
1303) 2025 Sep 10 05:09:37.309888 _relay_handle_packet: 1478 : dhcp_relay_handle_packet: DHCP UDP failed to relay packet back to client - Unknown error -1
1304) 2025 Sep 10 05:09:37.309874 _relayback_response: 929 : dhcp_relayback_response : option 82 not present. Drop the packet
1715) 2025 Sep 10 05:08:52.295721 _relay_handle_packet: 1478 : dhcp_relay_handle_packet: DHCP UDP failed to relay packet back to client - Unknown error -1
1716) 2025 Sep 10 05:08:52.295709 _relayback_response: 929 : dhcp_relayback_response : option 82 not present. Drop the packet

References

DHCP Relay deep troubleshooting in ACI Fabric - TACDCN-2017