DHCP Relay Issue

[Polina]

Hello,

Could you please help me with a DHCP Relay issue? We have a bond interface and several VLAN interfaces added to it. DHCP Relay is configured for each VLANs.
When traffic passes through one cluster node, DHCP Relay works correctly, but when another node becomes active, clients in one VLAN stop receiving addresses from the DHCP server. This issue only affects one VLAN; in all other networks, DHCP Relay works correctly on both cluster nodes.

We've already tried several SK methods to resolve the issue, but nothing works.
The strange thing is that both cluster members are configured identically.

Has anyone seen similar issues? Or are there any solutions for fixing this?

[WiliRGasparetto]

Hi Polina, have you already checked the DHCP traffic?

Check DHCP traffic arriving on the VLAN (client → gateway):

tcpdump -eni bondX.<VLAN> -vv udp port 67 or 68

Check DHCP relay traffic leaving towards the server (gateway → server):

tcpdump -eni <iface-to-dhcp-server> -vv udp port 67 or 68

If on member B you do not see the DHCPDISCOVER broadcast reaching the VLAN sub-interface, then it points to an L2/tagging/trunk issue.

If you can, please share the relevant tcpdump output/logs so we can review them and help you more effectively.

[Polina]

Hello
Currently, we've only checked traffic on the interface (gateway → server). We're awaiting results on the client-facing interface.

The dump shows DHCP Discover and DHCP Offer packets. The DHCP Offer packet arrives at the cluster's VIP address. I've attached a screenshot.

Checked fw ctl zdebug + drop:
fw_log_drop_ex: Packet proto=17 IP_DHCP Server :67 -> VIP address :67 dropped by fw_handle_first_packet Reason: fwconn_key_init_links (INBOUND) failed;

Scenario 3, sk97642, matches our symptoms. We added NAT rules as described in sk. But this did not bring results. Further, sk97642 (scenario 3) suggests changing the table.def file, but we're concerned that this change will disrupt DHCP Relay operation on other subnets where it works correctly.

On both cluster nodes:
# fw ctl get int fwx_dhcp_relay_nat
1

#routed:instance:default:bootpgw:interface:bondХ.Х t
#routed:instance:default:bootpgw:interface:bondХ.Х:primary VIP address
#routed:instance:default:bootpgw:interface:bond77.77:relayto:host: IP_DHCP Server t

Thank you for your answers.

[the_rock]

Based on that drop, did you make sure relevant services/protocols are configured in the rule?

[Polina]

Yes, we did.  We have edited the rules according to the documentation. And the rules are common (for all VLAN subnets). But only for one VLAN DHCP Relay does not work.

We are looking for workarounds to solve the problem.

[Alex-]

Ensure you are following the following SK:

sk104114 - Configuration of IPv4 BOOTP/DHCP Relay using new services

I've seen installations where misconfigured policies with Any or other worked until they didn't. That SK always worked.

[the_rock]

I would definitely open TAC case and do remote.

[WiliRGasparetto]

To tell you the “most likely root cause” without guessing, please send me these 4 pieces of evidence:

1. Version: R80.40 / R81.10 / R81.20 / R82 + JHF/take

2. Screenshot/output of the DHCP Relay configuration for the affected VLAN only (including Primary Address, if configured)

3. Output of fw ctl get int fwx_dhcp_relay_nat on both cluster members

4. Two 20–30 second tcpdumps during failover (good member vs bad member): one on the VLAN sub-interface and one on the interface towards the DHCP server

[the_rock]

This is the abbreviation often used when it comes to dhcp:

Detailed Breakdown of the DORA Process:

• Discover (DHCPDISCOVER): The client sends a broadcast message on the network to locate available DHCP servers.
• Offer (DHCPOFFER): The server receives the request and sends a unicast or broadcast offer containing an IP address, subnet mask, default gateway, and DNS information.
• Request (DHCPREQUEST): The client accepts the offer and broadcasts a request, notifying the server (and others) that it will use that specific IP address.
• Acknowledge (DHCPACK): The server sends a final confirmation (unicast) to the client, authorizing the lease and completing the process. 

Now, here is what I would do...can you run tcpdump/fw monitor and see where this exactly fails? That will 100% tell you why its not working. Also, I would make sure config matches the working firewall. Sometimes, people may forget, specially if they worked long time with another vendor (ie Fortinet for example), that in such case, there is no VIP, everything is replicated automatically from master to backup fw, unlike Check Point, where you would need to ensure it is the same.

[Vincent_Bacher]

Hi Andy,

For the sake of completeness, I would like to clarify something. It cannot be said outright that FortiGate does not have a VIP. Rather, in the standard configuration, there is ONLY the VIP located on the active node, and the nodes themselves do not have their own IP, but this can be configured.
brV

P.S. You really gave me a fright. I currently hear DORA as something completely different and it constantly annoys me, given the effort we're putting into it: Digital Operational Resilience Act.

[the_rock]

Haha...or the kid in us can reference it to Dora the explorer 😂

Anywho, you are absolutely right about Fortigate VIP. I more meant you dont need additional IP like you would in smart console to create cluster object and you would not need to enable management HA (believe they call it for Fortinet), unless you really wanted to be always able to log into backup firewall as well.

[Vincent_Bacher]

I had always preferred to be able to log in directly to both devices. However, as we also have less experienced colleagues, I have started to recommend doing it by default and then just execute ha manage …. To hop to the slave, then less can be broken.

[the_rock]

Honestly, Im the same way. That tripped me once so bad when my colleague and I (at my previous job) were troubloeshooting Cisco failover. We assumed (wrongly obviously) that it worked same way like Check Point, which it does not, so took us a bit of time to get it right. Im sure Cisco TAC guy had a laugh after, but hey, all good...once we got it, all was fine : - )

[the_rock]

Hey @Polina 

Any luck with this?