Hi all,
I've never really in depth troubleshooted secureXL, so I wanted to get some additional insight from the community. I'll be creating a TAC case for it as well.
We recently upgraded our customer from R81.20 to R82 JHF 44.
We did however encounter a major issue with VOIP traffic, specifically the RTP part.
Internally people could call each other, but couldn't hear one another. After some troubleshooting it seems an issue with RTP not being processed correctly on the firewall. Disabling secureXL resolves the issue.
Setup
=============
6200 cluster running R82 JHF 44
fwaccel is currently disabled and running in kppak mode.
firewall itself is in user mode
Kernel: 4.18.0-372.9.1cpx86_64
Voice vlan: 10.0.40.0/22
Voice border gateway: 172.16.51.20 (public IP 176.62.X.X)
Troubleshooting
==============
A VOIP telephone connects to the bordergateway public IP 176.62.X.X, which is natted to the internal IP 172.16.51.20. It establishes a SIP connection (TCP 5060/5061) and determines which UDP high port will be used for the data transfer (RTP) between the two users.
We use custom TCP protocols, to disable inspection/ALG issues. Nothing has changed to this setup prior to the upgrade.
This flow above worked in the R81.20 setup and also works in R82, if the voice user is not connected on an internal network.
Example: User working from home.
Flow: Public IP user <> 176.62.X.X <> 172.16.51.20 <> internal ip user 10.0.40.x/22
Capture of RTP traffic
------------------------
[vs_0][ppak_0] eth4:i[44]: 172.16.51.20 -> 10.0.40.38 (UDP) len=200 id=40609
UDP: 25076 -> 55114
[vs_0][ppak_0] eth4:I[44]: 172.16.51.20 -> 10.0.40.38 (UDP) len=200 id=40609
UDP: 25076 -> 55114
[vs_0][ppak_0] bond1.40:o[44]: 172.16.51.20 -> 10.0.40.38 (UDP) len=200 id=40609
UDP: 25076 -> 55114
[vs_0][ppak_0] bond1.40:O[44]: 176.62.X.X -> 10.0.40.38 (UDP) len=200 id=40609
UDP: 25076 -> 55114
This flow doesn't work in R82 when a user is working locally in the office.
Example: Both users have an IP in 10.0.40.0/22
Flow: 10.0.40.x/22 <> 176.62.X.X <> 172.16.51.20 <>10.0.40.x/22
Capture of RTP trafic
---------------------
[vs_0][ppak_0] eth4:i[44]: 172.16.51.20 -> 10.0.40.133 (UDP) len=200 id=42532
UDP: 26640 -> 55080
[vs_0][fw_2] eth4:i[44]: 172.16.51.20 -> 10.0.40.133 (UDP) len=200 id=42532
UDP: 26640 -> 55080
[vs_0][ppak_0] eth4:i[44]: 172.16.51.20 -> 10.0.40.133 (UDP) len=200 id=42548
UDP: 26640 -> 55080
[vs_0][fw_2] eth4:i[44]: 172.16.51.20 -> 10.0.40.133 (UDP) len=200 id=42548
UDP: 26640 -> 55080
Doesn't get processed after small "i", and gets dropped? by "fw_2".
fw ctl zdebug + drop, doesn't show any drops for these connections.
Disabling secureXL makes this flow work again.
I've tried setting up a fast_accel rule, but this doesn't seem to help. It's also more a quick fix, then an actual solution.
------------------------------------ FIREWALL FAST ACCEL TABLE ------------------------------------
# Source IP Destination IP D-Port Protocol Hit count
---- ------------------ ------------------ ------ -------- -----------
1) 10.0.40.0/22 172.16.51.20/32 any 17 0
2) 172.16.51.20/32 10.0.40.0/22 any 17 11
I'm not too knowledgeable on how the voice traffic actually works in depth.
I would assume if two users in the same subnet are calling each other, that the actual UDP traffic would be sent directly to one another (not passing the firewall), but it seems this isn't the case here and it's all relayed via the border gateway.
It might be related to NAT within secureXL someway, but unsure how to tackle this furhter.
Kr,
Bert
