Hi @KyleW00,
I have no experience with Azure Virtual Desktops ShortPath Public network UDP connections and our VMSS, but as you said, we are using double NAT (one in the LB and one on the GW instance).
It's worth a try to use Floating IP on the LB, which is disabled by default.
By enabling it the LB will not NAT the Destination IP, in this case you will see the Public IP on the Firewall and you can NAT accordingly.
You should be able to modify the load balancing rule to set/unset floating IP without redeploying the whole solution.
MS LB floating ip - https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-floating-ip
Thanks,
Natanel