Sounds like the RDP connection may be timing out (perhaps inappropriately) and the RDP client is reconnecting, first try increasing the timeout for the Remote_Desktop_Protocol service from the default of 3600. If that doesn't have an impact on the problem, enable TCP State Logging as specified here: sk101221: TCP state logging to get some more information about what is happening to those connections.
--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com
Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com