Long Buffer queue length

[Steve_Moran1]

I am seeing long buffer queue length's on my external interfaces of my gateways (only external).  As well output drops and pause input on the cisco 3850 10gb port connected to the device.

ifconfig tells me the txqueue length is 1000 (whatever units that is) while the cisco's max txqueue length is 2000.

Is it possible/advisable to make these match?  I looked into mulitqueue, which is currently disabled, and the cpu utilization on the proc alloctated to SND is 80-90% idle, so I wouldn't think its a candidate for multiqueue.  This is on a 15600, 10gb port, r80.10 take 112.  

I attached some screen shots of the info on the switch and the fw.  I did open a tac case, but cisco says its checkpoint's issue, while checkpoint says its cisco's issue.  

[PhoneBoy]

Wouldn't it be better to compare TX queue on one side to RX queue on the other?

I would think you'd want these to be similar, but I will defer to experts.

[Hugo_vd_Kooij]

The TX queue is a buffer between your core and the NIC. The RX queue is a buffer between your NIC and your core.

So you don't have to match them.

If your CPU is busy you may need to increase RX buffers to make sure you don't loose the data. On the other hand if you can do hardware handshaking you can just ask the other side to hold the phone for little while but it means they propably need larger TX buffers.

[_Val_]

I believe Dameon Welch Abernathy was talking about TX on the transferring device compared with RX on the receiving one. Of course it does not make sense comparing TX and RX on the same NIC

[Hugo_vd_Kooij]

Neither was I.

I was considering the path from core to core between 2 devices. The concept in switches is not that much different but it goes by other names then the ones I have used.

[_Val_]

I see your point, thanks for clarifying. 

[Timothy_Hall]

Matching the buffer sizes between two devices is not necessary, and in general you should only tune interface ring buffer sizes if you are having problems and even then only as a last resort.

A "pause input" is simply the Check Point attempting to implement Ethernet flow control because its RX ring buffer is nearly full.  Based on the two screenshots it looks like everything is fine.  Multi-Queue does not appear to be necessary either.  Just because the Check Point is requesting a pause (but not actually dropping any packets apparently) doesn't mean you need to necessarily tune anything.

The total output drops on the Cisco may be related to the class-based queueing QoS you are applying to the Cisco interface, please provide more information about that setup as traffic with low priority may be getting dropped when the queue fills up and incrementing that counter.

Also please provide the output of "netstat -ni" on the Check Point side and "show buffers" on the Cisco side. 

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

[Steve_Moran1]

We found that there was a device connected to the 3850 stack, which that device was configured as a port channel, however, on the 3850, it was just 2 ports.  There were 28 trillion drops on that port in like 12 hours.  I suppose its possible that blew out the buffers across the switch.  After correcting that, we're not seeing drops as bad.  We cleared the counters on the cisco last night after making that config change, and there have only be 108 pause input on the cisco. 

I believe according the marketing numbers, the 15600 is good for 13.6gb  throughput in threat prevention configuration. We hit high water marks of like 250mb on that 10gb interface.  So why would the box tell the switch to slow down?  

In regards to the QOS policy applied to the cisco port....

policy-map Firewall-ports

 class class-default

  bandwidth percent 100

  queue-buffers ratio 100

interface  tenGigabitEthernet 2/0/1

 service-policy output Firewall-ports

[Steve_Moran1]

We found that the 3850 can't handle the microburst traffic and its just being swamped.  We've been able to tune the switch to handle the traffic much better than it was, and i increased rx ringsize on the appliance and it hasn't dropped anything else, though the switch still suffers.  We're going to set up a port channel in order to increase the switch's ability to handle the traffic.  

[Steve_Moran1]

Here is the guide we used from cisco to tune the switch, in case anyone else sees something like this

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-3850-series-switches/200594-Catalyst-38...