Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Timothy_Hall
Legend Legend
Legend

Mellanox/NVIDIA ethtool RX ring buffer size request

Can someone who is using the following Mellanox/NVIDIA network cards please run an ethtool -g (interface) from expert mode on them and post the outputs?  I curious to see what the default RX ring buffer size is for these interfaces utilizing the mlx drivers.  Specifically I'm interested in seeing outputs for these connected line speeds:

  • 25Gbps
  • 40Gbps
  • 100Gbps

For Intel, it looks like the default RX ring buffer size for igb is 256 and ixgbe is 512.  Thanks!

Desired output should look like this:

[Expert@A-GW-01:0]# ethtool -g eth0

Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 4096
TX: 4096
Current hardware settings:
RX: 1024
RX Mini: 0
RX Jumbo: 256
TX: 1024

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
8 Replies
Danny
Champion Champion
Champion

Type: Check Point 16600HS Appliance
Line card: Model: CPAC-2-100/25F Type: 2 ports 100GbE QSFP28

Interface: ethsBP1-01
Identifier : 0x11 (QSFP28)
Transceiver type : 40G Ethernet: 40G Base-CR4
Length (Copper or Active cable) : 1m
Vendor name : Mellanox
Product Type : 100GBASE-CR4-1M
Speed : 100G
Check Point SKU : CPAC-DAC-100G-1M-B

 [Expert@CheckMates]# ethtool -g ethsBP1-01
Ring parameters for ethsBP1-01:
Pre-set maximums:
RX: 8192
RX Mini: 0
RX Jumbo: 0
TX: 8192
Current hardware settings:
RX: 1024
RX Mini: 0
RX Jumbo: 0
TX: 1024

Hugo_vd_Kooij
Advisor

We have cards set to a RX buffer of 512 after installing Jumbo Hotfix Take 41 on R81.20 where it was running Take 29 before this.

Actually it was set to 512 on all interfaces of the QLS450 appliance.

Adn we got loads of RX-DROPS

<< We make miracles happen while you wait. The impossible jobs take just a wee bit longer. >>
0 Kudos
John-Haynes
Participant

Ring parameters for eth1-01:
Pre-set maximums:
RX: 8192
RX Mini: 0
RX Jumbo: 0
TX: 8192
Current hardware settings:
RX: 1024
RX Mini: 0
RX Jumbo: 0
TX: 1024

 

Model: Check Point 29100

Line card 1 model: CPAC-2-40/100F-D
Line card 1 type: 2 ports 40/100GbE Fiber Rev 1.0

driver: mlx5_core
version: 5.8-2.0.3 (21 Jul 23)
firmware-version: 28.37.1014 (CP_0000000014)

Transceiver type : 100G Ethernet: 100G Base-SR4 or 25GBase-SR

Vendor name : PROLABS
Vendor OUI : 00:00:00
Vendor PN : 13B8NIY4456
Vendor rev : A2

 

Currently, any setting above 1024 with UPPAK enabled will cause crashes.

Timothy_Hall
Legend Legend
Legend

Thanks, looks like 1024 is more or less the default for 40/100 interfaces.

Currently, any setting above 1024 with UPPAK enabled will cause crashes.

Really?  That is very strange, are you using a Lightspeed appliance?  Are we talking a system-wide panic and reboot here or just crashing of the UPPAK SecureXL process?

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
John-Haynes
Participant

29100 with connectx-7 cards.  USIM coredumps and then it reboots.

0 Kudos
Timothy_Hall
Legend Legend
Legend

Yikes, thanks!

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
MattLeGrow
Employee
Employee

Hi John,

Was traffic passing through the device at the time of the ringsize change?   

As a general comment, the Ringsize settings for devices in User Space mode may differ from prior kernel mode defaults, even for the same card -  these variations are indeed deliberate.  

If not already, I recommend opening a case with Check Point TAC and providing the usim core dumps for analysis, as well as /var/log/usim_x86.elg*  and /var/log/messages* for additional context.  

0 Kudos
Timothy_Hall
Legend Legend
Legend

I couldn't envision how touching a ring buffer setting could ever crash SecureXL, but it looks like UPPAK/USIM mode is digging some tendrils into the Linux networking code in ways it never has before, which as a side effect is impacting use of the standard Linux ethtool command: sk181564: Differences using the 'ethtool' command with SecureXL Kernel Mode (KPPAK) and User Mode (U...

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events