Strange SNMP problems on GAIA

[NicoSeuss]

Hi guys,

since some time (can't tell exactly when it started) we encounter some strange problems with SNMP.

When I do an snmpwalk, it times out after a few hundred lines. Some attempts provide more, some less lines on the same machine.

However, that is not on all machines. E.g. on a cluster one machine answers just fine, the other don't.

All machines are on R80.40. Hardware is Check Point 5800 and Check Point 6200P. 

Most of the machines do NOT run VSX.

I made some tests and it looks like every attempt stops around MIB .1.3.6.1.2.1.25.3.2.1.1 

There is no special SNMP configuration, and identical on all machines:

Click to Expand

set snmp mode default
set snmp agent on
set snmp agent-version v3-Only
add snmp interface Mgmt
add snmp interface bond1
add snmp usm user FWxxxxx security-level authPriv auth-pass-phrase-hashed xxxxxxxx privacy-pass-phrase-hashed xxxxxxxx privacy-protocol DES authentication-protocol MD5
set snmp traps trap authorizationError disable
set snmp traps trap biosFailure disable
set snmp traps trap clusterXLFailover disable
set snmp traps trap coldStart disable
set snmp traps trap configurationChange disable
set snmp traps trap configurationSave disable
set snmp traps trap fanFailure disable
set snmp traps trap highVoltage disable
set snmp traps trap linkUpLinkDown disable
set snmp traps trap lowDiskSpace disable
set snmp traps trap lowVoltage disable
set snmp traps trap overTemperature disable
set snmp traps trap powerSupplyFailure disable
set snmp traps trap raidVolumeState disable
set snmp traps trap vrrpv2AuthFailure disable
set snmp traps trap vrrpv2NewMaster disable
set snmp traps trap vrrpv3NewMaster disable
set snmp traps trap vrrpv3ProtoError disable
set snmp contact "xxxxxxxxx"
set snmp location "xxxxxxxxxx"
set snmp traps advanced coldStart reboot-only off

Strange thing is, on machines, where everything works fine, I get around e.g. 14'000 lines of SNMPWALK, on machines where timeouts occure, sometimes I get above 65'000 or even over 100'000 lines back, before the timeout.

This way, one snmpwalk takes over half an hour...

One of the questions is: Why do I get so much different outputs on similar devices of a cluster?

BTW: Increasing the timeout setting of snmpwalk does not help (it simply takes much longer).

Versions are the same between the machines. E.g.

Click to Expand

This is Check Point CPinfo Build 914000215 for GAIA
[CPFC]
No hotfixes..

[MGMT]
HOTFIX_R80_40_JUMBO_HF_MAIN Take: 94

[IDA]
No hotfixes..

[FW1]
HOTFIX_GOT_TPCONF_AUTOUPDATE
HOTFIX_R80_40_MAAS_TUNNEL_AUTOUPDATE
HOTFIX_R80_40_JUMBO_HF_MAIN Take: 94

FW1 build number:
This is Check Point's software version R80.40 - Build 118
kernel: R80.40 - Build 104

[SecurePlatform]
HOTFIX_R80_40_JUMBO_HF_MAIN Take: 94

[PPACK]
HOTFIX_R80_40_JUMBO_HF_MAIN Take: 94

[CPinfo]
No hotfixes..

[AutoUpdater]
No hotfixes..

[CVPN]
HOTFIX_R80_40_JUMBO_HF_MAIN Take: 94

[CPUpdates]
BUNDLE_GENERAL_AUTOUPDATE Take: 12
BUNDLE_CPSDC_AUTOUPDATE Take: 19
BUNDLE_CORE_FILE_UPLOADER_AUTOUPDATE Take: 11
BUNDLE_GOT_TPCONF_AUTOUPDATE Take: 97
BUNDLE_R80_40_JUMBO_HF_MAIN_SC Take: 100
BUNDLE_HCP_AUTOUPDATE Take: 48
BUNDLE_R80_40_MAAS_TUNNEL_AUTOUPDATE Take: 44
BUNDLE_INFRA_AUTOUPDATE Take: 52
BUNDLE_DEP_INSTALLER_AUTOUPDATE Take: 23
BUNDLE_R80_40_JUMBO_HF_MAIN Take: 94

[CPDepInst]
No hotfixes..

[hcp_wrapper]
HOTFIX_HCP_AUTOUPDATE

[DIAG]
No hotfixes..

[core_uploader]
HOTFIX_CHARON_HF

[cpsdc_wrapper]
HOTFIX_CPSDC_AUTOUPDATE

A while ago everything was fine. We updated to R80.40 from 77.30 in the past, but as far as I see, the problems started a while after that, so I don´t see a correlation here.

Any ideas? 

Please let me know, if you need more/special information.

Thanks a lot in advance!

Best regards

Nico