R80.40 snmpv3

[Roy_Smith]

Hi

Just wondering if anyone else as seen an issue with SNMPv3 after upgrading to R80.40?

We have just upgraded our MDS servers and VSX clusters from R80.30 to R80.40 with JHF take48. We monitored the devices and virtual servers using SNMPv3. After upgrading to R80.40, our monitoring server, which is Solarwinds, cannot connect to the VSX clusters and virtual servers. However, it can connect to the MDS servers fine. 

If I enabled SNMP v2 on the VSX gateways, our Solarwinds server can connect to them. They just cannot connect using v3. Has anyone seen this? Is it likely to be a bug?

Thanks
Roy

[Martin_Valenta]

 how does your snmp settings looks like on vsx gateway?

[Roy_Smith]

Hi Martin

The SNMP settings are below. Other than using mode vs and vs-direct-access, all other settings are the same as on the MDS servers: 

set snmp mode vs
set snmp vs-direct-access on
set snmp agent on
set snmp agent-version any
add snmp traps receiver ***** version v3
add snmp usm user ***** security-level authPriv auth-pass-phrase-hashed ***** privacy-pass-phrase-hashed ***** privacy-protocol AES authentication-protocol SHA1
set snmp traps trap authorizationError disable
set snmp traps trap biosFailure enable
set snmp traps trap clusterXLFailover disable
set snmp traps trap coldStart enable
set snmp traps trap configurationChange disable
set snmp traps trap configurationSave disable
set snmp traps trap fanFailure enable
set snmp traps trap highVoltage enable
set snmp traps trap linkUpLinkDown enable
set snmp traps trap lowDiskSpace enable
set snmp traps trap lowVoltage enable
set snmp traps trap overTemperature enable
set snmp traps trap powerSupplyFailure enable
set snmp traps trap raidVolumeState disable
set snmp traps trap vrrpv2AuthFailure disable
set snmp traps trap vrrpv2NewMaster disable
set snmp traps trap vrrpv3NewMaster disable
set snmp traps trap vrrpv3ProtoError disable
set snmp contact *****
set snmp location *****
set snmp traps advanced coldStart reboot-only off

[shlomip]

Hi @Roy_Smith ,

We are looking into this. will update once i will have more information.

[JanVC]

I'm missing the line where it says:

set snmp usm user *** vsid 0-34,36,39-43

[Roy_Smith]

Jan

You are correct, the line was missing, probably due to me deleting and recreating the snmp user as part of my troubleshooting. We have 2 VSX clusters that this happened on, and the line was still there on the other cluster. 

Anyway, when I put the line 

set snmp usm user ***** vsid 0-13

I get the message:

NMSSNM9999 Timeout waiting for response from database server.

And when I check the config, I see the line as:

set snmp usm user ***** vsid 0-4

If I run the command again, I get the same error but then the config says vsid 0-9. And I'm still unable to connect from the Solarwinds server.

[michaelsc]

Can you please run the snmpwalk command locally on the VSX machine and see what response you get?

[Roy_Smith]

Michael

When I run snmpwalk, the response I get is:

snmpwalk: Unknown user name

[assafav]

To overcome this issue follow sk168180 . We are working on fix that will be part of future R80.40 JHF.

[Roy_Smith]

I followed the instructions in sk168180. This seems to work until I switch back to snmp mode VS. In default mode, I can successfully run snmpwalk and get a connection from our monitoring server to the VSX chassis, i.e. VS 0. But when I run "set snmp mode vs", snmpwalk fails with the same "snmpwalk: Unknown user name" message and the monitoring server loses connection. 

If I leave it in default mode, I cannot monitor the virtual systems, which is what we need to do.  

[mmunford_5701]

Hi Roy,

I just upgraded a R80.40 VSX and see the same issue that have reported here. I also tried the steps in SK168180 and snmpwalk responds correctly in default mode and then fails again in vs mode.

[Roy_Smith]

It's always good to know it's not just you with the problem. Not had any update from Check Point yet on fixing snmpv3 to work in VS mode. 

[assafav]

Hi,

sk168180 tested again in our lab and overcome this issue. I suggest try it once more , and if not working contact CP support for further analysis and assistance

[Friedrich_Recht · Accepted solution]

I had an similar issue with snmpd on an R80.40-JHF Take 48 VSX-Cluster.
The snmpd settings in GAIA are "mode vs" and "vs-direct-access off".

When i add an snmpv3 USM-User in GAIA the access with this User fails with "unknown user" after a some seconds.
I think the reason for this is that the multiple snpmd processes for VS0 and the virtual systems are overwriting the current engineID from the net-snmp persistent file /var/lib/net-snmp/snmpd.conf and then it does not match the engineID of the USM-User.
(The engineID ist random.)

I could only solve this with

1.) Stopping th snmpd-agent and all snmpd processes.
2.) Deleting /var/lib/net-snmp/snmpd.conf
3.) Deleting the USM-User in GAIA
4.) Setting the following entries in /etc/snmp/userDefinedSettings.conf.
engineIDType 3
engineIDNic Mgmt
5.) Adding the USM-User in GAIA
6.) Starting the agent
7.) Checking the file /var/lib/net-snmp/snmpd.conf (matching engineID)

[Kaspars_Zibarts]

**bleep**, got the same issue after our upgrade too, take 78. Tried @Friedrich_Recht method but did not work for me, still different engineIDs

@assafav - sk168180 is not available I'm afraid

This is pretty bad as we will be blind about box performance when morning comes!

[Kaspars_Zibarts]

Bingo you saved my day (night!) @Friedrich_Recht !

Had to add one additional step - when creating user, snmp should be in default mode, not vs. So:

1.) Stopping th snmpd-agent and all snmpd processes.
2.) Deleting /var/lib/net-snmp/snmpd.conf
3.) Deleting the USM-User in GAIA
4.) Setting the following entries in /etc/snmp/userDefinedSettings.conf.
engineIDType 3
engineIDNic Mgmt

5.) set snmp mode default

6.) Adding the USM-User in GAIA (don't forget to grant access to VSes)

7.) Starting the agent

8.) set snmp mode vs

9.) Checking the file /var/lib/net-snmp/snmpd.conf (matching engineID)

[Roy_Smith]

Guys

I'm afraid these steps have not resolved my issue. I followed them and now I have "snmp mode vs" I can monitor VS0, which I could not before. However, I still cannot connect to the other virtual systems. When I look at the snmpd.conf file EngineID for the user and the oldEngineID are the same. 

I'm still on JHF Take 48, but interesting to know the issue happens with Take 78 as well. I'm still waiting for this to get fixed. 

Thanks
Roy

[Guy_Grundman]

Hi,

An issues with similar symptoms was fixed in R80.40 JHF take 69.

[Kaspars_Zibarts]

Did you try to create user in default SNMP mode and then change it to VS? Step 5 and 8

[Kaspars_Zibarts]

Yapp, just created new test users and all is working good on T78 after implementing initial fix described above

Setting the following entries in /etc/snmp/userDefinedSettings.conf.
engineIDType 3
engineIDNic Mgmt

[redcrow]

Same problem here. Unfortunately not solved with community solution. Furthermore I've just updated to ongoing 80.40 T87, nothing changed: "Unknown user name"

[redcrow]

These are steps that worked for us (the order is fundamental):

# clish
> set snmp agent off
> exit
# mv /var/lib/net-snmp/snmpd.conf /home/admin
# clish
> delete snmp usm user youruser
> save config
> exit
# vim /etc/snmp/userDefinedSettings.conf
engineIDType 3
engineIDNic Mgmt
# clish
> set snmp mode default
> set snmp agent on
> set snmp mode vs
> add snmp usm user youruser security-level authNoPriv auth-pass-phrase yourpassword authentication-protocol MD5
> set snmp usm user youruser vsid all
> save config
> exit

[genisis__]

When clicking on the above SK link I get the following:

Sorry, this solution is deleted and can only be viewed by Check Point employees.

Can you please make this public available.

[Henrik_Noerr1]

Also know that VS monitoring has changed. There is no longer a vsxStatusCPUUsagePerCPUTable and other vsxStatus related tables in the MIB.

These are now found in ie. CHECKPOINT-MIB::fwInstancesCPU

Which is actually really nice so we can monitor the individual corexl instances.

/Henrik

[Johan_Rudberg]

Hello just upgraded our VSX gateways to R80.40 with HFA Take 91 and we have this snmp v3 issue on one of the gateways. However on the otherone it works just fine, and the configuration is idenical to both gateways. 

[michaelsc]

Hi Johan,

A few questions to better understand the problem:

Are you getting the 'unknown user' error?

Does the error happen after deleting and re-adding the snmp user?

Does the file /var/lib/net-snmp/snmpd.conf get updated when deleting and re-adding the snmp user?

Can you please share the content of the file /var/lib/net-snmp/snmpd.conf after deleting and re-adding the snmp user?

[Johan_Rudberg]

Yes I get the unkown user error on one of the gateways as of R80.40 JHFA Take 91

[Roy_Smith]

Hi

Finally got around to installing JHF89 but I still cannot get snmp monitoring of my virtual systems. I've now gone through the deletion/addition of snmp user many times with no luck. I can monitor VS0 and not the virtual systems. 

I noticed sk168180 is not available any more, which is curious. 

I'm not too familiar with using snmpwalk but when I try the following:

snmpwalk -v3 -l authPriv -u vsxsnmp -a SHA1 -A *** -x AES -X *** <IP or localhost> 1.3.6.1.2.1

I get "Timeout: No Response from localhost"

Any additional help would be appreciated

Thanks
Roy

[michaelsc]

Hi Roy,

The fix for the 'unknown user' error was integrated in R80.40 JHF as of take 91.

sk168180  was not effective, therefore was deleted.

As for the timeout issue, does it happen in snmp default mode as well as vs mode?

[Roy_Smith]

Hi Michaelsc

Take 91 was not available at the time of planning the upgrade. I'll look at getting it installed but it will be in January now. Hopefully that will resolve my issues.

The snmpwalk error happens in both default and vs modes

Thanks
Roy