SmartConsole Disconnect 81.10

[NorthernNetGuy]

Hi all,

I've been having issues for a couple weeks, where i am unable to reliably stay connected to my SMS server with smart console.

I received the message "The Connection with the server was lost. Any unsaved changes will be preserved". This consistently happens after 60s of connection time. If I attempt to open the logs&Monitoring tab, it immediately crashes. Same with Autonomous Policy.

my SMS is on r81.10 and the latest JHF.

Smart console version 410 (tried multiple versions)

I've gone through and removed significant load off of my SMS server, removing over 50% of all logging, I don't see significant load on the unit.

this isn't specific to a single computer, all clients i've tried face the same issue. I've also tried connecting directly from the same switch the SMS server is on, and see the same issue.

SCP connections are slow, and logging in to SSH takes a long time. the web GUI is also slow. once logged in to SSH it responds to commands.

I can pull policy from it fine, and export logs to SIEM are fine. if I disable my log exporting, I see no changes. I can make changes during my 60s connection window, publish and install policy, it just takes several log ins.

I've gone through multiple SK with no success. Hoping someone here might have some insight. 

[_Val_]

Please open a TAC request. Also, check CPU utilization, just to rule out performance issues. 

[NorthernNetGuy]

Hi Val,

I've had a TAC open for a couple weeks with little progress, so I was reaching out for some extra insight. Shortly after i posted I received my 3rd escalation. This tech was immediately able to find several issues.

Our database was corrupt, with bad entries. the domain work session entries were stuck at open state, but the sessions were also entered in as dead. Every 60s the core domain session mgmt svc checks the sessions. Seeing a dead session it would disconnect us, even with an open state. the state would remain open even when disconnected.

After cleaning up the bad DB entries, we found several cert issues, and cleaned those up. 

it appears that the reason for the database corruption was due to how the services interacted when running on low memory. When accessing SWAP and disconnecting from the network with unpublished changes the DB wasn't properly updating. 

As a temporary measure until we upgrade our memory, we've reduced memory load, and changed the FWM process so that logins occur using a different priority. We're also manually cleaning up disc space, as the auto cleanup features are also memory intensive.  

[_Val_ · Accepted solution]

It sounds like you are in a lot of pain. Yes, low RAM and high CPU would most probably get timeouts and disconnects and may cause DB corruption.

I would suggest finding a bigger box for your management server, and trying to restore it from a backup, if you have any. 

[NorthernNetGuy]

Just wrapping up this old thread, but we ended up upgrading our SMS server, which resolved our issues.

[Lesley]

What hardware you use? Is it maybe a VM or?