"Disconnected” status in a CMA

[Matlu]

Hello,

I have a problem between a MLM and a VSX.
Currently, 1 box of my VSX is filling its /var/log path on a recurring basis.
And as seen so far with the TAC, it is because the VS's that are in our VSX Cluster, are “seeing” their “Virtual Log Server” as “DISCONNECTED”, but this has happened from one moment to another.

TAC has not yet found a possible root-cause of the problem.
I have tried to restart the log related services from the MLM, applying an “evstop;evstart” but the problem continues.

I share for the post some data that I have been able to collect from the MLM.
The problem is that our main VSX cluster box is recurrently exceeding >90% usage of the /var/log path, and this should not happen because it is configured to forward the logs to the Log Server.

Any idea how to fix it please?
Thanks

[Lloyd_Braun]

not sure what version you're on but i don't think evstop;evstart will bounce all the clm services, that is just smartevent. did you try mdsstop_customer <clm name> mdsstart_customer <clm name> ?

[Matlu]

I'm on version R81.20 with JHF Take 82

This command is applied ... mdsstop_customer <clm name> mdsstart_customer <clm name> .... from the main MDS right?

Can the “;” character be used?
#mdsstop_customer <clm name>; mdsstart_customer <clm name>

[the_rock]

Yeah, you can do that.

Andy

[Matlu]

Do you know any command that in real time can give us a signal, if a particular VS, for example VS 2, is sending logs to the “virtual log server”?
tcpdump here is an option? or a fw monitor for this?

[the_rock]

Yes, you can do tcpdump on port 257.

[Amir_Senn]

I don't remember trying this on VSX but "cpstat fw -f log_connection" shows to which log servers they are sending logs.

In general for your issue, I recommend a few things:

a) Check if you have log forwarding configured on your VS. If not - configure. It should trigger sending local logging to log servers

b) See that the IP addresses from/to VS/log server are routed properly. In certain scenarios install policy can work well even if logging has issues arriving. Fetching the policy manually from VS/GW suffers in the exact same manner AFAIK so you can also validate this by trying to fetch it.

[Matlu]

Hello,

I executed the #mdsstop_customer <IP address or Name of Domain Management Server> .. commands mentioned in the documentation, and the recommendation I also received here, and what I observe is that now “nothing” appears connected.

As if there were no FW connected to the “Virtual Log Server”.

Is this normal?

Is it possible to try in some way to make the connected FW appear?

I attach a txt as proof of what I am saying.

Thank you.

[the_rock]

Right, but if you start it again, what does it show?

Andy

[Matlu]

I have already run it, repeatedly.
But when I check the “Virtual Log Server”, it doesn't even appear “Disconnected”, it just disappeared.
I am attaching the result in the post.

[the_rock]

Then Im not 100% sure...are you able to reboot MDS?

Andy

[Matlu]

Restart the box as well (MDS), and the behavior is maintained.

The problem is only with a “Virtual Log Server” because the rest of the virtual servers are working fine.

It is very strange.

[the_rock]

What did TAC say?

Andy

[Matlu]

They can't find the problem. 😐
I have a question, in which file are the events related to everything related to “Logging” saved?
It is in fwd.elg, right?

[the_rock]

You got it. You can also check /var/log/audit dir, but thats more for changes.

Andy