IPS Not Prevented

[biskit]

I'm repeatedly seeing an issue and wonder if anyone has any thoughts on this?

Each month I run a scheduled report on IPS.  I often get one or two things show in the report as "Not Prevented".  They are different IPS protections each time, so this is just an example using last month's report.

Notice one occurrence where it was "not prevented".  

Drill down in the logs and I see this.  Pages of "Prevent" for this particular protection, and a single random entry where it wasn't prevented.

When I open the log card I get this.  Note that when this happens there is no "destination" IP logged.

The packet capture mentioned in the log card doesn't exist anywhere (as per sk120773).

Last month I opened an SR with TAC for the same issue and they basically threw their hands in the air and said "dunno!".  They asked me to run tcpdump to capture it next time so they can investigate.  I replied saying it only happens once or twice a month, I don't know the source, or the protection that is going to fail, and there is no destination when it happens, so the only option is to leave tcpdump running with no filters, and wait for it to happen.  Not at all feasible.

This happened previously on R80.30.  I have since put in new gateway appliances on R80.40 T78 and it's still happening.

So before I go back to TAC and get the "dunno!" response again, does anybody have any ideas why this issue is happening?  If there's no destination IP in the packet, how it is getting to me in the first place?  

Thanks,

Matt

[TP_Master]

Hi @biskit 

Can you send me (privately? if you want) the SR# and other details you gathered. I would look into the logs first, possibly this is a glitch in the view (but that IPS did prevnt).

[biskit · Accepted solution]

Thanks again for the excellent help of you and your team.  The issue is now resolved. 

For the benefit of anyone else having the same problem, it was because of a communication issue between the gateway and the management/log server, whereby only part of the data was sent up to the management server for some reason.  The gateway in fact had several random log files left on it over time which hadn't properly uploaded to the Management server (the Mgmt server is out on the Internet so there is lots of potential for a short break in communication to prevent the log file uploading fully).  After copying the log files from the gateway to the management server and re-indexing, the missing data is now showing in the logs.

My particular situation was complicated by the fact that the gateway appliances were replaced between the last event happening and now, so we couldn't just do a "fetch" from the gateway.  Lucky I still had the old appliances untouched, so I was able to power this up offline and manually copy the logs files over.  This was all done on a remote session with R&D who did all sorts of other technical things as they went along.

The ultimate recommendation was to enable the Log Forwarding on the gateway, which I've now set to happen every 3 hours, so even if there is a blip in communication the gateway will re-try sending the logs to the Mgmt.  In the end a very easy fix.

Thanks again to CP for their time on the remote sessions 👍