Logs and Export

[Mohammed_Omin_B]

Even after enabling SmartEvent, we are not getting the logs for moredays say last month or so. It simply says no logs found but when we select the log file it is showing all the logs. However, if we select the log file then for sure it will not be of last month. What should be done in order to get the logs of a particular rule/src/dst/port of last month or so without selecting the log file and when we export the file in csv format it only exports the ones which are visible but we want all of them. 

[Clifton_Watts]

Here is a thread that addresses the export issue. https://community.checkpoint.com/thread/8991-how-to-export-all-result-for-the-smartlog-query-shows 

[Mohammed_Omin_B]

Thanks...

I tried it in SmartView but there are two issues now. First, is that sometimes its showing the logs for a particular rule/src/dst/port and sometimes says "No records found" and the other one is that while exporting the file, its size is in KBs say 111KB and opening it in .csv results in data loss. I am expecting the log file at least in MBs as there has to be huge traffic flowing in there. 

[PhoneBoy]

Pretty sure SmartEvent isn't required for exporting across multiple log files, but Log Indexing must be enabled on your management/log server:

Also what management version are you using?

What data is missing from the CSV?

[Mohammed_Omin_B]

Yep, the requirement was to export all possible logs in one shot along with the Events/ reports generation and found that it can be done with https://mgmt server IP/smartview. when we opened it, it was saying SmartEvent need to be enabled to view it. And after that it all started with different issues. 

last time when we enabled log indexing, mgmt server got hanged and was too slow. So this time, we haven't enabled it.

mgmt version: R80.10

while opening the .csv file, its empty and looks like it is getting corrupted or data loss

Here are some details which might help -

Symptoms:
============
Logs are Missing per rule 

Steps Taken:
============
-Rule wise logs unable to generate the .csv file is getting crash
-While login to Security Policy->Logs->unable to see the logs intermittently.
Getting the error "no logs" .
-Mohammed confirmed that when generated log report from Smart View ->logs are missing and also informed that in legecy smart tracker some of the rule logs are missing.
-#cpwd_admin list -all process are UP and running .
-Having enough disk space and memory verified #df -kh and free -m
-Its listening on port 257 verified #netstat -nap | grep 257
-#tcpdump -nnei any host <Firewall IP> and port 257
Firewall is sending logs to Management Server.
-#cpinfo -y all jumbo_take_112 installed on the appliance.
-#watch -d -n 2 "ls -l $FWDIR/log/fw.log" show the logs are storing on Management Server.

[PhoneBoy]

The only way you can work with log data that spans multiple log files is to have Log Indexing enabled.

What hardware are you running your management server on? How much memory installed?

[Mohammed_Omin_B]

Its deployed in cloud with 8 core Processor 16GB RAM. 

[PhoneBoy]

What "cloud" are you deployed in?

Allocating more RAM might be a good idea.

Disk I/O in the cloud may also be a blocking factor as well.