Pull GW local logs into MLM after MLM unavailable

[cem82]

Hi

We had to rebuild our log server so multiple GW were logging locally.  Once the MLM was up and running again they started sending logs there however what had been logging locally have stayed on the GWs.  It looks like filenames will likely be the same across multiple GW so can't just SCP them off and run fw repairlog / index on the MLM.

I thought they'd actually get imported over by itself but doesn't appear to be the case.  Does anyone have any suggestions?

Thanks

[Tomer_Noy · Accepted solution]

There is a configuration on the gateway / cluster object that determines if locally written logs will be uploaded automatically to the log server. It appears under "Logs => Additional Logging => Log Forwarding Settings".

Turn on the checkbox, select which log server should get the locally logged files and time interval. You can choose to upload in bulk at midnight, or create a new object for uploading every hour. Since local logging can accumulate to a  lot of data, choose the interval that makes sense to you in terms of latency of getting the files and whether you only want it to happen in off-hours.

Here's a screenshot of how it looks:

We recommend activating this setting.
If you want to do it in bulk for all your gateways, you can do it via a simple script using the Management API / CLI and the "set simple-gateway" or "set simple-cluster" command: https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/set-simple-gateway~v1.9.1%20 

For future versions, we're also looking into making this "on-by-default".

[cem82]

Thanks for that, overlooked that setting which is doing the job thanks! I've found one issue though, in the schedule option I created a new schedule which was 3am to avoid peak traffic and backup time etc. However I found as soon as I pushed policy the files started transferring. Time is correct as well as timezone on MDM / Log Servers / GW and was in the afternoon so had to revert that as it was causing 100% FWD CPU utilisation for extended period of time. After unticking the option to forward the logs and push policy the log files were still being transferred, due to the CPU issues things only returned to normal after temporarily moving to another location but will move back once I know how to get it to do late night.

Also where can you delete the schedules that are there as there's quite a few and also to verify the details of them as can only see the name and not edit them or delete? I've tried looking everywhere I can think of to do this in smartconsole, we're running R81.20 JHF 41.  I've also looked at the API guide and while I can see how to set this object within the GW properties, forward-logs-to-log-server-schedule-name however I can't find that listed anywhere else in the guide to add/show it

[Amir_Senn]

Those are legacy objects, this feature is very old. You won't be able to delete them AFAIK.

I details about the objects definitions using GUIDBedit (see Capture for example).

You can find references for the definitions in show simple GW:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/show-simple-gateway~v1.9.1%20

Search under log-settings (see Capture2)

[cem82]

Thanks @Amir_Senn  I suspected it was very legacy hence why I couldn't find it in smartconsole only dbedit.  Do you have any suggestions as to why even though I selected a schedule for 3am it actually started the transfer immediately as really don't want FWD to run at 100% for extended period of time while this happens.  Is it just that it is so legacy even though it is a required field to enable that it just doesn't honor it and more cosmetic to have to select it?  I did verify in dbedit that it is indeed set to 3am not just called that in the name 🙂

[Tomer_Noy]

The settings are not ignored (even though legacy...).

There is a slight chance that it started immediately after policy installation because it was the first time you activated the feature and it saw that there is a backlog that wasn't handled in the previous scheduled cycle. We'll look into it since it's not the preferred behavior.

For now, if you activate it and push policy a bit before you want the schedule to start, it will probably sort out and future uploads will happen at the schedule.

Also note that if there are a lot of historical local logging files, it may take time to upload them all and it may spill over into working hours. That's why it's best to have this setting on from the beginning.

[Amir_Senn]

Yeah, what he said=)