URL report by user

[Tim_Sueck]

Hey, all. We're using App/URL filtering on R77.30. I'm trying to get detailed external browsing history for a user - which URLs they explicitly visited during a given time frame. Any recommendations for this? I can get the Application Name (allowed_sites) or the App Category (Custom Application) or the destination (an external IP address), but the URL field is blank. I've been looking in SmartView Tracker and in SmartEvent NGSE with no luck. Any suggestions on this?

[Vladimir]

I suspect it is as designed. Consider the very long URLs, spanning the all allowed 2083 characters. it'll take a lot of space to store and display. Whereas Applications and Custom Apps are defined by more complex properties and require less space for identification in the logs.

[PhoneBoy · Accepted solution]

You only get all the URLs visited by ensuring the rule that matches user traffic has is set for "Complete" Log (versus either "Log" or "Extended).

[Vladimir]

Where would one find the "Complete" log settings?

  

Are the only options available.

[Timothy_Hall]

Hi Vladimir,

The naming of the various log options changed between R77.30 and R80 management, then changed again between R80 and R80.10.  Here is a summary of the changes from the third addendum for the first edition of my book:

• Network Log: (Replaced in R80.10 with Log) Generate a log with only the basic network information such as IP addresses and ports (application/category information will NOT be included). On a pre-R80 SMS, this setting is equivalent to setting the Track column to Log in the main rulebase (Firewall tab..Policy), but setting the Track column in the APCL/URLF policy rule to None.

• Log: Includes network-level, application/category, and Content Awareness logging. This setting is equivalent to setting the Track column to Log in the main rulebase, and the Track column to Log in the APCL/URLF policy on a pre-R80 SMS.

• Full Log: (in R80.10 this is called Detailed Log) For pre-R80 gateways, this is equivalent to the Log option described above. For R80.10 and later gateways, this option provides additional logging for application/category, even if an explicit application/category was not specified in the policy rule.

• Extended Log: (R80.10 only) Provides all individual URLs visited for a matching rule, and is the equivalent of setting Complete Log on a pre-R80 APCL/URLF policy rule. This logging option is likely to impact firewall performance and should be used sparingly.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

[Vladimir]

Thanks Tim!

Do you know how the log handles very long URLs?

Is there a limit on characters CP will log in the URL field?

[Timothy_Hall]

No limit that I'm aware of, and I've seen some pretty long URLs with this logging option, at least 512 bytes if not more.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

[PhoneBoy]

Since the original poster said R77.30, I used the name it should show in SmartDashboard (Complete log).

The equivalent in R80+ management is Extended log, as https://community.checkpoint.com/people/thalld401179d-0d5b-369d-a0f2-387c3ef54533‌ noted.

[Tim_Sueck]

Thank you https://community.checkpoint.com/people/dwelccfe6e688-522c-305c-adaa-194bd7a7becc‌ and https://community.checkpoint.com/people/thalld401179d-0d5b-369d-a0f2-387c3ef54533‌!

[Peter_Baumann]

The resulting URL can be viewed in the log at "Resource" but this field is not available when used in a view or Report (??)

[Gaurav_Pandya]

Hi Dameon/ Tim,

We have the same issue in Smart event with version R77.30. In the report, we are getting Application category but not exact URL. 

As you mentioned about Log setting but it is for R80. where we can do same setting in R77.30 

[Timothy_Hall]

I'm assuming you have R77.30 management and gateways.  In your APCL/URLF policy set Track field of the matching rule to "Extended Log" and reinstall policy.  If that doesn't work try setting Track to "Complete Log".

[Gaurav_Pandya]

Hi Tim,

Thanks for the reply.

I have done the changes of track setting as complete log but it did not help. Actually our requirement is to take report of specific application & based on that we should get destination URL/IP. 

As per TAC, there is limitation in R77.30. Need to upgrade to R80.10