Log Exporter - Inline Layer fields trimming

[Austin35]

Hello Everyone,

Looking for some help as I don't necessarily know where to go for this. Some back story, we recently overhauled our main firewall policy for the company. The old policy was flat with an ordered layer for APP/URL filtering. We moved that policy to a zone based policy using security zones and inline layers. The policy now also includes inline APP/URL filtering.

So, a basic web request flow from a production device to the internet would look something like this

InternalZone -> ExternalZone (Entry Layer)

          Production-WebAccess (Entry Layer) 443/80

                   Any -> Internet -> 443/80 -> Accept

with that in mind we were ingesting roughly 150Gb a day to splunk using log exporter before the change, now we are averaging 225Gb a day.

We are currently going through the logs and seeing what fields we need and which fields we don't need to try and reduce the volume of data. How ever with the new inline layers it replicates some of the fields. See example below.

layer_name=StCloud_Production Network
layer_name=StCloud-InternalToExternal
layer_name=Production-WebAccess
match_id=100
match_id=150995036
match_id=184549378
parent_rule=0
parent_rule=100
parent_rule=150995036
rule_action=Inline
rule_action=Inline
rule_action=Accept
rule_name=Entry Layer - Internal to External
rule_name=Entry Layer - Production HTTP/HTTPS Access
rule_name=HTTP/HTTPS Whitelist - Production

Does anyone know of a way either within log exporter or Splunk where we can trim the first two entries of each? 

[the_rock]

Search for this file on mgmtr server, may contain some of that config.

Andy

targetConfiguration.xml