Log query with Count? for export

[Flashy]

Hello Checkmates

Background - I am working on log analysis to remedy some of our anti-spoofing neglected gateways. With new approach accross our environment I am trying to set antispoofing to "defined by routes". 1st phase is to configure all interfaces into route-based AS, detect mode and after week exporting logs which are only relevant  to FW blade and contains word "spoofing" in message information column - that part works fine. I am, of course, exporting this data from browser-based smartview, as that provides most size for our exported log data - capped at 1M log entries. Excel log data is manually made into pivot tables which gives superior view customization for my needs based on what logs suggest.

Problem - when FW is traffic heavy, the amount of spoofing logs is so high that 1M limit is capped within one hour of log data, this of course does not give proper picture of traffic state, where I would need 7 days of data at least. Excluding particular src or dst host from log query,that are responsible for high amount, before exporting is out of question as there is no way to know this from small log window, or spending 1 hour on log export, reviewing and then excluding some players from it still does not guarantee we will arrive to 7 days overview in next batch..And having more exported log batches ..just not a solution.

Is there some solution to have log query reducing say 3000 log entries consisting of same src, dst and service, but with different timestamps, into 1 log entry, maybe even with information on count? this of course should occur before I export logs so 1M cap for logs wont be dominated by many log entries with same information, and I could have full 7 days insight in theory.

Thank you

[Chris_Atkinson]

sk65298 describes a way to split the export via CLI if encountering file size issues or similar and might help?