Importing Syslog Messages

[SkochilovIgnat]

Hello Team!

I am trying to import syslog messages from third-party device. I send logs directly to the SMS via UDP to 514 port. Everything worked until I tried to sending larger logs, I encountered packet fragmentation. So in the logs and in tcpdump (on port 514) I see only first part of the log, the rest part is not visible on port 514 and therefore not parsed.

The SMS version is R82.

Is there any workaround, or perhaps I’m doing something wrong? Thanks!

[Chris_Atkinson]

What JHF and is there an MTU issue along the intermediate path that you can resolve?

[SkochilovIgnat]

Thank you for your reply! 
I looked into this direction and increased the MTU on the interfaces on both sides, as well as on the virtual switch to which the machines are connected.
The thing is that both the SMS and my machine, from which the logs are sent, are deployed in a virtual environment. I managed to achieve that in tcpdump on the Check Point side I see the full message, i.e., now the packet is not fragmented.

However, in the logs on the Check Point, I still see only part of the full message as it was before increasing MTU. It’s as if the internal mechanism cannot process a message of that size or maybe there is a parameter for maximum syslog message size(

[elbergfeldt]

Hi! 

Can you describe the procedure you followed when setting this up? 

[SkochilovIgnat]

To setup this I followed the R82 Logging and Monitoring Administration Guide, specifically sk55020 - How to generate a log parser for third-party syslog records

I downloaded Eventia Log Parsing Editor archive with syslog parser in it. I attached the custom parsing file I created for my task, if you want to replicate the issue in your environment.
So:
1) I enabled the "Accept Syslog messages" in the SMS object (Logs -> Additional Logging) in Smart Console 
2) Installed Parser.C with the next command
addParsingFile -p /home/admin/Parser.C
3) For testing purpose I wrote the python script (in the attachements with test syslog messages). So currently I sent the syslog messages to SMS from Windows machine via this script, and see the same issue.
4) I increased the MTU size on both interfaces (SMS and Windows side), also I increased the MTU size on distributed switch in vSphere

So after that the results I can see is:
1) In the tcpdump (tcpdump_log.txt) I recieved full packects without fragmentation
2) In the logs the long message is not fully parsed and cutted, the short message is fully parsed.

 

I thought maybe in the SmartConsole log the raw message not fully shown, but the part shown in the log is parsed correctly, the rest of the message is not parsed at all.

The versions of SMS I tested this on is R82 Jumbo Hotfix Take 118 and R82.10 no JHF

[Chris_Atkinson]

If you need this investigated on the CP side please open an SR with TAC.

[elbergfeldt]

Am I understanding it correctly that it parses fine when the fields in "data" is "data":"{"information":"some_short_data"}"
But when the "data" message is long it appears like the log entry to the left? 

[SkochilovIgnat]

Not quite. The regex is "data.:.\{(.*)\}.,.", so it doesn’t matter what is inside the curly braces. The only difference here is the size of the string in the braces.