Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
JozkoMrkvicka
Mentor
Mentor

API show logs

Hello guys,

I want to use API call "show logs" to show me all logs. I want to see ALL logs, not only last 100. Is that possible ? Using following commands I am able to get only 100 logs, not more (within 7 days period):

mgmt_cli -s sid.txt show logs new-query.filter "src:10.20.30.40" new-query.time-frame last-7-days --format json
mgmt_cli -s sid.txt show logs query-id "$QUERYID" --format json

Variable QUERYID is fetched from first API call.

First 100 logs are shown, but then if I want to go to the "next page" I am getting following from query-id:

{
  "logs" : [ ],
  "logs-count" : 100,
  "query-id" : "WEB_API_3eb4f228-abff-4cbf-83bb-377bcc3272ad"
}

There are for sure more than 100 logs (checked within SmartView and SmartConsole).

Running latest Take of R80.30.

Kind regards,
Jozko Mrkvicka
0 Kudos
12 Replies
Tal_Paz-Fridman
Employee
Employee

Looking at the Management API Reference Guide it seems 100 is the limit:

https://sc1.checkpoint.com/documents/latest/APIs/index.html#cli/show-logs~v1.8%20

show logs.JPG

 

I'll forward this to R&D owners to see if this can be improved in future versions.

 

 

0 Kudos
JozkoMrkvicka
Mentor
Mentor

My guess would be to do not stress API with huge output - therefore maximum 100 logs per call.

Anyway, I managed to handle it with infinity loop (while true) where I am checking "log-count" value. If it is less than 100, the log search is over.

Also the issue with empty "query-id" was solved, but I dont know how 😄 Maybe the quotes were the issue...

I would like to have the same options like we have in SmartView GUI. For example, you can choose which columns you want to export (not all like in API call). Something like "set log-template" where you will be able to add/remove columns according your needs. Once set, add mandatory parameter in "show logs" to include the log template.

Kind regards,
Jozko Mrkvicka
0 Kudos
Amir_Senn
Employee
Employee

After your initial command you will also get session id/sid.

Please try this syntax:

mgmt_cli show-logs query-id <query-id> --session-id <session-id>

This should page further. You can repeat this command for further paging.

I think session is alive for 600 seconds after your initial command + credentials.

Kind regards, Amir Senn
0 Kudos
mohit_tater
Explorer

I am also facing a similar issue with the show-logs API. let's say I want to retrieve more than 100 logs, I make an API call. The first call returns me the log-count(100), list of logs event (size 100) and a query-id.

I use this query-id to make subsequent API call and this call returns me log-count(100) but the list of logs is empty []. Any API call after this point behaves similarly.

 

payload for the first request:

{
"new-query": {
"time-frame": "custom",
"custom-start": "2021-08-10T12:05:19.000Z",
"custom-end": "2021-09-10T11:20:31+0000",
"filter": "blade:\"Threat Emulation\"",
"type": "logs",
"max-logs-per-request" : 100
}
}
 
payload for subsequent requests
{
"query-id": "admin-api_cf0eac6a-883c-42e8-xxxxxxxxxxxxxxx"
}
 
I am not sure if I am doing something wrong here. Please help.
0 Kudos
PhoneBoy
Admin
Admin

Recommend a TAC case.

0 Kudos
Amir_Senn
Employee
Employee

I think it will show you 100 max logs per request regardless of number of results.

I suggest you try a few things:

Make sure that in SmartConsole you have results for that filter and timeframe.

Try to drop the filter/timeframe and do the query again and see if you have results.

Kind regards, Amir Senn
0 Kudos
JozkoMrkvicka
Mentor
Mentor

I have to run the script more times in order to get me the real logs based on filter set.

Sometimes the output from "show logs" is only 100 logs, sometimes 300 logs, sometimes all logs (lets say I have 1543 logs for filter in total).

Looks like the api call is warming up and is fully ready after multiple triggers 😄 Like starting your old car in the winter 😄

Kind regards,
Jozko Mrkvicka
0 Kudos
Douglas_Rich
Contributor

only 100 Logs?  What's the use case for the API then, if it's only 100 logs it seems not very useful. 

0 Kudos
JozkoMrkvicka
Mentor
Mentor

yeah, even on R81.10 with the latest jumbo, the API "show logs" looks buggy. But as I mentioned, if I run the script more times, it will get more logs than 100. But sometimes every run gives different number of logs.

Kind regards,
Jozko Mrkvicka
0 Kudos
PhoneBoy
Admin
Admin

Most of our API calls return a limited number of results by design to keep the API server performant.
You have to make multiple calls to the endpoint to retrieve all the logs, as noted.

0 Kudos
Paul_Hagyard
Advisor

Given the logic is already in SmartView to export logs in bulk (to 1M), it would make sense to include access for this in the API. Send an API request which runs the export on the management server and provide a request ID. Poll with the request ID to see when it's completed, and when completed return a URL for retrieving the file.

"show logs" doesn't even report the total number of logs matching the query (timeframe and query string), you have to just keep pulling data until a request returns nothing. Just return "logs_remaining: x" in each query.

Quicker to write an Autohotkey kb/mouse macro to login to Smartview and export logs 🙂

Paul_Hagyard
Advisor

Okay, the export does return logs remaining, not just the logs requested, sorry!

"logs-count" : 96,

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events