- CheckMates
- :
- Products
- :
- Quantum
- :
- Security Gateways
- :
- Re: RAD issues?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Are you a member of CheckMates?
×- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
RAD issues?
Hi!
We experience very slow internetaccess this morning and I suspect RAD being the root cause.
Feels like RAD service is not working well hosted by Check point?
Anyone else having the same issue this morning?
Flow ID = flow_8960_9582165
Flow Termination Status:Failed!
Flow Started (08:48:25)
Flow Ended (08:50:31)
Flow Total Time:126 seconds (Timed out)
Flow Steps:
Generic Trap Flow ( Queue Wait=0 / RunTime=0.001413 Seconds )
Cloud HTTP Access(IO) ( Queue Wait=46 / RunTime=0.118964 Seconds )
Handling Decrypted Response ( Queue Wait=1 / RunTime=0.108883 Seconds )
Cloud HTTP Access(IO) ( Queue Wait=79 / RunTime=0 Seconds )
End Of Flow Steps
Flow Items:
_indicator@trapper:vsid=2
_indicator@trapper:version=0
_indicator@trapper:session=
_indicator@trapper:service=malware
_indicator@trapper:resource=google.com/complete/search?output=chrome&q=test
_indicator@trapper:key_len=10
_indicator@trapper:is_ipv6=0
_indicator@trapper:flags=0
_indicator@cpradus:resend=1
Service=malware
Resource=google.com/complete/search?output=chrome&q=test
Malware-Restart=TRUE
FlowError=RAD request exceeded maximum handing time
FetchUrl=http://cws.checkpoint.com:80/Malware/malware/6.0?resource=Z29vZ2xlLmNvbS9jb21wbGV0ZS9zZWFyY2g/b3V0cH...
ActiveFlows=941
End of Flow Items
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I can confirm this behavior. We have many logs entries with the reason:
"Failed to fetch Check Point resources. Timeout was reached, check /opt/CPsuite-R81.20/fw1/log/rad_events/Errors/flow_.... For more details"
Those logs ends with:
...
[rad_curl_task.cpp:163] CRadCurlTask::performCurl: [INFO] enter to ...
[rad_curl_task.cpp:180] CRadCurlTask::performCurl: [INFO] Retry mode - pefrom curl totoal retries: 82
[rad_curl_task.cpp:209] CRadCurlTask::performCurl: [INFO] Maxumum retries reached - give up
[rad_curl_task.cpp:114] CRadCurlTask::run: [ERROR] handle: 0xf133ffc0 curl_easy_perform() failed: Timeout was reached
Kind regards
Sascha Mommert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can anyone from Check Point confirm if our problems are caused by issues on cws.checkpoint.com ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Same issue here. We can see dozens, hundreds of "rad_curl Maximum retries reached - give up" lines inside the flow logfiles as well.
CPU is climbing up on 2 cores to 90% again and again, but just for small peaks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We also see this behavior. It startet at around 9am (germany) and the RAD daemon also restarted right before that. This also shows in cpview as timeouts. The 48 days before the counter was at 0.
I also have the feeling that the issue startet on Thursday or Friday already but not that bad and maybe just slow response times. But I don't see that in the logs.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just got information that our users also noted that the behaviour started on friday
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just notices that all "System Alert" logs are related to Google searches like: Error occurred while accessing:google.com/complete/search?client=chrome-XXXXXXX
Can you confirm that? Maybe that is causing the issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
for our site its also a lot of google, but besides that we have also kleinenzeigen.de and other sites.
But yes, google is (as always if its about RAD processes) a prime process here as well
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
99% google.com for us too (not only searches) and always service=malware
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Same issue that impact CPU of the gateway and so all Internet Traffic.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
TAC just confirmed the issue to us. They are working on it with high prio.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We too have this issue. I created ticket and waiting for response.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are having the same issues, users started reporting Google not working this morning. For us it's also mostly Google searches but also other websites (but a lot less). google.nl seems to be working.
It also seems we get a lot more Blocks / Alerts with various "Internal system error" messages since upgrading from R81.20 T41 to T53. I'm not sure this is related to this issue but it's something we noticed.
Errors like:
Application Control - HTTPS Inspection error occurred (2)
Internal system error in HTTPS Inspection due to categorization service timeout
and:
URL Filtering - Internal system error. (0)
We got the first reports of sites not working last week (June 11th).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We have an active case open for this too since the 5th of June. Mostly google.com not working. No progress to date.
We have only seen it on the SMBs. Firmware version doesn't seem to matter
We too seemed to notice it around the 30th of May we think but we're not sure
t
Precise Error: Internal system error in HTTPS Inspection due to categorization service timeout
Reason: Application Control - HTTPS Inspection error occurred (2)
Action Reason: Blocking request as configured in engine settings of Application Control
Action: Block
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Mine shows updated, but EXACT same date/time as yours.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We also have that exact version.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What do you see from cpview?
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Workaround
caution lowers security level
In the meantime, it's possible to improve the situation by means of small workarounds.
Go to the Smartconsole's Manage and Settings menu, then to the "Blades" menu. You'll find several sub-menus, including "Threat Prevention" and "CheckPoint Online Web Service".
- Threat Prevention :
Fail Mode -> Switch to "Fail-open".
- CheckPoint Online Web Service:
App Control & URL Filtering -> switch to "Background
Fail Mode -> Switch to "Fail-open
This would temporarily relieve the flow processing and make it run more smoothly.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
But as you said, definitely lowers security and considering we all security oriented here, lets hope there is more permanent solution soon.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We were experiencing this too as of last Wednesday, then it started working, and then presented itself again sometime Thurs/Friday into today. It looks like it *might* be working again now...but our checkpoint ticket hasn't given any details. Anyone else have any information worth sharing?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't know why there is no official statement but the issue should be resolved now. We don't see any issues anymore and everything is fast as it should be.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For us it still isn't working and google.com / searches from the Chrome address bar are still giving problems. So not so sure if it's resolved for everyone.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If people want to PM me SRs they have open with TAC on this issue, I can investigate.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
similar situation here. Last warning was yesterday 21:12. Until now no new alert. But even no update by checkpoint
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This now appears to be fixed
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the spirit of the community, would you mind sharing how it got fixed? It can definitely help others in the future.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TJ_Aus informed me they are still awaiting for final response from TAC, so will update once thats provided.
Andy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
same here