Cloudguard Controller Error

[balmey]

Hi

I have a recurring Error on the Dashboard - Management Server Device Status shows:

Cloudguard Controller - Error: 1 out of 1 Data Centers is disconnected.

I had previously resolved this error by renewing the Service Principal Authentication on the AzureDCObject. But this is not the problem this time as it's Connected and test connection works fine.

Azure Cloudguard R81.20 Jumbo Hotfix Take 26

Any clues?

Steve

[Chris_Atkinson]

Anything interesting in the log: $FWDIR/log/cloud_proxy.elg

Note Take 41 also includes a new Cloud CA Bundle (sk172188).

[balmey]

Only some old entries:

This recurred repeatedly until the October 25th

25/10/23 01:14:04,764 ERROR datacenter.scanner.DcScanner [scanner-Azure-1014943195]: Error during scan - attempting to reconnect for
scanner of [AzureDCObject] in domainID 41e821a0-3720-11e3-aa6e-0800200c9fde
com.checkpoint.datacenter.util.exception.UnknownProblemException: Failed querying Azure, unknown problem
at com.checkpoint.datacenter.scanner.azure.AzureDeployment.getAzureResponse(AzureDeployment.java:226) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.azure.AzureScanner.innerRun(AzureScanner.java:136) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.DcScanner.run(DcScanner.java:50) [cms.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522) [?:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:277) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:1.8.0]
at java.lang.Thread.run(Thread.java:825) [?:2.9 (03-03-2022)]
25/10/23 01:14:04,764 ERROR scanner.util.DcScannerUtils [scanner-Azure-1014943195]: Exception while connecting to [AzureDCObject]. R
eturn unknown problem.
com.checkpoint.datacenter.util.exception.UnknownProblemException: Failed querying Azure, unknown problem
at com.checkpoint.datacenter.scanner.azure.AzureDeployment.getAzureResponse(AzureDeployment.java:226) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.azure.AzureScanner.innerRun(AzureScanner.java:136) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.DcScanner.run(DcScanner.java:50) [cms.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522) [?:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:277) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:1.8.0]
at java.lang.Thread.run(Thread.java:825) [?:2.9 (03-03-2022)]

then after 25th only these ones...

25/10/23 16:32:54,517 ERROR checkpoint.datacenter.Main [main]: === starting up ===
07/11/23 13:04:17,220 ERROR checkpoint.datacenter.Main [main]: === starting up ===
-end-

[tomlev]

Other than what @Chris_Atkinson suggested, do you see any errors in SmartConsole logs with this filter?

blade:"CloudGuard IaaS" AND severity:Critical

[balmey]

Nothing from that log filter...

Also, I've been uploading the log content from cloud_proxy.elg - just few lines - but the post keeps getting deleted... Any recommended way to do it?

[tomlev]

Do you see successful scans  of this Data Center if you remove the severity from the filter?

Any specific errors in cloud_proxy.elg? Not sure why is it not uploading for you.

[balmey]

There's this on SmartConsole Logs after i removed critical filter.

[Central License] VE-NGTP license pool core usage has exceeded the cores quota of the pool. The pool's total quota limit is 4, while the actual core usage is 8.

I did see there was an unclaimed licence on the Mngmt server which i added...  now there is 4 + 4 cores licenced on the mgmt server. I would expect the above log to not return now but the original issue is still remaining. 

Have tried attaching log entries on txt file. last few entries only, there's nothing recent there.

[the_rock]

I think what guys suggested is best place to start. Can you please also attach a screenshot, if possible?

Andy

[balmey]

Logs from cloud_proxy.elg... Nothing recent...

25/10/23 13:11:40,132 ERROR datacenter.scanner.DcScanner [scanner-Azure-1014943195]: Mapping of Data Center [AzureDCObject] failed . Next mapping is in 300 seconds.
25/10/23 13:37:10,895 ERROR util.process.ProcessExecutor [scanner-Azure-1014943195]: java.util.concurrent.TimeoutException
25/10/23 13:37:10,895 ERROR util.process.ProcessExecutor [scanner-Azure-1014943195]: Timeout reached: 1200 seconds, killing process
25/10/23 13:37:10,895 ERROR util.process.ProcessExecutor [pool-1806995-thread-1]: protectedWait: java.lang.InterruptedException
25/10/23 13:37:19,513 ERROR util.process.ProcessExecutor [Thread-302]: ProcessStreamReader: stderr - run: java.io.IOException: Stream closed
25/10/23 13:37:19,514 ERROR scanner.azure.AzureDeployment [scanner-Azure-1014943195]: com.checkpoint.datacenter.util.exception.ProcessExecutionException: Failed running process
25/10/23 13:37:19,514 ERROR datacenter.scanner.DcScanner [scanner-Azure-1014943195]: Error during scan - attempting to reconnect for scanner of [AzureDCObject] in domainID 41e821a0-3720-11e3-aa6e-0800200c9fde
com.checkpoint.datacenter.util.exception.UnknownProblemException: Failed querying Azure, unknown problem
at com.checkpoint.datacenter.scanner.azure.AzureDeployment.getAzureResponse(AzureDeployment.java:226) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.azure.AzureScanner.innerRun(AzureScanner.java:136) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.DcScanner.run(DcScanner.java:50) [cms.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522) [?:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:277) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:1.8.0]
at java.lang.Thread.run(Thread.java:825) [?:2.9 (03-03-2022)]
25/10/23 13:37:19,514 ERROR scanner.util.DcScannerUtils [scanner-Azure-1014943195]: Exception while connecting to [AzureDCObject]. Return unknown problem.
com.checkpoint.datacenter.util.exception.UnknownProblemException: Failed querying Azure, unknown problem
at com.checkpoint.datacenter.scanner.azure.AzureDeployment.getAzureResponse(AzureDeployment.java:226) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.azure.AzureScanner.innerRun(AzureScanner.java:136) ~[cms.jar:?]
at com.checkpoint.datacenter.scanner.DcScanner.run(DcScanner.java:50) [cms.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:522) [?:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:277) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) [?:1.8.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:1.8.0]
at java.lang.Thread.run(Thread.java:825) [?:2.9 (03-03-2022)]
25/10/23 13:37:19,514 ERROR datacenter.scanner.DcScanner [scanner-Azure-1014943195]: Mapping of Data Center [AzureDCObject] failed . Next mapping is in 300 seconds.
25/10/23 13:46:41,723 ERROR checkpoint.datacenter.Main [main]: === starting up ===
25/10/23 13:49:50,250 ERROR checkpoint.datacenter.Main [main]: === starting up ===
25/10/23 16:32:54,517 ERROR checkpoint.datacenter.Main [main]: === starting up ===
07/11/23 13:04:17,220 ERROR checkpoint.datacenter.Main [main]: === starting up ===

[Gil_Sudai]

In order to fix these errors:

25/10/23 13:37:10,895 ERROR util.process.ProcessExecutor [scanner-Azure-1014943195]: java.util.concurrent.TimeoutException
25/10/23 13:37:10,895 ERROR util.process.ProcessExecutor [scanner-Azure-1014943195]: Timeout reached: 1200 seconds, killing process

You need to increase the value of azure.scannerInterval in vsec.conf.  You can add 2-3 zeros to it. Then save the file and run "vsec stop ; vsec start" for it to take effect.

[balmey]

Screenshots

[the_rock]

When you search for data center in the object list and then right click -> where used, what does it show?

Andy

[balmey]

"No usages found." I assumed it was there only to authenticate the updatable objects process with Azure.

[the_rock]

If thats the case, I would say you can safely delete it, save (publish), install mgmt database and policy and Im positive error will be gone.

Best,

Andy

[balmey]

Thanks, I'll give it a go and update the thread

[the_rock]

Sounds good.

Andy