topic Re: Skyline -- Grafana -- Temperature Thresholds in OpenTelemetry/Skyline

Skyline -- Grafana -- Temperature Thresholds

David_Evans — Thu, 04 May 2023 13:19:55 GMT

The CPU, Inlet and Outlet temperature thresholds are different from each other on most models. I cannot work out in Grafana how to set these thresholds to each real temperature value.

hardware.temperature.max 50,50.100,100
hardware.temperature 30,27,64,71

Using Grafana, and the "Config from query results" in the transformation section, I can assign 50 to the Grafana built in variable of "threshold". However, it assigns the same value to ALL the hardware temperatures, and not matching each max value to its corresponding real value's "threshold" variable.

The threshold variable works in the graphs to do things like color coding and I assume alerting as well. However, I cannot assign a unique threshold to each of the temperatures.

This seems to be working as designed from what I can see as there is a note in the documentation that states.
"If you want to extract a unique config for every row in the config query result then try the rows to fields transformation."

The "rows to fields" transformation seems very broken at least in my install and doesn't give me any options to do anything like assigning the threshold values to each real value.

I was wondering how others were working around this issue or if I'm missing something simple.

Re: Skyline -- Grafana -- Temperature Thresholds

David_Evans — Thu, 01 Feb 2024 14:09:38 GMT

I have not found a good solution to this where I can alert on the different temperature threshold values for the different temps. I can display them like below but there are even some format issues with this where you have to really look to match the threshold to the current value. This is really a grafana issue and not a skyline specifically but when I google this for grafana, I can't find many examples where a dataset has various different thresholds for a single set of data.

Re: Skyline -- Grafana -- Temperature Thresholds

Alexander_Wilke — Mon, 05 Feb 2024 07:03:51 GMT

Hello,

I am not monitoring CPU temperature so maybe I am wrong how it is woprking.

But in the Grafan "Alert Rule"section you can create several different queries within one rule.
Maybe create a qeury for ecery type of firewalls which have the same cpu thresholds.

and in the "host_name" label you can do something like this:
"host_name=~".*fw1.*|.*fw2.*|.*fw3.*"

If you have 3 different models then you have 3 queries A, B and C.

Next use the "Math option" and select:

$A > 30 || $B > 50 || $C > 70

Re: Skyline -- Grafana -- Temperature Thresholds

David_Evans — Mon, 05 Feb 2024 19:03:12 GMT

These are different thresholds within the same firewall. ie On the same firewall, the intake temp threshold is different than the CPU temp threshold.

Re: Skyline -- Grafana -- Temperature Thresholds

Dorian — Thu, 15 Feb 2024 23:27:31 GMT

Hi David - when you say "However, I cannot assign a unique threshold to each of the temperatures. " - might be misunderstanding you but you could create separate Alert rules catering for each query/metric (ie one for CPUTemp, another one for CPU0Temp and so on.. see screenshot bellow.

I would probably create separate panels for CPU, Inlet, Outlet etc..

Hope this helps?

EDIT - just realised Alex already suggested something similar - hope the screenshots help

Re: Skyline -- Grafana -- Temperature Thresholds

David_Evans — Thu, 22 Feb 2024 14:28:56 GMT

I don't see in your examples where you are dynamically matching up the reported threshold for each temperature "type" on each firewall. I have many models of checkpoint firewalls ~~and~~ I've noticed that on my 6000XL's the reported thresholds are different across the models. I haven't dug into it but I'm thinking its when the BIOS updates on the 6000's was done to take care of the RAM boot issue, the reported max thresholds also changed. (not true on further investigation, it was one of my failed attempts at trying to get this to work messing with the data) I would really like to not have to generate a huge table with each model and each temperature type to do the alerting when the thresholds are right there in the provided data.

Re: Skyline -- Grafana -- Temperature Thresholds

David_Evans — Fri, 23 Feb 2024 14:31:05 GMT

For alerting I came up with this query.

max by(host_name, name) ((((hardware_temperature{environment="$d_environment", host_name="$d_hostname"}) * (9/5)) + 32) - (((hardware_temperature_max{environment="$d_environment", host_name="$d_hostname"} * (9/5)) + 32)))

That seems to correctly match the values for each monitored Temp sensor to its specific threshold value for for that sensor specifically. Subtracting them from each other, gives you a negative value as long as the temp is below the threshold. From there alerting is fairly easy. (if you ignore the C to F math)

What I haven't figured out is how to display that in grafana using those thresholds as a custom threshold for displaying in a dashboard.

IE so that its green below 80% of the value, orange between 80 and 90% and red over 90%...

That was my starting goal of the question, thinking the next easy step would be alerting. However, using the the custom threshold seems to be more difficult.