Showing top 5 CPU or Network delta from yesterday or last week.

David_Evans — Wed, 03 May 2023 12:37:58 GMT

This is what I like to call "anomaly detection lite".
I had a hard time figuring out how to do this in grafana at first as my searches led to a couple threads that seemed to indicate that this was not possible within grafana. Its actually fairly easy once you find the "offset" function.

The first 2 queries show the CPU % now with the same time window last week overlayed on the same graph. You have to play with the graph lines to make the display of the two different data sets visible. (one transparent fill, one a bold line of a different color)

Query real time:

avg(100-system_cpu_utilization{host_name="$d_hostname", environment=~"$d_environment", service_namespace=~"$d_namespace", state="idle"})

Query last week:

avg(100-system_cpu_utilization{host_name="$d_hostname", environment=~"$d_environment", service_namespace=~"$d_namespace", state="idle"} offset 1w)

This query shows your top 5 "changes" in CPU "now", compared to the same time frame last week. This can be on a quick checkout dashboard or the query can be used as the start of some alerting.

topk(5,avg by (host_name) ((100-system_cpu_utilization{host_name=~"$d_hostname", environment=~"$d_environment", service_namespace="vs_id_0", state="idle"})-((100-system_cpu_utilization{host_name=~"$d_hostname", environment=~"$d_environment", service_namespace="vs_id_0", state="idle"} offset 1w))))

topic Showing top 5 CPU or Network delta from yesterday or last week. in OpenTelemetry/Skyline

Showing top 5 CPU or Network delta from yesterday or last week.