topic Re: Quantum OpenTelemetry to prometheus in OpenTelemetry/Skyline

Quantum OpenTelemetry to prometheus

Witty — Fri, 28 Jun 2024 09:12:24 GMT

Hello, we have a checkpoint with version: Quantum Spark with Gaia embedded. From the administration console, we activated OpenTelemetry and pointed to a URL where we have a Prometheus. We have allowed outgoing traffic from the checkpoint to Prometheus through port 9090 in the firewall. We have also activated the flag "--web.enable-remote-write-receiver" in Prometheus version 2.45.0.

We have tested from the checkpoint console using OpenTelemetry and received an OK, but we are unable to extract metrics from Prometheus.

We are receiving this message in Prometheus:

"caller=write_handler.go:47 level=error component=web msg="Error decoding remote write request" err="snappy: corrupt input"

We are not sure if OpenTelemetry is sending data that Prometheus cannot integrate... the truth is that we are at a standstill right now.

Is it not compatible?

Re: Quantum OpenTelemetry to prometheus

PhoneBoy — Mon, 01 Jul 2024 15:39:14 GMT

I believe our support for OpenTelemetry on SMB appliances is still considered EA and requires R81.10.08 and above.

Re: Quantum OpenTelemetry to prometheus

Witty — Tue, 02 Jul 2024 13:07:07 GMT

We are using a 1570R Model of Checkpoint GAIA Embedded From the administration console we point without a user or certificate to the Prometheus remote_write through port 9090. Like: http://x.x.x.x.:9090/api/v1/write

We are seeing that we have metrics in Prometheus such as:
system_gaia_os_version and others, but we believe that open telemtry is sending the metrics to the remote write with a different protocol than the one used by Prometheus, that is why we do not have values in the metrics.
We cannot use skyline because we do not have a GAIA as such, but rather an embedded one.

Re: Quantum OpenTelemetry to prometheus

PhoneBoy — Wed, 03 Jul 2024 13:52:22 GMT

Skyline is the name of our project to integrate OpenTelemetry support into Quantum Security Gateways (which includes Spark appliances).
You're using it (or attempting to).

@Arik_Ovtracht any ideas?

Re: Quantum OpenTelemetry to prometheus

Witty — Thu, 04 Jul 2024 07:33:52 GMT

ok, but the question then is: If we have Opentelemetry in Spark as I mentioned, even though it doesn't have the skyline package, then I understand that this forum is not appropriate... correct?

Re: Quantum OpenTelemetry to prometheus

Chris_Atkinson — Thu, 04 Jul 2024 08:26:08 GMT

As was mentioned earlier R81.10.08 first introduced Skyline for Spark.

Per sk181079 "Skyline, monitoring agent (as EA)."

If you are running this firmware or higher it is available to use but is not yet considered GA.

Are you running R81.10.10 or something else?

Re: Quantum OpenTelemetry to prometheus

Witty — Fri, 05 Jul 2024 08:36:03 GMT

Hi, yes, we have this version:

1570R Appliance

Version
R81.10.10 (996002945)

WEB UI
GAIA EMBEDDED

I think the problem is that we have not enabled the skyline agent at the checkpoint, which is why I am not receiving the metrics correctly in Prometheus...

Re: Quantum OpenTelemetry to prometheus

Witty — Mon, 08 Jul 2024 11:12:02 GMT

I have other question, if necessary activate any feature on this version of checkpoint?

I no have, skyline commands, on cli, o REST.py file...

Re: Quantum OpenTelemetry to prometheus

Chris_Atkinson — Tue, 09 Jul 2024 11:37:04 GMT

If you have followed the documentation below and the problem persists please contact TAC for assistance

https://sc1.checkpoint.com/documents/Appliances/Skyline/Content/Topics-AG/Configuration-on-Quantum-Spark.htm

Re: Quantum OpenTelemetry to prometheus

Witty — Tue, 09 Jul 2024 11:40:04 GMT

I finally got it working, downgraded bitnami prometheus to prom/prometheus with these two options enabled:

- '--enable-feature=remote-write-receiver'
- '--web.enable-remote-write-receiver'

I restarted the Open telemetry services at the checkpoint and it started working.
Thanks for the help