topic Re: Skyline -- No Data being received Promethues not fully loading in OpenTelemetry/Skyline

Skyline -- No Data being received Promethues not fully loading

ptuttle_2 — Fri, 05 May 2023 13:16:53 GMT

Hello;

We are trying to setup up Skyline in a VM lab. We are following the Check Point sk178566. Everything seems to install ok but I see this when checking promethues.service status

May 04 18:51:24 cp-ubuntu-skyline prometheus[1409]: ts=2023-05-04T18:51:24.302Z caller=dedupe.go:112 component=remote level=error remote_name=3e5277 url=http://127.0.0.1:9090/api/v1/write >
May 04 18:51:39 cp-ubuntu-skyline prometheus[1409]: ts=2023-05-04T18:51:39.305Z caller=dedupe.go:112 component=remote level=error remote_name=3e5277 url=http://127.0.0.1:9090/api/v1/write >

I tried adding the remote write receiver to the promethues.service but does not change the flag in the webui either.

Not sure if this is a bug in promethues 2.38 ?

Any advice would be appreciated.

Thanks

-pat

Re: Skyline -- No Data being received Promethues not fully loading

the_rock — Fri, 05 May 2023 13:24:11 GMT

What version of CP? I tested this in R81.20 jumbo 10 lab and worked fine.

Andy

Re: Skyline -- No Data being received Promethues not fully loading

ptuttle_2 — Fri, 05 May 2023 13:35:20 GMT

Thanks I forgot to put that. We are also running R81.20 jumbo 10. But I don't think I'm getting that far if the receiver isn't loading.

-pat

Re: Skyline -- No Data being received Promethues not fully loading

the_rock — Fri, 05 May 2023 13:40:43 GMT

Apologies, I had to wipe out that lab and build brand new one recently, so have not set it up yet 2nd time, but I followed the video thats in that sk and did not have any problems. I can try set it up again later and see what happens...whats exact command you are running where you get that error?

Cheers,

Andy

Re: Skyline -- No Data being received Promethues not fully loading

ptuttle_2 — Fri, 05 May 2023 14:11:24 GMT

Thanks, I'm hoping its just a simple setting change. I have included what our configurations look like.

When I check status of the service, its running and I can web to the server. When looking at the command-line flags in the portal its showing web-enable-remote-write-receiver as false. My understanding is this needs to be true (enabled) in order to process the info coming from the Check Point Gateway.

ptuttle@cp-ubuntu-skyline:/tmp/prometheus$ sudo systemctl status prometheus.service
? prometheus.service - Prometheus Server
Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2023-05-05 12:59:02 UTC; 46min ago
Docs: https://prometheus.io/docs/introduction/overview/
Main PID: 5360 (prometheus)
Tasks: 8 (limit: 2234)
Memory: 37.6M
CPU: 24.773s
CGroup: /system.slice/prometheus.service
+-5360 /home/ptuttle/prometheus-2.38.0.linux-amd64/prometheus --config.file=/home/ptuttle/prometheus-2.38.0.linux-amd64/prometheus.yml

May 05 13:41:05 cp-ubuntu-skyline prometheus[5360]: ts=2023-05-05T13:41:05.261Z caller=dedupe.go:112 component=remote level=error remote_name=02f67d url=http://10.112.129.210:9090/api/v1/w>
May 05 13:41:25 cp-ubuntu-skyline prometheus[5360]: ts=2023-05-05T13:41:25.262Z caller=dedupe.go:112 component=remote level=error remote_name=02f67d url=http://10.112.129.210:9090/api/v1/w>

Here is the "promethues.service" I know its a little different from the sk but it was identical to the sk and We were seeing the same thing.

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After-network=online.target

[Service]
User=root
Restart=on-failure
ExecStart=/home/ptuttle/prometheus-2.38.0.linux-amd64/prometheus --config.file=/home/ptuttle/prometheus-2.38.0.linux-amd64/prometheus.yml
--enable-feature=remote-write-receiver \
--web.listen-address=0.0.0.0:9090 \
--web.external-url=
--storage.tsdb.path=/data \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries \
--web.listen-address=0.0.0.0:9090 \
--web.enable-lifecycle

[Install]
WantedBy=multi-user.target
~

Here is the "promethues.yml" file

# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).

remote_write:
- url: "http://10.112.129.210:9090/api/v1/write"

# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: "prometheus"

# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.

static_configs:
- targets: ["localhost:9090"]
~

Re: Skyline -- No Data being received Promethues not fully loading

the_rock — Fri, 05 May 2023 14:39:39 GMT

Let me see if I can find my old config, but on surface, looks right.

Andy

Re: Skyline -- No Data being received Promethues not fully loading

ptuttle_2 — Fri, 05 May 2023 14:52:00 GMT

Looks like I'm finally getting somewhere. I placed the "--web.enable-remote-write-receiver" in the ExecStart= line and now it works !!. it does not work putting it below. Maybe it needs to be in a different column? but its now working.

I appreciate the quick responses.

-pat

Re: Skyline -- No Data being received Promethues not fully loading

the_rock — Fri, 05 May 2023 14:54:36 GMT

Interesting, I never recall having to do so...good job regerdless 👍