Skyline: adding multiple export targets

[a-pomaskin]

Hello everyone,

I'm currently attempting to deploy Skyline, but I've run into a problem with adding multiple export targets to the config file. I would greatly appreciate any advice you may have on this matter. Thank you.

[Arik_Ovtracht]

Hi @a-pomaskin,

can you share more details? What are you trying to do exactly?

[a-pomaskin]

Hi @Arik_Ovtracht,

I am currently working on deploying Skyline based on sk178566. However, I am facing an issue with adding multiple export targets to the config file.
Please take a look at my current payload-tls.json:

{
    "enabled": true,
    "export-targets": {"add": [
        {
            "client-auth": {
                "basic": {
                    "username": "checkpoint",
                    "password": "<PASSWORD>"
                }
            },
            "enabled": true,
            "server-auth": {
                "ca-public-key": {
                    "type": "PEM-X509",
                    "value":"<CERT>"
                }
            },
			"type": "prometheus-remote-write",
            "url": "https://u40.prometheus-collector.service.rockset-s.local/api/v1/write"
			},
		{
            "client-auth": {
                "basic": {
                    "username": "checkpoint",
                    "password": "<PASSWORD>"
                }
            },
            "enabled": true,
            "server-auth": {
                "ca-public-key": {
                    "type": "PEM-X509",
                    "value":"<CERT>"
                }
            },
			"type": "prometheus-remote-write",
            "url": "https://m7.prometheus-collector.service.rockset-s.local/api/v1/write"
			}
    ]}
}

Additionally, the output of the "/opt/CPotelcol/GetOTDynamicConfig.sh" command shows that only the last export target is being displayed. The output is as follows:

{"exporters": {"prometheusremotewrite": {"tls": {"ca_file": "/opt/CPotelcol/certs/ca-bundle.crt"}, "headers": {"Authorization": "Basic "}, "endpoint": "https://m7.prometheus-collector.service.rockset-s.local/api/v1/write"}}, "service": {"pipelines": {"metrics": {"exporters": ["prometheusremotewrite"]}}}}

I am wondering if there is an error in my Skyline json configuration.

I would greatly appreciate any advice or suggestions you may have regarding this issue.

[a-pomaskin · Accepted solution]

I opened an SR with TAC regarding this question, but unfortunately, TAC informed me that currently it is not possible to use multiple export targets in SkyLine.

[Arik_Ovtracht]

I see.

Yes, that is correct, Skyline currently does not support multiple export targets with the same type (i.e. both Prometheus server).

We will add this support soon though, so keep an eye out for the next Skyline version.

[eltonsimoes]

@Arik_Ovtracht 

Do we already have the new version of Skyline that allows sending to multiple Prometheus servers?

Best Regards,

Elton Simões

[Elad_Chomsky]

Hi @eltonsimoes ,

Yes, it is now supported, see sk178566. It is also recommended to add the "name" key-value to the payload for each exporter definition, with a unique name per target. 

{
    "enabled": true,
    "export-targets": {"add": [
        {
            "server-auth": {
                "sigv4auth": {
                  "region":"<Region>",
                  "aws-access-key-id": "<Access Key ID>",
                  "aws-secret-access-key": "<Access Key>",
                  "session-token": "<Seesion Token>"
                 }
            },
            "enabled": true,
            "type": "prometheus-remote-write",
            "url": "https://<IP1>:9090/api/v1/write",
            "name" : "my-target-1"
        },
        {
            "enabled": true,
            "type": "prometheus-remote-write",
            "url": "http://<IP2>:9090/api/v1/write",
            "name" : "my-target-2"
        }
    ]}
}

[eltonsimoes]

Hi @Elad_Chomsky 

Thanks for the answer. It was out of date about sk178566. I read it again and found the necessary configuration for sending to multiple servers. However, in sk I noticed that it does not have a unique name and in the example given above it uses the unique name. Should I use it with a single name? Is it necessary to modify something in Prometheus? Thank you for now!

[Elad_Chomsky]

Hi @eltonsimoes , 

The correct approach is with a unique name, we will update the sk during the week, to the new format.

[eltonsimoes]

Hi @Elad_Chomsky 

Thanks for answer! But I need help with a configuration, I used the payload below, still without using SSL. I have the following scenario, the firewall already sends data to TARGET-1, but now I need to send the information to the new TARGET-2. However, I don't see the data arriving at TARGET-2 and sometimes I stop receiving information at TARGET-1.

My doubt is whether I am making the correct configuration in the payload. The configurations for TARGETs are standard, according to sk178566.

{
    "enabled": true,
    "export-targets": {"add": [
        {
            "enabled": true,
            "type": "prometheus-remote-write",
            "url": "https://IP-PROMETHEUS-1:9090/api/v1/write",
			"name": "TARGET-1"
        },
        {
            "enabled": true,
            "type": "prometheus-remote-write",
            "url": "http://IP-PROMETHEUS-2:9090/api/v1/write",
			"name": "TARGET-2"
        }
    ]}
}

[Elad_Chomsky]

Hi @eltonsimoes ,

Please contact me on private on eladch@checkpoint.com, and we will try to assist you.