Who rated this post

cancel
Showing results for 
Search instead for 
Did you mean: 
crescentwire
Employee
Employee

Hey, @bob111 , I'm guessing you're seeing this sort of gibberish in your Splunk dashboard:

splunk_tls_fail.png

I have this working in a lab environment, so let me explain how I did it:

Splunk server

Assuming Splunk is installed under /opt/splunk, you'll want to create a new directory under /opt/splunk/etc/apps:

 

cd /opt/splunk/etc/apps

mkdir -p sms-tls/{certs,local}

 

 (I'm using a specific "app" called "sms-tls", but you can choose any name you like.)

Now, depending on what you're using for your signed certificates (external CA, internal CA, etc.), you'll want to issue a new certificate signed by that CA. I'm using an internal CA (OpenSSL on Linux), so here are the steps for that:

 

openssl genrsa -out server-splunk-key.pem 2048

openssl req -new -key server-splunk-key.pem -out server-splunk-req.pem

openssl x509 -req -in server-splunk-req.pem -CA cacert.pem -CAkey private/cakey.pem -CAcreateserial -out server-splunk-cert.pem -days 365 -sha256

 

Let me know if you'd like a description of what these commands do, but we're simply creating a new private key called server-splunk-key.pem, generating a CSR and signing it with our private key, and then asking the CA to generate a new certificate, signed with its private key. The final certificate we'll use on Splunk is server-splunk-cert.pem.

Next, we want to concatenate the generated certificate and the generated private key into one single PEM file, like so:

 

cat server-splunk-cert.pem server-splunk-key.pem >> server-splunk-cert-combined.pem

 

Copy the following files to /opt/splunk/etc/apps/sms-tls/certs:

  • server-splunk-cert-combined.pem
  • cacert.pem (your CA's certificate, likely generated when you established your internal CA; an external CA will provide this to you separately after satisfying your CSR)

Next, navigate to the sms-tls directory with cd /opt/splunk/etc/apps/sms-tls/local and create a new file called inputs.conf. Using vi or nano, paste the following contents inside:

 

# NOTE: I'm restricting TLS versions to 1.1 and 1.2; 1.0 is not supported thanks to many vulnerabilities over the years
[SSL]
serverCert = /opt/splunk/etc/apps/sms-tls/certs/server-splunk-cert-combined.pem
sslVersions = tls,-tls1.0
requireClientCert = true

# NOTE: I'm using TCP port 6514, but you can choose any you like
[tcp-ssl:6514]

 

Notice the serverCert value is the new combined cert file we created earlier. Make sure this value matches the directory structure and filename exactly!

Next, create another new file called server.conf in the same directory (local). Using vi or nano, paste the following contents inside:

 

[sslConfig]
serverCert = /opt/splunk/etc/apps/sms-tls/certs/server-splunk-cert-combined.pem
sslRootCAPath = /opt/splunk/etc/apps/sms-tls/certs/cacert.pem
sslVerifyServerCert = true

 

Finally, your directory structure on Splunk should resemble this:

 

splunk@splunk-test:/opt/splunk/etc/apps/sms-tls$ tree ./
./
├── certs
│   ├── cacert.pem
│   ├── server-splunk-cert-combined.pem
├── local
    ├── inputs.conf
    └── server.conf

2 directories, 4 files

 

Restart your Splunk service and proceed to configuring the SMS!

 

# NOTE: May need to prefix with 'sudo'
/opt/splunk/bin/splunk restart 

 

 

SMS server

You probably have this config correct, but I'll include it anyway. Create a new cp_log_export instance using the destination IP address and TCP port you intend to use on your Splunk indexer:

 

cp_log_export add name splunk target-server 10.5.1.191 target-port 6514 protocol tcp format splunk

 

Though you will be prompted, do not restart the Log Exporter service yet; proceed to configure TLS.

 

mkdir $EXPORTERDIR/targets/splunk/tls

 

Copy the CA's certificate and the generated certificate for the SMS (in PKCS12/P12) format to this new tls directory. If you're not sure how generate a P12 certificate, you can use these steps (assuming you have access to the SMS's private key and generated certificate in PEM format):

 

# NOTE: Use 'cpopenssl' if on the SMS itself; otherwise, use 'openssl' 
cpopenssl pkcs12 -inkey client-sms-key.pem -in client-sms-cert.pem -export -out client-sms-cert.p12

 

Next, edit the XML configuration file:

 

vi $EXPORTERDIR/targets/splunk/targetConfiguration.xml

 

Between the <transport> tags, we'll need to configure TLS. Look for this section towards the top:

 

<transport>
    <security></security>
    <!--clear/tls-->
    <!-- the following section is relevant only if <security> is tls -->
    <pem_ca_file></pem_ca_file>
    <p12_certificate_file></p12_certificate_file>
    <client_certificate_challenge_phrase></client_certificate_challenge_phrase>
</transport>

 

And insert the following values between the various tags:

 

<transport>
    <security>tls</security><!--clear/tls-->
    <!-- the following section is relevant only if <security> is tls -->
    <pem_ca_file>/opt/CPrt-R81.20/log_exporter/targets/splunk/tls/cacert.pem</pem_ca_file>
    <p12_certificate_file>/opt/CPrt-R81.20/log_exporter/targets/splunk/tls/client-sms-cert.p12</p12_certificate_file>
    <client_certificate_challenge_phrase>changeme</client_certificate_challenge_phrase>
</transport>

 

Obviously, don't use "changeme" as your decryption password, but you'll want to type whatever password you used to decrypt the private key contained within the P12 file. 

Save changes to the file and quit using :x or :wq!.

Finally, restart the cp_log_export instance to begin sending logs to Splunk:

 

cp_log_export restart name splunk

 

Back on Splunk, validate log delivery and indexing with this query:

 

index="main" <SMS-IP-Address>

 

You can also monitor the cp_log_export instance's status with:

 

tail -f $EXPORTERDIR/targets/syslog-ng/log/log_indexer.elg

 

 

Let me know if you still run into trouble or if this is working for you. Hopefully it's helpful!

 

(1)
Who rated this post