White Paper - Integrating Custom IOC Feeds

[_Val_]

Author:

@Jonathan_Sande1 

Abstract:

This White Paper describes how to integrate and consume custom Indicators of Compromise (IOC) feeds from various 3rd parties, such as SANS, the Multi-State Information Sharing and Analysis Center (MS-ISAC), etc.

For the full list of White Papers, go here. 

[Juan_Concepcion]

I followed the whitepaper and am not sure what I'm missing -- the CP sk on debugging is not very clear at all and the log files only contain "started session" -- "ended session" nothing useful.

Version R80.30

Site pulling from: https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset

(Tested via wget and can def get file)

Syntax used:  ioc_feeds add --feed_name remote_stix_file --transport https --resource "https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset" --status false --format [type:ip,value:1] --comment "#" --delimiter "\n" --test true

Results:

[Expert@exodus-fw:0]# ioc_feeds add --feed_name remote_stix_file --transport https --resource "https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset" --state false --format [type:ip,value:1] --comment "#" --delimiter "\n" --test true
Default value for feed_action is: prevent

Feed Name: remote_stix_file
Feed is not Active
File will be fetched via HTTPS
Resource: https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset
Action: Prevent

Testing new feed connectivity and parsing methods. Feed will not be fetched
Feed Name: remote_stix_file
[============================================================] 100.0% ...Getting file from the server
Could not fetch file. Please solve before trying again
Deleting feed remote_stix_file

[_Val_]

@Jonathan_Sande1  do you want to take this?

[Juan_Concepcion]

I wanted to add I ran the parameter to " export EXT_IOC_NO_SSL_VALIDATION=1" with same result.  Had to revert to http for this to work.  This is not ideal as some of these feeds are coming from sensitive entities and therefore http connections are not an option.

[Tim_McColgan]

I'd love to hear the outcome of this. I'm following a few different guides here and all are not complete. First and foremost - does this need to be configured on the management server, gateways or both? 

I am looking at sk132193. 

Plus to two attached pdfs. 

I've added feed from sans using: 

ioc_feeds add --feed_name sans_domains --transport https --resource https://isc.sans.edu/feeds/suspiciousdomains_High.txt --format [type:domain,value:1] --comment "#, Site"

But I have no clue where to look to see the contents of the feed and if they downloaded and pushed properly to the gateways. 

ioc_feeds show looks like this: 

Feed Name: sans_domains
Feed is Active
File will be fetched via HTTPS
Resource: https://isc.sans.edu/feeds/suspiciousdomains_High.txt
Action: Prevent

Hey @Aaron_Vivadelli any experience with this 🙂

[Juan_Concepcion]

What to implement on:

So this is implemented on gateways only.

How to see if it was successful:

Search in logs for ioc and you will have entries if it was installed successfully or not.  Another thing is that you can watch the messaging when you download to see if it was successful or not.

I would be very careful when implementing as I did this in a test lab and the feed i was given was to general in scope and ended up killing all communication.

you can also run debugs to see if everything is working correctly: ioc_feeds -d -f

Juan

[Aitor_Carazo]

Hi,

I am trying to configure IOCs and i have the SSL problem too and i didn't solved.

Also I have a question. Wich kind of feed ioc_feeds need?

I mean, if i want to add every week IOCs, this file shoud have all the IOCs or just the new ones?

CC/ @Eduardo_Eiros 

Regards

[Ryan_Ryan]

If your server has a self signed SSL cert, you need to add the cert to the cert store manually on the gateways.

The feed list must contain all objects you want to block (not just a delta of the ones you want to add). If the object is no longer in the list the firewall will remove it from the the block at the next refresh. (by default 5 mins). 

Once you have the feed setup, be sure to regularly check your $FWDIR/log/ioc_feeder.elg file for any errors, there were a few bugs we hit that caused the fetch to fail and the gateway would start allowing the traffic through to the malicious IP's and domains.

[Dariusz_Maslane]

Could you tell me how add manually cert to store on security gateway?

[Mehmet_Sefa_Tec]

Is there any answer for the last question ? I am facing such a problem.

ioc_feeds add --feed_name TORBLOCK --transport http --resource "https://secureupdates.checkpoint.com/IP-list/TOR.txt" --test true
Enter feed format. Should be cp_csv/stix_1.x: stix_1.x
Only STIX 1.X format files will be loaded
Adding
Default value for feed_action is: prevent

Feed Name: TORBLOCK
Feed is not Active
File will be fetched via HTTP
Resource: https://secureupdates.checkpoint.com/IP-list/TOR.txt
Action: Prevent
Feed type: stix_1.x

Testing new feed connectivity and parsing methods. Feed will not be fetched
Feed Name: TORBLOCK
[============================================================] 100.0% ...Getting file from the server
Could not fetch file. Please solve before trying again
Deleting feed TORBLOCK

[Claudiu3]

Hi,

Since it's a  .txt  file you should use: --format [value:1,type:ip]

Also I used transport https ,I am not sure if https would work.

ioc_feeds add --feed_name TORBLOCK --transport https --resource "https://secureupdates.checkpoint.com/IP-list/TOR.txt" --format [value:1,type:ip] --test true