IOC FEED import does not work

[Prashant_YADAV1]

Hello ,

i am using the checkpoint IOC feed import feature for some known IOC feeds .

one of the know IOC feed is at location 

https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset

this is from firehol

when i try to add in gateway using below command it gives me error 

ioc_feeds add --feed_name Firehol --transport https --resource "https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset" --format [value:1,type:ip] --comment ["#"] 

$FWDIR/bin/ioc_feeder -d -f

gives below

Feed status Firehol :: IOC_FAILED_WHILE_PARSING

cat $FWDIR/log/ioc_feeder.elg | grep Firehol

gives below info

packFeeds: [WARN] Feed Firehol cannot be pushed.
Firehol: Feed format problem. Feed format not supported" severity 0
 Feed status Firehol :: IOC_FAILED_WHILE_PARSING
Firehol: Feed format problem. Feed format not supported

The gateway is R81.10 take 55

there is case open with checkpoint support but as of now they can not tell me reason why it is not workin .

[PhoneBoy]

That file is not in the correct format and thus won’t work with ioc_feeder.
The formats supported are described here: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut...

That file might be suitable for the Network Feed feature available in R81.20 (currently in public EA): https://community.checkpoint.com/t5/Product-Announcements/R81-20-Public-EA-Program/ba-p/150291

[Tobias_Moritz]

Normally, I would never argue with PhoneBoy, but I think he is wrong here.

Your feed seems supported and working (even on R80.40 where this IOC feed feature is missing some features). When you look at the sk132193  PhoneBoy links to, it is even shown as example "Original CSV structure is a list of IP addresses in CIDR format"

I think your problem is not the feed format itself.

Please post your $FWDIR/conf/ioc_feeder.conf.

I guess it is missing the comment statement you provided within your ioc_feeds add command. This is known bug at least in R80.40, R&D is currently working on (yes, I have a TAC case running for this). Maybe you see this also on R81.10.

TLDR:

I got this feed working with the same ioc_feeds add command, you used. The only thing I did: I added the missing comment line to $FWDIR/conf/ioc_feeder.conf:

{
    "external_ioc": "on",
    "interval": "300",
    "ioc_bundle": "/database/ca_bundle.pem",
    "feeds": {
        "Firehol": {
            "feed_action": "prevent",
            "resource": "https://raw.githubusercontent.com/ktsaou/blocklist-ipsets/master/firehol_level1.netset",
            "format": "[value:1,type:ip]",
            "comment": "#",
            "input_name": "Firehol_https",
            "active": "true",
            "feed_format": "custom_csv",
            "transport": "https"
        }
    }
}

After that, I refetched the feeds with:

[Expert@gateway:0]# $FWDIR/bin/ioc_feeder -d -f
Convert your csv format to Check Point's supported csv format. Supported fields: [name,value,type,confidence,severity,product,comment]
All content coming after  ['#']  will be ignored

[Name, Value, Type]
observ1,0.0.0.0-0.255.255.255,ip range,,,,
observ2,1.10.16.0-1.10.31.255,ip range,,,,
observ3,1.19.0.0-1.19.255.255,ip range,,,,
observ4,1.32.128.0-1.32.191.255,ip range,,,,
observ5,2.56.192.0-2.56.195.255,ip range,,,,
observ6,2.57.185.0-2.57.185.255,ip range,,,,
observ7,2.57.186.0-2.57.187.255,ip range,,,,
observ8,2.57.232.0-2.57.235.255,ip range,,,,
observ9,2.59.200.0-2.59.203.255,ip range,,,,
observ10,5.134.128.0-5.134.159.255,ip range,,,,
observ11,5.180.4.0-5.180.7.255,ip range,,,,

Successfully converted
IPS package: Compiled OK.
Signatures loaded successfully

Working fine.

[PhoneBoy]

Always happy to be wrong if the right answer comes out as a result 😁

[Nir_Naaman]

I tried it the easy way - using Infinity NDR Intel.

There are 2,538 IoCs here - all of them get imported cleanly if you define an input feed on this URL.

You can see the output feed from my test domain - published at: https://feeds.now.checkpoint.com/public_feeds/testIOCs-firehol_level1-detect.csv. Should be compatible with R80.30 and above.

Here's all I did - defined the feed as single-type list (IP) without header, and the IOCs started to populate automatically:

 

 

[Prashant_YADAV1]

Thanks a Lot Nir, i will try and see if this works

[Prashant_YADAV1]

Thanks a Tobias, i will try and see if this works.