IOC Feeds does not work properly

[oktaycebeci]

Hello everyone.

One of our customer asked for adding IOC feed on R81 version (firewall's Anti-bot, Anti-virus and IPS blades are enabled). First, we tried to import the file Indicators. However, we failed because of file size which is approximately 10 MB. Then, we tried to  separate files, which file sizes are lower than 4 MB, and add them via "ioc_feeds add ..." command. The first ioc feed added successfully. Showed us no error but when we tried to add the second one, feed showed us "Signatures load failed" and "Status: General Error" (images are from our test environment).

On the smart console, firewall status shows us an error that "Anti-Bot: Failed to prepare reputation DB".

Any advise about this problem?

Best regards

[Ruan_Kotze]

How many entries in your file?  I'm guessing you're running into the limit for R81, having said that, I have not seen a hard limit published anywhere.

For what it's worth, R81.20 does bring support for "a significantly increased capacity in the number of observables for URLs, Domains, IP addresses, and Hashes - 2 million and up to hardware limit"

Regards,
Ruan

[oktaycebeci]

Hello Ruan,

In total, all of our three .csv files have 215000 entries for now and the entry number increases everyday (for example 5-10 new malicious links everyday).

Best Regards

Oktay

[PhoneBoy · Accepted solution]

Upgrade to R81.20, which has substantially upgraded infrastructure to support large numbers of indicators.

In R81.10 and earlier, the Pattern Matcher is used, which is also used by other features (IPS, App Control, etc).
The PM itself has a character limit, which you are surely exceeding with more than 215000 entries. 

[oktaycebeci]

Hello PhoneBoy and Ruan_Kotze,

Thank you for all informations you have shared.

Best Regards

Oktay