Files on Threat Emulation

[Giancarlo_Cotta]

Hi

When a file arrives at the emulator its hash is compared with a list of hash in the emulator.

If it is a file that is not yet known its hash is stored to check it with the hashes of future files.

Is the file also stored to send a clean file to the client faster?

For how long time the file is stored on the emulator?

Thanks a lot

[Vincent_Bacher]

Hi. The file itself is not stored in the cache. Just the hash. And how long the hash is stored.... dunno. Believe you can modify the number of file hashes to save in local cache, if I don't remember wrong. 

[Giancarlo_Cotta]

Hi

I'm looking this book: CP_R77_ThreatPrevention_AdminGuide.pdf.

In page 25 I can see:

******************
Optimizing File Emulation

Files have unique cryptographic hashes, these file hashes are stored in a database after emulation
is complete. Before emulation is run on a file, the appliance compares the file hash to the
database:
   If the hash is not in the database, the file is sent for full emulation
   If the hash is in the database, then it is not necessary to run additional emulation on the file
This database helps to optimize emulation and give better network performance.

******************

Please, I need to understand "optimize emulation and give better network performance".

When one file is extracted: I suppose that if one file was previously emulated and extracted is not necessary to emulate and extract this file again.

But if is not necessary emulate this file again, I suppose that the this file (after extraction) was store when it was emulated.

So, is ready to send to user when the Sandblast receive its hash again.

Is it correct or I lost something?

Or the "optimization" is valid only for the files that is not necessary to emulate again because are "clean" without extraction?

Thansk a lot

Giancarlo

[Vincent_Bacher]

optimize emulation and give better network performance

That means, if a file hash is already stored in the cache, when a file with the same hash arrives in an email attachment or to be downloaded from web, there is no need to send it to the te for full Emulation again.
This reduces network traffic to a local sandblast appliance or checkpoint cloud. And if it has not to be emulated again, emulation is optimized.

But if is not necessary emulate this file again, I suppose that the this file (after extraction) was store when it was emulated.

Not the file itself, just the file hash.

So, is ready to send to user when the Sandblast receive its hash again.

The gateway detects a file, calculates the hash and compares it with ist file hash cache. If it's a known hash, the file is passed or dropped according to the last scan/emulation result. The sandblast will not be involved again.
"Sandblast" may be located on the gateway itself (emulate locally), in the cloud or on a local, separate device.

Or the "optimization" is valid only for the files that is not necessary to emulate again because are "clean" without extraction?

Not just the clean ones. When a file "arrives" at the Gateways which has a hash which is stored in the cache and already being classified as malicious, it will be immediately be dropped, i assume.

If i am wrong, Checkpoint staff and CheckMates, please correct me, thanks

Cheers

Vincent

[Giancarlo_Cotta]

Hi

I'm looking in the Threat Extraction configuration.

I can see that in the section "Resource Allocation" there is:

"Delete stored original files older than ..."

Which kind of file are considered in this option?

Thanks

Please see attach image

[Vincent_Bacher]

Suspicious / malicious files are stored in quarantine area on the sandblast appliance for further examination. For instance to check if there are false positives. 

[Giancarlo_Cotta]

Hi, thanks for your answers.

I’m looking this document:

Day2-01-SandBlast training-SandBlast Local emulation-v1.0.pdf

On page 39 I can see:

• Files will receive a TTL (Time To Live) in cache of 7 days
• After 7 days cache entries will automatically be removed

I suppose that the default time to live that files are stored in the cache is 7 days.

But in the previously image that I attached before I can see default value is 14 days.

-->      "Delete stored original files older than ..."

Are these different kind of files?

Is there only one cache for all the VM images or one cache for all VM image?

Thanks

[Vincent_Bacher]

Never had a Sandblast training so I don't know this document. Questions regarding ttl may be replied by Checkmates from Checkpoint  

And I assume there is one cache independent of VM. 

[Thomas_Werner]

Hi Giancarlo,

There is a confusion between Threat Emulation and Threat Extraction here.

1) Threat Extraction does not have a cache

If a file should be extracted by policy it will always be extracted. Even if the same file arrives later on the extraction process will run again (it only takes a few seconds). The 14 days default storage option in the GUI is for the "original files". So if you get a Threat Extracted file you have 14 days to use the "get original file" option to retrieve the original file from the gateway

2) Threat Emulation has a cache

Threat Emulation has a local cache (gateways and emulators). You can check the content of the cache by running

[Expert@R7730Cloud:0]# tecli cache dump all

Images Uid List
===============

|sha1 |file type |image |verdict |confidence|severity |date |hits |ttl |comment
|----------------------------------------|----------|------------------------------|----------|----------|----------|----------|-----|----------|----------------------------------------
|5b03ccec77b416805d6d8e270d33942aaedcc6dd|pdf |Win7,Office 2013,Adobe 11 |benign |None |None |5-7-2018 |1 |5-14-2018 |
|5b03ccec77b416805d6d8e270d33942aaedcc6dd|pdf |WinXP,Office 2003/7,Adobe 9 |benign |None |None |5-7-2018 |1 |5-14-2018 |
|1f25f4840b104a503cfe5c7f7b6b4a30e09002cb|pdf |Win7,Office 2013,Adobe 11 |benign |None |None |5-7-2018 |1 |5-14-2018 |
|1f25f4840b104a503cfe5c7f7b6b4a30e09002cb|pdf |WinXP,Office 2003/7,Adobe 9 |benign |None |None |5-7-2018 |1 |5-14-2018 |

You can also manipulate the cache via some # tecli cache .... commands.

Check out this great SK for a full reference of "tecli cache":

ATRG: Threat Emulation 

This cache is queried each time a new file arrives. If the SHA1 is found in the cache the related verdict and actions will be immediately taken without re-remulating the file (this improves performance and throughput). You can check the cache hit rate via # tecli show statistics.

[Expert@R7730Cloud:0]# tecli s s
Last day Last week Last 30 days

General Information:
--------------------
Scanned files: 0 0 0
Malicious files: 0 0 0
Files filtered by static analysis: 0 0 0
Files error count: 0 0 0
Files filtered by local cache: 0 0 0

The default cache live time for an entry is 7 days - meaning if the same SHA1 hash is not seen for more than seven days the entry will be removed from cache. Each time the entry will be seen again the cache life time is extended by 7 days. The cache life time (TTL) can also be manipulated via # tecli cache ttl ...

Regards Thomas 

[Giancarlo_Cotta]

Thanks for answer, now is clear!

Giancarlo