SandBlast and links inside email

[Shahar_Grober]

I have an ongoing case with TAC about emulation of links inside emails 

R80.10 with MTA take 25 

Issue: TE doesn't block Links with PDF inside emails 

Scenario: I took a malicious PDF from the  Threat Emulation POC (http://poc-files.threat-cloud.com/demo/poc/)

Test #1:  download the malicious file through web browser - AV found it as malicious and blocked the connection (see first log) 

Test #2: I took the same link that I downloaded and copy it to an email and forward it via the MTA - TE emulates the link and finds the email is benign (second log)

I also tried it with other files which are not part of the TE POC. As you can see the file is emulated in the link and is forwarded to the recipient

SK's that I already tried 

sk109573

sk115313

I am not sure if it is a configuration issue as TAC managed to reproduce it and cannot find any issue with the TE configuration 

can anyone approve if this feature is working on his environment or can try to reproduce it?

[18568]

The files from the POC require a password to be downloaded, therefore the MTA has no way to access it and emulate it.

[Shahar_Grober]

I tried also with other links without password. 

In addition the file was emulated by TE and found clean as you can see in the log

[18568]

The first log shows an emulation from the web browser so the password is not an issue (the user - you I guess - entered the password and the file is already downloading so Sandblast can intercept it), but it does not show being clean, it is correctly detected as malicious, you can see the action is "Prevent" (blue shield).

If you tried with other links without password in emails could you please provide the logs for example, including the more detailed logs when you double click on the line.

[Chinmaya_Naik]

Shahar Grober As testing, you send a different link (fresh link) password protected pdf file (different pdf file not as previous) and also that file till not downloaded through the web browser.

Please check and update.

And also update the Threat Emulation Engine if it's not updated.

Refer below Sk   

sk95235 |  sk92509

#Chinmaya Naik

[Shahar_Grober]

- the version is updated to the latest version (I already checked it with support)

Correct me if I am wrong but if a file is inspected as malicious by AV, shouldn't it be caught by TE even without emulating the file even if it is password protected - using file hash?

+ the verdict of the file is Benign

if it is password protected - TE should give an error that file cannot be emulated and not benign 

[Benoit_Verove]

Hi,

I think that the PDF file in not password protected BUT the web access to the PDF file is password protected.

Have to tried to host the PDF file on another location ?

Regards,

Benoit

[Shahar_Grober]

Not yet, but working on it.

What I don't understand is why the files are emulated and the verdict is benign although it is password protected 

[Thomas_Werner]

Hi Shahar,

if you follow emulation with VNC you might see that we simply open PDFs and Office documents and in case password protected the password challenge is displayed. So within the emulation nothing happens so the file´s verdict is benign.

Afaik it is not easy to determine if a file is encrypted pre-emulation.

Regards Thomas

[Shahar_Grober]

Hi Thomas,

The pdf is not password protected but the website is, you can check:

http://poc-files.threat-cloud.com/demo/poc/

I would expect that if the file couldn't be accessed then it should show verdict other then benign 

I am trying to put the file on another website which is not password protected but didn't have time to do it yet. 

[Thomas_Werner]

Hi Shahar,

I think at a certain point trying to solve weblink detections in email with any APT solution will always be only a half-baked solution. We are trying to protect the web vector in email which will never achieve 100% coverage.

The real solution is to implement proper sandboxing in your HTTP/S vector which is possible in prevent mode with SandBlast. And for certain other attacks you would also need proper Endpoint protection (e.g. SandBlast Agent).

Regards Thomas

[Shahar_Grober]

Hi Thomas, 

I have this ongoing case with support and I think that the design here is wrong. 

1. TE MTA will stop emails with malicious URL only if the file is downloadable 

2. if the file is not downloadable but the URL is still malicious - it will not be stopped by the TE  

This is bad behavior in my opinion!

Even if the URL is a link to malicious PDF and the link is detected as malicous and prevented by the AV and by URL Filtering when downloaded by the client or when passing through the GW, if it cannot be downloaded by the MTA it will be marked as benign by TE 

Lets take the following scenario: 

1. I sent an email with a malicious PDF but the download link is not active or contains a password to the website which I attach to the email 
2. After the email arrives in the MTA it is scanned by TE and pass as benign because the malicious document cannot be downloaded
3. Once the user gets the email and opens the file with the password to the website or when I activate the link after it passes through the MTA. 
4. Since the file is malicious the endpoint is exposed to the malicious file because it is not detected by the MTA (the file has to be stopped in 

This behavior can be prevented already in the MTA and be avoided by the users by simple AV and URL Filtering scan in the MTA 

[Thomas_Werner]

Hi Shahar,

"2. if the file is not downloadable but the URL is still malicious - it will not be stopped by the TE" 

This is correct for TE but with the new MTA and AV inside MTA available the EMail's URL will be neutralized if it is known as malicious to our AV URL reputation. 

Regards Thomas 

[Shahar_Grober]

hi Thomas,

This is exactly what I thought but I don't see malicious URLs being blocked in the new MTA  (Take 28) 
This is exactly what I argue with support about

I will give you an example with the following URL I tested 

the above URL is categorized  as phishing

It was caught by our AV via HTTP

When sending this URL in an email it goes through the MTA without even being scanned 

So maybe I am missing something but this is not the behavior I expect when sending malicious URL in an email 

[Shahar_Grober]

Thomas, 

According to support "Only URLs with supported file extension at the end of the actual URL will be scanned by the Threat Emulation blade."

So what the point of adding the AV to the MTA? 

1. MTA AV + TE cannot stop malicious URLs inside emails if the URL  doesn't contain a file

2. If the file cannot be downloaded it will bypass the MTA TE and AV 

3. If the URL contains a malicious file, The file extension has to be supported by MTA - so files which are supported by AV but not MTA will not be scanned 

I would expect from the MTA to also scan malicious URLs as any mail relay does

Can you please check it with R&D. I want to understand how the "Link Inside email" functionality works and what the AV inside the MTA adds to it if it is unable to scan malicious URL's (URL Reputation)?

[Thomas_Werner]

Hi Shahar,

Shahar Grober wrote:

[...]

So what the point of adding the AV to the MTA? 

1. MTA AV + TE cannot stop malicious URLs inside emails if the URL  doesn't contain a file

2. If the file cannot be downloaded it will bypass the MTA TE and AV 

3. If the URL contains a malicious file, The file extension has to be supported by MTA - so files which are supported by AV but not MTA will not be scanned 

I would expect from the MTA to also scan malicious URLs as any mail relay does

1)

As support said TE will only try to download URLs/links where the link ends with a TE supported file extension, e.g.  https://www.test.com/test.pdf. AV is supposed to check all links in the email with our URL Reputation service.

2) 

Not in general. If you set Engine Mode to "Fail-close" TE will block all emails with links pointing to supported file extensions but "undownloadable" content. If you have "Fail-Open" mode (as it is default) the emails will pass.

AV does not rely on file download ! It simply checks the URL against our reputation service.

3) 

Correct. That would require AV to support "link download". This would be a RFE.

As said above AV is supposed to check URL reputation for all links inside the email subject and body.

If this does not work for you it is a bug and TAC should handle it.

Regards Thomas

[Shahar_Grober]

"As said above AV is supposed to check URL reputation for all links inside the email subject and body.

If this does not work for you it is a bug and TAC should handle it." 

Thanks for the clarification Thomas - this is exactly what I thought the behavior should be and this is what I am trying to explain to TAC. 

I even verified it with the scenario above.

The problem is that TAC thinks that malicious URL's inside emails shouldn't be scanned using  AV and URL reputation. maybe because it is a new functionality they are not aware that this is now part of the MTA and claim that only downloadable files should be scanned by MTA. 

[Thomas_Werner]

Hi Shahar,

I got it confirmed by RnD that AV in MTA should scan URLs with reputation.

So you can tell TAC to contact the relevant RnD owner.

Regards Thomas

[Thomas_Werner]

Hi Shahar,

just an update because I ran some tests in my lab.

What might cause your issue is a degradation in URL inspection during the last TE engine.

In maillog you should find something similar to this:

Jan 28 10:40:10 2019 R8020SA postfix/smtp[16350]: 43p4Np04F8z8d0G: to=<linda@acme.com>, relay=127.0.0.1[127.0.0.1]:10025, delay=0.1, delays=0.03/0.02/0.01/0.04, dsn=4.0.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 Temporary scan failure. Email Session ID: {5C4ECDFA-0-FEFE1BAC-1D10} (in reply to end of DATA command))

The fix is deployed already with the latest TE engine update 58.990000298.

Threat Emulation Engine Update - What&apos;s New? 

TE engine deployment for local TE appliances is expected to be finsihed by mid of this week..

Regards Thomas

[Shahar_Grober]

Thanks thomas, 

I have an open SR with TAC on that issue as well, I asked them if it might be connected to the face that malicious links are not scanned. I am waiting for the MTA fix in order to be able to test it 

[Thomas_Werner · Accepted solution]

Hi Shahar,

we finally tracked it down.

If you enable full debug of MTA (ATRG: Mail Transfer Agent (MTA)  you will see this log for your testing URL:

[mtad 3200 3780316048]@R8020SA[30 Jan 17:43:56]  [TE_IS (TD::All)] te_is::CurlSender::Send: <response> http error code: 200, url: https://rep.checkpoint.com:443/url-rep/service/v2.0/query?resource=http://www.trendhure.com/top10.ph..., data:

{"response":[{"status":{"code":2001,"label":"SUCCESS","message":"Succeeded to generate reputation"},"resource":"http://www.trendhure.com/top10.php?id=63435","reputation":{"classification":"Volatile Website","severity":"Medium","confidence":"High"},"risk":50,"context":{"protection_name":"Phishing_website.bebwd"}}]}

So in general URL reputation works fine in MTA.

But we decided to only handle URLs in prevent mode if the risk level of a URL is 80 or greater.

In this case it is 50 - that´s why the URL is not blocked.

This decision was taken due to less false positive rate in a first stage when enabling AV in MTA.

Currently the level is not adjustable.

Regards Thomas

[Supporto_Checkp]

Good Morning Thomas,

I' am in R81.10 ; what file i should check for have your same debug's output?

[anstelios]

@PhoneBoy 

Hello,

Is there documentation somewhere describing this behavior because I am not able to find it.

We have noticed that on new R81.10 installations, the "Risk" of the link must be above 90 in order to be blocked.
Anything below 90 is "Detect".

Is this behavior configurable?

[PhoneBoy]

It should be pretty clear from the information linked above how it works (namely it looks up the URL against our ThreatCloud).
Not aware of a way to adjust the false positive threshhold.

[Shahar_Grober]

Hi Thomas,

Nice observation!!

I just finished a call with TAC and they explain the same to me. 

Here is the explanation:

">> AV over MTA works based on confidence and this URL has a very low confidence therefore we do not block it in AV or MTA. This new design is made to prevent FP incidents. 
>>AV over HTTP does not have the same capabilities yet and therefore browsing to this link was blocked"

However, in my opinion, there is a flaw in the design here which can still be used by attackers or lead to massive confusion for security administrators.

Let's say I am sending a malicious low-risk link to the user. the flow here is :

Email scanned by MTA --> the link found as benign due to low-risk level (less than 60)  --> email passed on to the user --> user clicks on the link inside the email --> link is stopped by URL filtering/AV reputation or by the endpoint AV /SBA (or not)

so what happens is that the email will pass the MTA but the link will be stopped in the network layer (or worse if the user is outside of the network perimeter and not protected by the gateway it will allow access to the malicious website and then we are in the hands of the endpoint AV/SB agent)

So we have 3 scenarios which lead to bad user experience in case of: 

1. False Negative by MTA / False Positive by the Network AV/URLF

the email arrived to the user and he clicked on the link which was blocked by the network AV/URLF

link shouldn't be blocked because it is a false positive 

2. True Negative by MTA / True positive by the Network AV/URLF 

the email arrived to the user and he clicked on the malicious link which was blocked by the network AV/URLF 

the user shouldn't get the email on the first place because it is malicious (the MTA doesn't do its job) 

3. True negative by MTA / user is outside of the network perimeter 

The email arrived to the user, he clicked on the link and the malicious content is downloaded to the PC. let's hope that the Endpoint AV/SBA/forensics can identify it as malicious 

The inconsistency between the same engines verdict on different attack vectors is problematic for security administrators. the whole issue started because I didn't understand why a malicious link was not stopped by one vector (email) but was stopped by another vector (web link). there is also no way to understand or compare the difference between the different engines. this is why I opened the TAC case which took more than 2 months to figure out by the support engineers.

I can send an email with malicious links which will not be blocked by the MTA and might or might not be stopped by the gateway/endpoint

inconsistency = inability to know if the link is malicious or not. 

I understand that the goal is to lower the false positive rate, but I would also would like to prevent my users to get links which are already been scanned by the MTA and can be malicious. In my opinion, allowing users to click on links that might be malicious is defying real-time prevention since it can be too late. It should be stopped by MTA 

[Chinmaya_Naik]

Hii Team,

Please suggest if I am wrong.

So based on the above discussion :

In MTA, Threat Emulation work only if that URL end with any file extension, like http://abc.com/xyz.pdf also http://abc.com leads to the PDF file to download (xyz.pdf)

It did not scan if it's not to leads any PDF or any known extension like simple http://abc.com

When enabling AV in MTA then URL reputation is checked over MTA base on the risk level. So if the risk level is  80 or below 80 then that malicious URL is not blocked even that the malicious URL have severity": "Medium", "confidence": "High".

As on the above scenario, URL is bypass but If the customer is using Checkpoint URL Filtering then when the user is open that malicious link its BLOCK by Checkpoint URL Filtering. Because CP URL filtering is working base on severity and confidence level, not by Risk level.

@Chinmaya

[Shahar_Grober]

This is an exact description of the MTA functionality

IMHO malicious links should be stopped in the MTA