MTA and Non Delivery Report

[Olga_Kuts]

Should the MTA send a NDR (Non delivery report) to the sender if the mail was not delivered? Is there documentation about this? Can this be set up using postfix in CheckPoint?

[PhoneBoy]

According to this SK, it seems it should do so: Anti-Spam blade sends Non-Delivery Report (NDR) when Mail Transfer Agent (MTA) is enabled on Securit... 

It is the default behavior.

[Olga_Kuts]

Thanks. I also saw this sk, but can there be experts in the community, who uses this function? Because this function has never worked by default. Or are there any nuances of her work? MTA not generating the "undeliverable message" when an email is rejected is not relevant for me, I think. The situation is as follows: it is necessary that the MTA send a NDR if the internal mail server is unavailable.

[PhoneBoy]

If I'm reading this SK correctly, it seems we're only generating NDR messages if we reject the message.

That said, if the remote server is unreachable, it seems reasonable that we should generate an NDR message.

Will check.

[Biju_Nair]

If I am assuming it correctly, you are expecting an NDR from CP gateway(MTA) device to be send to original sender(internet).

sk109339 says that Mail Transfer Agent (MTA) can be configured on the Check Point Security Gateway only as an incoming relay.

I also had a scenario, wherein the TE appliance was deployed in MTA mode and an NDR was sent out to the original sender(internet) by relaying it through the mail server.

[Olga_Kuts]

Tell me, please, in more detail, how you organized such scheme, and what settings are needed.

[Biju_Nair]

In my case, the scenario was Internet>>Email gateway>>>TE appliance(MTA)>>Exchange Mail server.

and the exchange email server decided to reject the message due to attachment size limit. NDR was sent to the original user(internet) by check point MTA because there was a forwarding rule in check point MTA allowing to send the NDR to that sender domain. In my case it was  (*) hence I guess it worked.

Also I believe the settings in the email server is also to be considered while sending out this kind of reject messages.

Whether the email server wish to sent the NDR directly or through our MTA just before it.

[Shahar_Hazut]

Hi,

1. An email is being bounced in case of an unrecoverable error. So in case of an attachment size limit, it is considered by the mail server/MTA next hop as an unrecoverable error, while "mail server unavailable" is considered as temporary failure.
In case of temporary failure, the email would be kept on "deferred" queue for 5 minutes, and then MTA will try to communicate with the next hop again. In case of another failure, the "penalty" of this email to be left on queue would be doubled (10 minutes), and would exponentially grow with any additional failure.
The maximal time an email would be reside on MTA queue is 5 days by default.

2. As a bounce email is actually a new email sent back to the sender, we should have a relevant rule on MTA forwarding rules for the sender domain. This is why a common practice is to use "*" as another rule, which would be the rule for relaying all the rest of domains, including the sender domain.

Hope it clarifies things.

Let me know if something isn't clear or missing.

Thanks,

Shahar Hazut

Mail security Team Leader

[Olga_Kuts]

Shahar,

Thanks a lot for reply!

Few questions for you:

1. Where can I change default parameters, which you described? I mean time of "penalty" and the maximal time an email would be reside on MTA queue.

2. How should my rules for mail look if I need to:
a. Decrypt the mail for the test.com domain (SMTP over TLS) and forward it to the internal mail server. Inspect these emails with Threat Emulation.
b. Do not touch mail from other domains, don't emulate them.
c. Have the ability to send a NDR.

[Shahar_Hazut]

Hi Olga,

1. The "penalty" is being controlled by Postfix starting with "minimal_backoff_time" (5 minutes by default), up to "maximal_backoff_time" (~66 minutes by default)
The maximal time an email would reside in queue is "maximal_queue_lifetime" (5 days by default)
You could manually configure Postfix parameters using sk101870.

2. In case the MTA receives the emails directly from internet, you should configure the MTA with the DNS server, so that it would be able to relay bounce emails according to the sender domain.
In this case the only rule should be used is of test.com as the domain with the required next hop.

In case the MTA is internal, receives the emails from a preliminary MTA (e.g. Anti Spam), then it is the preliminary MTA responsibility to reject emails with recipient other than "test.com", and in this case you could either use the same rule from above, so that bounce emails would be relayed based on DNS queries, or create another rule with "*" as the domain, for directing all other emails (that are only bounce emails in this case) to a specific next hop.

[G_W_Albrecht]

Thank you, that is quite helpfull !

[Olga_Kuts]

Shahar,

One more question: can I sent NDR report to sender not directly, but via external mail server? I need this NDR to come to an external mail server, and from an external server to the sender. If it is possible, I must create a second rule with domain * and with next-hop "my external mail server", correctly?

[Shahar_Hazut]

Hi Olga,

Correct, this is exactly what should be done to achieve this behavior.

[Olga_Kuts]

Shahar,

Thanks for help. It's pleasure to work with you!

[Thomas_Eichelbu]

Hello, 

first of all this is a very good thread here!

but i have some follow up questions:

for example:
A customer,  GW with MTA and two TE appliances as a cluster.
The MTA as "email routes" for all his email domains pointing to the Exchange server.
But NO "*" route back to the anti spam GW to send out any Bounce or NDR messages ...
so someday the Exchange server is unavailable, for some hours!
it turned out, ALL emails over night got lost, nothing was kept in a queue for 5days
in the Smartlog i saw all emails bounced, but no sender ever got a NDR or bounce message back.
/var/log/maillog was overwritten due log rotation many time, ok bad luck too. no real logging evidence of what has happend is present anymore.

so where are those mails? 
as u you wrote correctly the max value for maximal_queue_lifetime" in „/opt/postfix/etc/postfix/master.cf.defaults" is 5d!
but, this configuration file, i think its not used directly in first place, it just represent the default or possible values.
the main configuration file is „/opt/postfix/etc/postfix/main.cf" it seems this file gets created new on every policy install.
and all those values set in „/opt/postfix/etc/postfix/main.cf" overide „/opt/postfix/etc/postfix/master.cf.defaults" .
And the settings are derived from the MTA userinterface of the GW properties.
And there are settings for "maximal_queue_lifetime" are 0d, not 5. So why 0 days?
so how many hours are 0 days?  white duration is 0days?

why does Check Point reduces this value to 0 days? Does it mean, if the mail delivery is unsuccesful, bad luck all mails gets
deleted. if no "*" back to the sender exist you have a bad day explaining what happend.

if you want to override it, you have to follow sk101870 and create: $FWDIR/conf/mta_postfix_options.cf

but why?
why does Check Point set this value in its default to 0 and allows all emails to be lost?

for the end user it means, to dig into all postfix options and search for the correct values to tweak $FWDIR/conf/mta_postfix_options.cf to make emails survive.

or can you provide us with correct values which controls the queuing of mails?
+ how often to retry sending mails per time interval?
+ how long to keep a retry queue
+ a log when emails got deleting, and not just bounced.

the ATRG MTA is really good, but this part is missing.
how to avoid loss of mails in case delivery is not successful!

please update us here on this topic.
this is often asked my customer and we often run into situation were we lost mails, because of this issues.

best regards
Thomas

[cstueckrath]

I have to post a #metoo here, as we had observed this behaviour as well.

[Thomas_Eichelbu]

Hello Christian,

i already made TAC ticket for this ..
they said:

From previous experience what is causing this is that hopcount_limit(which is 50) is reached before maximal_queue_lifetime (which is 5d).

The only way to fix this is to change the configurations to do retry/requeue lest often and then it will take longer to reach the hopcount_limit. We cannot cancel the requeue mechanism as we need an extra header to be added to the mail upon retry for the next-hop and changing the hopcount_limit can have effects on the next hop which will receive mails with many headers.

Please try playing with these 3 configurations:

1) queue_run_delay(How often the queue manager scans the queue for deferred mail)
2) minimal_backoff_time(The minimal amount of time a message won't be looked at, and the minimal amount of time to stay away from a "dead" destination)
3) maximal_backoff_time(The maximal amount of time a message won't be looked at after a delivery failure)

You can start with setting #1 to 2h meaning it should reach the hopcount_limit after 100 hours(50*2) and see if it meets the customers request.

so i is refering to this article:
http://www.postfix.org/QSHAPE_README.html#deferred_queue

strange stuff ... 
we will try to find the best values for 
queue_run_delay
minimal_backoff_time
maximal_backoff_time
to set the settings in "$FWDIR/conf/mta_postfix_options.cf" and see how it performs

lets see what happens.

[Itall]

Recommended tunning , play with postfix parameters

queue_run_delay
minimal_backoff_time
maximal_backoff_time

in "$FWDIR/conf/mta_postfix_options.cf"

Does not working in R81.20 (maybe also R81.10), the same thing happened to us a few days ago (loss of mail after about 1 hour 40 minutes).  We have also TAC ticket opened... 😞 

[Thomas_Eichelbu]

Hello, 

well bad to hear.
For some customers this is still an ongoing issue, no real solution by now.
A RFE was raised, but already 2 years ago, i hope for the best ... 
I fear Check Point will resolve it by declaring all onprem TE appliances obsolete and move all functionalities to the cloud.


please post your solution if TAC manages to fix it!

[Itall]

Hello,

It's also bad for us. Given that we are dealing with an internal relay, I would expect the possibility of configuration so that postfix accepts at least the default status of holding the queue for 5 days in case of unavailability of the relay host, or even more (depending on the storage of the SBLast appliance and the number of emails, which can be found out from the statistics in some time). We have the same unavailability situation replicated in the LAB on the stable distribution of redhat with the latest stable postfix, and there it behaves as it should (according to the backoff values, it quadratically increases the intervals of further attempts and as soon as max_backoff is reached, further attempts are made in these intervals until the max queue lifetime is reached. Whatever parameters SBlast has set, it goes its own way, it performs 2 - 3 attempts with different intervals, which in total is about an hour, and then there are attempts every minute, which corresponds to the default value of the intervals in case relay transport is unavailable, but even if this we set the value to a larger one, so it does not accept it). For now, we are solving it with the local CheckPoint representative, who could hopefully renew the possible RFE (we'll see). In the event that such a situation occurs on a holiday or on a weekend, we lose a lot of mail, which is not very acceptable, because we also receive various confirmations via e-mail, which we are obliged by law to archive (the sending party cannot otherwise). We will definitely share if there is a solution.

[Itall · Accepted solution]

Hello Thomas, 

good news 🙂

today we deployed the requested custom hotfix HOTFIX_R81_20_MTA_T13_085 on the R81.20 JHFA T26 to one of our sblast gateway and it seems to be working.
Method:
1/ request an MTA hotfix according to the gateway version at CheckPoint (request to CHeckPoint support)
2/ after installation, the "maximal_retry_timeOut" parameter must be set in the MTA configuration ( file /var/$FWDIR/conf/mail_security_config )
3/ policy installation on the Sand Blast/MTA gateway

Our currently tested scenario (seems working):

maximal_retry_timeOut=300 (5 minutes) in mail_security_config
in /opt/CPsuite-R81.20/fw1/conf/mta_postfix_options.cf (must be created, not exist in default install)
hopcount_limit = 50

Maximum delay (SMC gateway object in the MTA section) 90 minutes (we plan decrease to 60 minutes, it is our default value before this case)

current behavior from the log:
The sent dummy email is held in the queue for the set 90 minutes (settings from SMC)
and subsequently repeated every 1 plus 5 minutes 50 times and email has been deferred/scratched after 6 hours instead 2 hours 🙂 , we do tune to extend the time.
(it is necessary to adjust the times and repetitions according to the necessary scenario/need)

CheckPoint promises that it should be added to the RFE so that each customer can
choose whether they want the default settings or whether they want to set their own behavior.

regards
Ladislav

[Chris_Atkinson]

I concur, interesting / helpful discussion.

To add something that hasn't already been mentioned here for others that may come across the discussion later...

1. Within the MTA Advanced Settings there are parameters concerning 'delay' & 'disk usage' that can be configured.

2. The postfix parameter: smtp_defer_if_no_mx_address_found (default: no) may also be relevant to some.