URL and Application blade issues

[Martin_Valenta]

We had issues with app control and url filtering blade today and this fixed our issues per recommendation from TAC..

cd $FWDIR/appi/update/
curl_cli -O http://secureupdates.checkpoint.com/appi/v4_0_1/gw/appi_db.C.tmp
mv appi_db.C.tmp appi_db.C
fw load_sigs

[Vato_Chantladze]

Hi!

This started yesterday. I saw this issue customers running on R80.10

Support should give you valid appi db file and after this is recommendation to move it to current one:

1.    Download the following file: appi_db.C.tmp

2.    Copy it on the gw to: $FWDIR/appi/update/

3.    Rename it to appi_db.C

4.    Load signatures with “fw load_sids”

BR

Vato

[Shahar_Grober]

What did it effect exactly,

we had a problem with SMTP over TLS which Check Point started to drop yesterday afternoon without any clear reason

[Martin_Valenta]

It basically didn't match anything on application layer policies, everything got dropped on clean up rule..

[Vato_Chantladze]

The same here. Everything as you described.

If you restart the firewall it will block all the traffic. You need to change db file without reboot

[Shahar_Grober]

I just talked to support 

you can verify the issue with the following command:

grep -i appi_app_db_get_app_log_data /var/log/messages*

and get

"pp_db_get_app_log_data: failed getting application object from hash"

Nasty bug!!!

Does it occur only in R80.10 or also other versions?

[Peter_Lyndley]

We have only seen it on r80.10 in our customers estates at the moment.

Does a policy push then overwrite these values/files  ? or are we safe to carry on as normal once the TAC workaround is in place?

thanks

Peter

[Vato_Chantladze]

So far I saw only R80.10 with automatic updates enabled on the gateway side.

There is no sk for this. support center should create it for other customers as soon as possible. It is not a lite bug it makes a big problem to the production environment

BR
Vato

[Mor_Himi]

Hi Shahar,

nice seeing you here 🙂
The issue has been resolved yesterday IL time, can you update if if suffer form this now?
Thanks...
--Mor

[Shahar_Grober]

Hi Mor,
Good to see you as well.
Yesterday we had to manually change some rules from Appi objects to legacy ports which solved the issue.
At the moment we don't experience it but we don't know if we have the issue until we go back to Appi objects in policy. I get confuse messages from TAC about if it still exists or not. as you see from my previous post it seems that the issue still exists

I can confirm that my package version is:

pkg_version (240419_3)

[Mor_Himi]

Any WA you have applied is not needed anymore.

TAC is getting their updates from me.

Your package version is fine, please let me know if you suffer  anything related to this case.

[Mor_Himi]

Hi Vato,

Can you please enable online online update and clarify if the issue has been resolved for you?

There is no need to apply any workarounds any this phase.

Thanks....

   --Mor

[LukeOxley]

The fix has worked for us - has anyone verified if a reboot / policy install overwrites the fix?

[Mor_Himi]

Reboot/Policy install will not affect the fix since the online package has been updated yesterday night (IL timezone).
this has been verified in our lab.

[Mor_Himi]

Hi All,

The issue has been resolved ~15 hours ago.

No need for manual overrides or turning off Automatic Updates.

Thanks...

   --Mor

[Martin_Valenta]

I didn't disable auto updates, i just applied fix and afterwards it started to work again..

[Mor_Himi]

Hi Martin,

Can you re-enable online update and reply if the issue has been resolved for you?

Thanks...

   --Mor

[Vato_Chantladze]

In our case, we tried to turn-off turn-on automatic updates but it was not getting it. Only db replacement is solving it. 

[Mor_Himi]

Can you please share the output of:

cat $FWDIR/appi/update/Version 

[Martin_Valenta]

cat $FWDIR/appi/update/Version
(
:pkg_file_name ("appi_urlf_db_pkg.tar")
:md5sum ("fc4ceae34aecb0394c17109687dcff1a")
:pkg_version (240419_3)
:appi_version ("140419_1")
:urlf_version ("240419_3")
:pkg_timestamp (1556091487)
:appi_filename ("appi_db.C.tmp")
:urlf_filename ("urlf_db.bin.tmp")

[Mor_Himi]

10x, looking great.

Any pkg version => 240419_1 (yours is 240419_3) should work without any issues.

[Martin_Valenta]

@Mor_Himi  shoulnd't be here https://status.checkpoint.com/ any sort of note ?

[Mor_Himi]

Good question.

Since the issue and its resolution didn't involve any of our cloud services it doesn't seem like the correct place.

I will however check internally how best to communicate such incidents going forward.

[Martin_Valenta]

Thanks, at least some official statement, since we'll need to do RCCA on our side for this.

[Mor_Himi · Accepted solution]

As the relevant R&D group manager, I can share that:

1. An updated package whit a resolution was available online at Tuesday, April 23, 2019 11:30:00 PM GMT+03:00 DST
2. The issue only affected a small subset of our URLF customers on R80.X.
3. We're conducting our own RCCA, but that will take some time.
4. I will share more information about our findings early next week.

[_Val_ · Accepted solution]

Hello all,

The issue with the update is expected to be resolved now. However, if you are still experiencing it, here is the workaround (mostly mentioned above already).

Firstly, the symptoms:

-        APPI is no longer working with the following error messages:

[ERROR]: appi_app_db_get_app_log_data: failed getting application object from hash

-        Issue occurs after URLF/APPI database update

The workaround:

Local revert procedure [NOTE: This is only relevant for R80.X GWs]:

1. Download the following file: http://secureupdates.checkpoint.com/appi/v4_0_1/gw/appi_db.C.tmp
   for example, by using the mentioned command: 
   curl_cli -O http://secureupdates.checkpoint.com/appi/v4_0_1/gw/appi_db.C.tmp
2. Copy it on the GW to: $FWDIR/appi/update/
3. Rename it to appi_db.C
4. Load signatures with “fw load_sigs”

Some clarifications from developers:

1. The workaround above won’t affect the online update. Once the unified URLF & APPI package is ready it will auto download as usual
2. There is a chance that the workaround will be needed twice since the most common case GW check for updates every 2 hours while the Unified package takes up to 4.5 hours to be online.
3. The best way to verify that you have fixed online package is to:
4. Run “cat $FWDIR/appi/update/Version”
5. You should see something like:

(
   :pkg_file_name ("appi_urlf_db_pkg.tar")
   :md5sum ("0e8f4e2b5d1172413fce76d31728e3cb")
   :pkg_version (230419_5)
   :appi_version ("220419_1")
   :urlf_version ("230419_5")
   :pkg_timestamp (1556039502)
   :appi_filename ("appi_db.C.tmp")
   :urlf_filename ("urlf_db.bin.tmp")
)

The workaround is needed until the package version (highlighted) shows 230419_6 / 240419_1 or higher

If after all the steps above the issue is not resolved, please open a TAC case at once.

[Martin_Valenta]

what about on r77.30?

[_Val_]

Should work just fine

[Mor_Himi]

indeed as Valeri mentioned, R77.X wasn't affected.

[Vato_Chantladze]

It sounds like a non-technical question but what is the best we can answer to customers who involved this issue and asking: How can we be sure it will not happen again?

BR
Vato