Gaia R81.20 Titan Zero-Phishing is breaking Apple iCloud Private Relay, how to whitelist and debug

[RamGuy239]

Greetings,

I'm struggling with Apple iCloud Private Relay not working. After much trial and error, I've pinpointed the issue to be the new "Zero-Phishing" blade introduced with R81.20.

The only way for me to get to this conclusion was by disabling blades one-by-one while testing using iOS 16.4.1 and seeing how it would behave. Upon disabling "Zero-Phishing" iCloud Private Relay is working. Upon re-activating Zero-Phishing it will claim iCloud Private Relay is unavailable, and it will also claim "WiFi is not connected to the Internet."

There are zero logs from the Zero Phishing Blade. I also tried to run fw ctl zdebug + drop | grep *IP address of the iPhone used for testing*, and nothing will show in zdebug either.

I noticed sk177924 for Zero Phishing debugging. With full debug logs enabled, the log is getting bombarded with so much information making it difficult to parse. But I noticed this:

[6948 4043859968]@barad-dur[11 Apr 11:02:34] T_event_mainloop_iter_epoll: epoll get signals for N=1 events
[6948 4043859968]@barad-dur[11 Apr 11:02:34] T_event_mainloop_iter_epoll: epoll epoll_wait reports socket/event 32/1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: piranha is activated on the session: 0x7f64bd2ddba8
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: conn_id: 0 session_id: 58846, is_gzip 0, is_deflate 0, is_chunked 0
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: injection method: IMMEDIATE, file_size according to content length: 68, URL: http://netcts.cdn-apple.com/
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:0 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: Performing code injection at offset 47
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: written all allocated bytes: 963
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: zph operation finished, total written bytes: 963
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_mngr_trap_cb: called. request_type:E_ZPH_TRAP_LOG_MESSAGE (1), Conn:0 session:58846 instance:1 dev_type:1
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] zph_log_message: TRAP message: Cleaning up inline_phishing_session. inline_phishing_session: 0x7f64bd2ddba8
[6948 4043859968]@barad-dur[11 Apr 11:02:34] [ZPH (TD::All)] ZphEvents::writeLog: Called
[6948 4043859968]@barad-dur[11 Apr 11:02:34] kmsg_read_local: 7 kmsgs handled

But to be honest, I don't really understand what is happening according to the logs. Without debug flags, I wouldn't see anything so debug flags are seemingly needed to notice anything affecting iCloud Private Relay in the file.

Zero-Phishing seems like a nifty feature. But I have several issues with this behaviour. First, nothing is logged under "blade:"Zero Phishing" in Smart Console. This makes it extremely difficult to understand when Zero Phishing might be breaking things.

Secondly, I don't know how to whitelist this effectively. I could always create a global exception by adding blade: Zero Phishing. But I don't want to disable Zero Phishing for all traffic. The best thing I can come up with for iCloud Private Relay specifically would be to have this global exception:

Protected Scope: Any, Src: Any, Dst: .apple.com (NON-FQDN), .icloud.com (NON-FQDN), .cdn-apple.com (NON-FQDN), Blade: Zero Phishing, Action: Inactive.

But using NON-FQDN has never been recommended when it comes to performance. I'd bet the same goes for using them in Threat Prevention as Global Exception?

And the fact that this traffic is breaking without easy-to-read logs for me to create whitelists makes me uncomfortable. I have no idea what else might seemingly break by this blade when the only way for me to notice is by reading debug-level logs on the process level.

Has anyone else been testing Zero Phisting on R81.20, and can you share some experiences?

[RamGuy239]

Getting a global exception going isn't that easy, either. I attempted to do:

scope: any, src: any, dst: .apple.com, .icloud.com, .cdn-apple.com, blade: zero-phishing, action: inactive.

But this is not enough. If I change it to:

scope: any, src: any, dst:any, blade: zero-phishing, action: inactive.

It's working. Doing a cat $FWDIR/log/zphd.elg* | grep IMMEDIATE only shows references towards netcts.cdn-apple.com and spectrum.s3.amazonaws.com.

Creating domain objects for those two specifically and adding them as dst in the global exception is not enough. I have to disable the blade completely or have a global exception putting the entire blade as inactive regardless of src and dst for it to work.

[RamGuy239]

I followed the recommendations in https://support.checkpoint.com/results/sk/sk180536.

Now the traffic is no loger being logged as

[6948 4043859968]@barad-dur[11 Apr 11:34:06] [ZPH (TD::All)] zph_log_message: TRAP message: injection method: IMMEDIATE, file_size according to content length: 68, URL: http://netcts.cdn-apple.com/

And iCloud Private Relay is working even without the exception.

No idea why it behaves like this. Seems like not having "Enable deep inspection scanning" on the Anti Virus profile makes the Zero Phishing feature incapable of working correctly for this traffic result it in breaking.

[Ruan_Kotze]

Don't have access to a box right now so not sure if you can use it as a destination in a TP exclusion - but how about creating a custom site using regex?  None of the performance hits associated with non-fqdn objects.

[RamGuy239]

Thanks for the tip. But I think it's the other way around. There is seemingly no way to add custom sites and applications to Threat Prevention Profiles. But I can add domain objects, both FQDN and NON-FQDN.

I tried with NON-FQDN, but it didn't work. But luckily, after following https://support.checkpoint.com/results/sk/sk180536 the traffic is no longer breaking, so I'm in no need of any exceptions.