Traffic through HTTPS inspection stopped working

[tjoll]

Hi Guru's,

I'm having quite a strange issue lately. Hopefully you all can help me out.

I have a R82JHF118 gateway running the following blades:
fw vpn cvpn urlf av appi ips identityServer SSL_INSPECT anti_bot ThreatEmulation content_awareness Scrub zero_phishing.

HTTPS inspection is enabled on one inbound connection from the internet towards an web application with Websocket capabilities. For some reason, the inbound HTTPS inspection connection broke. I checked the logs on the gateway but there were no issues and errors seen. Traffic was coming in correctly and, according to the logs, the flow was inspected without issues. 

I ran a TCPdump and found out that the connection was able to establish on the front and back-end of the gateway, but when the first application data was sent, the gateway immediately reset the connection towards the server and the client at the same time. Eventually I found out that HTTPS inspection was not passing the traffic correctly because configuring a bypass rule for this flow, solved the issue. I ran some WSTLSD debugs but nothing extraordinary in the logs. When I did a fw ctl zdebug + drop, I found the following error message:

Click to Expand

@;8807957.284726;11Aug2026 22:07:19.512533;[vs_0];[tid_4];[fw4_4];[x.x.x.x:38942 -> y.y.y.y:9985] [ERROR]: parsers_is_activate_protocol_if_needed: activate_mux_app_cb failed for mux app TLS_PARSER. Can't activate mux application.;
@;8807957.284727;11Aug2026 22:07:19.512535;[vs_0];[tid_4];[fw4_4];[x.x.x.x:38942 -> y.y.y.y:9985] [ERROR]: parsers_is_mux_read_handler: parsers_is_activate_protocol_if_needed failed;

This gave direction that my issue was related to the MUX module. So I did a kdebug on MUX with the following results:

Click to Expand

@;8807957.284722;11Aug2026 22:07:19.512528;[vs_0];[tid_4];[fw4_4];mux_app_manager_allocate_extra_app_arr_if_needed: ERROR: Already reached max apps from other(4).;
@;8807957.284723;11Aug2026 22:07:19.512529;[vs_0];[tid_4];[fw4_4];mux_add_extra_app: ERROR: mux_app_manager_create_new_app_if_needed() failed.mux_add_app: ERROR: Failed to add app TLS_PARSER_LAST from other, mux_state=0x7f553f304408.;
@;8807957.284724;11Aug2026 22:07:19.512530;[vs_0];[tid_4];[fw4_4];tls_activate_mux_app: ERROR: Failed to add TLS_PARSER_LAST to Mux ;
@;8807957.284725;11Aug2026 22:07:19.512530;[vs_0];[tid_4];[fw4_4];tls_parser_parsers_is_activate_tls_parser: ERROR: tls_activate_mux_app failed for MUX_APP_TLS_PARSER_LAST.;
@;8807957.284726;11Aug2026 22:07:19.512533;[vs_0];[tid_4];[fw4_4];[x.x.x.x:38942 -> y.y.y.y:9985] [ERROR]: parsers_is_activate_protocol_if_needed: activate_mux_app_cb failed for mux app TLS_PARSER. Can't activate mux application.;

@;8807957.284727;11Aug2026 22:07:19.512535;[vs_0];[tid_4];[fw4_4];[x.x.x.x:38942 -> y.y.y.y:9985] [ERROR]: parsers_is_mux_read_handler: parsers_is_activate_protocol_if_needed failed;
@;8807957.284728;11Aug2026 22:07:19.512536;[vs_0];[tid_4];[fw4_4];mux_task_handler: ERROR: Failed to handle task. task=0x7f54fd4cc278, app_id=0 (PARSERS_IS), mux_state=0x7f553f304408.;

Apparently, there is a hardcoded kernel limit were the firewall does not allow more than 4 parsers attached to a flow. While the TLS_PARSER_LAST was the fifth, the connection was dropped before it was sent out of the interface. I tried a lot of stuff and configuration combinations but was not able to fix this issue. Hereby my list:
- Used inline layer and normal fw rule.
- Changed services object to custom object with and without handler.
- Disabled features like app cntrl, urlf, content awareness on the inline layer.
- Disabled IPS.
- Changed IPS settings from autonomous Policy to custom.
- Created a global exception for IPS/Threat Prevention.
- Created a exception for app cntrl and urlf.
- Changed https inspection in fail-open/fail-closed mode.

The only workaround that worked was settings a kernel parameter: fw ctl set int tls_parser_enable 0

But this workaround prevents some deep inspection features, so it's not the best solution.

Do you guys have any idea what happened and how to solve this issue?

Thanks for the help.

Best regards,
Mitchel

[the_rock]

I recall back during Covid 19 time another kernel parameter breaking ssl inspection user check page. Maybe unrelated, but will check when i get up. Its only 3.23 am lol

[PhoneBoy]

Sounds like TAC should be involved here.