R81.20 JHF 89 Fails (0-1-200008 & 0-1-2000107) on CP3200

[Perry_McGrew]

Updated our Mgt and GWs to JHF 89 due to some current issues with Identity Awareness.  After any JHF update, I do a test Policy install (no changes) to verify.   Policy installs failed on the 3200s.  Tried "fw fetch" also from a CP3200 which fails as well.  Ran the Policy.sh debug script on the Mgt server and sent it to TAC.   Tried running it on one of the CP3200 and it just ran forever....had to cancel it when "/" went from 68% to over 92%.  Canceling it automatically removes all the logs it created in /tmp. 

JHF 89 on the 5800 HA Cluster worked fine and I can install policy.  

Typically, these error codes are memory related.   TAC verified memory was available on a 3200.   Waiting for TAC to get back with diagnosis and hopefully a fix.  

[PhoneBoy]

What did fw fetch have to say as far as an error?

[Perry_McGrew]

I don't recall exactly.   But it seemed to track the same as the Policy push -- some sort of memory allocation issue.  fw fetch takes less resources than the Push from the Management.   

I let the Policy.sh debug script run on the CP3200 until the "/" mount point went to 92% and was still climbing.   I grep'd dmesg for ERROR and it was full of these types of messages:

Oct  7 13:08:46 2024 BVILLE-3200 kernel:[fw4_0];[71.245.91.32:56490 -> 1.1.1.1:53] [ERROR]: cmik_loader_fw_context_match_cb: match_cb for CMI APP 31 - DNS_DATA_SOURCE failed on context 201, executing context 366 and adding the app to apps in exception

Oct  7 13:08:46 2024 BVILLE-3200 kernel:[fw4_0];[71.245.91.32:57990 -> 68.237.161.12:53] [ERROR]: cmik_loader_fw_context_match_cb: match_cb for CMI APP 31 - DNS_DATA_SOURCE failed on context 201, executing context 366 and adding the app to apps in exception

Oct  7 13:26:42 2024 BVILLE-3200 kernel:[fw4_0];[192.168.8.117:60971 -> 192.168.20.254:443] [ERROR]: fwk_install_policy_app_load_prepare: fwk_atomic_load_prepare() failed, error: (14)

Oct  7 13:26:42 2024 BVILLE-3200 kernel:[fw4_0];[192.168.8.117:60971 -> 192.168.20.254:443] [ERROR]: install_policy_mgr_k_load_prepare: load_prepare failed for app: (FW), app_id: (1), app_position: (2)

Oct  7 13:29:47 2024 BVILLE-3200 kernel:[fw4_0];[192.168.20.243:57946 -> 142.250.65.219:443] [ERROR]: fwk_install_policy_app_load_prepare: fwk_atomic_load_prepare() failed, error: (14)

Oct  7 13:29:47 2024 BVILLE-3200 kernel:[fw4_0];[192.168.20.243:57946 -> 142.250.65.219:443] [ERROR]: install_policy_mgr_k_load_prepare: load_prepare failed for app: (FW), app_id: (1), app_position: (2)

TAC asked for a df -h  

Filesystem                       Size  Used Avail Use% Mounted on

/dev/mapper/vg_splat-lv_current   32G   16G   15G  52% /

I have plenty of free space in "/" .   Have a session with Tier 2 today.