topic Re: identity awareness debugging commands fail on R81.20 in Firewall and Security Management

identity awareness debugging commands fail on R81.20

sorinstf — Tue, 05 Mar 2024 20:15:10 GMT

Hello,

I need to collect logs from gateways running on R81.20 as we encounter a problem after upgrading to R81.20. (gateway is authenticating the user, but it only pulls "All users" group, not all the groups the users is part of. FW security policies based on Access roles fail. Anyone else ?

I had a meeting earlier today with a TAC engineer, he was able to gather the debug logs from one gateway. He's off shift, so I have to collect debug & logs from another gateway. When I run the same commands on another gateway (R81.20 JHA T38 // 3800 series), I get the following:

[Expert@pcpfw04:0]# pdpd debug on
fwasync_create_socket_bindopt: failed to bind to a reserved port: Address already in use

[Expert@pcpfw04:0]# pdpd debug status
fwasync_create_socket_bindopt: failed to bind to a reserved port: Address already in use

[Expert@pcpfw04:0]# pepd debug on
Unable to open '/vs0/dev/fw6v0': Connection refused

Any idea what I'm doing wrong?

Full debug list for IA - just to have it at hand

updated commands as per official guide pdp debug (checkpoint.com)

#Enable debugs: #================= pdp debug on pdp debug set all all pep debug on pep debug set all all echo "======DEBUG_START======" >> $FWDIR/log/fwd.elg fw debug fwd on TDERROR_DBG_OPT=time,host,prog,topic,pid,tid fw debug fwd on TDERROR_ALL_ALL=5 fw debug fwd on OPSEC_DEBUG_LEVEL=3 NOW CONNECT IA AGENT #Disable debugs: #================== fw debug fwd off TDERROR_ALL_ALL=0 fw debug fwd off OPSEC_DEBUG_LEVEL=0 echo "=======DEBUG_STOP======" >> $FWDIR/log/fwd.elg pdp debug off pep debug off pep debug reset ++ tcpdump -nei to save pcap with connections between the gateway and LDAP.

Re: identity awareness debugging commands fail on R81.20

Lesley — Tue, 05 Mar 2024 16:23:41 GMT

Try without the extra 'd'

pdp debug on

pdp debug status

pep debug on

Re: identity awareness debugging commands fail on R81.20

sorinstf — Tue, 05 Mar 2024 16:25:58 GMT

Right! as per the manual ... pdp debug (checkpoint.com)

Re: identity awareness debugging commands fail on R81.20

Nadezhda — Thu, 14 Nov 2024 08:09:55 GMT

Hello!

We upgraded to 81.20 and got user identification errors too.
Did you find a solution to this problem ?

Re: identity awareness debugging commands fail on R81.20

sorinstf — Thu, 14 Nov 2024 12:13:37 GMT

In my case the incidents were related to an undocumented bug in PDP daemon which was crashing. I had to kill pdpd process and watchdog will restart it.

I've received a couple of private fixes from CP TAC / R&D which stabilized the solution, but AFAIK it was not merged in the main release. PRHF-33541 fixed my issue for JHF T41. later edit - PRHF merged in JHF Take 89, see sk182220

"pdp br st -e" returned "daemon did not respond or not running!", but there was a process associated . Run "pgrep -f pdpd", then kill it with "kill -9 PID_ID".

Notes taken during TAC case meetings:

Further details on core dump analysis:

The further core dump analysis points to a thread that was in wait state.
The class members output shows that "num_writers_waiting" is 1 and will continue to wait.
It looks like one thread was waiting on the write lock.
This behavior shows lock contention, and the lock will never be granted to either the read or write requestors in this scenario.
The process appears to be stuck when performing pdp::BrokerPublisherCleaner::RunTask:
The pdpd logs for affected gateways show the following logs before the process hangs:
1. 1. 19 Apr 23:24:52] [TRACKER]: #1006717 -> INTERNAL -> TASK_STARTED -> Starting task 2: Revoke all identities received from publisher: (10.15.25.21)

Re: identity awareness debugging commands fail on R81.20

Nadezhda — Thu, 14 Nov 2024 09:26:02 GMT

Thanks a lot for the info!
It looks like this fix is included in JHF T89.
Thank you for your reply.