identity awareness debugging commands fail on R81.20

[sorinstf]

Hello, 

I need to collect logs from gateways running on R81.20 as we encounter a  problem after upgrading to R81.20. (gateway is authenticating the user, but it only pulls "All users" group, not all  the groups the users is part of. FW security policies based on Access roles fail. Anyone else ? 

I had a meeting earlier today with a TAC engineer, he was able to gather the debug logs from one  gateway. He's off shift, so I have to collect debug & logs from another gateway.  When I run the same commands on another gateway (R81.20  JHA T38 // 3800 series), I get the following: 

[Expert@pcpfw04:0]# pdpd debug on
fwasync_create_socket_bindopt: failed to bind to a reserved port: Address already in use

[Expert@pcpfw04:0]# pdpd debug status
fwasync_create_socket_bindopt: failed to bind to a reserved port: Address already in use

[Expert@pcpfw04:0]# pepd debug on
Unable to open '/vs0/dev/fw6v0': Connection refused

Any idea what I'm doing wrong?

Full debug list for IA  - just to have it at hand

updated commands as per official guide pdp debug (checkpoint.com)

#Enable debugs:
#=================
pdp debug on
pdp debug set all all

pep debug on
pep debug set all all

echo "======DEBUG_START======" >> $FWDIR/log/fwd.elg
fw debug fwd on TDERROR_DBG_OPT=time,host,prog,topic,pid,tid
fw debug fwd on TDERROR_ALL_ALL=5
fw debug fwd on OPSEC_DEBUG_LEVEL=3

NOW CONNECT IA AGENT

#Disable debugs:
#==================
fw debug fwd off TDERROR_ALL_ALL=0
fw debug fwd off OPSEC_DEBUG_LEVEL=0

echo "=======DEBUG_STOP======" >> $FWDIR/log/fwd.elg
pdp debug off

pep debug off
pep debug reset

++ tcpdump -nei  to save pcap with connections between the gateway and LDAP. 

[Lesley · Accepted solution]

Try without the extra 'd'

pdp debug on

pdp debug status

pep debug on

[sorinstf]

Right! as per the manual ... pdp debug (checkpoint.com)

[Nadezhda]

Hello!

We upgraded to 81.20 and got user identification errors too.
Did you find a solution to this problem ?

[sorinstf · Accepted solution]

In my case the incidents were related to an undocumented bug in PDP daemon which was crashing. I had to kill pdpd process and watchdog will restart it.

I've received a couple of private fixes from CP TAC / R&D which stabilized the solution, but AFAIK it was not merged in the main release.  PRHF-33541 fixed my issue for JHF T41. later edit - PRHF merged in JHF Take 89, see sk182220

"pdp br st -e" returned  "daemon did not respond or not running!", but there was a process associated . Run "pgrep -f pdpd", then kill it  with "kill -9 PID_ID".

Notes taken during TAC case meetings:

Further details on core dump analysis:

1. The further core dump analysis points to a thread that was in wait state.
2. The class members output shows that "num_writers_waiting" is 1 and will continue to wait.
3. It looks like one thread was waiting on the write lock.
4. This behavior shows lock contention, and the lock will never be granted to either the read or write requestors in this scenario.
5. The process appears to be stuck when performing pdp::BrokerPublisherCleaner::RunTask:
6.  The pdpd logs for affected gateways show the following logs before the process hangs:
   1. 1. 19 Apr 23:24:52] [TRACKER]: #1006717 -> INTERNAL -> TASK_STARTED -> Starting task 2: Revoke all identities received from publisher: (10.15.25.21)

[Nadezhda]

Thanks a lot for the info!
It looks like this fix is included in JHF T89.
Thank you for your reply.