Identity Awareness getting DC Name only

[Vladimir]

Any suggestions what I may have done to cause this?

This is a lab environment, so I can do whatever I want with it.

Freshly installed 2016 domain controller.

IA enabled with wizard and successfully created LDAP account unit.

adlog a query all returns just DC, no users or other domain member PCs.

Gateway and management are R80.10.

[Kaspars_Zibarts]

Haven't used AD query since 2014..  all our environment relies on IDC. Forgotten most things about AD queries. But show us what you see, which commands / logs you are looking at?

adlog a dc

adlog a q a 

pdp m a

pep s p a

[Vladimir]

Thank you Kaspars.

[Expert@GW8010:0]# ps aux | grep pdp
admin 8456 0.0 1.0 213856 39432 ? Sl Nov12 0:24 pdpd 0 -t
admin 17748 0.0 0.0 1736 500 pts/2 S+ 08:23 0:00 grep pdp
[Expert@GW8010:0]# adlog a dc
Domain controllers:
Domain Name IP Address Events (last hour) Connection state
============================================================================================================
higherintelligence.com 10.101.25.16 142 has connection

Ignored domain controllers on this gateway:
No ignored domain controllers found.

[Expert@GW8010:0]# adlog a q a
ip: 10.101.25.16 --> Machines: dc2016@higherintelligence.com;

[Expert@GW8010:0]# pdp m a

Session: a28c7f3c
Session UUID: {489CFED8-015F-BB42-9E52-1AF88BDFC3EE}
Ip: 10.101.25.16
Machine:
dc2016@higherintelligence.com {64948b77}
Groups: All Machines
Roles: Internet_Users;TestRole1
Client Type: AD Query
Authentication Method: Trust
Distinguished Name: CN=DC2016,OU=Domain Controllers,DC=higherintelligence,DC=com
Connect Time: Mon Nov 12 18:36:27 2018
Next Reauthentication: Tue Nov 13 20:22:18 2018
Next Connectivity Check: Tue Nov 13 20:22:18 2018
Next Ldap Fetch: Tue Nov 13 10:30:57 2018

Packet Tagging Status: Not Active
Published Gateways: Local
************************************************************************************

[Expert@GW8010:0]# pep s p a
Command: root->show->pdp->all
Source IP ID Connect time Users
=========================================================
127.0.0.1 0x0 12Nov2018 18:35:06 1

[Expert@GW8010:0]#

The infuriating thing is that when I am going through the access role creation process, and querying the directory for users and groups, I can see them just fine:

So the query must be working.

It is just the actual IA that does not and the diagnostics seem to indicate that query returns only DC data.

The DC is configured to perform the Audit for Success and Failure in Domain Controller and on the Local Machine.

And I suspect that the "adlog a q a" should pull all the data available from AD.

[Kaspars_Zibarts]

Did you see those AD event logs generated in DC for one of the computers / users (4624, 4768, 4769 and 4770) as those are the ones that will trigger updates to firewall?

Try adding portal so you can log in manually from a machine  that should push your ID out to gateway

[Vladimir]

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 12:26:15 PM
Event ID: 4624
Task Category: Logon
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
An account was successfully logged on.

Subject:
Security ID: NULL SID
Account Name: -
Account Domain: -
Logon ID: 0x0

Logon Information:
Logon Type: 3
Restricted Admin Mode: -
Virtual Account: No
Elevated Token: Yes

Impersonation Level: Delegation

New Logon:
Security ID: HI\aduser1
Account Name: aduser1
Account Domain: HIGHERINTELLIGENCE.COM
Logon ID: 0x1104289
Linked Logon ID: 0x0
Network Account Name: -
Network Account Domain: -
Logon GUID: {248b4494-396f-2538-be76-9e0773f89dea}

Process Information:
Process ID: 0x0
Process Name: -

Network Information:
Workstation Name: -
Source Network Address: 10.101.25.1
Source Port: 10044

Detailed Authentication Information:
Logon Process: Kerberos
Authentication Package: Kerberos
Transited Services: -
Package Name (NTLM only): -
Key Length: 0

This event is generated when a logon session is created. It is generated on the computer that was accessed.

The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The impersonation level field indicates the extent to which a process in the logon session can impersonate.

The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4624</EventID>
<Version>2</Version>
<Level>0</Level>
<Task>12544</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T17:26:15.547179600Z" />
<EventRecordID>45317</EventRecordID>
<Correlation ActivityID="{36945420-7AF4-0000-2B54-9436F47AD401}" />
<Execution ProcessID="648" ThreadID="688" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="SubjectUserSid">S-1-0-0</Data>
<Data Name="SubjectUserName">-</Data>
<Data Name="SubjectDomainName">-</Data>
<Data Name="SubjectLogonId">0x0</Data>
<Data Name="TargetUserSid">S-1-5-21-2916104119-1630943108-1692471290-1103</Data>
<Data Name="TargetUserName">aduser1</Data>
<Data Name="TargetDomainName">HIGHERINTELLIGENCE.COM</Data>
<Data Name="TargetLogonId">0x1104289</Data>
<Data Name="LogonType">3</Data>
<Data Name="LogonProcessName">Kerberos</Data>
<Data Name="AuthenticationPackageName">Kerberos</Data>
<Data Name="WorkstationName">-</Data>
<Data Name="LogonGuid">{248B4494-396F-2538-BE76-9E0773F89DEA}</Data>
<Data Name="TransmittedServices">-</Data>
<Data Name="LmPackageName">-</Data>
<Data Name="KeyLength">0</Data>
<Data Name="ProcessId">0x0</Data>
<Data Name="ProcessName">-</Data>
<Data Name="IpAddress">10.101.25.1</Data>
<Data Name="IpPort">10044</Data>
<Data Name="ImpersonationLevel">%%1840</Data>
<Data Name="RestrictedAdminMode">-</Data>
<Data Name="TargetOutboundUserName">-</Data>
<Data Name="TargetOutboundDomainName">-</Data>
<Data Name="VirtualAccount">%%1843</Data>
<Data Name="TargetLinkedLogonId">0x0</Data>
<Data Name="ElevatedToken">%%1842</Data>
</EventData>
</Event>

---

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 12:51:26 PM
Event ID: 4768
Task Category: Kerberos Authentication Service
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
A Kerberos authentication ticket (TGT) was requested.

Account Information:
Account Name: WIN10NET30$
Supplied Realm Name: higherintelligence.com
User ID: HI\WIN10NET30$

Service Information:
Service Name: krbtgt
Service ID: HI\krbtgt

Network Information:
Client Address: ::ffff:10.101.25.1
Client Port: 10105

Additional Information:
Ticket Options: 0x40810010
Result Code: 0x0
Ticket Encryption Type: 0x12
Pre-Authentication Type: 2

Certificate Information:
Certificate Issuer Name:
Certificate Serial Number:
Certificate Thumbprint:

Certificate information is only provided if a certificate was used for pre-authentication.

Pre-authentication types, ticket options, encryption types and result codes are defined in RFC 4120.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4768</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>14339</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T17:51:26.755926600Z" />
<EventRecordID>45760</EventRecordID>
<Correlation />
<Execution ProcessID="648" ThreadID="2628" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserName">WIN10NET30$</Data>
<Data Name="TargetDomainName">higherintelligence.com</Data>
<Data Name="TargetSid">S-1-5-21-2916104119-1630943108-1692471290-1106</Data>
<Data Name="ServiceName">krbtgt</Data>
<Data Name="ServiceSid">S-1-5-21-2916104119-1630943108-1692471290-502</Data>
<Data Name="TicketOptions">0x40810010</Data>
<Data Name="Status">0x0</Data>
<Data Name="TicketEncryptionType">0x12</Data>
<Data Name="PreAuthType">2</Data>
<Data Name="IpAddress">::ffff:10.101.25.1</Data>
<Data Name="IpPort">10105</Data>
<Data Name="CertIssuerName">
</Data>
<Data Name="CertSerialNumber">
</Data>
<Data Name="CertThumbprint">
</Data>
</EventData>
</Event>

---

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 12:26:15 PM
Event ID: 4769
Task Category: Kerberos Service Ticket Operations
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
A Kerberos service ticket was requested.

Account Information:
Account Name: aduser1@HIGHERINTELLIGENCE.COM
Account Domain: HIGHERINTELLIGENCE.COM
Logon GUID: {4cfa961d-3e55-139f-f480-8f0d932fbb67}

Service Information:
Service Name: DC2016$
Service ID: HI\DC2016$

Network Information:
Client Address: ::ffff:10.101.25.1
Client Port: 10045

Additional Information:
Ticket Options: 0x40810000
Ticket Encryption Type: 0x12
Failure Code: 0x0
Transited Services: -

This event is generated every time access is requested to a resource such as a computer or a Windows service. The service name indicates the resource to which access was requested.

This event can be correlated with Windows logon events by comparing the Logon GUID fields in each event. The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.

Ticket options, encryption types, and failure codes are defined in RFC 4120.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4769</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>14337</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T17:26:15.544915700Z" />
<EventRecordID>45316</EventRecordID>
<Correlation />
<Execution ProcessID="648" ThreadID="1780" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserName">aduser1@HIGHERINTELLIGENCE.COM</Data>
<Data Name="TargetDomainName">HIGHERINTELLIGENCE.COM</Data>
<Data Name="ServiceName">DC2016$</Data>
<Data Name="ServiceSid">S-1-5-21-2916104119-1630943108-1692471290-1000</Data>
<Data Name="TicketOptions">0x40810000</Data>
<Data Name="TicketEncryptionType">0x12</Data>
<Data Name="IpAddress">::ffff:10.101.25.1</Data>
<Data Name="IpPort">10045</Data>
<Data Name="Status">0x0</Data>
<Data Name="LogonGuid">{4CFA961D-3E55-139F-F480-8F0D932FBB67}</Data>
<Data Name="TransmittedServices">-</Data>
</EventData>
</Event>

---

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 12:12:24 PM
Event ID: 4627
Task Category: Group Membership
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
Group membership information.

Subject:
Security ID: NULL SID
Account Name: -
Account Domain: -
Logon ID: 0x0

Logon Type: 3

New Logon:
Security ID: HI\aduser1
Account Name: aduser1
Account Domain: HIGHERINTELLIGENCE.COM
Logon ID: 0x10E73C2

Event in sequence: 1 of 1

Group Membership:
HI\Domain Users
Everyone
BUILTIN\Users
BUILTIN\Pre-Windows 2000 Compatible Access
NT AUTHORITY\NETWORK
NT AUTHORITY\Authenticated Users
NT AUTHORITY\This Organization
Authentication authority asserted identity
Mandatory Label\Medium Plus Mandatory Level

The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

This event is generated when the Audit Group Membership subcategory is configured. The Logon ID field can be used to correlate this event with the corresponding user logon event as well as to any other security audit events generated during this logon session.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4627</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>12554</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T17:12:24.939948800Z" />
<EventRecordID>45182</EventRecordID>
<Correlation ActivityID="{36945420-7AF4-0000-2B54-9436F47AD401}" />
<Execution ProcessID="648" ThreadID="4656" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="SubjectUserSid">S-1-0-0</Data>
<Data Name="SubjectUserName">-</Data>
<Data Name="SubjectDomainName">-</Data>
<Data Name="SubjectLogonId">0x0</Data>
<Data Name="TargetUserSid">S-1-5-21-2916104119-1630943108-1692471290-1103</Data>
<Data Name="TargetUserName">aduser1</Data>
<Data Name="TargetDomainName">HIGHERINTELLIGENCE.COM</Data>
<Data Name="TargetLogonId">0x10e73c2</Data>
<Data Name="LogonType">3</Data>
<Data Name="EventIdx">1</Data>
<Data Name="EventCountTotal">1</Data>
<Data Name="GroupMembership">
%{S-1-5-21-2916104119-1630943108-1692471290-513}
%{S-1-1-0}
%{S-1-5-32-545}
%{S-1-5-32-554}
%{S-1-5-2}
%{S-1-5-11}
%{S-1-5-15}
%{S-1-18-1}
%{S-1-16-8448}</Data>
</EventData>
</Event>

---

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 6:49:34 AM
Event ID: 4770
Task Category: Kerberos Service Ticket Operations
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
A Kerberos service ticket was renewed.

Account Information:
Account Name: Administrator@HIGHERINTELLIGENCE.COM
Account Domain: HIGHERINTELLIGENCE.COM

Service Information:
Service Name: krbtgt
Service ID: HI\krbtgt

Network Information:
Client Address: ::1
Client Port: 0

Additional Information:
Ticket Options: 0x10002
Ticket Encryption Type: 0x12

Ticket options and encryption types are defined in RFC 4120.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4770</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>14337</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T11:49:34.012766500Z" />
<EventRecordID>41867</EventRecordID>
<Correlation />
<Execution ProcessID="648" ThreadID="2084" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserName">Administrator@HIGHERINTELLIGENCE.COM</Data>
<Data Name="TargetDomainName">HIGHERINTELLIGENCE.COM</Data>
<Data Name="ServiceName">krbtgt</Data>
<Data Name="ServiceSid">S-1-5-21-2916104119-1630943108-1692471290-502</Data>
<Data Name="TicketOptions">0x10002</Data>
<Data Name="TicketEncryptionType">0x12</Data>
<Data Name="IpAddress">::1</Data>
<Data Name="IpPort">0</Data>
</EventData>
</Event>

---

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 11/13/2018 12:10:16 PM
Event ID: 4634
Task Category: Logoff
Level: Information
Keywords: Audit Success
User: N/A
Computer: DC2016.higherintelligence.com
Description:
An account was logged off.

Subject:
Security ID: HI\WIN10NET30$
Account Name: WIN10NET30$
Account Domain: HI
Logon ID: 0x10D31B1

Logon Type: 3

This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4634</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>12545</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2018-11-13T17:10:16.469434300Z" />
<EventRecordID>45136</EventRecordID>
<Correlation />
<Execution ProcessID="648" ThreadID="688" />
<Channel>Security</Channel>
<Computer>DC2016.higherintelligence.com</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserSid">S-1-5-21-2916104119-1630943108-1692471290-1106</Data>
<Data Name="TargetUserName">WIN10NET30$</Data>
<Data Name="TargetDomainName">HI</Data>
<Data Name="TargetLogonId">0x10d31b1</Data>
<Data Name="LogonType">3</Data>
</EventData>
</Event>

---

But while these are pertinent for IA to work, I am still uncertain why adlog a q a does not pull the directory objects in.

[Kaspars_Zibarts]

Try portal too. Just out of curiosity. Else hard to say - haven't used ad query approach for many years. I believe IDC is not an option to you?

[Kaspars_Zibarts]

Here's bit more from old stuff that I have saved  check that you see AD event logs in bold in your DC

After being configured, AD Query (ADQ) begins communication with all of the Domain Controllers in the configured account unit. The communication is a WMI query (over DCE-RPC), registering to receive all relevant Security Event logs from the Domain Controller. (For Windows 2003 Domain Controllers, events 672,673 and 674 are fetched, while for Windows 2008 Domain Controllers events 4624, 4768, 4769 and 4770 are fetched.) From then on, any new relevant Security Event log will be sent to the Security Gateway, seconds from its creation time. AD Query (ADQ) does pull the Security Event log from the Active Directory every second (1 second) or the Active Directory pushes the latest 100 events to the Security Gateway that runs AD Query (ADQ).

The registration mechanism allows AD Query (ADQ) to receive new Security Event logs in a timely manner (up to several seconds from generation time). It does not generate a lot of burden on the Domain Controller, as it already has the message in its memory, when it is sent to AD Query (ADQ). Real life deployments on running AD environments, measured up to a 3% increase in CPU usage on the Domain Controllers, with an average of much less than 1%.

Most of the problems with AD Query (ADQ) happen in the communication phase since WMI runs over DCE-RPC, which is a complicated and non firewall-friendly protocol (it starts on port 135 but later moves on to a dynamically coordinated port). The first thing that you want to check when AD Query (ADQ) is not working is if something, on the local Security Gateway, or on the way to the Domain Controller is blocking this traffic. Refer to sk58881 (AD Query traffic dropped by Check Point Security Gateway), for further details about how to diagnose and handle such problems.

[Vladimir]

Thank you Kaspars. Unfortunately, you can see that the DC is logging for all its worth but to no avail.

 Royi Priov‌, can I get your input on this issue?

BTW:

[Expert@GW8010:0]# $FWDIR/bin/test_ad_connectivity -o /var/log/test_ad_connectivity.txt -d higherintelligence.com -i 10.101.25.16 -u checkpoint -D CN=checkpoint,CN=Users,DC=higherintelligence,DC=com -a
Please enter checkpoint (DN=CN=checkpoint,CN=Users,DC=higherintelligence,DC=com) password:

[Expert@GW8010:0]# cat /var/log/test_ad_connectivity.txt
(
:status (SUCCESS_LDAP_WMI)
:err_msg ("ADLOG_SUCCESS;LDAP_SUCCESS")
:ldap_status (LDAP_SUCCESS)
:wmi_status (ADLOG_SUCCESS)
:timestamp ("Thu Nov 15 14:48:32 2018")
)
[Expert@GW8010:0]#

Thank you,

Vladimir

[Kaspars_Zibarts]

Royi is the right call.. I can't remember much about adlog queries and we rub R77.30 back then..  

[Royi_Priov]

Hi Vladimir,

I have reviewed most of the information written in this thread. 
Let's summarize the facts (correct me if I'm missing something):

1. ADQ was configured with one DC server.
2. A user is logged in but you don't see the event on the GW side (adlog a q a)

Few things to check to see where the issue falls:

1. Is indeed the user indeed logs in front of 10.101.25.16 DC? can be tested with "echo %LOGONSERVER%" on the endpoint PC.
2. please review $FWDIR/log/pdpd.elg files (you can start without debugging) and see the event processing. We might get the event but drops it for some reason.

Thanks,

Royi Priov

Team Leader, Identity Awareness R&D.

[Vladimir]

Hi Royi. Thank you for looking into this.

Client Was rebooted and logged onto domain successfully:

With DNS traffic filtered out in logs, the communication between Win10Net30 and DC2016 looks like:

While the only IA events visible are:

[8456 4105705232]@GW8010[18 Nov 9:03:16] [TRACKER]: #1624 -> INCOMING -> ADQUERY_ASSOCIATION ->
Association
ip: 10.101.25.16
user:
machine: dc2016
domain: higherintelligence.com
reason: 0
[8456 4105705232]@GW8010[18 Nov 9:08:16] [TRACKER]: #1625 -> INCOMING -> ADQUERY_ASSOCIATION ->
Association
ip: 10.101.25.16
user:
machine: dc2016
domain: higherintelligence.com
reason: 0
[8456 4105705232]@GW8010[18 Nov 9:13:16] [TRACKER]: #1626 -> INCOMING -> ADQUERY_ASSOCIATION ->
Association
ip: 10.101.25.16
user:
machine: dc2016
domain: higherintelligence.com
reason: 0
[8456]@GW8010[18 Nov 9:15:33] Warning:cp_timed_blocker_handler: A handler [0xf78c5bb0] blocked for 6 seconds.
[8456]@GW8010[18 Nov 9:15:33] Warning:cp_timed_blocker_handler: Handler info: Library [/opt/CPsuite-R80/fw1/lib/libpdplib.so], Function offset [0x102bb0].
[8456]@GW8010[18 Nov 9:15:33] Warning:cp_timed_blocker_handler: Handler info: Nearest symbol name [_ZN3pdp13ClusterHelper10s_ReportUpEPv], offset [0x102bb0].
[8456]@GW8010[18 Nov 9:17:20] Warning:cp_timed_blocker_handler: A handler [0xf78c5bb0] blocked for 16 seconds.
[8456]@GW8010[18 Nov 9:17:20] Warning:cp_timed_blocker_handler: Handler info: Library [/opt/CPsuite-R80/fw1/lib/libpdplib.so], Function offset [0x102bb0].
[8456]@GW8010[18 Nov 9:17:20] Warning:cp_timed_blocker_handler: Handler info: Nearest symbol name [_ZN3pdp13ClusterHelper10s_ReportUpEPv], offset [0x102bb0].
[8456 4105705232]@GW8010[18 Nov 9:18:17] [TRACKER]: #1627 -> INCOMING -> ADQUERY_ASSOCIATION ->
Association
ip: 10.101.25.16
user:
machine: dc2016
domain: higherintelligence.com
reason: 0
[Expert@GW8010:0]#

[Vladimir]

Any more suggestions?

[Royi_Priov]

Hi Vladimir Yakovlev‌

I just noticed that the IP you are querying for - 10.101.25.16, is the DC itself.

AD Query (and also IDC) are automatically not associating this IP as this is the DC IP.

If the problem is with all machines / users in the environment, please open a ticket with TAC as this needs to be further investigated.

If you want to start first investigating it on your own, I can suggest debugging PDP and try to see where the issue falls.

1. enable debug:

# pdp d s all all

2. replicate the issue - login with the user

3. disable debug:

# pdp d u all

4. search for this IP / user / machine in $FWDIR/log/pdpd.elg* files (up to 10 files).

Thanks,

Royi Priov

Team Leader, Identity Awareness R&D.

[Vladimir]

Hi Royi Priov . Sorry for taking so long to return to this thread: I was traveling on business and away from my lab.

The attempt at debug yields:

[Expert@GW8010:0]# grep -rnw $FWDIR/log/pdpd.elg -e 'aduser1'
1408:Username : aduser1
1409:Log Username : ADUser1 (aduser1)
1460:user: aduser1
1470:Username : aduser1
1471:Log Username : ADUser1 (aduser1)
2502:user: aduser1
2682:user: aduser1
2692:Username : aduser1;Administrator
2693:Log Username : ADUser1 (aduser1)
2761:user: aduser1
2771:Username : aduser1
2772:Log Username : ADUser1 (aduser1)
2994:Username : Administrator;aduser1
2995:Log Username : ADUser1 (aduser1)
3066:Username : aduser1
3067:Log Username : ADUser1 (aduser1)
3127:Username : aduser1
3128:Log Username : ADUser1 (aduser1)
3193:Username : aduser1
3194:Log Username : ADUser1 (aduser1)
3261:Username : aduser1
3262:Log Username : ADUser1 (aduser1)
3336:Username : aduser1
3337:Log Username : ADUser1 (aduser1)
3619:Username : aduser1
3620:Log Username : ADUser1 (aduser1)
3697:Username : aduser1
3698:Log Username : ADUser1 (aduser1)
3769:Username : aduser1
3770:Log Username : ADUser1 (aduser1)
3816:Username : aduser1
3817:Log Username : ADUser1 (aduser1)
4037:Username : aduser1
4038:Log Username : ADUser1 (aduser1)
4355:user: aduser1
5213:user: aduser1
5223:Username : aduser1
5224:Log Username : ADUser1 (aduser1)
5371:Username : Administrator;aduser1
5372:Log Username : ADUser1 (aduser1)
5681:Username : aduser1
5682:Log Username : ADUser1 (aduser1)
5750:Username : Administrator;aduser1
5751:Log Username : ADUser1 (aduser1)
5774:Username : aduser1
5775:Log Username : ADUser1 (aduser1)
5858:Username : aduser1
5859:Log Username : ADUser1 (aduser1)
5926:Username : aduser1
5927:Log Username : ADUser1 (aduser1)
5941:Username : Administrator;aduser1
5942:Log Username : ADUser1 (aduser1)
5983:Username : aduser1
5984:Log Username : ADUser1 (aduser1)
6020:Username : aduser1
6021:Log Username : ADUser1 (aduser1)
6035:Username : Administrator;aduser1
6036:Log Username : ADUser1 (aduser1)
6195:Username : aduser1
6196:Log Username : ADUser1 (aduser1)
6290:user: aduser1
6490:Username : aduser1
6491:Log Username : ADUser1 (aduser1)
6577:Username : aduser1
6578:Log Username : ADUser1 (aduser1)
6637:Username : aduser1
6638:Log Username : ADUser1 (aduser1)
6697:Username : aduser1
6698:Log Username : ADUser1 (aduser1)
7387:Username : aduser1
7388:Log Username : ADUser1 (aduser1)
7466:user: aduser1
7888:Username : aduser1
7889:Log Username : ADUser1 (aduser1)
7956:Username : aduser1
7957:Log Username : ADUser1 (aduser1)
8003:Username : aduser1
8004:Log Username : ADUser1 (aduser1)
8247:Username : aduser1
8248:Log Username : ADUser1 (aduser1)
8396:Username : aduser1
8397:Log Username : ADUser1 (aduser1)
8444:Username : aduser1
8445:Log Username : ADUser1 (aduser1)
8512:Username : aduser1
8513:Log Username : ADUser1 (aduser1)
8664:Username : aduser1
8665:Log Username : ADUser1 (aduser1)
8718:Username : aduser1
8719:Log Username : ADUser1 (aduser1)
8770:Username : aduser1;Administrator
8771:Log Username : ADUser1 (aduser1)
8833:Username : aduser1
8834:Log Username : ADUser1 (aduser1)
8855:Username : aduser1;Administrator
8856:Log Username : ADUser1 (aduser1)
8939:Username : aduser1
8940:Log Username : ADUser1 (aduser1)
8986:Username : aduser1
8987:Log Username : ADUser1 (aduser1)
9033:Username : aduser1
9034:Log Username : ADUser1 (aduser1)
9167:Username : aduser1
9168:Log Username : ADUser1 (aduser1)
9214:Username : aduser1
9215:Log Username : ADUser1 (aduser1)
9261:Username : aduser1
9262:Log Username : ADUser1 (aduser1)
9308:Username : aduser1
9309:Log Username : ADUser1 (aduser1)
9390:Username : aduser1
9391:Log Username : ADUser1 (aduser1)
9437:Username : aduser1
9438:Log Username : ADUser1 (aduser1)
9544:Username : aduser1
9545:Log Username : ADUser1 (aduser1)
9580:Username : Administrator;aduser1
9581:Log Username : ADUser1 (aduser1)
9738:Username : Administrator;aduser1
9739:Log Username : ADUser1 (aduser1)
9943:Username : aduser1
9944:Log Username : ADUser1 (aduser1)
10847:Username : aduser1
10848:Log Username : ADUser1 (aduser1)
10870:Username : aduser1;Administrator
10871:Log Username : ADUser1 (aduser1)
11185:Username : aduser1
11186:Log Username : ADUser1 (aduser1)
11320:Username : aduser1
11321:Log Username : ADUser1 (aduser1)
11655:Username : aduser1
11656:Log Username : ADUser1 (aduser1)
11685:Username : Administrator;aduser1
11686:Log Username : ADUser1 (aduser1)
11763:Username : aduser1
11764:Log Username : ADUser1 (aduser1)
11796:Username : aduser1
11797:Log Username : ADUser1 (aduser1)
11825:Username : aduser1;Administrator
11826:Log Username : ADUser1 (aduser1)
11901:Username : aduser1
11902:Log Username : ADUser1 (aduser1)
11923:Username : aduser1;Administrator
11924:Log Username : ADUser1 (aduser1)
12050:Username : aduser1
12051:Log Username : ADUser1 (aduser1)
12153:Username : aduser1
12154:Log Username : ADUser1 (aduser1)
12217:Username : aduser1
12218:Log Username : ADUser1 (aduser1)
12344:Username : Administrator;aduser1
12345:Log Username : ADUser1 (aduser1)
12392:Username : aduser1
12393:Log Username : ADUser1 (aduser1)
12432:Username : aduser1
12433:Log Username : ADUser1 (aduser1)
12454:Username : aduser1;Administrator
12455:Log Username : ADUser1 (aduser1)
12510:Username : aduser1
12511:Log Username : ADUser1 (aduser1)
12532:Username : Administrator;aduser1
12533:Log Username : ADUser1 (aduser1)
12555:Username : aduser1
12556:Log Username : ADUser1 (aduser1)
12588:Username : aduser1
12589:Log Username : ADUser1 (aduser1)
12610:Username : Administrator;aduser1
12611:Log Username : ADUser1 (aduser1)
12640:Username : aduser1
12641:Log Username : ADUser1 (aduser1)
12673:Username : aduser1
12674:Log Username : ADUser1 (aduser1)
12702:Username : Administrator;aduser1
12703:Log Username : ADUser1 (aduser1)
12921:Username : aduser1
12922:Log Username : ADUser1 (aduser1)
14810:Username : aduser1
14811:Log Username : ADUser1 (aduser1)
14840:Username : Administrator;aduser1
14841:Log Username : ADUser1 (aduser1)
14996:Username : Administrator;aduser1
14997:Log Username : ADUser1 (aduser1)
15019:Username : aduser1
15020:Log Username : ADUser1 (aduser1)
17212:user: aduser1
17222:Username : aduser2;aduser1;Administrator
17223:Log Username : ADUser1 (aduser1)
17484:Username : aduser1;Administrator
17485:Log Username : ADUser1 (aduser1)
116078:user: aduser1
116088:Username : aduser1
116089:Log Username : ADUser1 (aduser1)
283239: Account Name: aduser1
283396: Account Name: aduser1
283458:$$START$$ nt_log_event: aduser1 $$END$$
283461:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_DCOM (TD::All)] ADLOG::DcomWmiLogicLayer::analyze: analyzing data: nt_log_event: aduser1
283541:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: Username = aduser1
283544:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: InsertionStrings[0] = aduser1
283602:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Full field text (aduser1)
283604:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::EventFieldSettings::processField: Field extracted value: (aduser1)
283606:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Field extracted value (aduser1)
283614:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) and value (aduser1)
283615:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) pattern (][\/":|<>+=;,?*@) on value (aduser1)
283687: Account Name: aduser1@HIGHERINTELLIGENCE.COM
283826: Account Name: aduser1@HIGHERINTELLIGENCE.COM
283893:$$START$$ nt_log_event: aduser1@HIGHERINTELLIGENCE.COM $$END$$
283896:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_DCOM (TD::All)] ADLOG::DcomWmiLogicLayer::analyze: analyzing data: nt_log_event: aduser1@HIGHERINTELLIGENCE.COM
283961:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: Username = aduser1@HIGHERINTELLIGENCE.COM
283964:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: InsertionStrings[0] = aduser1@HIGHERINTELLIGENCE.COM
284019:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Full field text (aduser1@HIGHERINTELLIGENCE.COM)
284021:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::EventFieldSettings::processField: Field extracted value: (aduser1)
284023:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Field extracted value (aduser1)
284031:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) and value (aduser1)
284032:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) pattern (][\/":|<>+=;,?*@) on value (aduser1)
284104: Account Name: aduser1@HIGHERINTELLIGENCE.COM
284243: Account Name: aduser1@HIGHERINTELLIGENCE.COM
284302:$$START$$ nt_log_event: aduser1@HIGHERINTELLIGENCE.COM $$END$$
284305:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_DCOM (TD::All)] ADLOG::DcomWmiLogicLayer::analyze: analyzing data: nt_log_event: aduser1@HIGHERINTELLIGENCE.COM
284370:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: Username = aduser1@HIGHERINTELLIGENCE.COM
284373:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: InsertionStrings[0] = aduser1@HIGHERINTELLIGENCE.COM
284428:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Full field text (aduser1@HIGHERINTELLIGENCE.COM)
284430:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::EventFieldSettings::processField: Field extracted value: (aduser1)
284432:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Field extracted value (aduser1)
284440:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) and value (aduser1)
284441:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) pattern (][\/":|<>+=;,?*@) on value (aduser1)
284603: Account Name: aduser1
284869: Account Name: aduser1
284971:$$START$$ nt_log_event: aduser1 $$END$$
284974:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_DCOM (TD::All)] ADLOG::DcomWmiLogicLayer::analyze: analyzing data: nt_log_event: aduser1
285102:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: InsertionStrings[5] = aduser1
285160:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Full field text (aduser1)
285162:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::EventFieldSettings::processField: Field extracted value: (aduser1)
285164:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Field extracted value (aduser1)
285170:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) and value (aduser1)
285171:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) pattern (][\/":|<>+=;,?*@) on value (aduser1)
286065: Account Name: aduser1
286330: Account Name: aduser1
286432:$$START$$ nt_log_event: aduser1 $$END$$
286435:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_DCOM (TD::All)] ADLOG::DcomWmiLogicLayer::analyze: analyzing data: nt_log_event: aduser1
286563:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG (TD::Events)] ADLOG::NtEventLogEntry::NtEventLogEntry: InsertionStrings[5] = aduser1
286621:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Full field text (aduser1)
286623:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::EventFieldSettings::processField: Field extracted value: (aduser1)
286625:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventParser::processEvent: Field extracted value (aduser1)
286631:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) and value (aduser1)
286632:[6490 3967998864]@GW8010[28 Dec 19:40:57] [ADLOG_EVENT_PROCESS (TD::Events)] ADLOG::EventValidator::validateField: Validaing field (username) pattern (][\/":|<>+=;,?*@) on value (aduser1)
[Expert@GW8010:0]#

But the only event attributed to IA blade is:

This lab is build under NFR for partner with no active support contract, so I cannot open SR on it.

[Royi_Priov]

Hi Vladimir Yakovlev‌,

it seems that the user is received on the PDP gateway.

The next phase in the user processing is to perform LDAP query to get additional attributes from the AD.

please run the following debug flag:

# pdp d s AUTH all

to disable:

# pdp d u AUTH

please grep for "AUTH" in pdp debug and look for the query where we are asking for aduser1 information.

Thanks,

Royi.

[Vladimir]

Royi Priov‌, there are no results with "AUTH" in debug output.

See attached pdpd.elg for details.

[Royi_Priov]

Hi Vladimir Yakovlev‌,

The debug contains only "TRACKER" logs, which is the default topic.

make sure you are running the debugging procedure as I wrote - you can verify the current debug running topics with "pdp d stat".

And as I wrote above - this should be handled with an SR with TAC, as it seems that we need to troubleshoot this to understand why this happens in your environment. Our support will be happy to assist.

Thanks,

Royi.