Active Directory range retrieval returns duplicates?

[christopher-b]

I'm using range retrieval on an AD server to load the members of a group containing around 5600 members. The resulting list of members contains duplicated entries, and is missing some entries that are members of the group. The number of duplicates is not consistent across runs.

Something strange is there are no new duplicates after around record 3670. Regardless of page size, all duplicates occur between the second page and whatever page loads a range containing a record around 3670. No new duplicates are found after that point.

This only happens on my production machine, not on my workstation. Both are running the same version of Ruby and the same version of this gem (0.11). So my hunch is that it might be a network issue, but I'm not sure what that might be or where to look.

Has anyone ever encountered anything like this? It's a bit baffling. If I can't figure out the cause of this error, I should be able to work around it by loading entries with memberOf=group_name, but I'd like to figure this out. Thanks in advance for any insight.

Here's the code I'm using to load the group membership, and the resulting output.

def load_members
  range_regex    = /member;range=\d+-(\d+|\*)/ # member;range=0-1499, member;range=1500-*
  remote_members = []
  start          = 0
  match          = nil
  loop do
    entry = search("member;range=#{start}-*").first
    range = entry.attribute_names.map(&:to_s).find { |attr| match = attr.match range_regex }
    break unless range
    puts "Found range: #{range}"
    remote_members.concat entry[range]
    puts "Added #{entry[range].size}"
    puts "Duplicates: #{remote_members.size-remote_members.uniq.size}"
    stop = match[1]
    break if stop == '*' # Halt if we're at the end of the records: member;range=1500-*
    start = stop.to_i + 1
  end
  remote_members
end

def search(attributes)
  puts "Searching for: #{attributes} "
  @ldap.search(
    base:       ou,
    filter:     Net::LDAP::Filter.eq('sAMAccountName', name),
    attributes: attributes
  )
end

Searching for: member;range=0-*
Found range: member;range=0-1499
Added 1500
Duplicates: 0
Searching for: member;range=1500-*
Found range: member;range=1500-2999
Added 1500
Duplicates: 53
Searching for: member;range=3000-*
Found range: member;range=3000-4499
Added 1500
Duplicates: 425
Searching for: member;range=4500-*
Found range: member;range=4500-*
Added 1170
Duplicates: 425
Members: 5670
Unique: 5245

[jch]

Do you notice this behavior with other LDAP clients? It would provide another data point to compare against to narrow down whether it's a bug in the library or elsewhere.

[astockwell]

Using @christopher-b 's code above (which was super helpful BTW), I get inconsistent results as well from our own Active Directory DC.

If I run the same script over and over on a group w/ 4477 users, sometimes it reports 0 duplicates, sometimes it reports as many as 46 (so far). Doing a dump of the member attribute from another LDAP GUI tool, it corroborates that there are -in fact- 4477 unique members.

That said, I still get the feeling this is an AD issue, not an issue with this gem. I'm much more inclined to believe the GUI tool is repeating the query until it gets no duplicates, than that this gem could occasionally mangle the results.

That's what I'll probably do as well - structure my queries so that, if duplicates are detected, it re-queries that attribute from scratch until it get the correct values. Not ideal at all, but then AD has always been a piece of 💩

[jch]

@astockwell thanks for the additional data point. Do you have another LDAP client available to confirm this is server side behavior?

[a2f0]

@christopher-b - this code is super useful, nice algorithm; thank you. FWIW I don't see any duplicates returning with net-ldap 0.11 on a group with ~2K members.