LDAP search failed: Result Code 4 "Size Limit Exceeded"

[eirslett]

Feature Request:
Setting size limit for LDAP queries

Environment:
Linux, Vault v0.9.5

Vault Config File:

vault write auth/ldap/config \
  url="ldaps://ldap.example.com:636" \
  userdn="OU=Users,OU=Org,OU=BusinessUnits,DC=example,DC=com" \
  groupdn="OU=AccountGroups,OU=Groups,OU=Org,OU=BusinessUnits,DC=example,DC=com" \
  groupfilter="(&(objectClass=group))" \
  groupattr="memberOf" \
  upndomain="example.com" \
  insecure_tls=false \
  starttls=false

vault write auth/ldap/groups/developers policies=dev

Startup Log Output:

Expected Behavior:
vault login -method=ldap username=<user> with password, should be able to login

Actual Behavior:
Error message: Result Code 4 "Size Limit Exceeded" from the LDAP server.

Important Factoids:
The LDAP environment has very many entries.

Is it possible to set a parameter to increase the limit of groups to return?

[jefferai]

What parameter would that be? Is it something standardized across ldap servers?

[eirslett]

At least Active Directory supports it: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc755809(v=ws.10)

[jefferai]

That's a very long article, what parameter are you referring to?

[eirslett]

You're right, I'm sorry to refer to such a long article. I tried to track it down in the go code instead. I suspect it's called SizeLimit here:

https://github.com/hashicorp/vault/blob/master/vendor/github.com/go-ldap/ldap/search.go#L241

In typical AD/LDAP setups the default is set server-side at 1000, while it can be overridden per query. (That is my understanding)

[aryak007]

Even after I set SizeLimit parameter to math.MaxInt64 or math.MaxInt32 or to any large value, it doesn't seem to work. I still get Code 4 "Size Limit Exceeded". What am I missing?

[aryak007]

Even after I set SizeLimit parameter to math.MaxInt64 or math.MaxInt32 or to any large value, it doesn't seem to work. I still get Code 4 "Size Limit Exceeded". What am I missing?

It was a pagination problem and it's working fine now.

[csquire]

@jefferai This is still a problem in 1.2.3. Vault needs to search with pagination and set the page size to a high value as well.

This search appears to need updated: https://github.com/hashicorp/vault/blob/master/sdk/helper/ldaputil/client.go#L214-L222

Should use something like this:

	result, err := conn.SearchWithPaging(&ldap.SearchRequest{
		BaseDN: cfg.GroupDN,
		Scope:  ldap.ScopeWholeSubtree,
		Filter: renderedQuery.String(),
		Attributes: []string{
			cfg.GroupAttr,
		},
		SizeLimit: math.MaxInt32,
	}, math.MaxInt32)