System.DirectoryServices.Protocols - Cannot use SortControl with custom attribute on linux

[null-d3v]

Description

On linux, using SortRequestControl with a custom attribute will cause the search request to fail with the following:

System.DirectoryServices.Protocols.DirectoryOperationException: The server does not support the control. The control is critical.
    at System.DirectoryServices.Protocols.LdapConnection.ConstructResponseAsync(Int32 messageId, LdapOperation operation, ResultAll resultType, TimeSpan requestTimeOut, Boolean exceptionOnTimeOut, Boolean sync)
    at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout)
    at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request)

I am able to use a sort control with something basic, like common name.

Sample:

var searchRequest = new SearchRequest(
    $"{LdapOptions.DefaultContainer},{LdapOptions.DnFragment}",
    $"(objectCategory={LdapConstants.UserObjectClass})",
    SearchScope.Subtree,
    LdapConstants.RequestedAttributes);

var sortRequestControl = new SortRequestControl(
    "customAttribute", false);

var vlvRequestControl = new VlvRequestControl(
    0,
    searchOptions.PageSize.Value - 1,
    (searchOptions.PageSize.Value * searchOptions.PageIndex.Value) + 1);

searchRequest.Controls.Add(sortRequestControl);
searchRequest.Controls.Add(vlvRequestControl);

var searchOptionsControl = new SearchOptionsControl(SearchOption.DomainScope);
searchRequest.Controls.Add(searchOptionsControl);

var searchResults = (SearchResponse)LdapConnection.SendRequest(searchRequest);

Configuration

• .NET 5.0.0-preview.820414.8
• aspnet:5.0-alpine docker image
• x86_64
• libldap 2.4
• Windows Server 2019 DC

Regression

The exact code is functional when executing from the docker container's Windows host.

[ericstj]

@joperezr any idea if this is a feature gap in OpenLDAP or something we could fix in our implementation?

[lscorcia]

+1 for this. Encountered the very same issue when querying AD (2008R2 domain) from Linux, using a sort control for the uid attribute. No issue when the same code is run from Windows.

I compared the exchanged packets on Linux and Windows and it seems that on Linux the attribute name gets truncated - possibly some marshalling issue?

[lscorcia]

I think I have a possible explanation for the issue. I built the S.DS.P library on Linux in Debug mode and referenced that in my application instead of the framework provided one in order to debug it using VS Code. Everything is fine until the call to

runtime/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/ldap/LdapConnection.cs

Line 1251 in e21eacb

byte[] byteControlValue = tempControl.GetValue();

This is the call that should produce the correct BER encoded value for the sort control. Stepping into it, we arrive at

runtime/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/common/DirectoryControl.cs

Line 719 in e21eacb

Marshal.StructureToPtr(_keys[i], sortPtr, false);

Here, the value of each sort key is marshalled to an unmanaged pointer. The SortKey structure is annotated as using Unicode marshalling:

runtime/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/common/DirectoryControl.cs

Lines 53 to 54 in 3ae8739

	[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
	public class SortKey

This is fine on Windows, as the Interop layer is referencing the Unicode APIs:

runtime/src/libraries/Common/src/Interop/Windows/Wldap32/Interop.Ldap.cs

Lines 206 to 208 in ab0de53

	[GeneratedDllImport(Libraries.Wldap32, EntryPoint = "ldap_create_sort_controlW", CharSet = CharSet.Unicode)]
	[UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
	public static partial int ldap_create_sort_control(ConnectionHandle handle, IntPtr keys, byte critical, ref IntPtr control);

But on Linux, we are actually invoking the ANSI ones:

runtime/src/libraries/Common/src/Interop/Linux/OpenLdap/Interop.Ldap.cs

Lines 191 to 192 in ab0de53

	[GeneratedDllImport(Libraries.OpenLdap, EntryPoint = "ldap_create_sort_control", CharSet = CharSet.Ansi)]
	public static partial int ldap_create_sort_control(ConnectionHandle handle, IntPtr keys, byte critical, ref IntPtr control);

So what is happening is that the API finds a null terminator right after the first character and generates the truncated value I found in the network trace. Unless I am mistaken in the details of the marshalling process, I guess this is really a bug and that there is no applicable workaround. I would love some feedback, at least to validate that what I learnt from this investigation is correct!

Also - is there a way to visualize the content of unmanaged memory during debugging in VS Code? It would be nice to have visual confirmation of the issue by inspecting the marshalled pointers.

[lscorcia]

Anyone that can triage this? Maybe @joperezr ?

[joperezr]

Hey @lscorcia so sorry for missing the notification and thank you a lot for the very detailed report. I do think what you describe about the difference in Ansi vs Unicode is the culprit and it does sound like a very reasonable explanation of why SortControl (and any other controls for that matter) won't work today in Linux. I also agree with you that this is a bug and that we should fix it.

I've adjusted the milestone to 7.0.0, but I don't currently have the cycles to work on this. @lscorcia Would you be interested in contributing a fix for this issue 😃 ?

[lscorcia]

Thank you @joperezr . I just committed a fix on my fork (see above), if it's ok for you I'll open a PR.
I also noticed #34679 which could be related. Should I touch that ActiveIssue attribute?

[joperezr]

Thanks for sharing that commit @lscorcia, there might be some small comments in there, but in general I think it has the right idea so please do open a PR with your fix. I assume you have been able to validate that this fix addresses the issue and now you are able to use the Sort Controls?

[lscorcia]

I tested it by calling it with my code and it works as expected, I did not run the full runtime test suite as it would be too heavy for my VM but I suppose that will run as a Github action when I submit the PR tomorrow. Thanks for your help!

[lscorcia]

PR #65548 is ready for feedback!