[core-client] x-www-form-urlencoded parameters not serialized

[jeremymeng]

Some ACR generated methods use application/x-www-form-urlencoded to send request. Currently an error is thrown when using those generated methods.

Here's new unit test that represents the scenario.

  it.only("should serialize x-www-form-urlencoded parameters", async () => {                                                                                                                                                                                                                                                                                        
    const httpRequest = createPipelineRequest({ url: "https://example.com" });                                                                                                                                                                                                                                                                                      
    const operationSpec: OperationSpec = {                                                                                                                                                                                                                                                                                                                          
      path: "/oauth2/exchange",                                                                                                                                                                                                                                                                                                                                     
      httpMethod: "POST",                                                                                                                                                                                                                                                                                                                                           
      responses: {                                                                                                                                                                                                                                                                                                                                                  
        200: {}                                                                                                                                                                                                                                                                                                                                                     
      },                                                                                                                                                                                                                                                                                                                                                            
      formDataParameters: [                                                                                                                                                                                                                                                                                                                                         
        {                                                                                                                                                                                                                                                                                                                                                           
          parameterPath: ["path1", "aadAccesstoken"],                                                                                                                                                                                                                                                                                                               
          mapper: {                                                                                                                                                                                                                                                                                                                                                 
            type: {                                                                                                                                                                                                                                                                                                                                                 
              name: "Composite",                                                                                                                                                                                                                                                                                                                                    
              modelProperties: {                                                                                                                                                                                                                                                                                                                                    
                grantType: {                                                                                                                                                                                                                                                                                                                                        
                  defaultValue: "access_token",                                                                                                                                                                                                                                                                                                                     
                  isConstant: true,                                                                                                                                                                                                                                                                                                                                 
                  serializedName: "grant_type",                                                                                                                                                                                                                                                                                                                     
                  type: {                                                                                                                                                                                                                                                                                                                                           
                    name: "String"                                                                                                                                                                                                                                                                                                                                  
                  }                                                                                                                                                                                                                                                                                                                                                 
                },                                                                                                                                                                                                                                                                                                                                                  
                service: {                                                                                                                                                                                                                                                                                                                                          
                  serializedName: "service",                                                                                                                                                                                                                                                                                                                        
                  required: true,                                                                                                                                                                                                                                                                                                                                   
                  type: {                                                                                                                                                                                                                                                                                                                                           
                    name: "String"                                                                                                                                                                                                                                                                                                                                  
                  }                                                                                                                                                                                                                                                                                                                                                 
                }                                                                                                                                                                                                                                                                                                                                                   
              }                                                                                                                                                                                                                                                                                                                                                     
            }                                                                                                                                                                                                                                                                                                                                                       
          }                                                                                                                                                                                                                                                                                                                                                         
        }                                                                                                                                                                                                                                                                                                                                                           
      ],                                                                                                                                                                                                                                                                                                                                                            
      headerParameters: [                                                                                                                                                                                                                                                                                                                                           
        {                                                                                                                                                                                                                                                                                                                                                           
          parameterPath: ["options", "contentType"],                                                                                                                                                                                                                                                                                                                
          mapper: {                                                                                                                                                                                                                                                                                                                                                 
            defaultValue: "application/x-www-form-urlencoded",                                                                                                                                                                                                                                                                                                      
            isConstant: true,                                                                                                                                                                                                                                                                                                                                       
            serializedName: "Content-Type",                                                                                                                                                                                                                                                                                                                         
            type: {                                                                                                                                                                                                                                                                                                                                                 
              name: "String"                                                                                                                                                                                                                                                                                                                                        
            }                                                                                                                                                                                                                                                                                                                                                       
          }                                                                                                                                                                                                                                                                                                                                                         
        }                                                                                                                                                                                                                                                                                                                                                           
      ],                                                                                                                                                                                                                                                                                                                                                            
      serializer: createSerializer()                                                                                                                                                                                                                                                                                                                                
    };                                                                                                                                                                                                                                                                                                                                                              
    serializeRequestBody(                                                                                                                                                                                                                                                                                                                                           
      httpRequest,                                                                                                                                                                                                                                                                                                                                                  
      {                                                                                                                                                                                                                                                                                                                                                             
        path1: {                                                                                                                                                                                                                                                                                                                                                    
          aadAccesstoken: {                                                                                                                                                                                                                                                                                                                                         
            grantType: "refresh_token",                                                                                                                                                                                                                                                                                                                             
            service: "myregistry.azurecr.io"                                                                                                                                                                                                                                                                                                                        
          }                                                                                                                                                                                                                                                                                                                                                         
        }                                                                                                                                                                                                                                                                                                                                                           
      },                                                                                                                                                                                                                                                                                                                                                            
      operationSpec                                                                                                                                                                                                                                                                                                                                                 
    );                                                                                                                                                                                                                                                                                                                                                              
    const response: PipelineResponse = {                                                                                                                                                                                                                                                                                                                            
      headers: createHttpHeaders({ location: "https://example.com/redirect" }),                                                                                                                                                                                                                                                                                     
      request: httpRequest,                                                                                                                                                                                                                                                                                                                                         
      status: 300                                                                                                                                                                                                                                                                                                                                                   
    };                                                                                                                                                                                                                                                                                                                                                              
    const policy = formDataPolicy();                                                                                                                                                                                                                                                                                                                                
    const next = (request: PipelineRequest) => {                                                                                                                                                                                                                                                                                                                    
      console.log(request.body);                                                                                                                                                                                                                                                                                                                                    
      assert.strictEqual(httpRequest.body, "grant_type=access_token&service=myregistry.azurecr.io");                                                                                                                                                                                                                                                                
      return Promise.resolve( response);                                                                                                                                                                                                                                                                                                                            
    }                                                                                                                                                                                                                                                                                                                                                               
    await policy.sendRequest(httpRequest, next);                                                                                                                                                                                                                                                                                                                    
  }); 

[ramya-rao-a]

What is the workaround that is being used at the moment?

[jeremymeng]

What is the workaround that is being used at the moment?

azure-sdk-for-js/sdk/containerregistry/container-registry/src/containerRegistryTokenCredential.ts

Lines 110 to 130 in 67a3401

	// const acrRefreshToken = await this.authClient.authentication.exchangeAadTokenForAcrRefreshToken(
	// {
	// aadAccesstoken: {
	// grantType: "access_token",
	// service,
	// aadAccesstoken: aadAccessToken,
	// }
	// }
	// );
	// TODO: (jeremymeng) revert custom sendOperationRequest call after FormData is working in core
	const payload = `grant_type=access_token&service=${encodeURIComponent(
	service
	)}&access_token=${encodeURIComponent(aadAccessToken)}`;
	const customOptions: CustomAuthOptions = {
	payload
	};
	const acrRefreshToken = await this.authClient.sendOperationRequest<AcrRefreshToken>(
	{ options: { ...options, ...customOptions } },
	customExchangeAadTokenForAcrRefreshTokenOperationSpec
	);

[jeremymeng]

The generated ACR operation spec:

azure-sdk-for-js/sdk/containerregistry/container-registry/src/generated/operations/authentication.ts

Lines 78 to 93 in 45a3fd3

	const exchangeAcrRefreshTokenForAcrAccessTokenOperationSpec: coreClient.OperationSpec = {
	path: "/oauth2/token",
	httpMethod: "POST",
	responses: {
	200: {
	bodyMapper: Mappers.AcrAccessToken
	},
	default: {
	bodyMapper: Mappers.AcrErrors
	}
	},
	formDataParameters: [Parameters.acrRefreshToken],
	urlParameters: [Parameters.url],
	headerParameters: [Parameters.contentType3, Parameters.accept4],
	serializer
	};

In serialization policy

azure-sdk-for-js/sdk/core/core-client/src/serializationPolicy.ts

Line 211 in 45a3fd3

const formDataParameterValue = getOperationArgumentValueFromParameter(

we get

formDataParameterPropertyName of "path1.aadAccesstoken", and serialized value of { grant_type: 'access_token', service: 'myregistry.azurecr.io' }, so we are setting

  request.formData["path1.aadAccesstoken"] = { grant_type: 'access_token', service: 'myregistry.azurecr.io' }

while I'd expect form data to be

  request.formData["grant_type"] = "access_token";
  request.formData["service"] = "myregistry.azurecr.io";

[jeremymeng]

@sarangan12 @joheredi do you know whether this is expected generated code? I looked at other languages but couldn't find similar things. They seems to just use form content (Python) or encoded content (.NET),

[sarangan12]

@jeremymeng

Looking at the spec (provided in the first issue description), the parameterPath: ["path1", "aadAccesstoken"], then it has to be

 request.formData["path1.aadAccesstoken"] = { grant_type: 'access_token', service: 'myregistry.azurecr.io' }

I am not clear about the reasoning about the expectation:

request.formData["grant_type"] = "access_token";
request.formData["service"] = "myregistry.azurecr.io";

Could you please explain a little more?

[jeremymeng]

I am not clear about the reasoning about the expectation:

This is what the service actually expects, which suggests that for form data, the parameter path path1.addAccesstToken doesn't matter.

For application/x-www-form-urlencoded content type, I'd expect a list of name-value string pairs, and they would be url encoded, each pair connected with "=" and all of them concatenated with "&" as delimitor. like "grant_type=access_token&service=myregistry.azurecr.io"

I wonder whether .NET or Python code gen have any special handling for form data parameters.

Right now one idea for fixing in core is to

• check in formDataPolicy.ts that if a form data key-value pair has an object value, then flatten it and add each property's name-value pair.

but I am still trying to understand the problem.

[jeremymeng]

kinda of related to #16007. It would be nice if we could generate parameters for form parameters for application/x-www-form-urlencoded content type too

[jeremymeng]

Not sure what has changed in the codegen since this is logged. But now the generated code is different so this might not be an issue any more.

  formDataParameters: [
    Parameters.service,
    Parameters.scope,
    Parameters.refreshToken1,
    Parameters.grantType1
  ],

[jeremymeng]

Fixed by #18560