Search documents migration

[sarangan12]

This PR is to move the search-documents SDK's generated code to the new v6 code generator and perform the corresponding custom code changes.

@xirzec @joheredi Please review and approve the PR.

@ramya-rao-a FYI....

[xirzec]

This is a bit tricky to review because of a lot of churn in result shapes, especially the way we've pulled out these seemingly not useful 'base' types.

My only real concern, however, is there are a few places were we were careful to deserialize special @odata properties separately from the user's document schema that appear to be broken. When we fix this, we should add tests that create a schema with properties like score, highlights, and text to check for these collisions.

[xirzec]

did these change to be extensible enums?

[sarangan12]

Yes. I have also exposed Known values.

[sarangan12]

@xirzec One of the questions you had was - What is the reason for this | null value at the end of dataChangeDetectionPolicy. To explain this, we need to go the swagger:

....................
....................
"SearchIndexerDataSource": {
  "properties": {
    "name": {
      "externalDocs": {
        "url": "https://docs.microsoft.com/rest/api/searchservice/Naming-rules"
      },
      "type": "string",
      "description": "The name of the datasource."
    },
    "description": {
      "type": "string",
      "description": "The description of the datasource."
    },
    "type": {
      "$ref": "#/definitions/SearchIndexerDataSourceType",
      "description": "The type of the datasource."
    },
    "credentials": {
      "$ref": "#/definitions/DataSourceCredentials",
      "description": "Credentials for the datasource."
    },
    "container": {
      "$ref": "#/definitions/SearchIndexerDataContainer",
      "description": "The data container for the datasource."
    },
    "dataChangeDetectionPolicy": {
      "$ref": "#/definitions/DataChangeDetectionPolicy",
      "x-nullable": true,
      "description": "The data change detection policy for the datasource."
    },
    "dataDeletionDetectionPolicy": {
      "$ref": "#/definitions/DataDeletionDetectionPolicy",
      "x-nullable": true,
      "description": "The data deletion detection policy for the datasource."
    },
    "@odata.etag": {
      "x-ms-client-name": "ETag",
      "type": "string",
      "description": "The ETag of the data source."
    },
    "encryptionKey": {
      "$ref": "#/definitions/SearchResourceEncryptionKey",
      "description": "................",
      "externalDocs": {
        "url": "https://aka.ms/azure-search-encryption-with-cmk"
      },
      "x-nullable": true
    }
  },
  "required": [
    "name",
    "type",
    "credentials",
    "container"
  ],
  "description": "Represents a datasource definition, which can be used to configure an indexer."
}

As you can see from the above swagger, the dataChangeDetectionPolicy has the property x-nullable: true. So, we need to add the | null value at the end.

What would happen if we do not add this?
If the SDK user wants to actually send in the null value (as the service allows it - assuming the value null has some special property on the server side.), it might not be possible to send it. Also, to reflect the swagger code closely, we need to add this | null at the end.

[xirzec]

I'm fine supporting something the server does -- I'm just curious how the server handles it. If you set null on an update, is that the proper way to remove the thing? And how is that functionally different from us removing that part of the API shape?

What's confusing me is null is how you'd blank out part of the payload in C#, but in JS we have undefined to do this same job. It's rare that undefined and null have different meanings. If they mean the same thing, do we need both?

[bterlson]

To me, nullable should mean optional. We really don't want users to have to think about the differences between {x: undefined}, {}, and {x: null} do we? I hope that no service treats these differently.

[xirzec]

I'm ok with the changes, though I'm a little confused why we need to have both null and undefined still. Like how would the actual REST call be different if you picked one or the other? Wouldn't they both be serialized as the empty string? Or is null going to actually send the string null or something?

[sarangan12]

I am going to discuss next week with @joheredi about this issue. At least, we could remove the null if the property is optional.

Now, I am merging this PR.