New SDK - SchemaRegistry (#13657)

* Generated the SchemaRegistry SDK. Set up project and custom client. Fixed original swagger and made a separate PR for it.

* Update to latest swagger with description fixes.

* Fixed changlog. Regen surface.

* Regen with correct swagger.

* Started adding client methods.

* Update SchemaId to be struct. Regen api.

* Initial attempt of serializer.

* Update swagger version. Regenerate rest client. Update public surface. Created first working test.

* Added happy path tests for each client method. Added recordings.

* Move serialization stuff into the Azure.Data.SchemaRegistry.Avro project. Created project, test project, and tool (to test Avro).

* Modify SchemaProperties to have smaller set of properties. Rerun and rename tests. Update api.

* Still attempting to get Avro library usable functionality. Some reflection-based usage is working. Unfortunately, we're encoding the avro file incorrectly (it includes schema).

* Initial serializer implementation. Only does specific records.

* Fixed issues with specific record schema de/serialization.

* Got GenericRecord working. Needs cleanup.

* Code cleanup begins!

* :-) Cleaned up code for serializers. Wrote proper async implementation.

* Renamed classes to be more accurate. Removal of commented code.

* Moved serializer over to main project. Removed other project and solution.

* Added 1 test for the serializer.

* Update api.

* Added back the separate solution. Named it appropriately.

* Addressing some changes from API view comments.

* Added some negative tests. Updated public surface descriptions.

* In progress writing readmes.

* Fixed position reset in deserialization stream.

* Made constants for wire format lengths/positions.

* Updated property description. Made static field for UTF8 encoding.

* Removed unused Directory.Build.props.

* Removed unused description.

* Continuing readme. Started adding snippets.

* First readme completed.

* Finally! Created readmes and snippets.

* Put credential scope string into a constant.

* Addressing a couple documentation PR comments and removed a code comment.

* Added global suppressions for namespace name.

* Forgot using for memory streams in tests and snippets.

* Fixed bugged diagnostic scope names. Put scope names into constants.

* Updated ci configuration. Removed resources template since resources cannot be provisioned currently.

* Added the other package to CI.

* Updated sample file names to match convention.

* Added missing readme links.

* Fixed the readme snippets so they don't format oddly.

* Updated schema components descriptions.

* Fixed command to acquire packages.

sdk/schemaregistry/Azure.Data.SchemaRegistry/Azure.Data.SchemaRegistry.sln

@@ -0,0 +1,37 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 16
+VisualStudioVersion = 16.0.29709.97
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Data.SchemaRegistry", "src\Azure.Data.SchemaRegistry.csproj", "{E33D09D9-D809-472C-82E6-6A26BDB86FC2}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Data.SchemaRegistry.Tests", "tests\Azure.Data.SchemaRegistry.Tests.csproj", "{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Core.TestFramework", "..\..\core\Azure.Core.TestFramework\src\Azure.Core.TestFramework.csproj", "{8052009B-2126-44A3-88CD-4F3B17894C64}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Release|Any CPU.Build.0 = Release|Any CPU
+		{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Release|Any CPU.Build.0 = Release|Any CPU
+		{8052009B-2126-44A3-88CD-4F3B17894C64}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{8052009B-2126-44A3-88CD-4F3B17894C64}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{8052009B-2126-44A3-88CD-4F3B17894C64}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{8052009B-2126-44A3-88CD-4F3B17894C64}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+	GlobalSection(ExtensibilityGlobals) = postSolution
+		SolutionGuid = {A97F4B90-2591-4689-B1F8-5F21FE6D6CAE}
+	EndGlobalSection
+EndGlobal

sdk/schemaregistry/Azure.Data.SchemaRegistry/CHANGELOG.md

@@ -0,0 +1,7 @@
+# Release History
+
+## 1.0.0-beta.1 (2020-09-08)
+- Added SchemaRegistryClient with 3 operations:
+  - RegisterSchema
+  - GetSchemaId
+  - GetSchema

sdk/schemaregistry/Azure.Data.SchemaRegistry/README.md

@@ -0,0 +1,158 @@
+# Azure Schema Registry client library for .NET
+
+The Azure Schema Registry service allows developers to provide and retrieve data schemas from a centralized repository for use in messaging systems.
+
+## Getting started
+
+### Install the package
+
+Install the Azure Schema Registry client library for .NET with [NuGet][nuget]:
+
+```bash
+dotnet add package Azure.Data.SchemaRegistry --version 1.0.0-beta.1
+```
+
+### Prerequisites
+
+* An [Azure subscription][azure_sub]
+* An [Event Hubs namespace][event_hubs_namespace]
+
+If you need to [create an Event Hubs namespace][create_event_hubs_namespace], you can use the Azure Portal or [Azure PowerShell][azure_powershell].
+
+You can use Azure PowerShell to create the Event Hubs namespace with the following command:
+
+```PowerShell
+New-AzEventHubNamespace -ResourceGroupName myResourceGroup -NamespaceName namespace_name -Location eastus
+```
+
+### Authenticate the client
+
+In order to interact with the Azure Schema Registry service, you'll need to create an instance of the [Schema Registry Client][schema_registry_client] class. To create this client, you'll need Azure resource credentials and the Event Hubs namespace hostname.
+
+#### Get credentials
+
+To acquire authenicated credentials and start interacting with Azure resources, please see the [quickstart guide here][quickstart_guide].
+
+#### Get Event Hubs namespace hostname
+
+The simpliest way is to use the [Azure portal][azure_portal] and navigate to your Event Hubs namespace. From the Overview tab, you'll see `Host name`. Copy the value from this field.
+
+#### Create SchemaRegistryClient
+
+Once you have the Azure resource credentials and the Event Hubs namespace hostname, you can create the [SchemaRegistryClient][schema_registry_client]. You'll also need the [Azure.Identity][azure_identity] package to create the credential.
+
+```C# Snippet:CreateSchemaRegistryClient
+string endpoint = "<event_hubs_namespace_hostname>";
+var credentials = new ClientSecretCredential(
+    "<tenant_id>",
+    "<client_id>",
+    "<client_secret>"
+);
+var client = new SchemaRegistryClient(endpoint, credentials);
+```
+
+## Key concepts
+
+### Schemas
+
+A schema has 6 components:
+- Group Name: The name of the group of schemas in the Schema Registry instance.
+- Schema Name: The name of the schema.
+- Schema ID: The ID assigned by the Schema Registry instance for the schema.
+- Serialization Type: The format used for serialization of the schema. For example, Avro.
+- Schema Content: The string representation of the schema.
+- Schema Version: The version assigned to the schema in the Schema Registry instance.
+
+These components play different roles. Some are used as input into the operations and some are outputs. Currently, [SchemaProperties][schema_properties] only exposes those properties that are potential outputs that are used in SchemaRegistry operations. Those exposed properties are `Content` and `Id`.
+
+## Examples
+
+The following shows examples of what is available through the SchemaRegistryClient. There are both sync and async methods available for these client operations.
+
+* [Register a schema](#register-a-schema)
+* [Retrieve a schema ID](#retrieve-a-schema-id)
+* [Retrieve a schema](#retrieve-a-schema)
+
+### Register a schema
+
+Register a schema to be stored in the Azure Schema Registry.
+
+```C# Snippet:RegisterSchema
+string schemaName = "<schema_name>";
+string groupName = "<schema_group_name>";
+SerializationType schemaType = SerializationType.Avro;
+// Example schema's content
+string schemaContent = @"
+{
+   ""type"" : ""record"",
+    ""namespace"" : ""TestSchema"",
+    ""name"" : ""Employee"",
+    ""fields"" : [
+    { ""name"" : ""Name"" , ""type"" : ""string"" },
+    { ""name"" : ""Age"", ""type"" : ""int"" }
+    ]
+}";
+
+Response<SchemaProperties> schemaProperties = client.RegisterSchema(groupName, schemaName, schemaType, schemaContent);
+```
+
+### Retrieve a schema ID
+
+Retrieve a previously registered schema ID from the Azure Schema Registry.
+
+```C# Snippet:RetrieveSchemaId
+string schemaName = "<schema_name>";
+string groupName = "<schema_group_name>";
+SerializationType schemaType = SerializationType.Avro;
+// Example schema's content
+string schemaContent = @"
+{
+   ""type"" : ""record"",
+    ""namespace"" : ""TestSchema"",
+    ""name"" : ""Employee"",
+    ""fields"" : [
+    { ""name"" : ""Name"" , ""type"" : ""string"" },
+    { ""name"" : ""Age"", ""type"" : ""int"" }
+    ]
+}";
+
+Response<SchemaProperties> schemaProperties = client.GetSchemaId(groupName, schemaName, schemaType, schemaContent);
+string schemaId = schemaProperties.Value.Id;
+```
+
+### Retrieve a schema
+
+Retrieve a previously registered schema's content from the Azure Schema Registry.
+
+```C# Snippet:RetrieveSchema
+string schemaId = "<schema_id>";
+
+Response<SchemaProperties> schemaProperties = client.GetSchema(schemaId);
+string schemaContent = schemaProperties.Value.Content;
+```
+
+## Contributing
+
+This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].
+
+When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
+
+This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][code_of_conduct_faq] or contact [[email protected]][email_opencode] with any additional questions or comments.
+
+![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Ftemplate%2FAzure.Template%2FREADME.png)
+
+<!-- LINKS -->
+[nuget]: https://www.nuget.org/
+[event_hubs_namespace]: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about
+[azure_powershell]: https://docs.microsoft.com/en-us/powershell/azure/
+[create_event_hubs_namespace]: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-powershell#create-an-event-hubs-namespace
+[quickstart_guide]: https://github.com/Azure/azure-sdk-for-net/blob/master/doc/mgmt_preview_quickstart.md
+[schema_registry_client]: src/SchemaRegistryClient.cs
+[azure_portal]: https://ms.portal.azure.com/
+[schema_properties]: src/SchemaProperties.cs
+[azure_identity]: https://www.nuget.org/packages/Azure.Identity
+[cla]: https://cla.microsoft.com
+[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
+[code_of_conduct_faq]: https://opensource.microsoft.com/codeofconduct/faq/
+[email_opencode]: mailto:[email protected]
+[azure_sub]: https://azure.microsoft.com/free/

sdk/schemaregistry/Azure.Data.SchemaRegistry/api/Azure.Data.SchemaRegistry.netstandard2.0.cs

@@ -0,0 +1,46 @@
+namespace Azure.Data.SchemaRegistry
+{
+    public partial class SchemaProperties
+    {
+        internal SchemaProperties() { }
+        public string Content { get { throw null; } }
+        public string Id { get { throw null; } }
+    }
+    public partial class SchemaRegistryClient
+    {
+        protected SchemaRegistryClient() { }
+        public SchemaRegistryClient(string endpoint, Azure.Core.TokenCredential credential) { }
+        public SchemaRegistryClient(string endpoint, Azure.Core.TokenCredential credential, Azure.Data.SchemaRegistry.SchemaRegistryClientOptions options) { }
+        public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> GetSchema(string schemaId, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+        public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> GetSchemaAsync(string schemaId, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+        public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> GetSchemaId(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+        public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> GetSchemaIdAsync(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+        public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> RegisterSchema(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+        public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> RegisterSchemaAsync(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+    }
+    public partial class SchemaRegistryClientOptions : Azure.Core.ClientOptions
+    {
+        public SchemaRegistryClientOptions(Azure.Data.SchemaRegistry.SchemaRegistryClientOptions.ServiceVersion version = Azure.Data.SchemaRegistry.SchemaRegistryClientOptions.ServiceVersion.V2017_04) { }
+        public enum ServiceVersion
+        {
+            V2017_04 = 1,
+        }
+    }
+    [System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
+    public readonly partial struct SerializationType : System.IEquatable<Azure.Data.SchemaRegistry.SerializationType>
+    {
+        private readonly object _dummy;
+        private readonly int _dummyPrimitive;
+        public SerializationType(string value) { throw null; }
+        public static Azure.Data.SchemaRegistry.SerializationType Avro { get { throw null; } }
+        public bool Equals(Azure.Data.SchemaRegistry.SerializationType other) { throw null; }
+        [System.ComponentModel.EditorBrowsableAttribute(System.ComponentModel.EditorBrowsableState.Never)]
+        public override bool Equals(object obj) { throw null; }
+        [System.ComponentModel.EditorBrowsableAttribute(System.ComponentModel.EditorBrowsableState.Never)]
+        public override int GetHashCode() { throw null; }
+        public static bool operator ==(Azure.Data.SchemaRegistry.SerializationType left, Azure.Data.SchemaRegistry.SerializationType right) { throw null; }
+        public static implicit operator Azure.Data.SchemaRegistry.SerializationType (string value) { throw null; }
+        public static bool operator !=(Azure.Data.SchemaRegistry.SerializationType left, Azure.Data.SchemaRegistry.SerializationType right) { throw null; }
+        public override string ToString() { throw null; }
+    }
+}

sdk/schemaregistry/Azure.Data.SchemaRegistry/src/Azure.Data.SchemaRegistry.csproj

@@ -0,0 +1,29 @@
+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <Description>Azure Schema Registry SDK</Description>
+    <AssemblyTitle>Azure Schema Registry SDK</AssemblyTitle>
+    <Version>1.0.0-beta.1</Version>
+    <PackageTags>Azure;Schema Registry;SchemaRegistry;.NET;Data;$(PackageCommonTags)</PackageTags>
+    <TargetFrameworks>$(RequiredTargetFrameworks)</TargetFrameworks>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="Apache.Avro" VersionOverride="1.10.0" />
+  </ItemGroup>
+
+  <!-- Shared source from Azure.Core -->
+  <ItemGroup>
+    <Compile Include="$(AzureCoreSharedSources)ArrayBufferWriter.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)ClientDiagnostics.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)ContentTypeUtilities.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)DiagnosticScope.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)DiagnosticScopeFactory.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)HttpMessageSanitizer.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)TaskExtensions.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)OperationHelpers.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+    <Compile Include="$(AzureCoreSharedSources)Argument.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
+  </ItemGroup>
+
+  <Import Project="$(MSBuildThisFileDirectory)..\..\..\core\Azure.Core\src\Azure.Core.props" />
+
+</Project>