Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New SDK - SchemaRegistry #13657

Merged
merged 51 commits into from
Sep 5, 2020
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
63045ea
Generated the SchemaRegistry SDK. Set up project and custom client. F…
MiYanni Jul 22, 2020
ce5b15c
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Jul 30, 2020
6b44ba3
Update to latest swagger with description fixes.
MiYanni Jul 31, 2020
e0b8961
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Aug 5, 2020
4c7a288
Fixed changlog. Regen surface.
MiYanni Aug 5, 2020
202090a
Regen with correct swagger.
MiYanni Aug 5, 2020
96e42d3
Started adding client methods.
MiYanni Aug 6, 2020
66a4426
Update SchemaId to be struct. Regen api.
MiYanni Aug 7, 2020
aa5b5cb
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Aug 11, 2020
86932ac
Initial attempt of serializer.
MiYanni Aug 12, 2020
7ff033c
Update swagger version. Regenerate rest client. Update public surface…
MiYanni Aug 13, 2020
ba0addd
Added happy path tests for each client method. Added recordings.
MiYanni Aug 13, 2020
3c8e1b1
Move serialization stuff into the Azure.Data.SchemaRegistry.Avro proj…
MiYanni Aug 21, 2020
3853114
Modify SchemaProperties to have smaller set of properties. Rerun and …
MiYanni Aug 21, 2020
1b5999c
Still attempting to get Avro library usable functionality. Some refle…
MiYanni Aug 27, 2020
eaec4d9
Initial serializer implementation. Only does specific records.
MiYanni Aug 29, 2020
8ca6a68
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Aug 31, 2020
af23793
Fixed issues with specific record schema de/serialization.
MiYanni Sep 1, 2020
8d04053
Got GenericRecord working. Needs cleanup.
MiYanni Sep 1, 2020
306fa4f
Code cleanup begins!
MiYanni Sep 1, 2020
d65d746
:-) Cleaned up code for serializers. Wrote proper async implementation.
MiYanni Sep 2, 2020
6daf008
Renamed classes to be more accurate. Removal of commented code.
MiYanni Sep 2, 2020
aff9ef7
Moved serializer over to main project. Removed other project and solu…
MiYanni Sep 2, 2020
52220c9
Added 1 test for the serializer.
MiYanni Sep 3, 2020
9940ec4
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Sep 3, 2020
817f11d
Update api.
MiYanni Sep 3, 2020
b61cd0e
Added back the separate solution. Named it appropriately.
MiYanni Sep 3, 2020
96887b6
Addressing some changes from API view comments.
MiYanni Sep 4, 2020
7b25995
Added some negative tests. Updated public surface descriptions.
MiYanni Sep 4, 2020
88339c2
In progress writing readmes.
MiYanni Sep 4, 2020
5194247
Fixed position reset in deserialization stream.
MiYanni Sep 4, 2020
0e1122b
Made constants for wire format lengths/positions.
MiYanni Sep 4, 2020
a46462b
Updated property description. Made static field for UTF8 encoding.
MiYanni Sep 4, 2020
2d19b51
Removed unused Directory.Build.props.
MiYanni Sep 4, 2020
54f3d48
Removed unused description.
MiYanni Sep 4, 2020
88d46a2
Continuing readme. Started adding snippets.
MiYanni Sep 4, 2020
608ada2
Merge branch 'master' into schemaregistry-new-sdk
MiYanni Sep 4, 2020
7008405
First readme completed.
MiYanni Sep 4, 2020
6317936
Finally! Created readmes and snippets.
MiYanni Sep 4, 2020
713b987
Put credential scope string into a constant.
MiYanni Sep 4, 2020
09adad4
Addressing a couple documentation PR comments and removed a code comm…
MiYanni Sep 4, 2020
f331a51
Added global suppressions for namespace name.
MiYanni Sep 5, 2020
747e336
Forgot using for memory streams in tests and snippets.
MiYanni Sep 5, 2020
3f1cd01
Fixed bugged diagnostic scope names. Put scope names into constants.
MiYanni Sep 5, 2020
c87bb3b
Updated ci configuration. Removed resources template since resources …
MiYanni Sep 5, 2020
e152c57
Added the other package to CI.
MiYanni Sep 5, 2020
ff50a0c
Updated sample file names to match convention.
MiYanni Sep 5, 2020
87348be
Added missing readme links.
MiYanni Sep 5, 2020
aa78861
Fixed the readme snippets so they don't format oddly.
MiYanni Sep 5, 2020
7d9a67f
Updated schema components descriptions.
MiYanni Sep 5, 2020
03d9a62
Fixed command to acquire packages.
MiYanni Sep 5, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.29709.97
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Data.SchemaRegistry", "src\Azure.Data.SchemaRegistry.csproj", "{E33D09D9-D809-472C-82E6-6A26BDB86FC2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Data.SchemaRegistry.Tests", "tests\Azure.Data.SchemaRegistry.Tests.csproj", "{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Core.TestFramework", "..\..\core\Azure.Core.TestFramework\src\Azure.Core.TestFramework.csproj", "{8052009B-2126-44A3-88CD-4F3B17894C64}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E33D09D9-D809-472C-82E6-6A26BDB86FC2}.Release|Any CPU.Build.0 = Release|Any CPU
{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Debug|Any CPU.Build.0 = Debug|Any CPU
{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Release|Any CPU.ActiveCfg = Release|Any CPU
{4F476D56-DDE7-43D3-8CB4-BA1E77F5A300}.Release|Any CPU.Build.0 = Release|Any CPU
{8052009B-2126-44A3-88CD-4F3B17894C64}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8052009B-2126-44A3-88CD-4F3B17894C64}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8052009B-2126-44A3-88CD-4F3B17894C64}.Release|Any CPU.ActiveCfg = Release|Any CPU
{8052009B-2126-44A3-88CD-4F3B17894C64}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {A97F4B90-2591-4689-B1F8-5F21FE6D6CAE}
EndGlobalSection
EndGlobal
7 changes: 7 additions & 0 deletions sdk/schemaregistry/Azure.Data.SchemaRegistry/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Release History

## 1.0.0-beta.1 (2020-09-08)
- Added SchemaRegistryClient with 3 operations:
- RegisterSchema
- GetSchemaId
- GetSchema
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!--
MiYanni marked this conversation as resolved.
Show resolved Hide resolved
Add any shared properties you want for the projects under this package directory that need to be set before the auto imported Directory.Build.props
-->
<Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory).., Directory.Build.props))\Directory.Build.props" />
</Project>
105 changes: 105 additions & 0 deletions sdk/schemaregistry/Azure.Data.SchemaRegistry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Azure Schema Registry client library for .NET

The Azure Schema Registry service allows developers to provide and retrieve data schemas from a centralized repository for use in messaging systems.

## Getting started

### Install the package

Install the Azure Schema Registry client library for .NET with [NuGet][nuget]:

```PowerShell
Install-Package Azure.Data.SchemaRegistry
MiYanni marked this conversation as resolved.
Show resolved Hide resolved
```

### Prerequisites

* An [Azure subscription][azure_sub].
* An [Event Hubs namespace][event_hubs_namespace]

If you need to [create an Event Hubs namespace][create_event_hubs_namespace], you can use the Azure Portal or [Azure PowerShell][azure_powershell].

You can use Azure PowerShell to create the Event Hubs namespace with the following command:

```PowerShell
New-AzEventHubNamespace -ResourceGroupName myResourceGroup -NamespaceName namespace_name -Location eastus
```

### Authenticate the client

If your library requires authentication for use, such as for Azure services, include instructions and example code needed for initializing and authenticating.

For example, include details on obtaining an account key and endpoint URI, setting environment variables for each, and initializing the client object.

## Key concepts

The *Key concepts* section should describe the functionality of the main classes. Point out the most important and useful classes in the package (with links to their reference pages) and explain how those classes work together. Feel free to use bulleted lists, tables, code blocks, or even diagrams for clarity.

## Examples


* [Register a schema][azconfig_setting_concepts]
* [Retrieve a schema ID][azconfig_asof_snapshot]
* [Retrieve a schema][azconfig_asof_snapshot]




Include code snippets and short descriptions for each task you listed in the [Introduction](#introduction) (the bulleted list). Briefly explain each operation, but include enough clarity to explain complex or otherwise tricky operations.

If possible, use the same example snippets that your in-code documentation uses. For example, use the snippets in your `examples.py` that Sphinx ingests via its [literalinclude](https://www.sphinx-doc.org/en/1.5/markup/code.html?highlight=code%20examples#includes) directive. The `examples.py` file containing the snippets should reside alongside your package's code, and should be tested in an automated fashion.

Each example in the *Examples* section starts with an H3 that describes the example. At the top of this section, just under the *Examples* H2, add a bulleted list linking to each example H3. Each example should deep-link to the types and/or members used in the example.

* [Create the thing](#create-the-thing)
* [Get the thing](#get-the-thing)
* [List the things](#list-the-things)

### Create the thing

Use the [create_thing](not-valid-link) method to create a Thing reference; this method does not make a network call. To persist the Thing in the service, call [Thing.save](not-valid-link).

```Python
thing = client.create_thing(id, name)
thing.save()
```

### Get the thing

The [get_thing](not-valid-link) method retrieves a Thing from the service. The `id` parameter is the unique ID of the Thing, not its "name" property.

```C# Snippet:GetSecret
var client = new MiniSecretClient(new Uri(endpoint), new DefaultAzureCredential());

SecretBundle secret = client.GetSecret("TestSecret");

Console.WriteLine(secret.Value);
```Python
things = client.list_things()
```

MiYanni marked this conversation as resolved.
Show resolved Hide resolved
## Troubleshooting

Describe common errors and exceptions, how to "unpack" them if necessary, and include guidance for graceful handling and recovery.

Provide information to help developers avoid throttling or other service-enforced errors they might encounter. For example, provide guidance and examples for using retry or connection policies in the API.

If the package or a related package supports it, include tips for logging or enabling instrumentation to help them debug their code.

## Next steps

* Provide a link to additional code examples, ideally to those sitting alongside the README in the package's `/samples` directory.
* If appropriate, point users to other packages that might be useful.
* If you think there's a good chance that developers might stumble across your package in error (because they're searching for specific functionality and mistakenly think the package provides that functionality), point them to the packages they might be looking for.

## Contributing

This is a template, but your SDK readme should include details on how to contribute code to the repo/package.

![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Ftemplate%2FAzure.Template%2FREADME.png)

<!-- LINKS -->
[nuget]: https://www.nuget.org/
[event_hubs_namespace]: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about
[azure_powershell]: https://docs.microsoft.com/en-us/powershell/azure/
[create_event_hubs_namespace]: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-powershell#create-an-event-hubs-namespace
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
namespace Azure.Data.SchemaRegistry
{
public partial class SchemaProperties
{
internal SchemaProperties() { }
public string Content { get { throw null; } }
public string Id { get { throw null; } }
}
public partial class SchemaRegistryClient
{
protected SchemaRegistryClient() { }
public SchemaRegistryClient(string endpoint, Azure.Core.TokenCredential credential) { }
public SchemaRegistryClient(string endpoint, Azure.Core.TokenCredential credential, Azure.Data.SchemaRegistry.SchemaRegistryClientOptions options) { }
public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> GetSchema(string schemaId, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> GetSchemaAsync(string schemaId, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> GetSchemaId(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> GetSchemaIdAsync(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public virtual Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties> RegisterSchema(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public virtual System.Threading.Tasks.Task<Azure.Response<Azure.Data.SchemaRegistry.SchemaProperties>> RegisterSchemaAsync(string groupName, string schemaName, Azure.Data.SchemaRegistry.SerializationType serializationType, string schemaContent, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
}
public partial class SchemaRegistryClientOptions : Azure.Core.ClientOptions
{
public SchemaRegistryClientOptions(Azure.Data.SchemaRegistry.SchemaRegistryClientOptions.ServiceVersion version = Azure.Data.SchemaRegistry.SchemaRegistryClientOptions.ServiceVersion.V2017_04) { }
public enum ServiceVersion
{
V2017_04 = 1,
}
}
[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public readonly partial struct SerializationType : System.IEquatable<Azure.Data.SchemaRegistry.SerializationType>
{
private readonly object _dummy;
private readonly int _dummyPrimitive;
public SerializationType(string value) { throw null; }
public static Azure.Data.SchemaRegistry.SerializationType Avro { get { throw null; } }
public bool Equals(Azure.Data.SchemaRegistry.SerializationType other) { throw null; }
[System.ComponentModel.EditorBrowsableAttribute(System.ComponentModel.EditorBrowsableState.Never)]
public override bool Equals(object obj) { throw null; }
[System.ComponentModel.EditorBrowsableAttribute(System.ComponentModel.EditorBrowsableState.Never)]
public override int GetHashCode() { throw null; }
public static bool operator ==(Azure.Data.SchemaRegistry.SerializationType left, Azure.Data.SchemaRegistry.SerializationType right) { throw null; }
public static implicit operator Azure.Data.SchemaRegistry.SerializationType (string value) { throw null; }
public static bool operator !=(Azure.Data.SchemaRegistry.SerializationType left, Azure.Data.SchemaRegistry.SerializationType right) { throw null; }
public override string ToString() { throw null; }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<Description>Azure Schema Registry SDK</Description>
<AssemblyTitle>Azure Schema Registry SDK</AssemblyTitle>
<Version>1.0.0-beta.1</Version>
<PackageTags>Azure;Schema Registry;SchemaRegistry;.NET;Data;$(PackageCommonTags)</PackageTags>
<TargetFrameworks>$(RequiredTargetFrameworks)</TargetFrameworks>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Apache.Avro" VersionOverride="1.10.0" />
</ItemGroup>

<!-- Shared source from Azure.Core -->
<ItemGroup>
<Compile Include="$(AzureCoreSharedSources)ArrayBufferWriter.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)ClientDiagnostics.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)ContentTypeUtilities.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)DiagnosticScope.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)DiagnosticScopeFactory.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)HttpMessageSanitizer.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)TaskExtensions.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)OperationHelpers.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
<Compile Include="$(AzureCoreSharedSources)Argument.cs" Link="Shared\%(RecursiveDir)\%(Filename)%(Extension)" />
</ItemGroup>

<Import Project="$(MSBuildThisFileDirectory)..\..\..\core\Azure.Core\src\Azure.Core.props" />

</Project>

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading