Release .NET Clients version 1.0.0 to NuGet

[joshuali925]

Is your feature request related to a problem? Please describe

Since NuGet account is created, we would like to release the .NET clients from opensearch-net repo. Here is the list of packages and descriptions:

• OpenSearch.Client -- Strongly typed interface to OpenSearch. Fluent and classic object initializer mappings of requests and responses. Uses and exposes OpenSearch.Net.
• OpenSearch.Net -- Exposes all the OpenSearch API endpoints but leaves you in control of building the request and response bodies. Comes with built in cluster failover/connection pooling support.
• OpenSearch.Net.VirtualizedCluster -- Provides a way to assert client behavior through a rule engine backed VirtualClusterConnection.
• OpenSearch.Client.JsonNetSerializer -- OpenSearch.Client uses an internal serializer. This package enables serialization for your documents using your own custom Json.NET serializer

The artifacts are generated by this workflow: https://github.com/opensearch-project/opensearch-net/actions/workflows/release.yml

Describe the solution you'd like

Infra's help to release the packages to NuGet, please let me know if something is required

Describe alternatives you've considered

No response

Additional context

Linking release issue opensearch-project/opensearch-net#14

[bbarani]

@zelinh @prudhvigodithi Please proceed with publishing the client

[prudhvigodithi]

[Triage] @zelinh already have some progress with nuget account setup, need a way to sign and publish the package
https://docs.microsoft.com/en-us/nuget/create-packages/sign-a-package
https://docs.microsoft.com/en-us/nuget/nuget-org/publish-a-package

[anirudha]

@prudhvigodithi , @zelinh what are the next steps on this ? thanks

[prudhvigodithi]

Hey @anirudha and @joshuali925 let me know if you are ok with reserved prefixed as OSC and OpenSearch, so that glob patterns OSC.* and OpenSearch.* should also work.

[joshuali925]

@prudhvigodithi That should work. Since there are two prefixes, is it still possible to see them together in one place on NuGet? maybe by going to the owner/publisher profile, or search for OpenSearch, would both show up?

[prudhvigodithi]

Hey @joshuali925 yes
Expecting something like
https://www.nuget.org/packages/OSC
https://www.nuget.org/packages/OpenSearch
https://www.nuget.org/packages/OpenSearch.Net

[anandbprem]

Hi Team,
Please keep us updated with the NuGet package release plan. We are eagerly waiting for it.

Please let us know if you need any help.

[prudhvigodithi]

Hey thanks @anandbprem, working closely on this issue with Microsoft(nuget.org) and awsdotnet teams to get a reserved prefixes for OpenSearch and OSC, this includes getting three Glob patterns OpenSearch.* OSC.* OSC and creating a release flow to sign the generated dll's and update the nukpg packages securely. Multiple phases :)

[anandbprem]

Thanks for your response, also we need to plan to upgrade from .net 5.0 to 6.0 :) :)

[prudhvigodithi]

Hey @joshuali925, following are the dll files generated when i run dotnet build OpenSearch.sln --configuration Release, Please let me know if i'm missing anything, we should be signing all generated dll's and run dotnet pack OpenSearch.sln --configuration Release --no-build to pack them into .nukpg file.

  OpenSearch.Net -> src/OpenSearch.Net/bin/Release/net461/OpenSearch.Net.dll
  OpenSearch.Net -> src/OpenSearch.Net/bin/Release/netstandard2.1/OpenSearch.Net.dll
  OpenSearch.Net -> src/OpenSearch.Net/bin/Release/netstandard2.0/OpenSearch.Net.dll
  OpenSearch.Net.dll -> src/Osc/bin/Release/netstandard2.0/OpenSearch.Net.dll
  OpenSearch.Net.dll -> src/Osc/bin/Release/net461/OpenSearch.Net.dll
  OpenSearch.Net.dll -> src/OpenSearch.Net.VirtualizedCluster/bin/Release/net461/OpenSearch.Net.dll
  OpenSearch.Net.dll -> src/OpenSearch.Net.VirtualizedCluster/bin/Release/netstandard2.0/OpenSearch.Net.dll 
  OpenSearch.Net.dll -> src/Osc.JsonNetSerializer/bin/Release/net461/OpenSearch.Net.dll 
  OpenSearch.Net.dll -> src/Osc.JsonNetSerializer/bin/Release/netstandard2.0/OpenSearch.Net.dll


  OpenSearch.Net.VirtualizedCluster -> src/OpenSearch.Net.VirtualizedCluster/bin/Release/netstandard2.0/OpenSearch.Net.VirtualizedCluster.dll
  OpenSearch.Net.VirtualizedCluster -> src/OpenSearch.Net.VirtualizedCluster/bin/Release/net461/OpenSearch.Net.VirtualizedCluster.dll


  Osc -> src/Osc/bin/Release/netstandard2.0/Osc.dll
  Osc -> src/Osc/bin/Release/net461/Osc.dll
  Osc -> src/Osc.JsonNetSerializer/bin/Release/net461/Osc.dll
  Osc -> src/Osc.JsonNetSerializer/bin/Release/netstandard2.0/Osc.dll
   
  Osc.JsonNetSerializer -> src/Osc.JsonNetSerializer/bin/Release/netstandard2.0/Osc.JsonNetSerializer.dll
  Osc.JsonNetSerializer -> src/Osc.JsonNetSerializer/bin/Release/net461/Osc.JsonNetSerializer.dll

or just the following that actually get packed into the nuget packages ?

src/Osc/bin/Release/net461/Osc.dll
src/Osc/bin/Release/netstandard2.0/Osc.dll
src/OpenSearch.Net/bin/Release/net461/OpenSearch.Net.dll
src/OpenSearch.Net/bin/Release/netstandard2.0/OpenSearch.Net.dll
src/OpenSearch.Net/bin/Release/netstandard2.1/OpenSearch.Net.dll
src/OpenSearch.Net.VirtualizedCluster/bin/Release/net461/OpenSearch.Net.VirtualizedCluster.dll
src/OpenSearch.Net.VirtualizedCluster/bin/Release/netstandard2.0/OpenSearch.Net.VirtualizedCluster.dll
src/Osc.JsonNetSerializer/bin/Release/net461/Osc.JsonNetSerializer.dll
src/Osc.JsonNetSerializer/bin/Release/netstandard2.0/Osc.JsonNetSerializer.dll

[joshuali925]

From the existing .nupkg files i see just these ones:

OpenSearch.Net/lib/net461/OpenSearch.Net.dll
OpenSearch.Net/lib/netstandard2.0/OpenSearch.Net.dll
OpenSearch.Net/lib/netstandard2.1/OpenSearch.Net.dll
OpenSearch.Net.VirtualizedCluster/lib/net461/OpenSearch.Net.VirtualizedCluster.dll
OpenSearch.Net.VirtualizedCluster/lib/netstandard2.0/OpenSearch.Net.VirtualizedCluster.dll
Osc/lib/net461/Osc.dll
Osc/lib/netstandard2.0/Osc.dll
Osc.JsonNetSerializer/lib/net461/Osc.JsonNetSerializer.dll
Osc.JsonNetSerializer/lib/netstandard2.0/Osc.JsonNetSerializer.dll

but I'm not sure, @MaxKsyunz could you check which ones are needed? thanks

[MaxKsyunz]

@prudhvigodithi only the DLLs that get packed need to be signed -- your second list.

[prudhvigodithi]

thanks @MaxKsyunz and @joshuali925
I do see another issue OSC.JsonNetSerializer.1.0.0.nupkg packages
Response status code does not indicate success: 409 (The package ID is reserved. You can upload your package with a different package ID. Reach out to [email protected] if you have questions.).
I'm able to test sign and upload OpenSearch.Net, OpenSearch.Net.VirtualizedCluster, OSC
https://www.nuget.org/packages/OpenSearch.Net/
https://www.nuget.org/packages/OpenSearch.Net.VirtualizedCluster/
https://www.nuget.org/packages/OSC/
But not OSC.JsonNetSerializer can you please check if anything wrong with OSC.JsonNetSerializer?
Thank you

[prudhvigodithi]

I think I got the error
OSC.JsonNetSerializer.1.0.0.nupkg
The package ID is reserved. You can upload your package with a different package ID
can we use a different package ID?
@joshuali925 @MaxKsyunz

[joshuali925]

Update: we renamed OSC and OSC.JsonNetSerializer to OpenSearch.Client and OpenSearch.Client.JsonNetSerializer respectively to avoid conflict

[prudhvigodithi]

Hey @joshuali925 and @MaxKsyunz published the clients to nuget.org, please test/validate the clients, if required we can still unlist them from opensearchproject profile and republish them back again.
https://www.nuget.org/profiles/opensearchproject
Thank you.

[anandbprem]

@prudhvigodithi , the URL shared does not have any packages listed.

[INeerav]

any dates are we targeting @joshuali925

[joshuali925]

the packages were taken down as there were some issues found during testing and validation. @prudhvigodithi could you provide an update on current status please? thanks

[INeerav]

@prudhvigodithi all we wanted to know is if the publish will happen in next couple of days or it would take more than a month to help us plan our dev work

[prudhvigodithi]

Hey @INeerav @anandbprem the published packages were unlisted due to reported validation tests failure, @anirudha @joshuali925 are we good to republish the packages again? if so can you please share the 1.x version branch information.
Thank you

[anirudha]

pending release processes for a final publish. thanks @prudhvigodithi

[bbarani]

@anirudha What do you mean by pending release process? Are you still validating the packages? Can you provide a brief overview of the current status?

[msoler8785]

Looks like OpenSearch 2.0.x release has broken compatibility with elasticsearch-net library, so this is going to be needed soon for some people.

[INeerav]

@joshuali925 any updates?

[vmjrojas]

@joshuali925 Any updates?

[jamesdoran]

@prudhvigodithi, @anirudha, @joshuali925 is there any update on when the nuget packages will be re-released please?

[prudhvigodithi]

Hey @anirudha @joshuali925 just wanted to follow up again, can you post an update when you get a chance?
Thank you

[joshuali925]

currently in progress of security review

[PaulDMendoza]

How long does it take to get this nuget package published? We really need this nuget package published.

[jamesdoran]

currently in progress of security review

@joshuali925, @prudhvigodithi, @anirudha - is there anything I can do to help this process along?

[anirudha]

All done! last friday :) finally! planning release activites to nuGet. we should have this posted in under week now.
@PaulDMendoza @jamesdoran

[anirudha]

I posted the current status above, but I also want to acknowledge, and apologize for failing to provide a regular status update earlier. Thank you for your patience.

[CEHENKLE]

Thanks @anirudha! Can you share what the remaining steps are before we can release?

[prudhvigodithi]

We just shipped OpenSearch 1.0.0 .NET clients to nuget repo https://www.nuget.org/profiles/opensearchproject.
Thanks @MaxKsyunz @joshuali925 @anirudha @Yury-Fridlyand @acarbonetto @bbarani

[jamesdoran]

Thanks everyone. Much appreciated 🥳

[bbarani]

Closing this issue as we have released .Net client 1.0.0 to NuGet.