Cosmos track 2 migration #4351
Merged
Cosmos track 2 migration #4351
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* inital commit for V3 port from vsts to github * Hash v2 for SDK v3 bumped version to 3.0.0-beta-3 for release * Modified Pom file to copy jar files to required location * updated sdk-version property * removed additions to pom * Removing unused code Changing version to 3.0.0-SNAPSHOT
Fixed CODEOWNERs syntax
Added support for partition split testing and CI experience improvement Azure#143
* inital commit for V3 port from vsts to github * Hash v2 for SDK v3 bumped version to 3.0.0-beta-3 for release * Modified Pom file to copy jar files to required location * updated sdk-version property * removed additions to pom * Removing unused code Changing version to 3.0.0-SNAPSHOT * Initial check in with reactor netty library * Initial commit for Non partitioned collection support for V3 async Java SDK * updated git ignore * updated git ignore * removed conflict markers in comments * removed unnecessary files * Removed public constructor that took PartitionKeyInternal * fixed javadoc errors * Work In Progress * Work in progress * Work In Progress, Added Reactor Netty Client and other interfaces for Reactor HTTP Client * Reverted test configuration: * Fixed Gateway Store Model Test * Removed unused classes, added headers to new classes. Added configuration to Reactor Netty Http Client * Implemented Reactor HTTP Client for HttpTransportClient * Fixed HttpTransportClient store response subscribe * Removed unused code, removed some rxnetty usages * Fixed Proxy Tests with correct log level for reactor netty client * Fixed Gateaway Address Cache Tests * Fixed test cases, optimized imports, and other refactoring changes * Removed propogating error, instead just throw the error * Storing Connection provider to close resources later * Fixed empty body issue with Flux not allowing null values * Disposing just connection provider on http client shutdown * increased used memory limit for document leak resource test. Fix it later * Incorporated port in Http Request * Replacing read and write time out handlers with idle state handlers * Removing experimental feature of pooled connection timeout * initial commit for fixing tetss (will break) * Removed timeout feature * Calling http client shutdown in HttpTransportClient * Fixed Document resource leak test by closing the channel when done * Experimenting changes * Commenting out old ssl provider * Setting ssl timeout explicitly * Trying out default ssl * Trying out default ssl * Trying out ssl config * Trying out default ssl * Using vanila ssl * fixed testsuitebase * Improving the configuration for http client * Merging v3_reactor_netty_client_experiment changes * Reverting Test Configurations change * Implemented code review suggestions as well as some optimizations and refactoring * Handling delete operation * Fixing Delete operation * Refactoring CosmosClient adding a Builder Removed cosmos configuration Adding getters for scripts Adding CosmosUser Changing Options to composition * WIP: Work in progress * - Added CosmosUser, get APIs for scripts - Removed CosmosConfiguration and moved builder to CosmosClient * Added retry utility for Backoff retry utility * - Ported Cosmos item changes * fixed more tests * Minor refactoring Adding listUsers and queryUsers * Compilation work. Work in progress * Refactoring in Reactor Netty Client: * fixed database tests * Fixed RxGatewayStoreModelTests * Fixed Store reader, consistency reader, writer, address selector test cases * fixed more tests * Replaced Func1 with java Function * - Fixed more tests - rx.internal tests run using asyncDocumentClient * Updated store and consistency reader writers, Collection caches, partition caches, helpers, address resolvers, global endpoint managers, retry policies with reactor-core changes * Updated query contexts, document service requests, cosmos client changes * Fixed bechmarks, and some tests in commons * fixed more tests * fixed more tests. commented out some tests for compilation * Implementing PR comments * Fixed Direct Implementation test cases * Refactoring internal tests to Reactor * Work in progress, fixing location cache test * Fixing Async benchmark after Mo's changes * Fixed Examples source code and test cases * Fixing javadoc warnings * Refactoring internal query tests to Reactor * Refactoring gateway tests to Reactor * Removed unused code, refactored to use internal test suite base * Fixed concurrency issue with internal test suite base, imported internal test suite base in rx.internal tests * Minor changes, removed invalid usage of TransportClient * Fixed gateway tests, removed verbose test subscriber, using basic test subscriber for now * Handling IO Exception for create attachment and media. Fixing Consistency and Retry Create Document tests * Fixing typo in doc * Work in progress * reverted changes to examples utils * Added paginator prototype code * Merging V3 * Code-complete draft port to reactor. The Direct TCP-related code compiles and is not yet testable because other significant portions of the code do not currently compile. * Code-complete draft port to reactor. The Direct TCP-related code compiles and is not yet testable because other significant portions of the code do not currently compile. * Renamed Reactor Transport Client to Transport Client, removed rx.Observable and rx.Single usages completely * fixed more tests made shared resources private * Addressed some error handling ommissions in RntbdRequestManager and verified locally on the V2 branch that the long test suite passes. * Fixed more tests to use getSharedDatabase and similar methods * Addressed some error handling ommissions in RntbdRequestManager and verified locally on the V2 branch that the long test suite passes. * Fixed buffer size issue * Added Cosmos Request options * Renamed COSMOS.PROTOCOL as cosmos.directModeProtocol and move ConfigsTests to the package that Configs is defined * Fixing tests and refactored the way to get values * Icreased Ssl handshake timeout for reactor netty client * Ensured that our dependency on org.hamcreate:hamcrest-all:1.3 is respected by mockito which brings in hacrest-all:1.1. Also: sorted dependency version properties and dependency lists. * Addressed RntbdTransportClient test faiures * Minor test fixes * We no longer skip the DocumentClientResourceLeakTest when Direct TCP is in use * We no longer permit VeryLargeDocumentQueryTest to fail when Direct TCP is in use. Also improved logger.info messages in DocumentClientResourceLeakTest * We no longer permit DCDocumentCrudTest.crossPartitionQuery to fail when Direct TCP is in use. * We no longer permit BackPressureCrossPartition.query to fail when Direct TCP is in use. * skipped tcp tests * removed unused imports * Resolves a long-standing TODO: Ensure that the Configs instance created by DirectHttpsClientUnderTest specifies Protocol.Https (now that Direct TCP is the default connection mode) * changes to ConsistencyLevel * TestSuiteBase.beforeMethod correctly logs the connection mode: Direct Tcp, Direct Https, or Gateway Https * Merged enable TCP default protocol changes * Fixed Address Resolver Tests * Disabling connection pool exhausted tests * Fixing Config Tests * removed changes to DocumentCrudTests edited manner in which ConsistencyLevel.toString is implemented * removed unused import * reverted changes to documentcrud tests * fixed failing test * made more ConsistencyLevel.name -> toString changes * changed line endings * Removed rx Composite Exception and used reactor Exceptions utility class to handle multiple exceptions * Made more changes to enums removed cosmos tests * more enum fixes * more enum changes * Test tweaks * Made more enum changes * fixed line endings in includepathtest * Fixed pagination and order by utils bugs * Swapped rx Http Client with Reactor Http Client in Cosmos Partition Key Tests * Removed rx java dependency * Fixing tests * Removed throws Exception and unused imports * Optimized imports in Microsoft style * Added parallel run code for bulk insert blocking method in Test Suite Base * Minor test fixes * Trying default Http Client * re-using https client * Disabling document resource leak test * Fixing up schedulers * fixed CosmosPartitionKeyTests and ConsistencyTests null pointer issues * Added logging to netty client and fixed logger category * Fixing spy wire test * fixed failing tests * Removing unused imports * Minor test changes * Removed usages of bodyAsString method from http response as its not working correctly, implemented toString to convert byteBuf flux to Mono string * Increased timeout * Replaced flux with mono across consistency layer to avoid confusion * Reverting test configurations changes * Fixed Parallel Document Query Context * Fixed Protocol tests * Replaced exceptions by Mono / Flux Error * Incresaed wait to capture requests * Added block instead of subscribe to capture response * Fixed ReadMyWrites test with emitting just one value * Modified the order of client builders and reverted flux / mono change for consistency stack * Fixed Read my writes workflow * Removed unnecessary stack trace * Fixed Benchmarks to use BaseSubscriber * Adding conflict api (Azure#192) Fixing Permission tests Making id method of Resource package private and adding id on individual resource classes to enable fluent composition * Removed unused ConfigsTests.java class * Fixed Read latency and write latency benchmarks * Fixed metrics version in parent pom.xml * Revised DocumentCrudTest and TestSuiteBase to address test breaks * Using Step Verifier, removed RetryAnalyzer * POM + test tweaks. * Eliminating some tests to test fast integration tests * Reverting the experiment * Fixed test break (see changes to CosmosPartitionKeyTests), tidied some test code (see changes to StoredProcedureCrudTest) and deleted two extraneous files * Removed unnecessary call * Code review comments
… object node return types (Azure#204)
* Add test case for change feed processor. The test creates a bunch of documents and compare them with what change feed processor receives. Fix couple bugs in the implementation of change feed processor including an issue related to continuation tokens which were previously not correctly retrieved. * Cleanup public interfaces (remove "with") and other tweaks. * Move ChangeFeedProcessor implementation inside "internal". Only two public interfaces/classes are exposed to the users, ChangeFeedProcessor and ChangeFeedProcessorOptions. * Update CFP Builder interface to accept a Consumer for handling the feed changes. * Remove CheckpointFrequency setting from the user facing public APIs. * Remove blocking calls from ChangeFeedProcessorBuilderImpl implementation * Fix documentation capture. * Cleanup/fixes for Mono.error() calls. Renamings/refactoring. * Test cleanup; add extra coverage for ChangeFeedProcessorOptions argument. * Test fix * Change feed test refactoring. Disable resourceLeak() test that it causes runs to timeout * increase the "AfterMethod" timeout. * Refactor test assets. Add test case for ChangeFeedProcessorOptions "startTime" option. * Increase timeouts; move map initialization inside the test. * Rename syncHandleChanges to handleChanges. Return an actual instance of the ChangeFeedProcessor rather than a Reactor Mono. Some initialization code will need to be moved to from builder to start of the processing in a later check-in. * cleanup * Updates * Fix for release of a lease. * switch to publishing on a different scheduler to avoid blocking on netty threads.
* Hiding Json Serializable set and remove methods and object mapper and object node return types * Renamed Settings suffixes to Properties suffixes * Refactored settings method to properties * Removed sdk - CosmosUserSettings, renamed commons - CosmosUserSettings to CosmosUserProperties
* Remove CosmosResource * Naming and JavaDocs updates * Move Scripts to CosmosScripts object * Add header * Fix JavaDoc build error * Fix up CosmosScripts + tests
* Collection to list changes, moved old types to internal, pom file change to exclude BridgeInternal and CosmosBridgeInternal * Fix build issues with scripts * fix partition key bug * JsonSerializable handle private constructors * Add test coverage IndexingPolicy deserialization * Remove Collection in favor of List * Code Review changes, excluded all BridgeInternal from API docs * Fixed constructor scope
* FeedOptions refactoring * Remove FeedOptionsBase
…e shutdown since this will stop any future execution using it. (Azure#212)
* Moved exceptions to commons module, made most of the constructors package private * Moved PartitionKey back to public scope
* merge modules * fix tests * move direct mode internal * Optimized imports, fixed dependencies for benchmarks and sdk
* Wrapped Client Side Request Statistics in Cosmos Response Diagnostic Statistics. Exposed only toString and request latency * Code review comments for Cosmos Response Diagnostics * Wrapped Query Metrics Map in Feed Response Diagnostics. Exposed toString through Feed response diagnostics * Reducing scope of setter to package private
* Hide AsyncDocumentClient * Remove Attachment * Cleanup ChangeFeedOptions * Hide Conflict * Cleanup ConsistencyPolicy * Cleanup CosmosClient * Cleanup CosmosClientBuilder * Cleanup CosmosConflict * Cleanup CosmosConflictProperties * CosmosConflictResponse cleanup * Cleanup CosmosContainer * Cleanup CosmosContainerRequestOptions * Cleanup CosmosContainerResponse * Cleanup CosmosContainerProperties * Cleanup CosmosDatabase * Cleanup CosmosDatabaseRequestOptions * Remove CosmosFeedResponse * Cleanup CosmosItemRequestOptions * Cleanup CosmosPermissionsRequestOptions * Remove CosmosRequestOptions * Cleanup CosmosStoredProcedure * Cleanup CosmosStoredProcedureRequestOptions * Cleanup CosmosTrigger * Cleanup CosmosUser * Cleanup CosmosUDF * Cleanup CosmosUDFResposne * Clean up CosmosUserResponse * Cleanup CosmosUserProperties * Hide old v2 types * Hide Json constructors * Remove MediaReadMode * Hide Undefined * list to readAll * Fix json serialization
g2vinay
requested review from
JonathanGiles,
weshaggard,
kushagraThapar and
mitchdenny
JonathanGiles
approved these changes
Jul 10, 2019
weshaggard
reviewed
Jul 10, 2019
| @@ -0,0 +1,188 @@ | |||
| <!-- | |||
There was a problem hiding this comment.
The folder here should be sdk/cosmos/microsoft-azure-cosmos-parent assuming this is track 1 based on the group id being com.microsoft.azure. @JonathanGiles is that correct?
There was a problem hiding this comment.
will do the folder renames in the follow up PR, which will wire up the builds to central parent pom, add Emulator CI + live tests CI.
There was a problem hiding this comment.
Just wondering, why is benchmark folder to be renamed to microsoft-azure-cosmos-parent
There was a problem hiding this comment.
Generally speaking we want the folder names containing the sources for a package to match the artifact name.
weshaggard
reviewed
Jul 10, 2019
| @@ -0,0 +1,159 @@ | |||
| <!-- | |||
There was a problem hiding this comment.
For similar reasons as already stated this folder should be sdk/cosmos/microsoft-azure-cosmos-examples
There was a problem hiding this comment.
will cover it in follow up PR too.
weshaggard
reviewed
Jul 10, 2019
| @@ -0,0 +1,280 @@ | |||
| <!-- | |||
There was a problem hiding this comment.
Folder should be sdk/cosmos/microsoft-azure-cosmos
There was a problem hiding this comment.
will cover it in follow up PR too.
|
Looks like you need to update docsettings to fix the readme errors and I left a few comments about directory names otherwise this looks reasonable for an initial port. We will of course want to more integrate it into the repo as we start to do work on it and clean-up some things. It would be good to file an issue (or set of issues) to track that work. |
kushagraThapar
approved these changes
Jul 10, 2019
g2vinay
force-pushed
the
cosmos-track2-migration
branch
from
d1c8db1 to
58adead
Compare
g2vinay
force-pushed
the
cosmos-track2-migration
branch
from
58adead to
6ef1b37
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Moves Cosmos Track 2 code (V3 branch) as it is to 'sdk/cosmos/' location in central repo.
Follow up PR to: