Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Storage] Dec19 - Quickquery #9112

Merged
merged 5 commits into from
Jun 9, 2020
Merged

[Storage] Dec19 - Quickquery #9112

merged 5 commits into from
Jun 9, 2020

Conversation

XiaoningLiu
Copy link
Member

No description provided.

@XiaoningLiu XiaoningLiu requested a review from ljian3377 May 26, 2020 11:00
XiaoningLiu
XiaoningLiu commented May 26, 2020
View reviewed changes
sdk/storage/storage-blob/review/storage-blob.api.md
@@ -769,6 +769,44 @@ export interface BlobProperties {
tagCount?: number;
}

// @public
export interface BlobQueryCsvTextConfiguration extends BlobQueryTextConfiguration {
Copy link
Member Author

@XiaoningLiu XiaoningLiu May 26, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @bterlson Here is API reference change for incoming feature QuickQuery. If necessary, I can create another separate API review PR for your convenience.

XiaoningLiu
XiaoningLiu commented May 26, 2020
View reviewed changes
sdk/storage/storage-blob/review/storage-blob.api.md
kind: "json";
}

// Warning: (ae-forgotten-export) The symbol "BlobQueryHeaders" needs to be exported by the entry point index.d.ts
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BlobQueryResponseModel is a generated mode

@XiaoningLiu XiaoningLiu changed the title Quickquery [Storage] Dec19 - Quickquery May 26, 2020
@XiaoningLiu XiaoningLiu added Client This issue points to a problem in the data-plane of the library. Storage Storage Service (Queues, Blobs, Files) labels May 26, 2020
@jiacfan
Copy link
Member

jiacfan commented May 27, 2020

"version": "12.1.2",

minor: version across the change after it's finalized @lin

Refers to: sdk/storage/storage-blob/package.json:4 in c228dfc. [](commit_id = c228dfc, deletion_comment = False)

@XiaoningLiu
Copy link
Member Author

"version": "12.1.2",

minor: version across the change after it's finalized @lin

Refers to: sdk/storage/storage-blob/package.json:4 in c228dfc. [](commit_id = c228dfc, deletion_comment = False)

I think @ljian3377 will do version change.

bterlson
bterlson reviewed May 29, 2020
View reviewed changes
sdk/storage/storage-blob/review/storage-blob.api.md
@@ -769,6 +769,44 @@ export interface BlobProperties {
tagCount?: number;
}

// @public
export interface BlobQueryCsvTextConfiguration extends BlobQueryTextConfiguration {
columnSeparator: string;
Copy link
Member

@bterlson bterlson May 29, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be defaulted to ,?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's me confirm from service feature side, if optional is acceptable, let's me make it optional with default values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bterlson Make it as optional. Please refer to #9484

bterlson
bterlson reviewed May 29, 2020
View reviewed changes
sdk/storage/storage-blob/review/storage-blob.api.md
columnSeparator: string;
escapeCharacter?: string;
fieldQuote?: string;
hasHeaders: boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be defaulted to false? My feeling is that generally speaking this is an optional parameter in CSV parsers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's me confirm from service feature side, if optional is acceptable, let's me make it optional with default values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bterlson Make it as optional. Please refer to #9484

@XiaoningLiu XiaoningLiu added the APIChange This PR contains an addition or change to the API signature and must be reviewed by an architect. label Jun 1, 2020
XiaoningLiu
XiaoningLiu commented Jun 1, 2020
View reviewed changes
sdk/storage/storage-blob/test/node/blobclient.spec.ts Outdated
});
});

it.only("query should work with fatal error event", async () => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bterlson Just for example of how a fetal error is.

XiaoningLiu
XiaoningLiu commented Jun 1, 2020
View reviewed changes
sdk/storage/storage-blob/test/node/blobclient.spec.ts Outdated
assert.deepStrictEqual(await bodyToString(response), "\n");
});

it.only("query should work with non fatal error event", async () => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bterlson Just for example of how a non fetal error is.

@kasobol-msft kasobol-msft mentioned this pull request Jun 2, 2020
Closed
@XiaoningLiu
Copy link
Member Author

XiaoningLiu commented Jun 5, 2020 •
edited
Loading

Hi @bterlson @jeremymeng We may need to merge this PR into stg73 feature branch.

@ljian3377 needs to sync up feature branch with master branch, before that we'd like to merge pending PRs to feature branch to avoid conflicts. Finally, there will be a PR from feature branch to master for another review.

All comments in this PR will keep following.

@ljian3377 ljian3377 merged commit 680d952 into Azure:storage/stg73base Jun 9, 2020
@ljian3377
Copy link
Member

https://apiview.dev/Assemblies/Review/2cdf1e8e47874f8681106b24a9002bdf?diffRevisionId=2a2d3cd11460478cb21ea91b3f8bd010
.NET seems have some wrapping in DataLake for quick query, could you check if we need to do the similiar?

@XiaoningLiu
Copy link
Member Author

https://apiview.dev/Assemblies/Review/2cdf1e8e47874f8681106b24a9002bdf?diffRevisionId=2a2d3cd11460478cb21ea91b3f8bd010
.NET seems have some wrapping in DataLake for quick query, could you check if we need to do the similiar?

Thanks! Let's move it to next release for datalake changes, as the limitied schedule.

@XiaoningLiu XiaoningLiu mentioned this pull request Jun 11, 2020
Merged
bterlson
bterlson reviewed Jun 11, 2020
View reviewed changes
sdk/storage/storage-blob/src/utils/BlobQuickQueryStream.ts
if (this.onError) {
const fatal = (obj as any).fatal;
if (typeof fatal !== "boolean") {
throw Error("Invalid fatal in avor error record.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed "avor" here, can be fixed in a couple places.

bterlson
bterlson reviewed Jun 12, 2020
View reviewed changes
Copy link
Member

@bterlson bterlson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API changes look good with the new optional properties.

ljian3377 added a commit that referenced this pull request Jul 1, 2020
@ljian3377 @XiaoningLiu
@jeremymeng @bterlson
[storage] STG73 features preview (#9532)
410dbb8 
* storage-internal-avro parser (#8893)

* IReadable

* Avro compiles

* use storage-internal-avro as shared source in storage-blob

* local build config

* test and bug fix

* rename IReadable to AvroReadable, separate the stream implementation with the interface declaration we need different implementation in browsers

* add "private": true to pacakge.json

* use Record

* me

* fix

* tentative: use string enum instead of string union to avoid two copies of constant string

Co-authored-by: Lin Jian <[email protected]>

* [Storage][Blob] Regenerate with Dec19 swagger

* avor parser offset fix; Readable interface update (#9088)

* [Storage] Regenerate API review

* [storage] avro parser fix (#9154)

* fix

* resolve PR comments: remove useless assignment

* AvroReadableFromStream: removeListener when read succeeds

* remove listener when error callback triggered

Co-authored-by: Lin Jian <[email protected]>

* [Storage] Dec19 - Quickquery (#9112)

* [Storage] Quick query

* [Storage] Quick query API review

* [Storage] Quick query recording

* [Storage] Quick query test case update

* [Storage] Resolve comments

* regenerate storage-blob

* re-generate storage-file-share

* regenrate datalake and undo adding mode to storageClient and storageClientContext

* update datalake service version to 2019-12-12

* [Storage] QuickQuery - Make BlobQueryCsvTextConfiguration properties optional (#9484)

* [storage][blob] blob versioning (#7886)

* blob versioning

* support 'bv' SAS

* add response headers to getProperties & api-extract

* fix comments

* add versionId to getBlob response headers

* fix broken undelete test

* undo 'add snapshot to options'

* replace withVersionId() with getURLWithVersionId()

* Revert "replace withVersionId() with getURLWithVersionId()"

This reverts commit eb1d90d.

* hide the versionId option in convenience layer

* add more tests

* fix comments

* change version to preview

* fix merge error

Co-authored-by: Lin Jian <[email protected]>

* [storage] added change feed (#9151)

* fix

* change feed with mininal live tests

* mocking tests

* format code & rename buildAvroReader parameter name

* add README.md

* change eventTime type from string to Date

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* PR comments

* Rename eTag to etag in BlobChangeFeedEventData

* remove hns only properties in change event record

* fix minor bug in avro

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Brian Terlson <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Brian Terlson <[email protected]>

* rename to BlobChangeFeedGetChangesOptions and fix comments

* update api

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Brian Terlson <[email protected]>

* [storage][blob & datalake] Jumbo blob (#9480)

* Jumbo putBlock, putBlob, datalake file append

* undo upload ArrayBuffer

* minor edit

Co-authored-by: Lin Jian <[email protected]>

* [Storage] Support blob tags (recording will be added later) (#9440)

* [Storage] Support blob tags (recording will be added later)

* [Storage] Unify blob tags type Tags everywhere in options and responses

* [Storage] Blob tags - resolve comments

* update service version to 2019-12-12

* blob tags test record

* fix listContainer options

* wip

* record skipped test for Jumbo

* set default formatter

* me

* wip

* blob record node

* record

* add version check for x permission

* record for browser

* wip

* wip

* wip

* wip

* add f for account SAS permission

* add permission f, and x for container

* wip

* me

* me

* PR comments

* me

* record changefeed

* record with blobversioning disabled

* record skipped test for browser

* wip

* avro compatible with Node v8

* fix test record

* fix record

* fix ae-forgotten-export

* lint

* fall back to util for old node.js TextEncoder

* release pipeline for change feed

* [storage][file] large smb files (#9624)

* create large smb files

* wip

* record test

* me

Co-authored-by: Lin Jian <[email protected]>

* [storage] added set Expiry (#9560)

* add set expiry

* add check for input

* use Date instead of a time string

* call blob endpoint directly in datalake

* undo evenhub package.json

* add alternative version and use number instead of decimal string for
expiresOn relative

* add timeToExpireInMs to option

* use string literals instead of enum

* keep old enum for backward compatibility

* nit

Co-authored-by: Lin Jian <[email protected]>

* merge fix

* omit doc in storage-internal-avro

* Revert "[storage] added set Expiry (#9560)"

This reverts commit c3b4414.

* nit

* skip live tests for blob tags and quick query

* change path for ts-samples

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: XiaoningLiu <[email protected]>
Co-authored-by: Xiaoning Liu <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Brian Terlson <[email protected]>
ljian3377
ljian3377 reviewed Jul 10, 2020
View reviewed changes
sdk/storage/storage-blob/src/Clients.ts
* blobContext provided by protocol layer.
*
* Note. Ideally BlobClient should set BlobClient.blobContext to protected. However, API
* extractor has issue blocking that. Here we redecelare _blobContext in BlockBlobClient.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redecelare [](start = 48, length = 10)

redecelare

ljian3377
ljian3377 reviewed Jul 10, 2020
View reviewed changes
sdk/storage/storage-blob/src/BlobQueryResponse.ts
* @class BlobQueryResponse
* @implements {BlobDownloadResponseModel}
*/
export class BlobQueryResponse implements BlobDownloadResponseModel {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BlobDownloadResponseModel [](start = 42, length = 25)

Why don't we implements the underlying BlobQueryResponse?
Note that with the rehydrate priority feature, underlying BlobQueryResponse and BlobDownloadResponseModel now diverge.

tzhanl added a commit that referenced this pull request Jul 15, 2020
@tzhanl @jeremymeng
@kristapratico @sarangan12 @chradek @richardpark-msft @HarshaNalluru @ramya-rao-a @witemple-msft @schaabs @ymasaoka @mitchdenny @changlong-liu @sadasant @southpolesteve @azure-sdk @KarishmaGhiya @joheredi @zzhxiaofeng @AndreasHassing @mikeharder @qiaozha @zfoster @chunyu3 @chidozieononiwu @xirzec @Abyssknight @bterlson @mahdiva @joshfree @abouroubi @shawncx @msxichen @ljian3377 @XiaoningLiu @danieljurek @stfaul @ellismg
Rebase Master to Identity-ENVVARs branch (#10078)
fbaacbc 
* [FormRecognizer] whitelist header names and query parameters for logging (#9149)

* [FormRecognizer] Update CHANGELOG for preview.3 (#9117)

* [FormRecognizer] Update CHANGELOG for preview.3

* parameter names for preview.3 should be `formUrl` and `receiptUrl`

* More renaming

* data => form/receipt

* Update sdk/formrecognizer/ai-form-recognizer/CHANGELOG.md

Co-authored-by: Krista Pratico <[email protected]>

* logging whitelist

* useTrainingLabels parameter now required

* intervalInMs => updateIntervalInMs

* Remove USReceipt for preview.3

Co-authored-by: Krista Pratico <[email protected]>

* Modified changelog for the June Release (#9317)

* Modified changelog for the June Release

* Response to PR Comments

* Dummy Commit I

* Revert Dummy Commit

* [event-hubs] doc improvements (#9342)

* [event-hubs] explain empty events

* [event-hubs] readme update for 8798

* [event-hubs] migration-guide checkpointing 8419

* update changelog for 5.2.1

* Update sdk/eventhub/event-hubs/migrationguide.md

Co-authored-by: Richard Park <[email protected]>

* Decided to be more ambitious for the release date

Co-authored-by: Richard Park <[email protected]>

* [Service Bus] Token Credential Overload for ATOM Mgmt Client (#9316)

* Update Queue APIs aper the .NET proposal

* remove @internal @ignore tags wherever required and generate API report

* more exports

* more and more exports

* Add token credential constructor overlaod

* doesExist() for queue, topic and subscription

* generate API report

* api extractor and ts-config (TO BE REVERTED)

* feedback from the service team

* update index.ts

* update API report

* Update queue related imports in tests

* revert api-extractor and tsconfig changes to allow compiling tests

* fix build failures of ATOM API tests

* changes to rule options and interfaces

* fix test failures

* changes to Topic APIs

* fix build failures

* generate API report

* subscription APIs updated

* fix build failures

* generate API report

* fix API report warnings

* changes to rule APIs

* fix build failures

* generate API report

* getNamespaceProperties API

* namespace info improvement + API report

* fix warnings in the API report

* generatre API report

* resolve merge conflicts and organize imports

* fix the warnings from API report

* API report

* tests for getRuntimeInfo API for a single entity

* remove messageCount from *Description APIs

* remove unused helper

* test for Get namespace properties

* tests for entityExists

* tests for getEntitiesRuntimeInfo

* Add changelog

* Changelog

* tokenProvider attempt to fix

* added servicebus scope in the getToken attribute

* mark "@azure/identity" as external

* Token credential

* test for Token credential - DefaultAzureCredential from `@azure/identity`

* update the name

* Update sdk/servicebus/service-bus/src/serviceBusAtomManagementClient.ts

Co-authored-by: chradek <[email protected]>

* Update sdk/servicebus/service-bus/src/serviceBusAtomManagementClient.ts

Co-authored-by: chradek <[email protected]>

* enter one more code path in the tests for gettoken invocation

* refactoring the constructor as suggested by Chris and fixing tests

* fix trailing slash issue

* Update the test to cover more cases

* fix test for AAD

* remove unneeded !

* fix endpoint and endpointWithProtocol attributes

* Add getToken for SasServiceCredential

* merge tokenProvider into credentials

* remove       "@azure/ms-rest-nodeauth" from the external list in rollup config for tests

* Update sdk/servicebus/service-bus/rollup.base.config.js

Co-authored-by: Ramya Rao <[email protected]>

* private sharedKeyCredential

Co-authored-by: chradek <[email protected]>
Co-authored-by: Ramya Rao <[email protected]>

* [ai-text-analytics] Various documentation-related changes (#9333)

* [ai-text-analytics] Various documentation-related fixes

* Fixed use of showStats instead of includeStatistics

* Added alternative TA product slug

* [ai-text-analytics] Change versions in package.json and constants to 1.0.0 (#9346)

* Removed a couple of old recordings

* Changed all preview version indicators to 1.0.0

* Switch to using the CredentialUnavailable error (#8127)

* Switch to using the CredentialUnavailable error

* exporting CredentialUnavailable and fixing tests

* updating api spec

* fixed missed test update

Co-authored-by: Scott Schaab <[email protected]>

* [Cosmos] Update Containers createIfNotExists() docs (#9355)

The content of the comment in the method is the same as that described in `Databases.createIfNotExists()`, and it does not match the content of `Containers.createIfNotExists()`, so it was corrected.

* Remove pipeline config for eventgrid, graphrbac, operationalinsights and servicefabric. (#9334)

* Sdk automation/@azure arm mediaservices (#9192)

* Update from master

* version

Co-authored-by: SDK Automation <[email protected]>

* New aka.ms/azsdk links (#9034)

* Duplicated ThirdPartyNotices in eventhub

* no more https://azure.github.io/azure-sdk-for-js links

* other link fixes

* new aka.ms links

* docs/<service> to <service>/docs, as agreed upon over Teams

* Changed https://aka.ms/azure-sdk-preview1-js to https://aka.ms/azsdk/releases/July2019Preview

* removed the versions

* Update README.md

* lowercase july2019preview

* lowercase july2019preview

* [Cosmos] Use constants for auth headers returned by generateHeaders function (#9354)

* [event-hubs] fix flaky disconnect test (#9350)

* [Event Hubs] Update Identity tests to use public clients (#9343)

* [service-bus] T2 preview.3 readme and changelog update with date and version. (#9340)

* [event-hubs] migration guide and test updates (#9367)

* [event-hubs] improve migration guide checkpoint section

* [event-hubs] make receciver disconnect even more tolerant!

* Apply suggestions from code review

Co-authored-by: Harsha Nalluru <[email protected]>

Co-authored-by: Harsha Nalluru <[email protected]>

* [FormRecognizer] Update June release date in CHANGELOG (#9368)

* Increment package version after release of azure-search-documents (#9370)

* Increment package version after release of azure-service-bus (#9372)

* Minor change in the sample (#9373)

* Minor change in the sample

* Additional changes

* [Event Hubs] Use EventHubConsumerClient in misc.spec.ts test file (#9341)

* Increment package version after release of azure-event-hubs (#9377)

Co-authored-by: azure-sdk <[email protected]>

* [Service Bus] Sample for SBManagementClient (#9369)

* ts sample

* js sample

* update samples readmes

* readme updates for ATOM

* Update sdk/servicebus/service-bus/samples/javascript/README.md

Co-authored-by: Richard Park <[email protected]>

Co-authored-by: Richard Park <[email protected]>

* [core-http] Fix constant flattened property serialization (#8658)

* Fix constant flattened property serialization

* Update changelog

* Update changelog

* [Identity] Default credential error msg (#9374)

* 123

* update

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Simplify azure-identity error messaging

* Azure.Identity simplify exception messaging

* Azure.Identity simplify exception messaging

* Azure.Identity simplify exception messaging

* Simple change to try to re-run CI

* Get building and test passing

* Simplify error message and properly use CredentialUnavailable

* Fix test

Co-authored-by: Ziheng Zhou <[email protected]>
Co-authored-by: zzhxiaofeng <[email protected]>

* [Service Bus] Tests for SBClient token credential overload (#9347)

* Add "@azure/identity" to the external list for test rollup

* undo AAD test comments

* Update sdk/servicebus/service-bus/test/serviceBusClient.spec.ts

Co-authored-by: Richard Park <[email protected]>

* remove unnecessary check

Co-authored-by: Richard Park <[email protected]>

* [ai-text-analytics] Final doc fixes before GA (#9382)

* [ai-text-analytics] Remove dangling code fence

* [ai-text-analytics] De-localize links.

* [ai-text-analytics] add caveat note about TS 3.1 (#9407)

* [ai-text-analytics] add caveat note about TS 3.1

* patch

* [service-bus] Sender.createBatch cancellation support (#9233)

- Instrumenting abortSignal futher in the sender (going through init()).
- Replacing the _checkAndSetupAbortSignalCleanup() pattern in _trySend() with the newer shared utility waitForTimeoutOrAbortOrResolve function.
- Update launch.json so you can just hit F5 and run the unit tests target.

Part of #4375

* [service-bus] Fixing some issues with the samples (#9412)

- Needed to add in AsyncIterable
- `useProxy.ts` was incorrectly loading ws, causing it to not actually use a proxy.
- `useProxy.ts` wasn't compiling against later versions of https-proxy-agent

* [service-bus] Fix formatting issues for samples (#9421)

* Sync eng/common directory with azure-sdk-tools repository (#9411)

* [FormRecognizer] Update release date in CHANGELOG (#9422)

* [FormRecognizer] Update sample in README and ref docs (#9441)

* Update changelog for identity 1.1.0-preview.4 (#9356)

* [Event Hubs] Use consumer & producer client in disconnects.spec.ts (#9427)

* Fix sample optimisticConcurrencyViaEtag (#9451)

* Min-Max Dependency testing (#8919)

* Delete unused file "eng/pipelines/docs.yml" (#9453)

* [Event Hubs] Update startingFromHere helper to stop using EventHubClient (#9431)

* Update default Node version from 10 to 12 (#9454)

* Update dev dependency "rollup-plugin-visualizer" (#9330)

* Update runtime dependency "tslib" (#9322)

* [appconfig] Updating JavaScript sample to include changes from #9451. (#9461)

- Adding in a check that'll at least print when the results are incorrect so we don't run into this again.
- Fixing package.json - we don't have a next tag now that AppConfig is GA.

* Update dev dependency "puppeteer" (#9328)

* Increment package version after release of azure-ai-form-recognizer (#9447)

* Release for mgmt hdinsight (#9464)

* [service-bus] Throw an error if the user tries to just send a string (sender.send(string)) (#9381)

Throw an error if a user accidentally passes something that isn't a ServiceBusMessage (or it's accepted variations)

Also:
- Fixing some fairly obvious issues that eslint found
- Making it we use the various methods for sending, not just single message. Mostly just to get some simple test coverage in our smoke testing.

Fixes #9167

* rush update --full (#9463)

* [FormRecognizer] rename includeTextDetails to includeTextContent (#9486)

* [FormRecognizer] rename includeTextDetails to includeTextContent

in the custom form/receipt recognition options.  This was missed in the
consistency push.

* Update changelog

* [event-hubs] fix subscription deadlock (#9491)

* [service-bus] T2: Enabling abort in receiveBatch and subscribe (#9284)

- Plumbing abortSignal through so it can be used to cancel the init() call in subscribe() and any part of receiveBatch()
- Standardizing our AbortError message for all the new abortSignal work for Sender and Receiver.
- Factoring out the cleanup code.

Part of #4375

* [core-amqp] ensure sendRequest timeout is cleared for all responses (#9494)

* [core-amqp] ensure sendRequest timeout is cleared for all responses

* Update sdk/core/core-amqp/test/requestResponse.spec.ts

Co-authored-by: Richard Park <[email protected]>

* address feedback

Co-authored-by: Richard Park <[email protected]>

* release for mgmt cosmosdb (#9465)

* Increment package version after release of azure-cosmos (#9366)

* [FormRecognizer] Update live test resource region (#9419)

This is to avoid busy region like westus2

* Verify node version after install (#9498)

* [Cosmos] 3.7.1 Fixes null authToken for deleting items and IE browser crypto bug (#9512)

* 3.7.1

* Remove throw

* Restore old auth token behavior

* Add second bugfix

* Update sdk/cosmosdb/cosmos/test/functional/authorization.spec.ts

Co-authored-by: Zachary Foster <[email protected]>

* Assert status code

* Remove only

Co-authored-by: Zachary Foster <[email protected]>

* Increment package version after release of azure-cosmos (#9517)

* [Event Hubs] Use consumer & producer client in hubruntime.spec.ts (#9428)

* Update swagger_to_sdk_config.json (#9527)

upgrade typescript code-gen to latest version

* [Event Hubs] Remove EventHubClient dependency in ProducerClient (#9520)

* Update rush (#9542)

* Release for mgmt datafactory (#9437)

* [Cosmos] Fix bug where abort signal was being called immediately if passed by the user (#9540)

* Increment package version after release of azure-cosmos (#9561)

* Sync eng/common directory with azure-sdk-tools repository (#9489)

* Update pnpm to v5 (#9547)

* rush update --full (#9566)

* private feed publish (#9306)

* Renames and Refactorings (#9553)

* Change analyzer-analyzerName in AnalyzeTextOptions

* Change tokenizer-tokenizerName to CustomAnalyzer

* Chg includeTotalResultCount-includeTotalCount

* Minor Comment Added

* [Cosmos] Remove references to PK on options objects (#9546)

* Move text from method paramter to options bag (#9570)

* Reference tools repo version (#9577)

* Add initial Refs for repositiory

* Initial Changes to refs JS

* Switch template referencing to use extends

* Remove resources from archetype-sdk-integration.yml

* [Event Hubs] Test err scenarios against EventHubConsumerClient, not EventHubConsumer (#9521)

* [Event Hubs] Remove the middle layer of EventHubProducer (#9548)

* Update codeowners (#9576)

* [KeyVault] Common folder (#8866)

* Update challengeBasedAuthenticationPolicy.ts

* Common folder for the challenge based auth

During testing, I get this error:

Error: Dynamic requires are not currently supported by
@rollup/plugin-commonjs

* deleting this from certificates again

* wip

* fixes from https://github.com/ljian3377/azure-sdk-for-js/commit/0ce33f73d14516e9abe2206e113381c98275c882

* other changes that make sense after rebasing

* making sure keyvault-common is up to date

* same changes on keys and secrets

* formatting

* this .gitignore is not needed

* bringing the hotfix to this PR and making sure it passes CI

* new pnpm-lock after merging master

* package.json and eslintrc updates

* fixed end of file on keyvault-common/src/index.ts

* some of the feedback

* standard tsconf on keyvault-common

* tsconfig.json cleanup

* removed unnecessary scripts from keyvault-common

* [event-hubs] update EventHubConsumerClient.close to close subscriptions (#9567)

* [event-hubs] update EventHubConsumerClient.close to close subscriptions

* keep references to Subscriptions instead of EventProcessors and update test

* fix EventHubConsumerClient waiting 1 min before exit (#8598)

* add test

* update changelog

* [Key Vault] Separating public and internal folders (and fixing the sample paths) (#9584)

* [Key Vault] Separating public and internal folders

* TestMinMax: true

* cleanups after merging the latest master

* broke the build by mistake while fixing conflicts

* aligning secrets to the path changes to keys and certificates

* samples path fix

* [core-https] Port request policies to new pipeline model (#9468)

Port over existing policies from core-http.

* [Service Bus] ATOM API - Merging the return types (#9432)

* merge response interfaces

* update exports

* API report

* extends Response and update DOC comments

* NamespaceResponse -> NamespacePropertiesResponse

* API Report

* listRequestOptions -> options for Richard!

* API report

* Pin dev dependency "follow-redirects" (#9609)

- [email protected] adds a peer dependency on "debug"
- [email protected] depends on follow-redirects@^1.0.0, but does not depend on debug, which causes an error due to missing peer dependencies
- rush update --full

* [event-hubs] Fixing websockets sample (typescript one needed actual changes to both tsconfig.json and websockets.ts). (#9613)

Propagating the fixes we found for #9412 (websockets/proxy samples were broken) to the sample here.

Some small issues:
- The tsconfig.json needed to have esModulesInterop turned on
- package.json for both the JS and TS samples were referencing a tag we no longer publish (changed to 'latest' rather than 'next')
- Small changes for the TS so it can compile properly using `import`, rather than `require`.

Fixes #9415

* Unpin dev dependency "follow-redirects" (#9614)

- [email protected] removes peer dependency "debug"
- Fixes #9612

* [Service Bus] Convert ISO-8601 timestamp string into Date type (#9434)

* Helper method to convert ISO-8601 time into Date type

* createdAt, accessedAt, updatedAt

* update Tests

* API Report

* getDateOrUndefined -> getDate

* API report

* "At" -> "On"

* API Report

* release for arm synapse (#9573)

Co-authored-by: Shixun Guan <[email protected]>

* [Identity] Remove unneeded devDependency "test-utils-recorder" (#9617)

* remove unneeded devDependency

* fix karma.conf

* Update azure-sdk-tools Repository Resource Refs in Yaml files (#9606)

* Update azure-sdk-build-tools Repository Resource Refs in Yaml files (#9605)

* release for arm vmware (#9502)

update client name for python and js

Co-authored-by: SDK Automation <[email protected]>

* [FormRecognizer] fix JavaScript copy model sample (#9460)

to remove the TypeScript keyword from copy-n-paste

* Increment package version after release of azure-ai-text-analytics (#9414)

* Add @sarangan12 as a search owner (#9636)

* [FormRecognizer] Simplify the samples to get poller results (#9488)

* [core-http] Run formatter (#9631)

* [core-http] Update copyright text (#9632)

* Sync eng/common directory with azure-sdk-tools repository (#9610)

* update logic to registry check (#9640)

* Remove unneeded awaits for PagedAsyncIterableIterators (#9065)

* [storage-blob] clean up container iteration readme

The async iterator is returned synchronously, so no need for the `await`.

* Fix other storage readmes

* Fix a bunch of needless awaits

* Fix overzealous deawaiting

Co-authored-by: Ramya Rao <[email protected]>

* [Event Hub] Update ConsumerClient, Processor, PumpManager & Pump to not use EventHubClient (#9600)

* [core-http] change browser telemetry header key to x-ms-useragent (#9490)

Based on the analysis at https://github.com/Azure/azure-sdk-for-js/issues/6530#issuecomment-642326789

* KeyVault service doesn't support CORS yet which means no browser support
* Storage doesn't send x-ms-command-name or x-ms-useragent for browsers as they
are not using core-http user agent policy
* Event Hubs/Service Bus don't use http
* Rest of services allows the x-ms-useragent header key

it is safe to change the header key for browser user agent information from
`x-ms-command-name` to `x-ms-useragent` in core-http.

Updates to AppConfiguration and Storage libraries will be in separate PRs.

* [Service Bus] Bug Fix - Same seq number returned for parallel schedule requests (#9503)

* Add message_id to the request to be sent

* backup message_id

* test for parallel scheduled messages

* backup correlation_id

* Add a test - request without `message_id` should not match a response with `undefined` correlationId

* remove reqst.correlation_id check

* Update sdk/core/core-amqp/src/requestResponseLink.ts

Co-authored-by: Ramya Rao <[email protected]>

* update test with an additional check

* Update the test as Ramya suggested

* check message bodies too

* update the _makeManagementRequest to add message_id

Co-authored-by: Ramya Rao <[email protected]>

* [core-http] Fix simple linting errors & warnings (#9663)

* Code changes to nest user defined fields into a document property (#9655)

* Code changes to nest user defined fields into a document property

* Code changes to address PR Comments

* [core-client] Create package for AutoRest specific code (#9620)

Create a new package we can use for helpers for generated code.

* [search-documents] Add sentinel value to connection string (#9672)

* Add sentinel value to connection string

* Update sdk/search/search-documents/src/searchIndexerClient.ts

Co-authored-by: Jeff Fisher <[email protected]>

Co-authored-by: Jeff Fisher <[email protected]>

* [Service Bus] Fix "disconnect" tests in browser with rhea's bug-fix update  (#9586)

* unskip tests

* update rhea versions

* lock file

* revert rhea caret updates

* pnpm-lock from master

* Only verify error.message if it exists

* remove .only

* Modified Readme file similar to .NET Readme (#9687)

* Modified Readme file similar to .NET Readme

* Minor change

* Minor changes

* Change Readme sample (#9697)

* [Tables] Add the initial files for the track 2 Tables library (#9565)

* Add the initial files for the track 2 tables library

* Update the package.json file

* Update pnpm-lock.yaml file

* Fix small nits

* Remove articats from ci.yml

* Add downlevel types

Co-authored-by: Mahdi Varposhti <[email protected]>

* [Service Bus] Renames to use the messages suffix and remove singular alternatives (#9678)

* [Event Hubs] Remove the middle layer of EventHubConsumer (#9623)

* [Event Hubs] Run formatter (#9650)

* add docs/rules back for eslint-plugin (#9705)

* Avoid passing unused credentials to generated client constructors (#9635)

* Add missing single quote (#9709)

* Ignore src/generated folders when running check-format/format (#9670)

* [Service Bus] Add changelog for recent service bus PRs (#9703)

* Add changelog for recent service bus PRs

* minor updates

* request -> send request

* Update sdk/core/core-amqp/CHANGELOG.md

Co-authored-by: Ramya Rao <[email protected]>

Co-authored-by: Ramya Rao <[email protected]>

* [app-configuration] Fix issue with usages of 'crypto' package for browsers. (#9648)

Our usage of the `crypto` package in appconfig causes an issue in browsers.

This adds functions to hash and hmac  that use the appropriate implementation for browser (WebCrypto) and the node.js crypto package for node.

Fixes #9585

* [Service Bus] More info in the doc strings for ISO-8601 duration format (#9723)

* Add link regarding ISO-8601 duration format

* prefix the statement with "More on "

* [Service Bus] Renaming ListRequestOptions {top, skip} -> {maxCount, skip} (#9664)

* top -> maxCount

* update tests

* API Report

* improve helper method for test

* changelog

* Update sdk/servicebus/service-bus/CHANGELOG.md

Co-authored-by: Ramya Rao <[email protected]>

* fix changelog format

Co-authored-by: Ramya Rao <[email protected]>

* Regeneration and Other changes (#9711)

* Regeneration and Other changes

* Test Build Fix

* Minor Regenerate Change

* [Core AMQP] Remove ThirdPartyNotices.txt for core-amqp (#9728)

* Bump dependency core-http version to 1.1.4 (#9539)

* Bump dependency core-http version to 1.1.4

to ensure that all packages have the `x-ms-useragent` browser header key change
for telemetry.

* Make exception for storage

* [event-hubs] ensure error stack traces only logged in verbose mode (#9738)

* [appconfig] Use core-http userAgentPolicy() (#9534)

* [core-amqp] ensure error stack traces only logged in verbose mode (#9736)

* [service-bus] Missing the necessary link for "what is service bus?" in the readme (#9740)

We were missing the necessary link for "what is service bus?". Leading people to wonder "what is what is service bus?".

Fixes #9683

* [core-auth] Update CHANGELOG for July release (#9741)

* [core-amqp] prepare for 1.1.4 release (#9743)

* [event-hubs] prepare for 5.2.2 release (#9745)

* [keyvault-common,table] bump core-http dep version to 1.1.4 (#9739)

* [keyvault-common,table] bump core-http dep version to 1.1.4

These two libraries were not updated in #9539, possibly due to the exception and
tables change comes after the the exception was made.

* Disable PR validation for Tables for now

* Remove SkillSetCounter Py in ServiceCounters model (#9747)

* [event-hubs][service-bus] update core-amqp to 1.1.4 (#9750)

* [FormRecognizer] Renames based on Arch Board feedback (#9744)

* [FR] Rename in swagger transformations

* [FormRecognizer] Renames based on Arch Board feedback

- Rename properties `requestedOn` to `trainingStartedOn` and `completedOn` to `trainingCompletedOn` in `CustomFormModel` and `CustomFormModelInfo` types.

* Minor tweaks on log output

* Update doc comments

* [FormRecognizer] Add test coverage for Azure AD credential (#9752)

* Enable AAD credential for recorded clients

* Add a couple of recognition test to cover AAD credential

* Add one training client test to cover AAD credential

* [KeyVault] Some name cleanups and formattings (#9693)

* [KeyVault] Some name cleanups and formattings

* package.json types cleanup

* keyvault-keys node recordings

* keyvault-secrets browser and node recordings

* keyvault-certificates node recordings

* keys recordings

* karma port changes

* cleaning things up

* keys recordings

* browser recordings for certificates

* missing keys recordings

* fixing keyvault-keys tests by not running the crypto tests in the browser

* better comment about local cryptography on the local crypto test

* [KeyVault-Keys] The CryptographyClient indeed works with a KeyVaultKey (#9647)

* [KeyVault-Keys] The CryptographyClient indeed works with a KeyVaultKey

* sample readme fixes

* undoing irrelevant change

* no need for another sample file

* https://github.com/Azure/azure-sdk-for-js/pull/9647#discussion_r444006990

* formatting

* I forgot to check this in

* removed unnecessary comment

* documentation on the key paramter of the CryptographyClient constructor

* I collapsed File Name into FileName in one of the READMEs by mistake

* undoing the sample readme changes, theyre unnecessary

* Update sdk/keyvault/keyvault-keys/src/cryptographyClient.ts

Co-authored-by: Ramya Rao <[email protected]>

* Update sdk/keyvault/keyvault-keys/samples/javascript/cryptography.js

Co-authored-by: Ramya Rao <[email protected]>

* Update cryptography.js

* Update sdk/keyvault/keyvault-keys/samples/javascript/cryptography.js

Co-authored-by: Ramya Rao <[email protected]>

* Update sdk/keyvault/keyvault-keys/samples/typescript/src/cryptography.ts

Co-authored-by: Ramya Rao <[email protected]>

Co-authored-by: Ramya Rao <[email protected]>

* [KeyVault] apiVersion documentation (#9656)

* [KeyVault-Keys] apiVersion documentation

* undid the samples changes

* Update sdk/keyvault/keyvault-keys/README.md

Co-authored-by: Ramya Rao <[email protected]>

* Update sdk/keyvault/keyvault-keys/README.md

Co-authored-by: Ramya Rao <[email protected]>

* other clients

Co-authored-by: Ramya Rao <[email protected]>

* [KeyVault Admin] Generated client (#9668)

* base files

* rush.json addition

* rbac generated code and other relevant changes

* small fix

* wip

* no api-extractor for now

* it builds

* cleanups

* prettier, eslint and file path fixes

* two secret matches

* moving things from core to generated

Related to https://github.com/Azure/azure-sdk-for-js/issues/9069

* add-credentials to false

* autorest fixes, from https://github.com/Azure/azure-sdk-for-js/pull/9668#discussion_r445201408

* One last instance of "Secret" in the README

* no more build:minify

* feedback from https://github.com/Azure/azure-sdk-for-js/pull/9668#discussion_r445149145

* 1.0.0-preview.1 since this preview is so far behind

* copyright headers

* tests.yml

* Delete ThirdPartyNotice.txt

* [KeyVault] Documenting the lack of browser support (#9716)

* [KeyVault] Documenting the lack of browser support

Here's what comes to mind. Feedback appreciated.

Once approved:
Fixes #9033

* typo

* feedback, and same note in other READMEs

* colon

* feedback by Ramya

* release for arm digitaltwins (#9681)

* Generated from 0859f6ca9d516e1ae64f47d13c417a898fa6281f

Update Typescript readme for SDK generation

As requested by Azure JS SDK team

* update version

Co-authored-by: SDK Automation <[email protected]>
Co-authored-by: Shixun Guan <[email protected]>

* [Service Bus] Update relevant Docs for the update methods and and remove internal get calls (#9751)

* Add Docs and remove get calls from the update requests

* minor fix in docs

* Changelog

* fix update tests

* final fix

* fix updateSubscription tests

* remove .only

* update docs for update methods as Ramya suggested

* Update sdk/servicebus/service-bus/CHANGELOG.md

Co-authored-by: Ramya Rao <[email protected]>

Co-authored-by: Ramya Rao <[email protected]>

* Bump core-auth dependency version to 1.1.3 (#9754)

for the ES5 compatibility fix.

* [FormRecognizer] Enable copy model test (#9749)

* [FormRecognizer] Enable copy model test

and adding recordings

* Remove some creating polling state from recording

to shorten playback test run duration.

* FR resource should also be created in westcentralus

* Revert "Remove some creating polling state from recording"

This reverts commit 669c0e44f27edd6d8a34547db9e126b6cf37717e.

* Use model id from old recordings for copy model recorded tests

* reverting what i accidentally changed in previoud PR (#7902)

* Sync eng/common directory with azure-sdk-tools repository (#9720)

* Update SECURITY.md (#9721)

* Sync eng/common directory with azure-sdk-tools repository (#9790)

* Generate id before extracting partition key (#9735)


Co-authored-by: Abdou <[email protected]>
Co-authored-by: Steve Faulkner <[email protected]>
Co-authored-by: Zachary Foster <[email protected]>

* [core-http] Share common code for exponential backoff (#9667)

* [core-http] Refactor to share common code for exponential backoff

for `ExponentialRetryPolicy` and `SystemErrorRetryPolicy`

Address #4201 and #9569

* Update CHANGELOG

* Address CR feedback

* Update the release date in CHANGELOG

* [FormRecognizer] address issues/feedbacks from UX study (#9673)

* [FormRecognizer] address issues/feedback from UX study

- Add ref doc to FormField.value and FormField.valueType

* - Filter RecognizedForm.pages to those relevant to the recognized form

* Default confidence to 1 if missing

* - Remove `||` from sample snippets

* [Cosmos] Increment package version after release of azure-cosmos (#9795)

* [core-http] Upgrade to @azure/eslint-plugin-azure-sdk@^3.0.0 (#8687)

* [core-http] Upgraded ESLint

* Fixing the tests

* formatting

* These ESLint rules cant be applied in this repo

* some of the feedback by Jeff

* formatting

* undoing this review API change

* [event-hubs] Fixing flaky tests. (#8684)

Fixing two flaky tests:
- createBatch() was dumping out a Diagnostic-Id property into its messages. The issue was random tests overwriting the global tracer object with their test tracer. Now all those tests use the same utility function and cleanup after they're done.
- There were a few tests that were using the date to choose their start position which ends up being unreliable in the face of clock skew. Changed over to use the same sequenceNumber positioning trick as other tests with a new testutils method.

* [event-hubs] Fixing broken sample (#8695)

Fixing missing 'async' keyword for snippet (and wrong client name) in readme.

* undoing the eqeqeq changes on src/serializer

* fixing the browser tests for core-http

* formatting

* no more jshint comments

* Feedback from Jeremy regarding using Promise<never>

* feedback from Jeff on test/mockHttp

* stricter types on some serializers (from Jeremy)

* comparison to undefined feedback (Ramya)

* formatting

* I couldnt do Jeffs advice

* better solution for timeout

* more improvements after updating from master

* fixes after I applied eslint to the latest version of core-http

* package.json fix after updating this branch based on the latest master

* deserializeResponseBody is back as a non-async function, though it still returns a Promise

* feedback from https://github.com/Azure/azure-sdk-for-js/pull/8687/files#r444542455

* Using RequestPolicyFactory in more places

* eslint fixes after updating this branch based on master

* very small lint fixes

* Ignore unused arguments with _ prefix

* headers.get() only returns string or undefined

Co-authored-by: Richard Park <[email protected]>
Co-authored-by: Ramya Achutha Rao <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>

* Release for arm network (#9529)

Co-authored-by: SDK Automation <[email protected]>
Co-authored-by: Shixun Guan <[email protected]>
Co-authored-by: qiaozha <[email protected]>

* Release for iotcentral 535 (#9803)

Co-authored-by: SDK Automation <[email protected]>
Co-authored-by: xichen <[email protected]>

* [storage] add convenience methods exists, createIfNotExists, deleteIfExists (#8964)

* [storage-blob] add CreateIfNotExists and DeleteIfExists for Container and Blob

* [storage-flie-share] add CreateIfNotExists and DeleteIfExists

* restore package.json

* [storage-queue] add createIfNotExists and deleteIfExists

* add exists for storage-file-share

* add createIfNotExists and deleteIfExists for storage-file-datalake

* record tests

* fix typo

* change return type - blob

* change return type - datalake

* change return type - file

* change return type - queue

Co-authored-by: Lin Jian <[email protected]>

* Make sure we properly handle processError being treated as async. (#9797)

Errors thrown _from_ processError will only be printed, not routed back into processError to prevent possible cascading failures.

Fixes #7836

* Added Typescript Samples for Search Documents (#9809)

* [FormRecognizer] Support passing in continuation token for next page (#9235)

* Add mocked tests for listing multiple pages

* [FormRecognizer] Fix broken ref docs about supported content types (#9518)

* Mention file formats instead.

* [Cosmos] Properly escape DEL character in partition key (#9822)

* [Service Bus] Remove check for session exists when creating session receiver (#9766)

* [Service Bus] Tracing expired sessions is not needed (#9765)

* [app-configuration] Fixing readme with feedback from Cala. (#9823)

* Fixing headings for the readme to match the standard.
* prettier made some minor formatting changes that looked good.

* [Service Bus] peekMessages to have maxMessageCount mandatory (#9798)

* Increment package version after release of azure-cosmos (#9825)

* [Service Bus] Add isServiceBusMessage checks to all methods that take messages (#9708)

* [Service Bus] Change suffix for time stamp attributes from "On" to "At"  (#9807)

* remove duplicate copyright header

* "On" -> "At"

* Changelog

* API Report

* [ai-form-recognizer] Fix a silent merge conflict (#9827)

* [Service Bus] Remove redundant docs between interfaces and implementations (#9826)

* [storage] STG73 features preview (#9532)

* storage-internal-avro parser (#8893)

* IReadable

* Avro compiles

* use storage-internal-avro as shared source in storage-blob

* local build config

* test and bug fix

* rename IReadable to AvroReadable, separate the stream implementation with the interface declaration we need different implementation in browsers

* add "private": true to pacakge.json

* use Record

* me

* fix

* tentative: use string enum instead of string union to avoid two copies of constant string

Co-authored-by: Lin Jian <[email protected]>

* [Storage][Blob] Regenerate with Dec19 swagger

* avor parser offset fix; Readable interface update (#9088)

* [Storage] Regenerate API review

* [storage] avro parser fix (#9154)

* fix

* resolve PR comments: remove useless assignment

* AvroReadableFromStream: removeListener when read succeeds

* remove listener when error callback triggered

Co-authored-by: Lin Jian <[email protected]>

* [Storage] Dec19 - Quickquery (#9112)

* [Storage] Quick query

* [Storage] Quick query API review

* [Storage] Quick query recording

* [Storage] Quick query test case update

* [Storage] Resolve comments

* regenerate storage-blob

* re-generate storage-file-share

* regenrate datalake and undo adding mode to storageClient and storageClientContext

* update datalake service version to 2019-12-12

* [Storage] QuickQuery - Make BlobQueryCsvTextConfiguration properties optional (#9484)

* [storage][blob] blob versioning (#7886)

* blob versioning

* support 'bv' SAS

* add response headers to getProperties & api-extract

* fix comments

* add versionId to getBlob response headers

* fix broken undelete test

* undo 'add snapshot to options'

* replace withVersionId() with getURLWithVersionId()

* Revert "replace withVersionId() with getURLWithVersionId()"

This reverts commit eb1d90d34783b7717ca0a6a651ba2c53a637be48.

* hide the versionId option in convenience layer

* add more tests

* fix comments

* change version to preview

* fix merge error

Co-authored-by: Lin Jian <[email protected]>

* [storage] added change feed (#9151)

* fix

* change feed with mininal live tests

* mocking tests

* format code & rename buildAvroReader parameter name

* add README.md

* change eventTime type from string to Date

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Jeremy Meng <[email protected]>

* PR comments

* Rename eTag to etag in BlobChangeFeedEventData

* remove hns only properties in change event record

* fix minor bug in avro

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Brian Terlson <[email protected]>

* Update sdk/storage/storage-blob-changefeed/README.md

Co-authored-by: Brian Terlson <[email protected]>

* rename to BlobChangeFeedGetChangesOptions and fix comments

* update api

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Brian Terlson <[email protected]>

* [storage][blob & datalake] Jumbo blob (#9480)

* Jumbo putBlock, putBlob, datalake file append

* undo upload ArrayBuffer

* minor edit

Co-authored-by: Lin Jian <[email protected]>

* [Storage] Support blob tags (recording will be added later) (#9440)

* [Storage] Support blob tags (recording will be added later)

* [Storage] Unify blob tags type Tags everywhere in options and responses

* [Storage] Blob tags - resolve comments

* update service version to 2019-12-12

* blob tags test record

* fix listContainer options

* wip

* record skipped test for Jumbo

* set default formatter

* me

* wip

* blob record node

* record

* add version check for x permission

* record for browser

* wip

* wip

* wip

* wip

* add f for account SAS permission

* add permission f, and x for container

* wip

* me

* me

* PR comments

* me

* record changefeed

* record with blobversioning disabled

* record skipped test for browser

* wip

* avro compatible with Node v8

* fix test record

* fix record

* fix ae-forgotten-export

* lint

* fall back to util for old node.js TextEncoder

* release pipeline for change feed

* [storage][file] large smb files (#9624)

* create large smb files

* wip

* record test

* me

Co-authored-by: Lin Jian <[email protected]>

* [storage] added set Expiry (#9560)

* add set expiry

* add check for input

* use Date instead of a time string

* call blob endpoint directly in datalake

* undo evenhub package.json

* add alternative version and use number instead of decimal string for
expiresOn relative

* add timeToExpireInMs to option

* use string literals instead of enum

* keep old enum for backward compatibility

* nit

Co-authored-by: Lin Jian <[email protected]>

* merge fix

* omit doc in storage-internal-avro

* Revert "[storage] added set Expiry (#9560)"

This reverts commit c3b4414b09539c02266cc16b192f87f3d686dd8e.

* nit

* skip live tests for blob tags and quick query

* change path for ts-samples

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: XiaoningLiu <[email protected]>
Co-authored-by: Xiaoning Liu <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Brian Terlson <[email protected]>

* [storage] version bump up for stg73 preview release (#9838)

* use "rc" instead of preview and bump up versions for queue

* nit

Co-authored-by: Lin Jian <[email protected]>

* add 'pack' script for avro as js - test - utils CI fail (#9841)

Co-authored-by: Lin Jian <[email protected]>

* Fix release artifacts parameter name for passing to stage-artifacts.yml  (#9820)

* rush update --full (#9848)

* Sync eng/common directory with azure-sdk-tools repository (#9816)

* [Service Bus] Bug Fix: userProperties aren't populated while using createRule() from ATOM API (#9794)

* fix ruleResourceSerializer w.r.t userProperties

* add test for userProperties

* changelog

* update error messages

* remove console.log

* Update sdk/servicebus/service-bus/CHANGELOG.md

Co-authored-by: Ramya Rao <[email protected]>

* remove extra types

* Update enum names to TypeMaps

* fix the 3 failed tests

* keyValueTypeXMLTag

* buildRawSqlParameter -> buildRawKeyValuePairFromSqlParameter

* more docs and renames

* Add test for array

* InternalRawKeyValuePairs, refactoring and much more

* scope the test to user-properties

* refactor parameter.type

* update error message

* remove .only

* fix error

* unneeded .valueOf

* Add boolean type

* Validations

* tests for Type validation errors on Correlation user property inputs

* Update sdk/servicebus/service-bus/src/serializers/ruleResourceSerializer.ts

* Update sdk/servicebus/service-bus/src/serializers/ruleResourceSerializer.ts

* fix test failures

Co-authored-by: Ramya Rao <[email protected]>

* Release for appconfiguration 551 (#9832)

* Generated from fbb8071c4a5769e8365fe4722a6ab3ef97720346

Update go readme since it was using 2019-11-01-preview API version.

* release for appconfiguration 551

Co-authored-by: SDK Automation <[email protected]>
Co-authored-by: xichen <[email protected]>

* [Service Bus] Support for OperationOptions in management operations (#9654)

* OperationOptionsForHTTP

* API Report

* OperationOptionsForHTTP

* plug in OperationOptionsForHTTP

* Make OperationOptionsForHTTP optional

* fix build failures

* API report

* Fixes plugging in the options to webResource

* TYPO

* OperationOptions -> OperationOptionsForAMQP,
OperationOptionsForHTTP -> OperationOptions

* API Report

* add some tests

* OperationOptionsForAMQP -> AMQPOperationOptions

* revert core-http changes

* API Report

* tests

* fix build failures

* Typos

* API Report

* AMQPOperationOptions -> OperationOptionsBase

* API Report

* Update sdk/core/core-http/src/webResource.ts

* Changelog

* fix operOptions for createSubscription

* export { OperationOptions } from "@azure/core-http";

* API Report

* Options description

* Update docs as per the feedback

* Update sdk/servicebus/service-bus/src/modelsToBeSharedWithEventHubs.ts

Co-authored-by: chradek <[email protected]>

* Update custom headers test, ensure the custom headers are populated as expected

* fix test failures

* API Report

* fix test from merge resolution

Co-authored-by: chradek <[email protected]>

* Adding JS Samples (#9837)

* Adding JS Samples

* Dummy Commit

* Revert Dummy Commit

* [storage][stg73preview] use preview version (#9844)

* undo .only

* use preview version

* common-version

Co-authored-by: Lin Jian <[email protected]>

* [Cosmos] Use tslint & fail build on lint errors (#9861)

* [core-http] Fix a regression in exponential retry policy (#9852)

The refactoring in PR #9667 introduce a behavior change: when the response is
undefined we retry.  This is different from the previous behavior that we don't
retry on undefined response or response.status.

Also the `delay()` and `_nextPolicy.sendRequest()` call should be inside the
try-block.

* Add tests

address CR feedback

* Update eslint plugin version to 3.0 (#9862)

* [service-bus] Adding in support for stopping a Subscription for non-session `subscribe` (#9802)

This first PR is just going to handle non-session closing of the result of a `subscribe()`* call. 

When close()'d we drain the receiver so no new messages will be received and leave the link _open_ so the
user can still settle against it. To close the link they must close the outer `Receiver` object.

*Normally we'd call it a subscription but that's already the name of an entity within service bus

* Enable ChangeLog Verification in Repo (#9819)

* [event-hubs] adds loadBalancingOptions and greedy load balancing strategy (#9706)

* [event-hubs] adds EventHubConsumerClientOptions

* fix conflicts from rebase

* refactor load balancer to get all claimable partitions

* add balanced and greedy load balancer strategies

* add unbalancedLoadBalancingStrategy

* update pumpManager to expose receivingFromPartitions

* updates EventProcessor to use LoadBalancingStrategies

* add tests for the partitionOwnershipExpirationIntervalInMs

* add functional load balancing tests

* update docs

* update version to 5.3.0-preview.1

* add changelog

* update pnpm-lock

* address feedback

* add explicity existance check to partitionOwnership.lastModifiedTimeInMs

* trashing -> thrashing

* be smarter about else-if statements

* explain the magic number 6

* identifyPartitionsToClaim -> getPartitionsToClaim

* identifyClaimablePartitions -> listAvailablePartitions

* add better summary for EventHubConsumerClientOptions

* add better docs around greedy and balanced strategies

* remove superfluous doc from CommonEventProcessorOptions

* add comment around why we have abandoned partitions

* throw AbortError instead of silent return

* remove unneeded receivingFrom method

* update pnpm-lock.yaml

* rush update

* [Service Bus] Removing `isReceivingMessages` from the Receiver API Surface (#9875)

* Remove `isReceivingMessages` from the API surface

* API Report

* fix build failures

* remove `as any`

* Add ### Breaking Changes sub-title

* Update sdk/servicebus/service-bus/CHANGELOG.md

Co-authored-by: Ramya Rao <[email protected]>

Co-authored-by: Ramya Rao <[email protected]>

* [service-bus] Adding in support for stopping a Subscription for session `subscribe` (#9849)

Adding in support for stopping a session-based subscribe() call. 

Fixes #9493.

* [Service Bus] Minor changelog updates (#9884)

* Minor changelog updates

* release date = Tuesday

* [ai-form-recognizer] Anticipating support for non-text field elements (#9882)

* [ai-form-recognizer] Renames to support non-text elements.

* Samples

* Fixed some missing renames in exception texts, fixed a typo in changelog.

* [ai-form-recognizer] Use Date intead of number for expiresOn (#9876)

* [app-configuration] updating changelog with release date (#9886)

* Use subscription configuration template pattern that .NET and Java use (#9891)

* [Cosmos] Add missing copyright text (#9887)

* [Identity] Fix warning on unresolved dependencies (#9855)

* [Identity] Fix warning on unresolved dependencies when building tests (#9897)

* [arm-recoveryservicesbackup] Re-generate to fix missing nextLink (#9851)

* [Key Vault Certificates] Happy path to avoid unstable live tests (#9879)

* [Key Vault Certificates] Happy path to avoid variable behavior in live tests

* forgot these .only

* added timeout on the package.json somewhere I missed before

* Sync eng/common directory with azure-sdk-tools repository (#9856)

* [event-hubs] prepare for 5.3.0-preview.1 release (#9911)

* [event-hubs] prepare for 5.3.0-preview.1 release

* add unreleased entry for checkpointstore

* [ai-form-recognizer] Remove RecognizedReceipt type (#9853)

* [ai-form-recognizer] De-specialize receipt recognition

* Add sample showing how to coerce the type of Form repsonses

* Format

* Fix ae-missing-export warnings

* [Service Bus] Update preview.3 to preview.4 (#9913)

* [Service Bus] Update preview.3 to preview.4

* update versions and hyperlinks

* migration guide updates

* Release Custom Vision Prediction & Training - July 2020 (#9906)

* Changes to Prediction SDK

* Code changes for TRaining SDK

* Change version of Training to 5.1.0

* Increment package version after release of azure-core-http (#9907)

* [Storage] avoid import fs directly in exporting modules (#9877)

Put them in utils.node instead, and add stubs for browsers.

* Increment package version after release of azure-event-hubs (#9930)

* [ai-form-recognizer] Samples usability improvements (#9917)

* [ai-form-recognizer] Samples usability improvements

* Tweaked instructions

* Fixes

* Increment version for core releases (#9815)

* Increment package version after release of azure-core-auth

* Increment package version after release of azure-core-amqp

* Increment version for storage releases (#9893)

* Increment package version after release of azure-storage-queue

* Increment package version after release of azure-storage-blob

* Increment package version after release of azure-storage-file-datalake

* Increment package version after release of azure-storage-file-share

* Increment package version after release of azure-storage-blob-changefeed

* use GA versions

* update versions in build:autorest script

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>

* Increment package version after release of azure-search-documents (#9908)

* [ai-form-recognizer] Last minute samples fixes from release testing. CHANGELOG release date. (#9936)

* Increment package version after release of azure-service-bus (#9931)

* [Cosmos] Remove Lazy Index Tests (#9932)

Co-authored-by: Steve Faulkner <[email protected]>

* [Storage] bump core-http dependency version to 1.1.4 (#9737)

Revert "Make exception for storage"

This reverts commit a3035f8256e11105d9d16615f7e7809aa40fd1ff.

* [servicebus] docs: fix type in doc comment (#9941)

There was disagreement about the returned type between the doc comment
and the code itself.

* Clean up Changelog and ci files (#9915)

* Sync eng/common directory with azure-sdk-tools repository (#9934)

* [Service Bus] atomXml.spec.ts has internal tests (#9946)

* [storage] enalbe live tests for new features (#9892)

* queue: use ./.env

* file: skip large file test except playback

* blob: un-skip live tests ex. browser, quick query

* record undelete

* fix intermittent test failure: wait 30s after containerClient.setAccessPolicy()

* Add enableVersioning parameter to test-resources.json

* Add ARM template parameter to enable versioning

* "boolean" -> "bool"

Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: Daniel Jurek <[email protected]>

* Fix issue when debugging storage recorded tests (#9940)

There's some difference between `npm run unit-test:node`/`npx _mocha
...` and `node ./node_modules/mocha/bin/_mocha` which we used to
launch the debugger. The former did a probe to find a `mocha`
executable in a set of paths and in our case it finds
`./node_modules/.bin/mocha` and execute it.
`./node_modules/.bin/mocha` is actually a shell script wrapping around
`./node_modules/mocha/bin/mocha` and setting the `NODE_PATH`
environment variable. One of the path included in this variable is
particularly important:
`<project-root>/common/temp/node_modules/.pnpm/node_modules`. It
helped to resolve the indirect dependency of `nock` which otherwise
wouldn't resolve under storage libraries' `./node_modules/`.  So just
launching `node ./node_modules/mocha/bin/_mocha` would lead to an
error of `Cannot find module 'nock'`.

This change adds a `NODE_PATH` environment variable to the VS Code
launch configuration using `ts-node`.  It should be safe to only
include the `common/temp/node_modules/.pnpm/node_modules` because in
our repository, all libraries' dependencies are symbol links to
packages under pnpm's `node_modules` directory.

The other configuration of debugging unit tests does not have this
issue, possibly because the test file is a bundle.

Fixes #9901.

* [Service Bus] Clean up SubscriptionRuleManager (#9952)

* fixing the logic to add to the common-versions.json (#9967)

* Update rush and pnpm (#9965)

* rush update --full (#9969)

* Clean up resources (#9945)

* Sync eng/common directory with azure-sdk-tools repository (#9964)

* fix unstable test case (#9957)

Co-authored-by: Lin Jian <[email protected]>

* update readme of keyvault (#9978)

* Remove exporting the DAC credentials list (#9972)

* [Service Bus] Remove unused helper methods (#9990)

* [Service Bus] Avoid using Dictionary (#9943)

* release-for-mgmt-dns (#9979)

* [core-amqp] close session after sender and receiver in requestResponseLink (#9998)

* Preventing dev feed for private package on script level and with artifact parameter option in pipeline (#9948)

* Sync eng/common directory with azure-sdk-tools repository (#10012)

* [Service Bus] Renaming `userProperties` to `properties` - Both ServiceBusMessage and ATOM API (#10003)

* Rename userProperties under ManagementClient too

* fix test utils

* rename userProps in the tests

* one leftover

* API Report

* Changelog

* fix build failure

* Increment package version after release of azure-identity (#10017)

Co-authored-by: azure-sdk <[email protected]>

* verification of links pipelines (#9216)

* fix eventhub link (#10019)

* [Test Recorder] Fix browser custom console logging when recording (#10018)

In function `customConsoleLog()` we save `window.console.log` to a
local variable and use this variable to log to real console.  The
problem is that when `customConsoleLog()` is called multiple times,
`window.console.log` is no longer the original one, and we end up
nesting the modified `log` function again and again.

This PR saves the original console log function to a module level
variable instead so it's not wrapped multiple times.

* Allow overriding consoleLog for testing

* [Test Recorder] Fix an issue where not all browser requests are recorded (#10013)

We use an async function to add a recorded request to an array then
save the array to files when recorder stops.  However we did not wait
for the async function to complete, which leads to recording missing
requests. This issue only happens in the browser case. In Node it is
done in synchronous way.

This change makes recorder to wait until all the recorded
requests are added before saving them.

All the tests have been updated to await for `recorder.stop()`.

Co-authored-by: Harsha Nalluru <[email protected]>

* [Key Vault Certificates] Test timeout need to be increased since now we wait for all pollers (#10031)

* [Key Vault Certificates] Test timeout need to be increased since now we wait for all pollers

* dependency-testing timeout

* Adding dev-tool to the repo (#7872)

* Azure SDK dev-tool first pass

* [dev-tool] dev-tool dev-samples and fixes to all commands

* Nested command structure

* Better argument parsing, type-checking, and recursive command structure. Added support for running a single sample.

* Added dev-tool README

* [ai-text-analytics] Update package.json to use new script.

* Removed some development cruft

* prettier + eslint

* Quick fix to ParsedOptions type

* WIP

* Command framework improvements

* Basic unit-tests for package resolution.

* One more test, assorted changes

* README update

* Migrated all packages with sample code to use dev-tool

* Added dummy integration-test stub

* Added dummy integration-test stub for eslint plugin

* Added dummy integration-test:browser stubs

* Added dev-tool dependency to packages using it

* Corrected build:samples step in package.json

* WIP

* [dev-tool] ts-to-js command

* [dev-tool] leaf command test

* Fixed more deeply nested samples due to shared code

* [Service Bus] Invalid param tests don't need live resources (#9947)

* [Service Bus] Pick TestClientType randomly (#9950)

* generate and versioning (#9198)

Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Krista Pratico <[email protected]>
Co-authored-by: Sarangan Rajamanickam <[email protected]>
Co-authored-by: chradek <[email protected]>
Co-authored-by: Richard Park <[email protected]>
Co-authored-by: Harsha Nalluru <[email protected]>
Co-authored-by: Ramya Rao <[email protected]>
Co-authored-by: Will Temple <[email protected]>
Co-authored-by: Jonathan Turner <[email protected]>
Co-authored-by: Scott Schaab <[email protected]>
Co-authored-by: まっぴぃ⊿ <[email protected]>
Co-authored-by: Mitch Denny <[email protected]>
Co-authored-by: changlong-liu <[email protected]>
Co-authored-by: SDK Automation <[email protected]>
Co-authored-by: Daniel Rodríguez <[email protected]>
Co-authored-by: Steve Faulkner <[email protected]>
Co-authored-by: Azure SDK Bot <[email protected]>
Co-authored-by: KarishmaGhiya <[email protected]>
Co-authored-by: azure-sdk <[email protected]>
Co-authored-by: Jose Manuel Heredia Hidalgo <[email protected]>
Co-authored-by: Ziheng Zhou <[email protected]>
Co-authored-by: zzhxiaofeng <[email protected]>
Co-authored-by: Andreas Bjørn Hassing <[email protected]>
Co-authored-by: Mike Harder <[email protected]>
Co-authored-by: Qiaoqiao Zhang <[email protected]>
Co-authored-by: Zachary Foster <[email protected]>
Co-authored-by: chunyu3 <[email protected]>
Co-authored-by: Chidozie Ononiwu <[email protected]>
Co-authored-by: Jeff Fisher <[email protected]>
Co-authored-by: Abyssknight <[email protected]>
Co-authored-by: Shixun Guan <[email protected]>
Co-authored-by: Brian Terlson <[email protected]>
Co-authored-by: Jeff Fisher <[email protected]>
Co-authored-by: Mahdi Varposhti <[email protected]>
Co-authored-by: Mahdi Varposhti <[email protected]>
Co-authored-by: Jeremy Meng <[email protected]>
Co-authored-by: Josh Free <[email protected]>
Co-authored-by: abouroubi <[email protected]>
Co-authored-by: Abdou <[email protected]>
Co-authored-by: Ramya Achutha Rao <[email protected]>
Co-authored-by: qiaozha <[email protected]>
Co-authored-by: xichen <[email protected]>
Co-authored-by: xichen <[email protected]>
Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: Lin Jian <[email protected]>
Co-authored-by: XiaoningLiu <[email protected]>
Co-authored-by: Xiaoning Liu <[email protected]>
Co-authored-by: Daniel Jurek <[email protected]>
Co-authored-by: Steve Faulkner <[email protected]>
Co-authored-by: Matt Ellis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiacfan jiacfan jiacfan left review comments

@ljian3377 ljian3377 ljian3377 left review comments

@jeremymeng jeremymeng jeremymeng left review comments

@bterlson bterlson bterlson left review comments

@HarshaNalluru HarshaNalluru Awaiting requested review from HarshaNalluru

@vinjiang vinjiang Awaiting requested review from vinjiang

Assignees
No one assigned
Labels
APIChange This PR contains an addition or change to the API signature and must be reviewed by an architect. Client This issue points to a problem in the data-plane of the library. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@XiaoningLiu @jiacfan @ljian3377 @bterlson @jeremymeng