[Storage] Binary data blobs #19607

kasobol-msft · 2021-03-04T03:56:28Z

check-enforcer · 2021-03-04T17:24:31Z

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

What if I am onboarding a new service?

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:
/azp run prepare-pipelines
This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:
/azp run java - [service] - ci

gapra-msft · 2021-03-04T18:45:09Z

...rage/azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobClientBase.java

        DownloadRetryOptions options, BlobRequestConditions requestConditions, boolean getRangeContentMd5,
        Duration timeout, Context context) {
        StorageImplUtils.assertNotNull("stream", stream);
        Mono<BlobDownloadResponse> download = client
-            .downloadWithResponse(range, options, requestConditions, getRangeContentMd5, context)
+            .downloaFluxWithResponse(range, options, requestConditions, getRangeContentMd5, context)


typo: download

alzimmermsft · 2021-03-05T18:58:21Z

sdk/storage/azure-storage-blob/README.md

@@ -156,7 +158,7 @@ The following sections provide several code snippets covering some of the most c

 Create a `BlobServiceClient` using the [`sasToken`](#get-credentials) generated above.

-<!-- embedme ./src/samples/java/com/azure/storage/blob/ReadmeSamples.java#L30-L33 -->
+<!-- embedme ./src/samples/java/com/azure/storage/blob/ReadmeSamples.java#L31-L34 -->


@srnagar, do we know if embedme has resolved our issue about supporting named blocks so we don't have a painful experience when changing README sample files?

sdk/storage/azure-storage-blob/src/main/java/com/azure/storage/blob/BlobAsyncClient.java

alzimmermsft · 2021-03-05T19:04:16Z

sdk/storage/azure-storage-blob/src/main/java/com/azure/storage/blob/BlobAsyncClient.java

+            }
+
+            return overwriteCheck
+                .then(uploadWithResponse(Flux.just(ByteBuffer.wrap(data.toBytes())), null, null, null, null,


data.toBytes() will perform an array copy, which may be fine for now, but do we want to look into another API such as toReadOnlyByteBuffer()? @srnagar

Yeah. I was disappointed to not find a way to avoid this copy. It would be great improvement.

@alzimmermsft We should look into adding data.toFlux() method in BinaryData as we discussed offline to prevent array copy.

alzimmermsft · 2021-03-05T19:11:17Z

...azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobAsyncClientBase.java

+     *
+     * @return A reactive response containing the blob data.
+     */
+    @ServiceMethod(returns = ReturnType.COLLECTION)


nit: ReturnType.SINGLE

alzimmermsft · 2021-03-05T19:13:15Z

...rage/azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobClientBase.java

+     * @throws UncheckedIOException If an I/O error occurs.
+     */
+    @ServiceMethod(returns = ReturnType.SINGLE)
+    public BinaryData downloadContent() {


For the methods which return BinaryData do we want to document that these only support operations up to 2GB?

alzimmermsft · 2021-03-05T19:14:30Z

...storage-blob/src/samples/java/com/azure/storage/blob/BlobAsyncClientJavaDocCodeSnippets.java

+     */
+    public void downloadContentCodeSnippet() {
+        // BEGIN: com.azure.storage.blob.BlobAsyncClient.downloadContent
+        ByteArrayOutputStream downloadData = new ByteArrayOutputStream();


This doesn't appear to be used, can it be removed?

alzimmermsft · 2021-03-05T19:15:40Z

...zure-storage-blob/src/samples/java/com/azure/storage/blob/BlobClientJavaDocCodeSnippets.java

+    public void downloadContent() {
+        // BEGIN: com.azure.storage.blob.BlobClient.downloadContent
+        BinaryData data = client.downloadContent();
+        System.out.println("Download completed.");


Do we want to print the data.toString() value as we do in the async sample?

alzimmermsft · 2021-03-05T19:17:26Z

...b/src/samples/java/com/azure/storage/blob/specialized/BlobClientBaseJavaDocCodeSnippets.java

+    public void downloadStreaming() {
+        // BEGIN: com.azure.storage.blob.specialized.BlobClientBase.downloadStreaming#OutputStream
+        client.downloadStreaming(new ByteArrayOutputStream());
+        System.out.println("Download completed.");


Same question about printing the downloaded data

not for this one. we don't assume that bytes are string in this example nor we want to hint customer that they should be using this api to handle text blobs.

srnagar · 2021-03-05T18:58:28Z

...azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobAsyncClientBase.java

@@ -846,6 +868,29 @@ String tagsToString(Map<String, String> tags) {
        }
    }

+    /**
+     * Reads the entire blob. Uploading data must be done from the {@link BlockBlobClient}, {@link PageBlobClient}, or


The javadoc for this method should specify the limitation here that the blob size should not exceed 2GB and suggest using the overload that returns Flux<ByteBuffer> instead.

srnagar · 2021-03-05T19:21:33Z

...rage/azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobClientBase.java

+     * @throws NullPointerException if {@code stream} is null
+     */
+    @ServiceMethod(returns = ReturnType.SINGLE)
+    public BlobDownloadResponse downloadStreamingWithResponse(OutputStream stream, BlobRange range,


downloadStreamWithResponse?

srnagar · 2021-03-05T19:41:45Z

sdk/storage/azure-storage-blob/src/main/java/com/azure/storage/blob/BlobAsyncClient.java

+            }
+
+            return overwriteCheck
+                .then(uploadWithResponse(Flux.just(ByteBuffer.wrap(data.toBytes())), null, null, null, null,


@alzimmermsft We should look into adding data.toFlux() method in BinaryData as we discussed offline to prevent array copy.

srnagar · 2021-03-05T19:45:11Z

...rage/azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobClientBase.java

+     * @throws NullPointerException if {@code stream} is null
+     */
+    @ServiceMethod(returns = ReturnType.SINGLE)
+    public void downloadStreaming(OutputStream stream) {


downloadStream

srnagar · 2021-03-05T19:50:05Z

...azure-storage-blob/src/main/java/com/azure/storage/blob/specialized/BlobAsyncClientBase.java

+     * @return A reactive response containing the blob data.
+     */
+    @ServiceMethod(returns = ReturnType.SINGLE)
+    public Mono<BlobDownloadAsyncResponse> downloadFluxWithResponse(BlobRange range, DownloadRetryOptions options,


downloadStream seems like a better name to use in both sync and async clients. It lets the user know that the response is going to be streamed either through Flux or through OutputStream. So, we name this based on the concept of stream rather than the actual Java types that are used to represent the stream.

modify readme.typescripy.md for compute package (Azure#19607) Co-authored-by: ZiWei Chen (WICRESOFT NORTH AMERICA LTD) <[email protected]>

Compute update folder structure (Azure#19723) * update folder structure * small errors and CI check * [Language Text] Update swagger titles (Azure#19835) * [Language Text] Update swagger titles * edits * address feedback * Review request for Microsoft.ContainerService to add version 2022-06-01 (Azure#19848) * Adds base for updating Microsoft.ContainerService from version stable/2022-04-01 to version 2022-06-01 * Updates readme * Updates API version in new specs and examples * update readmes (Azure#19421) * Add key management service profile of a managed cluster for version 2022-06-01 (Azure#19529) * Add NetworkPlugin none option to 2022-06-01 (Azure#19510) * Add NetworkPlugin none option to 2022-06-01 * improve description for none value * Fix violated rule R4041 for 2022-06-01 managedCluster swagger (Azure#19581) * remove useless directive * fix R4041 * GA AKS support for Dedicated Host Group (Azure#19547) * GA AKS support for Dedicated Host Group * fit some format minor issue * add newline in end of files * Defender updates (Azure#19665) * Defender updates * sample * remove old description Co-authored-by: Bin Xia <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Jianping Zeng <[email protected]> Co-authored-by: Or Parnes <[email protected]> * Update resources.json (Azure#19861) * Pattts/predictive autoscale 20221001 (Azure#19765) * Rev Autoscale API from 2021-05-01-preview to 2022-10-01. * Fix minor swagger issue. Default should be 'false' * Fix swagger validation error * Defining a new package 2022-10 * AutoscaleSettingResource now reference allOf 'resource' which is the actual autoscaleSetting resource definition * Fix require property issue * Run prettier against autoscale_API.json * [Maps - Render & Spatial Services] add v2022-08-01 (Azure#19520) * copy old swagger to new folders * set api version to 2022-08-01 * apply api changes Co-authored-by: Gigi Grajo <[email protected]> * update appconfiguration (Azure#19330) * update appconfiguration * Update readme.python.md * Update readme.python.md * Update readme.python.md * Update readme.python.md * Update readme.python.md * Update readme.python.md * Update readme.python.md * [Hub Generated] Publish private branch 'main' (Azure#19852) * Add StorageMover specification for PubliC Preview * SpellCheck fix * Update custom-words * Update specification/storagemover/resource-manager/readme.md Co-authored-by: Abhishek Krishna <[email protected]> Co-authored-by: Dapeng Zhang <[email protected]> * Merging Dynatrace swagger in stable folder (Azure#19862) * Committing base swagger version * Removing preview tag from stable folder api version * Changing version name in all the files * Updating readme.go and readme.md files * [CDN] Fix customDomains property type in Endpoint (Azure#19788) * [CDN] Fix customDomains property type in Endpoint * Fix linter error Co-authored-by: Bo Zhang <[email protected]> * Adding Microsoft.Sql AdvancedThreatProtectionSettings APIs for MI on v5 tag for 2022-02-01-preview (Azure#19866) * update folders * [Hub Generated] Review request for Microsoft.KeyVault to add version stable/7.3 (Azure#19844) * Update description of exportable attribute and release_policy.data * Remove submodule azure-reset-api-specs * Fix description for exportable Co-authored-by: Sunny Solanki <[email protected]> * Add latest StorageMover updates (Azure#19876) * Add StorageMover specification for PubliC Preview * SpellCheck fix * Update custom-words * Update specification/storagemover/resource-manager/readme.md * Add latest StorageMover updates Co-authored-by: Abhishek Krishna <[email protected]> Co-authored-by: Dapeng Zhang <[email protected]> * Update readme.python.md (Azure#19899) * Update readme.python.md * Update readme.md * Moving files from azure-rest-spec-pr to azure-rest-spec repo after api review (Azure#19878) * Moving files from azure-rest-spec-pr to azure-rest-spec repo after api review * Add known words * Removed static IP allocation only from examples (Azure#19858) * Removed static IP allocation * Removed static only from examples Co-authored-by: Arpit Gagneja <[email protected]> * Update securityinsights readme.python (Azure#19903) * update securityinsights readme.python * Update readme.python.md Co-authored-by: Zhenbiao Wei (WICRESOFT NORTH AMERICA LTD) <[email protected]> * fix lint errors in Synapse trigger.json (Azure#19660) * put json files into RP folders * update readme * Add CONTRIBUTING.md (Azure#19257) * Add CONTRIBUTING.md * Apply suggestions from PR review Co-authored-by: Heath Stewart <[email protected]> * Apply suggestions from PR review Co-authored-by: Weidong Xu <[email protected]> * Regen toc for CONTRIBUTING.md * Address PR review comments * Address PR review comments Co-authored-by: Heath Stewart <[email protected]> Co-authored-by: Weidong Xu <[email protected]> * fix devcenter readme.go.md config (Azure#19906) * fix readme.go.md config * rename to SkuInfo * add annotation for labservices (Azure#19884) * add to description of OS state (Azure#19764) * [Hub Generated] Review request for Microsoft.KeyVault to add version preview/2021-06-01-preview (Azure#19767) * Updated the managed hsm resource manager spec to include two additional properties for private endpoint connection item * Update managed hsm private endpoint connection item in mhsm spec for latest api version * Address LRO_RESPONSE_HEADER violation for managed hsm Long running operations that are annotated with x-ms-long-running-operation:true must return location header or azure-AsyncOperation in response. Added the missing location header for managed hsm update command as well as to the corresponding examples. * [Microsoft.DeviceUpdate] Adding first stable API version (Azure#19846) * Copied the most recent preview version into the first stable version * Updated api version in new stable version, added the tag to readme.md * Fixed typo * Fixed the wrong path in readme.md * Reverted VS Code automatic breaking change * Fix broken link (Azure#19688) * add aadObjectId property to kustoPool (Azure#19856) Co-authored-by: Amit Elran <[email protected]> * Updata securityinsights readme.python (Azure#19917) * update securityinsights readme.python * update readme.python Co-authored-by: Zhenbiao Wei (WICRESOFT NORTH AMERICA LTD) <[email protected]> * modify readme.typescripy.md for compute package (Azure#19607) Co-authored-by: ZiWei Chen (WICRESOFT NORTH AMERICA LTD) <[email protected]> * Dev gubalasu frontdoor microsoft.network 2021 06 01 (Azure#19578) * Adds base for updating Microsoft.Network from version stable/2020-05-01 to version 2021-06-01 * Updates readme * Updates API version in new specs and examples * update waf configs and examples from 2020-11-01 * Update api version in new specs and examples * Migration api update * Update readme * Fix readme * update readme * Fix readme again * Fix readme * Fix readme * update waf policy provisioning state * Update securityinsights readme.python (Azure#19923) * update securityinsights readme.python * update readme.python * last modify * Update readme.python.md Co-authored-by: Zhenbiao Wei (WICRESOFT NORTH AMERICA LTD) <[email protected]> * sync with changes made to master * update folder structure * small errors and CI check * update folders * put json files into RP folders * update readme * resolve capitalization and number misplacement accident Co-authored-by: Theodore Chang <[email protected]> Co-authored-by: Deyaaeldeen Almahallawi <[email protected]> Co-authored-by: FumingZhang <[email protected]> Co-authored-by: Bin Xia <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Jianping Zeng <[email protected]> Co-authored-by: Or Parnes <[email protected]> Co-authored-by: Alexander Batishchev <[email protected]> Co-authored-by: PatrickTseng <[email protected]> Co-authored-by: gigi <[email protected]> Co-authored-by: Gigi Grajo <[email protected]> Co-authored-by: zhenbiao wei <[email protected]> Co-authored-by: Abhishek Krishna <[email protected]> Co-authored-by: Abhishek Krishna <[email protected]> Co-authored-by: Dapeng Zhang <[email protected]> Co-authored-by: Divyansh Agarwal <[email protected]> Co-authored-by: t-bzhan <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: Uriel Cohen <[email protected]> Co-authored-by: susolank <[email protected]> Co-authored-by: Sunny Solanki <[email protected]> Co-authored-by: Ralf Beckers <[email protected]> Co-authored-by: arpit-gagneja <[email protected]> Co-authored-by: Arpit Gagneja <[email protected]> Co-authored-by: Zhenbiao Wei (WICRESOFT NORTH AMERICA LTD) <[email protected]> Co-authored-by: YanjunGao <[email protected]> Co-authored-by: Mike Kistler <[email protected]> Co-authored-by: Heath Stewart <[email protected]> Co-authored-by: Weidong Xu <[email protected]> Co-authored-by: Jiahui Peng <[email protected]> Co-authored-by: j-zhong-ms <[email protected]> Co-authored-by: Tom FitzMacken <[email protected]> Co-authored-by: neeerajaakula <[email protected]> Co-authored-by: darkoa-msft <[email protected]> Co-authored-by: Roy Wellington <[email protected]> Co-authored-by: Amit Elran <[email protected]> Co-authored-by: Amit Elran <[email protected]> Co-authored-by: kazrael2119 <[email protected]> Co-authored-by: ZiWei Chen (WICRESOFT NORTH AMERICA LTD) <[email protected]> Co-authored-by: gubalasu <[email protected]>

kasobol-msft added 3 commits February 17, 2021 15:57

public area.

099db5b

more apis.

86f24f6

wip

ff8fa46

ghost added the Storage Storage Service (Queues, Blobs, Files) label Mar 4, 2021

kasobol-msft added 3 commits March 3, 2021 22:55

fix build.

3799686

tests

b1d0ecc

readme.

3fe671a

kasobol-msft marked this pull request as ready for review March 4, 2021 17:27

kasobol-msft requested review from alzimmermsft, amishra-dev, gapra-msft, jaschrep-msft and rickle-msft as code owners March 4, 2021 17:27

gapra-msft reviewed Mar 4, 2021

View reviewed changes

typo.

a48acec

gapra-msft approved these changes Mar 4, 2021

View reviewed changes

kasobol-msft added 3 commits March 4, 2021 12:00

readme changelog.

53513ed

Merge remote-tracking branch 'upstream/master' into binary-data-blobs

b737f16

Merge remote-tracking branch 'upstream/master' into binary-data-blobs

f2fdcd7

alzimmermsft reviewed Mar 5, 2021

View reviewed changes

alzimmermsft approved these changes Mar 5, 2021

View reviewed changes

srnagar approved these changes Mar 5, 2021

View reviewed changes

srnagar mentioned this pull request Mar 5, 2021

Add support to get Flux<ByteBuffer> from BinaryData #19665

Closed

pr feedback.

9a5106b

kasobol-msft merged commit 7e1cadb into Azure:master Mar 5, 2021

kasobol-msft mentioned this pull request Mar 11, 2021

[Storage] Use BinaryData.toBuffer in blobs SDK to avoid array copy. #19799

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Storage] Binary data blobs #19607

[Storage] Binary data blobs #19607

kasobol-msft commented Mar 4, 2021 •

edited

Loading

check-enforcer bot commented Mar 4, 2021

gapra-msft Mar 4, 2021

alzimmermsft Mar 5, 2021

alzimmermsft Mar 5, 2021

kasobol-msft Mar 5, 2021

srnagar Mar 5, 2021

alzimmermsft Mar 5, 2021

alzimmermsft Mar 5, 2021

alzimmermsft Mar 5, 2021

alzimmermsft Mar 5, 2021

alzimmermsft Mar 5, 2021

kasobol-msft Mar 5, 2021

srnagar Mar 5, 2021 •

edited

Loading

srnagar Mar 5, 2021

srnagar Mar 5, 2021

srnagar Mar 5, 2021

srnagar Mar 5, 2021

[Storage] Binary data blobs #19607

[Storage] Binary data blobs #19607

Conversation

kasobol-msft commented Mar 4, 2021 • edited Loading

check-enforcer bot commented Mar 4, 2021

What is Check Enforcer?

Why am I getting this message?

What should I do now?

What if I am onboarding a new service?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srnagar Mar 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kasobol-msft commented Mar 4, 2021 •

edited

Loading

srnagar Mar 5, 2021 •

edited

Loading