Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AzDatalake] Lease Clients Implementation #21297

Merged
merged 61 commits into from
Aug 1, 2023
Merged
Changes from 1 commit
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
e8167a2
Enable gocritic during linting (#20715)
jhendrixMSFT Apr 28, 2023
86627ae
Cosmos DB: Enable merge support (#20716)
ealsur Apr 28, 2023
8ac8c6d
[azservicebus, azeventhubs] Stress test and logging improvement (#20710)
richardpark-msft May 1, 2023
9111616
update proxy version (#20712)
azure-sdk May 1, 2023
d6bf190
Return an error when you try to send a message that's too large. (#20…
richardpark-msft May 1, 2023
e2693bd
Changes in test that is failing in pipeline (#20693)
siminsavani-msft May 2, 2023
03f0ac3
[azservicebus, azeventhubs] Treat 'entity full' as a fatal error (#20…
richardpark-msft May 2, 2023
838842d
[azservicebus/azeventhubs] Redirect stderr and stdout to tee (#20726)
richardpark-msft May 3, 2023
20b4dd8
Update changelog with latest features (#20730)
jhendrixMSFT May 3, 2023
745d967
pass along the artifact name so we can override it later (#20732)
azure-sdk May 3, 2023
6dfd0cb
[azeventhubs] Fixing checkpoint store race condition (#20727)
richardpark-msft May 3, 2023
ed7f3c7
Fix azidentity troubleshooting guide link (#20736)
chlowell May 3, 2023
b2cddab
[Release] sdk/resourcemanager/paloaltonetworksngfw/armpanngfw/0.1.0 (…
Alancere May 4, 2023
2a8d96d
add sdk/resourcemanager/postgresql/armpostgresql live test (#20685)
Alancere May 4, 2023
0d22aed
add sdk/resourcemanager/eventhub/armeventhub live test (#20686)
Alancere May 4, 2023
5fa7df4
add sdk/resourcemanager/compute/armcompute live test (#20048)
Alancere May 4, 2023
c005ed6
sdk/resourcemanager/network/armnetwork live test (#20331)
Alancere May 4, 2023
36f766d
add sdk/resourcemanager/cosmos/armcosmos live test (#20705)
Alancere May 4, 2023
9c9d62a
Increment package version after release of azcore (#20740)
azure-sdk May 4, 2023
8bc3450
[azeventhubs] Improperly resetting etag in the checkpoint store (#20737)
richardpark-msft May 4, 2023
e1a6152
Eng workflows sync and branch cleanup additions (#20743)
azure-sdk May 4, 2023
04b463d
[azeventhubs] Latest start position can also be inclusive (ie, get th…
richardpark-msft May 4, 2023
8849196
Update GitHubEventProcessor version and remove pull_request_review pr…
azure-sdk May 5, 2023
27f5ee0
Rename DisableAuthorityValidationAndInstanceDiscovery (#20746)
chlowell May 5, 2023
2eec707
fix (#20707)
Alancere May 6, 2023
22db2d4
AzFile (#20739)
souravgupta-msft May 8, 2023
0cbfd88
azfile: Fixing connection string parsing logic (#20798)
souravgupta-msft May 8, 2023
d54fb08
[azadmin] fix flaky test (#20758)
gracewilcox May 8, 2023
ad8ebd9
Prepare azidentity v1.3.0 for release (#20756)
chlowell May 8, 2023
e2a6f70
Fix broken podman link (#20801)
azure-sdk May 8, 2023
a59d912
[azquery] update doc comments (#20755)
gracewilcox May 8, 2023
bd3b467
Fixed contribution section (#20752)
bobtabor-msft May 8, 2023
132a01a
[azeventhubs,azservicebus] Some API cleanup, renames (#20754)
richardpark-msft May 8, 2023
8db51ca
Add supporting features to enable distributed tracing (#20301) (#20708)
jhendrixMSFT May 9, 2023
4a66b4f
Restore ARM CAE support for azcore beta (#20657)
chlowell May 9, 2023
7d4a3cb
Upgrade to stable azcore (#20808)
chlowell May 9, 2023
068c3be
Increment package version after release of data/azcosmos (#20807)
azure-sdk May 9, 2023
8e0f66e
Updating changelog (#20810)
souravgupta-msft May 9, 2023
ce926c4
Add fake package to azcore (#20711)
jhendrixMSFT May 9, 2023
1a145c5
Updating CHANGELOG.md (#20809)
siminsavani-msft May 9, 2023
90dfc5c
changelog (#20811)
tasherif-msft May 9, 2023
c7eda59
Increment package version after release of storage/azfile (#20813)
azure-sdk May 9, 2023
7fac0b5
Update changelog (azblob) (#20815)
siminsavani-msft May 9, 2023
498a2ef
[azquery] migration guide (#20742)
gracewilcox May 9, 2023
ccb967e
Increment package version after release of monitor/azquery (#20820)
azure-sdk May 9, 2023
f4e6a22
[keyvault] prep for release (#20819)
gracewilcox May 10, 2023
8fd8eda
Merge branch 'main' into feature/azdatalake
tasherif-msft May 11, 2023
c94fa00
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft May 11, 2023
fc0b2b5
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jun 12, 2023
6fb1694
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jun 19, 2023
4f7fe43
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jun 26, 2023
3dac9d0
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 4, 2023
a0a861b
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 7, 2023
124e27e
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 19, 2023
0f5a52c
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 24, 2023
81dabb1
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 27, 2023
d87e78b
Merge remote-tracking branch 'upstream/feature/azdatalake' into featu…
tasherif-msft Jul 31, 2023
450a48e
lease implementation and tests
tasherif-msft Jul 31, 2023
e8a41ef
tests
tasherif-msft Jul 31, 2023
c2891f5
handled errors
tasherif-msft Jul 31, 2023
767cbd9
nit
tasherif-msft Aug 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
azfile: Fixing connection string parsing logic (#20798)
* Fixing connection string parse logic

* Update README
  • Loading branch information
souravgupta-msft authored May 8, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 0cbfd88756304b154dc877a50f721b7ec454a0e8
6 changes: 3 additions & 3 deletions sdk/storage/azfile/README.md
Original file line number Diff line number Diff line change
@@ -245,7 +245,7 @@ or contact [[email protected]][coc_contact] with any
additional questions or comments.

<!-- LINKS -->
[source]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage
[source]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage/azfile
[docs]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/storage/azfile
[rest_docs]: https://docs.microsoft.com/rest/api/storageservices/file-service-rest-api
[product_docs]: https://docs.microsoft.com/azure/storage/files/storage-files-introduction
@@ -257,8 +257,8 @@ additional questions or comments.
[storage_account_create_portal]: https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal
[azure_sub]: https://azure.microsoft.com/free/
[azcore_response_error]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azcore#ResponseError
[file_error]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage
[samples]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/storage
[file_error]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage/azfile/fileerror/error_codes.go
[samples]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/storage/azfile/file/examples_test.go
[storage_contrib]: https://github.com/Azure/azure-sdk-for-go/blob/main/CONTRIBUTING.md
[cla]: https://cla.microsoft.com
[coc]: https://opensource.microsoft.com/codeofconduct/
2 changes: 0 additions & 2 deletions sdk/storage/azfile/file/client_test.go
Original file line number Diff line number Diff line change
@@ -3117,5 +3117,3 @@ func (f *FileRecordedTestsSuite) TestFileForceCloseHandlesDefault() {
}

// TODO: Add tests for retry header options

// TODO: fix links in README: source, file_error, samples
2 changes: 1 addition & 1 deletion sdk/storage/azfile/file/models.go
Original file line number Diff line number Diff line change
@@ -732,7 +732,7 @@ func (u *UploadStreamOptions) getUploadRangeOptions() *UploadRangeOptions {
}
}

// URLParts object represents the components that make up an Azure Storage Container/Blob URL.
// URLParts object represents the components that make up an Azure Storage Share/Directory/File URL.
// NOTE: Changing any SAS-related field requires computing a new SAS signature.
type URLParts = sas.URLParts

33 changes: 25 additions & 8 deletions sdk/storage/azfile/internal/shared/shared.go
Original file line number Diff line number Diff line change
@@ -100,20 +100,37 @@ func ParseConnectionString(connectionString string) (ParsedConnectionString, err
connStrMap[parts[0]] = parts[1]
}

accountName, ok := connStrMap["AccountName"]
if !ok {
return ParsedConnectionString{}, errors.New("connection string missing AccountName")
}

accountName := connStrMap["AccountName"]
accountKey, ok := connStrMap["AccountKey"]
if !ok {
sharedAccessSignature, ok := connStrMap["SharedAccessSignature"]
if !ok {
return ParsedConnectionString{}, errors.New("connection string missing AccountKey and SharedAccessSignature")
}
return ParsedConnectionString{
ServiceURL: fmt.Sprintf("%v://%v.file.%v/?%v", defaultScheme, accountName, defaultSuffix, sharedAccessSignature),
}, nil

fileEndpoint, ok := connStrMap["FileEndpoint"]
if !ok {
// We don't have a FileEndpoint, assume the default
if accountName != "" {
return ParsedConnectionString{
ServiceURL: fmt.Sprintf("%v://%v.file.%v/?%v", defaultScheme, accountName, defaultSuffix, sharedAccessSignature),
}, nil
} else {
return ParsedConnectionString{}, errors.New("connection string missing AccountName")
}
} else {
if !strings.HasSuffix(fileEndpoint, "/") {
// add a trailing slash to be consistent with the portal
fileEndpoint += "/"
}
return ParsedConnectionString{
ServiceURL: fmt.Sprintf("%v?%v", fileEndpoint, sharedAccessSignature),
}, nil
}
} else {
if accountName == "" {
return ParsedConnectionString{}, errors.New("connection string missing AccountName")
}
}

protocol, ok := connStrMap["DefaultEndpointsProtocol"]
40 changes: 40 additions & 0 deletions sdk/storage/azfile/internal/shared/shared_test.go
Original file line number Diff line number Diff line change
@@ -7,6 +7,7 @@
package shared

import (
"fmt"
"testing"

"github.com/stretchr/testify/require"
@@ -93,3 +94,42 @@ func TestCParseConnectionStringAzurite(t *testing.T) {
require.Equal(t, "dummyaccountname", parsed.AccountName)
require.Equal(t, "secretkeykey", parsed.AccountKey)
}

func TestParseConnectionStringSASAndCustomDomain(t *testing.T) {
testData := []struct {
connectionStr string
parsedServiceURL string
parsedAccountName string
parsedAccountKey string
err error
}{
{
connectionStr: "AccountName=dummyaccountname;SharedAccessSignature=fakesharedaccesssignature;FileEndpoint=http://127.0.0.1:10000/dummyaccountname;",
parsedServiceURL: "http://127.0.0.1:10000/dummyaccountname/?fakesharedaccesssignature",
},
{
connectionStr: "BlobEndpoint=https://dummyaccountname.blob.core.windows.net/;FileEndpoint=https://dummyaccountname.file.core.windows.net/;SharedAccessSignature=fakesharedaccesssignature",
parsedServiceURL: "https://dummyaccountname.file.core.windows.net/?fakesharedaccesssignature",
},
{
connectionStr: "BlobEndpoint=https://dummyaccountname.blob.core.windows.net;FileEndpoint=https://dummyaccountname.file.core.windows.net;SharedAccessSignature=fakesharedaccesssignature",
parsedServiceURL: "https://dummyaccountname.file.core.windows.net/?fakesharedaccesssignature",
},
{
connectionStr: "SharedAccessSignature=fakesharedaccesssignature",
err: fmt.Errorf("connection string missing AccountName"),
},
{
connectionStr: "DefaultEndpointsProtocol=http;AccountKey=secretkeykey;EndpointSuffix=core.windows.net",
err: fmt.Errorf("connection string missing AccountName"),
},
}

for _, td := range testData {
parsed, err := ParseConnectionString(td.connectionStr)
require.Equal(t, td.err, err)
require.Equal(t, td.parsedServiceURL, parsed.ServiceURL)
require.Equal(t, td.parsedAccountName, parsed.AccountName)
require.Equal(t, td.parsedAccountKey, parsed.AccountKey)
}
}