Skip to content

Commit

Permalink
Remove resources post test (#13379)
Browse files Browse the repository at this point in the history
* pruning tests.yml

* potential fix

* updated environment variables

* another change to tests.yml

* change

* edits to testcase

* setting storage_name and rg_kwargs to None to help identify problem

* forcing i_need_to_create_rg to be True

* flirting with version change

* reverting back to normal for both testcase and version

* Packaging update of azure-data-tables

* adding azure identity to dev reqs

* adding azure-identity to dev_reqs

* mirror updates from personal branch

* adding continue on error so we can see the env

* update shared test case rg_kwargs resource to mapping instead of NoneType

* updating dev_reqs

* random issues gone?

* build-test changes

* undoing all the bot changes from AutorestCI

* edits thanks to Kristas suggestion, hopefully stops AutorestCI from sabotage

* import directly from file, python2 can't find the module

* pylint fixes

* debugging why python 2.7 can't find azure data tables

* undoing last change

* reverting setup.py to master branch, adding space in inits just incase

* add comprehension of nspkg to azure-data-tables

* adding nspkg to shared_reqs

* updating shared_req to freeze nspkg at the same version

Co-authored-by: Azure SDK Bot <[email protected]>
Co-authored-by: Scott Beddall <[email protected]>
Co-authored-by: scbedd <[email protected]>
  • Loading branch information
4 people authored Sep 1, 2020
1 parent 2ef48bc commit e9bd025
Show file tree
Hide file tree
Showing 8 changed files with 40 additions and 61 deletions.
3 changes: 2 additions & 1 deletion sdk/tables/azure-data-tables/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
recursive-include tests *.py *.yaml
include *.md
include azure/__init__.py
include azure/data/__init__.py
include LICENSE.txt
recursive-include tests *.py
recursive-include samples *.py *.md
recursive-include samples *.py *.md
41 changes: 15 additions & 26 deletions sdk/tables/azure-data-tables/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
# Azure Data Tables client library for Python

Azure Data Tables is a NoSQL data storing service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.
Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
Azure Data Tables is a NoSQL data storing service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.
Tables scale as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
The Azure Data Tables client can be used to access Azure Storage or Cosmos accounts.

Common uses of Azure Data Tables include:

* Storing structured data in the form of tables
# Usage
* Storing structured data in the form of tables # Usage
* Quickly querying data using a clustered index

[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) | [Package (PyPI)](https://pypi.org) | [API reference documentation](https://aka.ms/azsdk/python/tables/docs) | [Product documentation](https://docs.microsoft.com/azure/storage/) | [Samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk)
For code examples, see [MyService Management](https://docs.microsoft.com/python/api/overview/azure/)
on docs.microsoft.com.

## Getting started

Expand Down Expand Up @@ -38,7 +39,6 @@ or [Azure CLI](https://docs.microsoft.com/azure/storage/common/storage-quickstar
# Create a new resource group to hold the storage account -
# if using an existing resource group, skip this step
az group create --name MyResourceGroup --location westus2

# Create the storage account
az storage account create -n mystorageaccount -g MyResourceGroup
```
Expand All @@ -51,12 +51,11 @@ you to access the account:

```python
from azure.data.tables import TableServiceClient

service = TableServiceClient(account_url="https://<myaccount>.table.core.windows.net/", credential=credential)
```

#### Looking up the account URL
You can find the account's table service URL using the
You can find the account's table service URL using the
[Azure Portal](https://docs.microsoft.com/azure/storage/common/storage-account-overview#storage-account-endpoints),
[Azure PowerShell](https://docs.microsoft.com/powershell/module/az.storage/get-azstorageaccount),
or [Azure CLI](https://docs.microsoft.com/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-show):
Expand All @@ -77,24 +76,23 @@ The `credential` parameter may be provided in a number of different forms, depen
```python
from datetime import datetime, timedelta
from azure.data.tables import TableServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions

sas_token = generate_account_sas(
account_name="<account-name>",
account_key="<account-access-key>",
resource_types=ResourceTypes(service=True),
permission=AccountSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=1)
)

table_service_client = TableServiceClient(account_url="https://<my_account_name>.table.core.windows.net", credential=sas_token)
```

2. To use an account [shared key](https://docs.microsoft.com/rest/api/storageservices/authenticate-with-shared-key/)
(aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys"
(aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys"
section or by running the following Azure CLI command:

```az storage account keys list -g MyResourceGroup -n mystorageaccount```

Use the key as the credential parameter to authenticate the client:
```python
from azure.data.tables import TableServiceClient
Expand All @@ -103,12 +101,11 @@ The `credential` parameter may be provided in a number of different forms, depen

#### Creating the client from a connection string
Depending on your use case and authorization method, you may prefer to initialize a client instance with a
connection string instead of providing the account URL and credential separately. To do this, pass the
connection string instead of providing the account URL and credential separately. To do this, pass the
connection string to the client's `from_connection_string` class method:

```python
from azure.data.tables import TableServiceClient

connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net"
service = TableServiceClient.from_connection_string(conn_str=connection_string)
```
Expand Down Expand Up @@ -139,14 +136,14 @@ Two different clients are provided to to interact with the various components of
this client represents interaction with a specific table (which need not exist yet). It provides operations to
create, delete, or update a table and includes operations to query, get, and upsert entities
within it.

### Entities
* **Create** - Adds an entity to the table.
* **Delete** - Deletes an entity from the table.
* **Update** - Updates an entities information by either merging or replacing the existing entity.
* **Query** - Queries existing entities in a table based off of the QueryOptions (OData).
* **Get** - Gets a specific entity from a table by partition and row key.
* **Upsert** - Merges or replaces an entity in a table, or if the entity does not exist, inserts the entity.
* **Upsert** - Merges or replaces an entity in a table, or if the entity does not exist, inserts the entity.

## Examples

Expand All @@ -162,7 +159,6 @@ Create a table in your account

```python
from azure.data.tables import TableServiceClient

table_service_client = TableServiceClient.from_connection_string(conn_str="<connection_string>")
table_service_client.create_table(table_name="myTable")
```
Expand All @@ -172,9 +168,7 @@ Create entities in the table

```python
from azure.data.tables import TableClient

my_entity = {'PartitionKey':'part','RowKey':'row'}

table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="myTable")
entity = table_client.create_entity(entity=my_entity)
```
Expand All @@ -184,9 +178,7 @@ Querying entities in the table

```python
from azure.data.tables import TableClient

my_filter = "text eq Marker"

table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="mytable")
entity = table_client.query_entities(filter=my_filter)
```
Expand Down Expand Up @@ -246,15 +238,12 @@ headers, can be enabled on a client with the `logging_enable` argument:
import sys
import logging
from azure.data.tables import TableServiceClient

# Create a logger for the 'azure.data.tables' SDK
logger = logging.getLogger('azure.data.tables')
logger.setLevel(logging.DEBUG)

# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)

# This client will log detailed information about its HTTP sessions, at DEBUG level
service_client = TableServiceClient.from_connection_string("your_connection_string", logging_enable=True)
```
Expand Down Expand Up @@ -287,7 +276,7 @@ Several Azure Data Tables Python SDK samples are available to you in the SDK's G
* Query entities
* Update entities
* Upsert entities

### Additional documentation
For more extensive documentation on Azure Data Tables, see the [Azure Data Tables documentation](https://docs.microsoft.com/azure/storage/tables/) on docs.microsoft.com.

Expand All @@ -296,4 +285,4 @@ This project welcomes contributions and suggestions. Most contributions require

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
3 changes: 2 additions & 1 deletion sdk/tables/azure-data-tables/dev_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@
../../core/azure-core
cryptography>=2.1.4
aiohttp>=3.0; python_version >= '3.5'

azure-identity
../azure-data-nspkg
2 changes: 2 additions & 0 deletions sdk/tables/azure-data-tables/sdk_packaging.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[packaging]
auto_update = false
5 changes: 3 additions & 2 deletions sdk/tables/azure-data-tables/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,16 @@
# Exclude packages that will be covered by PEP420 or nspkg
'azure',
'tests',
'azure.data',
]),
install_requires=[
"azure-core<2.0.0,>=1.2.2",
"msrest>=0.6.10"
# azure-data-tables
],
extras_require={
":python_version<'3.0'": ['futures'],
":python_version<'3.0'": ['futures', 'azure-data-nspkg<2.0.0,>=1.0.0'],
":python_version<'3.4'": ['enum34>=1.0.4'],
":python_version<'3.5'": ["typing"]
},
)
)
19 changes: 10 additions & 9 deletions sdk/tables/azure-data-tables/tests/_shared/testcase.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@
import time
from datetime import datetime, timedelta

from azure.data.tables import ResourceTypes, AccountSasPermissions
from azure.data.tables._table_shared_access_signature import generate_account_sas

try:
import unittest.mock as mock
except ImportError:
Expand All @@ -42,8 +39,8 @@
from io import StringIO

from azure.core.credentials import AccessToken
#from azure.data.tabless import generate_account_sas, AccountSasPermissions, ResourceTypes
from azure.mgmt.storage.models import StorageAccount, Endpoints
from azure.data.tables import generate_account_sas, AccountSasPermissions, ResourceTypes

try:
from devtools_testutils import mgmt_settings_real as settings
Expand Down Expand Up @@ -351,6 +348,9 @@ def storage_account():
i_need_to_create_rg = not (existing_rg_name or existing_storage_name or storage_connection_string)
got_storage_info_from_env = existing_storage_name or storage_connection_string

storage_name = None
rg_kwargs = {}

try:
if i_need_to_create_rg:
rg_name, rg_kwargs = rg_preparer._prepare_create_resource(test_case)
Expand Down Expand Up @@ -431,11 +431,12 @@ def build_service_endpoint(service):
TableTestCase._STORAGE_CONNECTION_STRING = storage_connection_string
yield
finally:
if not got_storage_info_from_env:
storage_preparer.remove_resource(
storage_name,
resource_group=rg
)
if storage_name is not None:
if not got_storage_info_from_env:
storage_preparer.remove_resource(
storage_name,
resource_group=rg
)
finally:
if i_need_to_create_rg:
rg_preparer.remove_resource(rg_name)
Expand Down
27 changes: 5 additions & 22 deletions sdk/tables/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,12 @@ jobs:
parameters:
BuildTargetingString: azure-data-tables
ServiceDirectory: tables
AllocateResourceGroup: 'false'
EnvVars:
STORAGE_ACCOUNT_NAME: $(python-storage-storage-account-name)
STORAGE_ACCOUNT_KEY: $(python-storage-storage-account-key)
STORAGE_DATA_LAKE_ACCOUNT_NAME: $(python-storage-data-lake-account-name)
STORAGE_DATA_LAKE_ACCOUNT_KEY: $(python-storage-data-lake-account-key)
BLOB_STORAGE_ACCOUNT_NAME: $(python-storage-blob-storage-account-name)
BLOB_STORAGE_ACCOUNT_KEY: $(python-storage-blob-storage-account-key)
REMOTE_STORAGE_ACCOUNT_NAME: $(python-storage-remote-storage-account-name)
REMOTE_STORAGE_ACCOUNT_KEY: $(python-storage-remote-storage-account-key)
PREMIUM_STORAGE_ACCOUNT_NAME: $(python-storage-premium-storage-account-name)
PREMIUM_STORAGE_ACCOUNT_KEY: $(python-storage-premium-storage-account-key)
OAUTH_STORAGE_ACCOUNT_NAME: $(python-storage-oauth-storage-account-name)
OAUTH_STORAGE_ACCOUNT_KEY: $(python-storage-oauth-storage-account-key)
ACTIVE_DIRECTORY_APPLICATION_ID: $(aad-azure-sdk-test-client-id)
ACTIVE_DIRECTORY_APPLICATION_SECRET: $(aad-azure-sdk-test-client-secret)
ACTIVE_DIRECTORY_TENANT_ID: $(aad-azure-sdk-test-tenant-id)
CONNECTION_STRING: $(python-storage-blob-connection-string)
BLOB_CONNECTION_STRING: $(python-storage-blob-connection-string)
PREMIUM_CONNECTION_STRING: $(python-storage-premium-connection-string)
TEST_MODE: 'RunLiveNoRecord'
AZURE_SKIP_LIVE_RECORDING: 'True'
AZURE_TEST_RUN_LIVE: 'true'
AZURE_TENANT_ID: $(aad-azure-sdk-test-tenant-id)
AZURE_SUBSCRIPTION_ID: $(azure-subscription-id)
AZURE_CLIENT_SECRET: $(aad-azure-sdk-test-client-secret)
AZURE_CLIENT_ID: $(aad-azure-sdk-test-client-id)
AZURE_CLIENT_SECRET: $(aad-azure-sdk-test-client-secret)
TEST_MODE: 'RunLiveNoRecord'
AZURE_SKIP_LIVE_RECORDING: 'True'
AZURE_TEST_RUN_LIVE: 'true'
1 change: 1 addition & 0 deletions shared_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ azure-common~=1.1
azure-core<2.0.0,>=1.2.2
azure-cosmosdb-table~=1.0
azure-datalake-store~=0.0.18
azure-data-nspkg<2.0.0,>=1.0.0
azure-eventhub<6.0.0,>=5.0.0
azure-eventgrid~=1.1
azure-graphrbac~=0.40.0
Expand Down

0 comments on commit e9bd025

Please sign in to comment.