Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python …
Browse files Browse the repository at this point in the history 
…into network_ga

* 'main' of https://github.com/Azure/azure-sdk-for-python: (63 commits)
  check only once whether artifact extension exists (Azure#29594)
  code and test (Azure#29530)
  fix: support data binding expression for resources.xxx (Azure#29559)
  code and test (Azure#29558)
  [ServiceBus] enable sovereign cloud testing (Azure#29202)
  [AutoRelease] t2-communication-2023-03-20-22792(can only be merged by SDK owner) (Azure#29453)
  [Identity] Allow configurable process timeouts (Azure#28290)
  Increment package version after release of azure-containerregistry (Azure#29582)
  Update optional (Azure#28435)
  [Monitor][Query] Add sample notebook for large queries (Azure#28148)
  Update CHANGELOG.md (Azure#29580)
  [ServiceBus] remove msrest (Azure#29324)
  Explicitly set PSNativeCommandArgumentPassing to Legacy for git push script (Azure#29554)
  [ML][Pipelines] Support queue settings in pipeline job (Azure#29209)
  add experimental decorator for datatransfer (Azure#29562)
  Featurestore/online store crud new pr (Azure#29546)
  switch tooling over to typespec (Azure#29501)
  set up base path for feature set in load function (Azure#29544)
  Address credential description in docstring (Azure#29549)
  [Monitor] Regen query and ingestion (Azure#29504)
  ...
  • Loading branch information
@iscai-msft
iscai-msft committed Mar 27, 2023
2 parents 1c157de + e1f4348 commit 7a46088
Show file tree
Hide file tree
Showing 2,169 changed files with 135,729 additions and 30,926 deletions.
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
View file
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -917,7 +917,7 @@
#/<NotInRepo>/ @ccmshowbackdevs

# ServiceLabel: %Monitor - Exporter %Service Attention
#/<NotInRepo>/ @@cijothomas @@reyang @@rajkumar-rangaraj @@TimothyMothra @@vishweshbankwar @@ramthi
#/<NotInRepo>/ @lzchen @jeremydvoss


# Management Plane
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ _azci_*.log
.artifacts
.tmp_whl_dir
.logs
.proxy

# tox environment folders
.tox/
Expand Down Expand Up @@ -147,4 +148,4 @@ sdk/cosmos/azure-cosmos/test/test_config.py
*_python.json

# temporary folder to refresh SDK with cadl
TempCadlFiles/
TempTypeSpecFiles/
22 changes: 21 additions & 1 deletion .vscode/cspell.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,25 @@
"whls"
],
"overrides": [
{
"filename": "doc/dev/test_proxy_migration_guide.md",
"words": [
"pytestmarkparametrize"
]
},
{
"filename": "tools/azure-sdk-tools/devtools_testutils/proxy_startup.py",
"words": [
"certifi",
"passenv"
]
},
{
"filename": "tools/azure-sdk-tools/tests/integration/test_proxy_startup.py",
"words": [
"spinup"
]
},
{
"filename": "sdk/remoterendering/**",
"words": [
Expand Down Expand Up @@ -603,7 +622,8 @@
"ierr",
"mymodel",
"heif",
"racwd"
"racwd",
"rrggbb"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion doc/dev/issues/resolve_issues_effectively.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Most issues can be classified into 3 categories. They are
## Usage error (For users)
If you are not familiar with the SDK usage of a service, you can find relevant examples in [this repo][sample repo] in most cases.

For some common errors, you can check [here](#Summary-Of-Common-Errors).
For some common errors, you can check [here](#summary-of-common-errors).

## Feature Request(For users)

Expand Down
18 changes: 18 additions & 0 deletions doc/dev/static_type_checking_cheat_sheet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,24 @@ def foo(
...
```

- Do use `Optional` if a parameter can be typed as `Any` or `None`. `Optional[Any]`, or `Union[Any, None]`, is __not__ equal to `Any`.

```python
from typing import Optional, Any

# Yes
def foo(
bar: Optional[Any] = None,
) -> None:
bar.append(1) # error caught at type checking time: Item "None" of "Optional[Any]" has no attribute "append"

# No
def foo(
bar: Any = None,
) -> None:
bar.append(1) # error caught at runtime: AttributeError: 'NoneType' object has no attribute 'append'
```

### Collections

- Do familiarize yourself with the supported operations of various abstract collections in the [collections.abc](https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes) docs.
Expand Down
51 changes: 9 additions & 42 deletions doc/dev/test_proxy_migration_guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Please refer to the [troubleshooting guide][troubleshooting] if you have any iss
- [Advanced details](#advanced-details)
- [What does the test proxy do?](#what-does-the-test-proxy-do)
- [How does the test proxy know when and what to record or play back?](#how-does-the-test-proxy-know-when-and-what-to-record-or-play-back)
- [Start the proxy manually](#start-the-proxy-manually)
- [Use pytest.mark.parametrize with migrated tests](#use-pytestmarkparametrize-with-migrated-tests)

## Update existing tests

Expand Down Expand Up @@ -96,10 +96,7 @@ Resource preparers need a management client to function, so test classes that us

### Perform one-time setup

1. Docker (or Podman) is a requirement for using the test proxy. You can install Docker from [docs.docker.com][docker_install], or install Podman at [podman.io][podman]. To use Podman, set an alias for `podman` to replace the `docker` command.
2. After installing, make sure Docker/Podman is running and is using Linux containers before running tests.
3. Follow the instructions [here][proxy_cert_docs] to complete setup. You need to trust a certificate on your machine in
order to communicate with the test proxy over a secure connection.
The test proxy uses a self-signed certificate to communicate with HTTPS. Follow the general setup instructions [here][proxy_cert_docs] to trust this certificate locally.

### Start the proxy server

Expand All @@ -120,8 +117,8 @@ def start_proxy(test_proxy):
return
```

The `test_proxy` fixture will fetch the test proxy Docker image and create a new container called
`ambitious_azsdk_test_proxy`, which will be deleted after test execution unless interrupted.
The `test_proxy` fixture will download a test proxy executable if one isn't available locally, start the tool, and stop
it after tests complete.

If your tests already use an `autouse`d, session-level fixture for tests, you can accept the `test_proxy` parameter in
that existing fixture instead of adding a new one. For an example, see the [Register sanitizers](#register-sanitizers)
Expand All @@ -141,6 +138,9 @@ Recordings for a given package will end up in that package's `/tests/recordings`
do. Recordings that use the test proxy are `.json` files instead of `.yml` files, so migrated test suites no longer
need old `.yml` recordings.

After migrating to use the test proxy, libraries can and are encouraged to use out-of-repo recordings. For more
information, refer to the [recording migration guide][recording_migration].

> **Note:** support for configuring live or playback tests with a `testsettings_local.cfg` file has been
> deprecated in favor of using just `AZURE_TEST_RUN_LIVE`.
Expand All @@ -153,7 +153,7 @@ Instead, sanitizers (as well as matchers and transforms) can be registered on th
`add_general_string_sanitizer`. Other sanitizers are available for more specific scenarios and can be found at
[devtools_testutils/sanitizers.py][py_sanitizers].

Sanitizers, matchers, and transforms remain registered until the proxy container is stopped, so for any sanitizers that
Sanitizers, matchers, and transforms remain registered until the proxy tool is stopped, so for any sanitizers that
are shared by different tests, using a session fixture declared in a `conftest.py` file is recommended. Please refer to
[pytest's scoped fixture documentation][pytest_fixtures] for more details.

Expand Down Expand Up @@ -403,36 +403,6 @@ Running tests in playback follows the same pattern, except that requests will be
The `recorded_by_proxy` and `recorded_by_proxy_async` decorators send the appropriate requests at the start and end of
each test case.

### Start the proxy manually

There are two options for manually starting and stopping the test proxy: one uses a PowerShell command, and one uses
methods from `devtools_testutils`.

#### PowerShell

There is a [PowerShell script][docker_start_proxy] in `eng/common/testproxy` that will fetch the proxy Docker image if
you don't already have it, and will start or stop a container running the image for you. You can run the following
command from the root of the `azure-sdk-for-python` directory to start the container whenever you want to make the test
proxy available for running tests:

```powershell
.\eng\common\testproxy\docker-start-proxy.ps1 "start"
```

Note that the proxy is available as long as the container is running. In other words, you don't need to start and
stop the container for each test run or between tests for different SDKs. You can run the above command in the morning
and just stop the container whenever you'd like. To stop the container, run the same command but with `"stop"` in place
of `"start"`.

#### Python

There are two methods in `devtools_testutils`, [start_test_proxy][start_test_proxy] and
[stop_test_proxy][stop_test_proxy], that can be used to manually start and stop the test proxy. Like
`docker-start-proxy.ps1`, `start_test_proxy` will automatically fetch the proxy Docker image for you and start the
container if it's not already running.

For more details on proxy startup, please refer to the [proxy documentation][detailed_docs].

### Use `pytest.mark.parametrize` with migrated tests

Migrating tests to use basic `pytest` tools allows us to take advantage of helpful features such as
Expand Down Expand Up @@ -493,12 +463,10 @@ client to the test.


[detailed_docs]: https://github.com/Azure/azure-sdk-tools/tree/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
[docker_install]: https://docs.docker.com/get-docker/
[docker_start_proxy]: https://github.com/Azure/azure-sdk-for-python/blob/main/eng/common/testproxy/docker-start-proxy.ps1

[env_var_loader]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/envvariable_loader.py

[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/README.md
[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md

[mgmt_recorded_test_case]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/mgmt_recorded_testcase.py

Expand All @@ -507,7 +475,6 @@ client to the test.
[parametrize_class]: https://github.com/Azure/azure-sdk-for-python/blob/d92b63b9976b0025b274016c49a250fb7c4d7333/sdk/keyvault/azure-keyvault-keys/tests/_test_case.py#L59
[pipelines_ci]: https://github.com/Azure/azure-sdk-for-python/blob/5ba894966ed6b0e1ee8d854871f8c2da36a73d79/sdk/eventgrid/ci.yml#L30
[pipelines_live]: https://github.com/Azure/azure-sdk-for-python/blob/e2b5852deaef04752c1323d2ab0958f83b98858f/sdk/textanalytics/tests.yml#L26-L27
[podman]: https://podman.io/
[proxy_cert_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/test-proxy/trusting-cert-per-language.md
[py_sanitizers]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/sanitizers.py
[pytest_collection]: https://docs.pytest.org/latest/goodpractices.html#test-discovery
Expand Down
30 changes: 7 additions & 23 deletions doc/dev/test_proxy_troubleshooting.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,15 @@ GitHub repository, and documentation of how to set up and use the proxy can be f
## Table of contents
- [Guide for test proxy troubleshooting](#guide-for-test-proxy-troubleshooting)
- [Table of contents](#table-of-contents)
- [General troubleshooting tip](#general-troubleshooting-tip)
- [Test collection failure](#test-collection-failure)
- [Errors in tests using resource preparers](#errors-in-tests-using-resource-preparers)
- [Playback failures from body matching errors](#playback-failures-from-body-matching-errors)
- [Recordings not being produced](#recordings-not-being-produced)
- [KeyError during container startup](#keyerror-during-container-startup)
- [ConnectionError during test startup](#connectionerror-during-test-startup)
- [ConnectionError during tests](#connectionerror-during-tests)
- [Different error than expected when using proxy](#different-error-than-expected-when-using-proxy)
- [Test setup failure in test pipeline](#test-setup-failure-in-test-pipeline)
- [Fixture not found error](#fixture-not-found-error)

## General troubleshooting tip

For any issue that may come up, it's generally a good idea to first try deleting any existing proxy container (which
will be called `ambitious_azsdk_test_proxy`) and creating a new one by running tests. This will fetch the latest tag of
the test proxy Docker container, meaning the latest version of the proxy tool will be used.

## Test collection failure

Because tests are now using pure `pytest` conventions without `unittest.TestCase` components, discovering tests with
Expand Down Expand Up @@ -56,22 +48,14 @@ matching enabled by default.

## Recordings not being produced

First, make sure that the environment variable `AZURE_SKIP_LIVE_RECORDING` isn't set to "true". If it's not and live
tests still aren't producing recordings, try deleting the `ambitious_azsdk_test_proxy` Docker container and re-running
tests. The recording storage location is determined when the test proxy Docker container is created. If there are
multiple local copies of the `azure-sdk-for-python` repo on your machine, the container could be storing recordings in
the wrong repo.

## KeyError during container startup

Try updating your machine's version of Docker. Older versions of Docker may not return a status to indicate whether or
not the proxy container is running, which the [proxy_startup.py][proxy_startup] script needs to determine.
Ensure the environment variable `AZURE_SKIP_LIVE_RECORDING` **isn't** set to "true", and that `AZURE_TEST_RUN_LIVE`
**is** set to "true".

## ConnectionError during test startup
## ConnectionError during tests

For example, you may see a `requests.exceptions.ConnectionError` when trying to contact URL `/Info/Available`. This
means that the test proxy tool wasn't started up properly, so requests to the tool are failing. Make sure Docker is
installed and is up to date, and ensure that Linux containers are being used.
For example, you may see a `requests.exceptions.ConnectionError` when trying to make service or sanitizer setup
requests. This means that the test proxy tool never started correctly; ensure the `test_proxy` fixture is being invoked
during test startup so that the tool is available during tests.

## Different error than expected when using proxy

Expand Down
18 changes: 5 additions & 13 deletions doc/dev/tests.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -169,10 +169,7 @@ To migrate an existing test suite to use the test proxy, or to learn more about

### Perform one-time test proxy setup

1. Docker (or Podman) is a requirement for using the test proxy. You can install Docker from [docs.docker.com][docker_install], or install Podman at [podman.io][podman]. To use Podman, set an alias for `podman` to replace the `docker` command.
2. After installing, make sure Docker/Podman is running and is using Linux containers before running tests.
3. Follow the instructions [here][proxy_cert_docs] to complete setup. You need to trust a certificate on your machine in
order to communicate with the test proxy over a secure connection.
The test proxy uses a self-signed certificate to communicate with HTTPS. Follow the general setup instructions [here][proxy_cert_docs] to trust this certificate locally.

### Set up test resources

Expand Down Expand Up @@ -383,11 +380,8 @@ to the process of updating recordings.

- The targeted library is already migrated to use the test proxy.
- Git version > 2.25.0 is to on the machine and in the path. Git is used by the script and test proxy.
- [Docker][docker_install] or [Podman][podman] is installed.
- Global [git config settings][git_setup] are configured for `user.name` and `user.email`.
- These settings are also set with environment variables `GIT_COMMIT_OWNER` and `GIT_COMMIT_EMAIL`, respectively (in your environment or your local `.env` file).
- The environment variable `GIT_TOKEN` is set to a valid [personal access token][git_token] for your user (in your environment or your local `.env` file).
- This token is necessary for authenticating git requests made in a Docker/Podman container.
- Membership in the `azure-sdk-write` GitHub group.

Test recordings will be updated if tests are run while `AZURE_TEST_RUN_LIVE` is set to "true" and
Expand All @@ -409,15 +403,16 @@ The recording directory in this case is `2Km2Z8755`, the string between the two

After verifying that your recording updates look correct, you can use the [`manage_recordings.py`][manage_recordings]
script from `azure-sdk-for-python/scripts` to push these recordings to the `azure-sdk-assets` repo. This script accepts
a verb and a **relative** path to your package's `assets.json` file. For example, from the root of the
`azure-sdk-for-python` repo:
a verb and a **relative** path to your package's `assets.json` file (this path is optional, and simply `assets.json`
by default). For example, from the root of the `azure-sdk-for-python` repo:
```
python scripts/manage_recordings.py push sdk/{service}/{package}/assets.json
python scripts/manage_recordings.py push -p sdk/{service}/{package}/assets.json
```

The verbs that can be provided to this script are "push", "restore", and "reset":
- **push**: pushes recording updates to a new assets repo tag and updates the tag pointer in `assets.json`.
- **restore**: fetches recordings from the assets repo, based on the tag pointer in `assets.json`.
- **reset**: discards any pending changes to recordings, based on the tag pointer in `assets.json`.

After pushing your recordings, the `assets.json` file for your package will be updated to point to a new `Tag` that
contains the updates. Include this `assets.json` update in any pull request to update the recordings pointer in the
Expand Down Expand Up @@ -714,8 +709,6 @@ Tests that use the Shared Access Signature (SAS) to authenticate a client should
[azure_portal]: https://portal.azure.com/
[azure_recorded_test_case]: https://github.com/Azure/azure-sdk-for-python/blob/7e66e3877519a15c1d4304eb69abf0a2281773/tools/azure-sdk-tools/devtools_testutils/azure_recorded_testcase.py#L44

[docker_install]: https://docs.docker.com/get-docker/

[engsys_wiki]: https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/48/Create-a-new-Live-Test-pipeline?anchor=test-resources.json
[env_var_loader]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/envvariable_loader.py

Expand All @@ -732,7 +725,6 @@ Tests that use the Shared Access Signature (SAS) to authenticate a client should
[mgmt_settings_fake]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/mgmt_settings_fake.py

[packaging]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/packaging.md
[podman]: https://podman.io/
[proxy_cert_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/test-proxy/trusting-cert-per-language.md
[proxy_general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
[proxy_migration_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md
Expand Down
Loading

0 comments on commit 7a46088

Please sign in to comment.