Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…into protocol_base

* 'master' of https://github.com/Azure/azure-sdk-for-python: (337 commits)
  [Container Registry] adding blurb about env vars to set (#17976)
  shared_requirements msrest verison update (#17938)
  Deprecate Azure-Synapse (#18027)
  add perf tests (#17519)
  Increment package version after release of azure-search-documents (#18017)
  [Communication]: Migrated python identity tests to use ARM template (#17890)
  [Container Registry] fixing up the test-resources-post script (#17883)
  [Communication]: Removed error code assertion from unauthorized phone number test (#17955)
  Add Conda Publishing (#17889)
  fix search fields (#18014)
  Sync eng/common directory with azure-sdk-tools for PR 1552 (#18011)
  [AppConfiguration] fix tests (#17917)
  updating generated code (#17874)
  Update document translation sample slug (#17989)
  update release date (#17988)
  [Communication] - SDK - Synced SMS and Phone Numbers shared folders (#17985)
  Remove unnecessary attribute access (#18007)
  [AutoRelease] t2-healthcareapis-2021-04-12-56802 (#17968)
  Revert retain. (#18004)
  Sync eng/common directory with azure-sdk-tools for PR 1553 (#18002)
  ...
  • Loading branch information
iscai-msft committed Apr 14, 2021
2 parents cd12344 + 7f94101 commit 3fd484f
Show file tree
Hide file tree
Showing 4,101 changed files with 502,267 additions and 205,323 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
8 changes: 7 additions & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
/sdk/storage/ @amishra-dev @zezha-msft @annatisch @xiafu-msft @tasherif-msft @kasobol-msft

# PRLabel: %App Configuration
/sdk/appconfiguration/ @xiangyan99
/sdk/appconfiguration/ @xiangyan99 @seankane-msft

/sdk/applicationinsights/azure-applicationinsights/ @divya-jay @geneh @alongafni
/sdk/loganalytics/azure-loganalytics/ @divya-jay @geneh @alongafni
Expand Down Expand Up @@ -107,12 +107,18 @@
# PRLabel: %Cognitive - Metrics Advisor
/sdk/metricsadvisor/ @xiangyan99 @kristapratico

# PRLabel: %Cognitive - Translator
/sdk/translation/ @kristapratico @mohamedshabanofficial

# PRLabel: %Tables
/sdk/tables/ @seankane-msft

# PRLabel: %Media
/sdk/media/ @naiteeks @bennage @giakas

# PRLabel: %Container Registry
/sdk/containerregistry/ @seankane-msft

# Smoke Tests
/common/smoketest/ @lmazuel @chlowell @annatisch @rakshith91 @shurd @southpolesteve

Expand Down
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@ Mypy install and run.
**Example: Invoke tox, breaking into the debugger on failure**
`tox -e whl -c ../../../eng/tox/tox.ini -- --pdb`

### Performance Testing

SDK performance testing is supported via the custom `perfstress` framework. For full details on this framework, and how to write and run tests for an SDK - see the [perfstress tests documentation](https://github.com/Azure/azure-sdk-for-python/blob/master/doc/dev/perfstress_tests.md).

### More Reading

We maintain an [additional document](https://github.com/Azure/azure-sdk-for-python/blob/master/doc/eng_sys_checks.md) that has a ton of detail as to what is actually _happening_ in these executions.
Expand Down
4 changes: 2 additions & 2 deletions ci_template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,5 @@ extends:
parameters:
ServiceDirectory: MyService
Artifacts:
- name: azure_mgmt_MyService
safeName: azuremgmtMyService
- name: azure-mgmt-MyService
safeName: azuremgmtMyService
97 changes: 97 additions & 0 deletions doc/dev/conda-builds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Azure SDK for Python Conda Distributions

## Local Environment Setup

Follow the instructions [here](https://docs.conda.io/projects/conda-build/en/latest/install-conda-build.html) to install `conda` and `conda-build`.

## CI Build Process

There will be a `CondaArtifact` defined in the `ci.yml` of each service directory. (`sdk/<service>`)

A Conda Artifact defines:
- The location of the `meta.yml`
- Which packages will be pulled into the combined artifact
- The name of the combined artifact
- Any other necessary details.

## How to Build an Azure SDK Conda Package Locally


### Create Your Build Directory
Given how Conda packages are comprised of multiple source distributions _combined_, the buildable source does not exist directly within the azure-sdk-for-python repo. Currently, there is _some_ manual work that needs to be done.

To begin, check your `ci.yml` for a `CondaArtifact`. Each these artifacts will become a single conda package. Let's use `storage/ci.yml` as an example.

```
- name: azure-storage
meta_source: meta.yml
common_root: azure/storage
checkout:
- package: azure-storage-blob
checkout_path: sdk/storage
version: 12.8.0
- package: azure-storage-queue
checkout_path: sdk/storage
version: 12.1.5
- package: azure-storage-file-share
checkout_path: sdk/storage
version: 12.4.1
- package: azure-storage-file-datalake
checkout_path: sdk/storage
version: 12.3.0
```

- `name: azure-storage`: will be the name of the "combined" sdist package that we generate.
- `meta_source: meta.yml`: this is the path (relative to the service directory) to the target conda package meta.yml.
- `common_root: azure/storage`: when generating the combined package, where will we begin combining? This is tightly bound to folder structure within the generated sdist.
- `checkout`: the `checkout` setting is a list of target packages that will go into the combined artifact. These targets will be individually sparse cloned, and copied into the conda build directory. Currently, this is a **manual step** in your local build. Reference `eng/pipelines/templates/get-tagged-code.yml` for exact details on how CI does it.

Before we continue, you should be aware of two primary locations that are necessary, but not referenced directly in the `ci.yml`.

The `build` folder and the `output` folder. The `build` folder (`$(Conda.Build)` variable in CI) is where we will...

- store the cloned package code
- generate the combined sdist

To locally repro without magic given a specific `checkout` artifact:

```
<cd sdk-for-python>
git checkout `<package>_<version>`
grab the entire <package> directory under the <checkout_path>. place into your `build` folder.
```

Given the `storage` example. This is what your `build` folder should look like prior to invoking `build_conda_artifacts.py`.

```
<build directory>/
azure-storage-blob/ <-- the package directly ripped from specified tag
azure-storage-file-datalake/
azure-storage-file-share/
azure-storage-queue/
```

### Create the Combined SDist

Once you have a directory assembled, invoke the script to build. The below command is formatted for visibility, recombine the lines however necessary for your chosen shell environment.


```
python `build_conda_artifacts.py`
-d "<output folder>"
-b "<build folder>"
-m "<resolvable path to sdk/storage/meta.yml>"
-r "azure/storage"
-n "azure-storage"
-s "storage"
```

### Generate the Conda Package

Locally, from the anaconda prompt, set the environment variable `STORAGE_SOURCE_DISTRIBUTION` to the location of the generated sdist. After that:

```bash
export STORAGE_SOURCE_DISTRIBUTION=<path/to/generated/sdist>
cd <meta.yml directory>
conda-build . --output-folder <conda.output>
```
Loading

0 comments on commit 3fd484f

Please sign in to comment.