Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files Browse the repository at this point in the history
…-worker into default
lzchen committed Jan 2, 2025
2 parents a69f01a + ed26bd0 commit c625306
Showing 176 changed files with 1,650 additions and 1,111 deletions.
2 changes: 1 addition & 1 deletion CODEOWNERS
Original file line number Diff line number Diff line change
@@ -10,4 +10,4 @@
# For all file changes, github would automatically
# include the following people in the PRs.

* @vrdmr @gavin-aguiar @YunchuWang @pdthummar @hallvictoria
* @vrdmr @gavin-aguiar @hallvictoria
33 changes: 16 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,34 @@
# <img src="https://raw.githubusercontent.com/Azure/azure-functions-python-worker/dev/docs/Azure.Functions.svg" width = "30" alt="Functions Header Image - Lightning Logo"> Azure Functions Python Worker

| Branch | Status | CodeCov | Unittests | E2E tests |
|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
| main | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=main)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=main) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/main/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=main) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=main) |
| dev | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=dev)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=dev) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=dev) |
| Branch | Build Status | CodeCov | Test Status |
|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| dev | [![Build Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | [![Test Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) |

Python support for Azure Functions is based on Python 3.6, 3.7, 3.8, 3.9, and 3.10 serverless hosting on Linux and the Functions 2.0, 3.0 and 4.0 runtime.
Python support for Azure Functions is based on Python 3.8, 3.9, 3.10, 3.11, and 3.12 serverless hosting on Linux and the Functions 4.0 runtime.

Here is the current status of Python in Azure Functions:

What are the supported Python versions?

| Azure Functions Runtime | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 |
|----------------------------------|------------|------------|------------|------------|-------------|-------------|
| Azure Functions 2.0 (deprecated) ||| - | - | - | - |
| Azure Functions 3.0 (deprecated) ||||| - | - |
| Azure Functions 4.0 | - | - |||||
| Azure Functions Runtime | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | Python 3.12 |
|----------------------------------|------------|------------|-------------|-------------|-------------|
| Azure Functions 3.0 (deprecated) ||| - | - | - |
| Azure Functions 4.0 ||||||

For information about Azure Functions Runtime, please refer to [Azure Functions runtime versions overview](https://docs.microsoft.com/en-us/azure/azure-functions/functions-versions) page.

### What's available?

- Build, test, debug and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
- Deploy Python Function project onto consumption, dedicated, or elastic premium plan.
- Deploy Python Function project in a custom docker image onto dedicated, or elastic premium plan.
- Triggers / Bindings : HTTP, Blob, Queue, Timer, Cosmos DB, Event Grid, Event Hubs and Service Bus
- Build, test, debug, and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
- Deploy Python Function project onto consumption, dedicated, elastic premium, or flex consumption plan.
- Deploy Python Function project in a custom docker image onto dedicated or elastic premium plan.
- Triggers / Bindings : Blob, Cosmos DB, Event Grid, Event Hub, HTTP, Kafka, MySQL, Queue, ServiceBus, SQL, Timer, and Warmup
- Triggers / Bindings : Custom binding support

What's coming?
### What's new?

- [Durable Functions For Python](https://github.com/Azure/azure-functions-durable-python)
- [SDK Type Bindings for Blob](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-sdk-type-bindings-for-azure-blob-storage-with/ba-p/4146744)
- [HTTP Streaming](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-support-for-http-streams-in-python-is-now-in/ba-p/4146697)

### Get Started

@@ -72,4 +71,4 @@ provided by the bot. You will only need to do this once across all repos using o

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
2 changes: 1 addition & 1 deletion azure_functions_worker/version.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

VERSION = '4.33.0'
VERSION = '4.34.0'
46 changes: 46 additions & 0 deletions eng/ci/emulator-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
trigger: none # ensure this is not ran as a CI build

pr:
branches:
include:
- dev
- release/*

schedules:
- cron: "0 8 * * 1,2,3,4,5"
displayName: Monday to Friday 3 AM CST build
branches:
include:
- dev
always: true

resources:
repositories:
- repository: 1es
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release
- repository: eng
type: git
name: engineering
ref: refs/tags/release

variables:
- template: /ci/variables/build.yml@eng
- template: /ci/variables/cfs.yml@eng
- template: /eng/templates/utils/variables.yml@self

extends:
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
parameters:
pool:
name: 1es-pool-azfunc
image: 1es-windows-2022
os: windows

stages:
- stage: RunEmulatorTests
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc
8 changes: 8 additions & 0 deletions eng/ci/official-build.yml
Original file line number Diff line number Diff line change
@@ -30,6 +30,8 @@ resources:
variables:
- template: /eng/templates/utils/variables.yml@self
- template: /eng/templates/utils/official-variables.yml@self
- name: codeql.excludePathPatterns
value: deps/,build/

extends:
template: v1/1ES.Official.PipelineTemplate.yml@1es
@@ -52,6 +54,12 @@ extends:
dependsOn: Build
jobs:
- template: /eng/templates/official/jobs/ci-e2e-tests.yml@self
- stage: RunEmulatorTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc
- stage: RunUnitTests
dependsOn: Build
jobs:
21 changes: 15 additions & 6 deletions eng/ci/public-build.yml
Original file line number Diff line number Diff line change
@@ -28,6 +28,14 @@ resources:

variables:
- template: /eng/templates/utils/variables.yml@self
- name: codeql.excludePathPatterns
value: deps/,build/
- name: codeql.compiled.enabled
value: true
- name: codeql.runSourceLanguagesInSourceAnalysis
value: true
- name: codeql.sourceLanguages
value: python, powershell

extends:
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
@@ -36,11 +44,6 @@ extends:
name: 1es-pool-azfunc-public
image: 1es-windows-2022
os: windows
sdl:
codeql:
compiled:
enabled: true # still only runs for default branch
runSourceLanguagesInSourceAnalysis: true
settings:
skipBuildTagsForGitHubPullRequests: ${{ variables['System.PullRequest.IsFork'] }}
stages:
@@ -50,4 +53,10 @@ extends:
- stage: RunUnitTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-unit-tests.yml@self
- template: /eng/templates/jobs/ci-unit-tests.yml@self
- stage: RunEmulatorTests
dependsOn: Build
jobs:
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
parameters:
PoolName: 1es-pool-azfunc-public
6 changes: 5 additions & 1 deletion eng/templates/jobs/build.yml
Original file line number Diff line number Diff line change
@@ -21,4 +21,8 @@ jobs:
python -m pip install .
displayName: 'Build python worker'
# Skip the build stage for SDK and Extensions release branches. This stage will fail because pyproject.toml contains the updated (and unreleased) library version
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
- bash: |
pip install pip-audit
pip-audit -r requirements.txt
displayName: 'Run vulnerability scan'
100 changes: 100 additions & 0 deletions eng/templates/jobs/ci-emulator-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
jobs:
- job: "TestPython"
displayName: "Run Python Emulator Tests"

pool:
name: ${{ parameters.PoolName }}
image: 1es-ubuntu-22.04
os: linux

strategy:
matrix:
Python37:
PYTHON_VERSION: '3.7'
Python38:
PYTHON_VERSION: '3.8'
Python39:
PYTHON_VERSION: '3.9'
Python310:
PYTHON_VERSION: '3.10'
Python311:
PYTHON_VERSION: '3.11'
Python312:
PYTHON_VERSION: '3.12'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: $(PYTHON_VERSION)
- task: UseDotNet@2
displayName: 'Install .NET 8'
inputs:
version: 8.0.x
- bash: |
chmod +x eng/scripts/install-dependencies.sh
chmod +x eng/scripts/test-setup.sh
eng/scripts/install-dependencies.sh $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install dependencies and the worker'
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
- task: DownloadPipelineArtifact@2
displayName: 'Download Python SDK Artifact'
inputs:
buildType: specific
artifactName: 'azure-functions'
project: 'internal'
definition: 679
buildVersionToDownload: latest
targetPath: '$(Pipeline.Workspace)/PythonSdkArtifact'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- bash: |
chmod +x eng/scripts/test-sdk.sh
chmod +x eng/scripts/test-setup.sh
eng/scripts/test-sdk.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install test python sdk, dependencies and the worker'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- task: DownloadPipelineArtifact@2
displayName: 'Download Python Extension Artifact'
inputs:
buildType: specific
artifactName: $(PYTHONEXTENSIONNAME)
project: 'internal'
definition: 798
buildVersionToDownload: latest
targetPath: '$(Pipeline.Workspace)/PythonExtensionArtifact'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- bash: |
chmod +x eng/scripts/test-setup.sh
chmod +x eng/scripts/test-extensions.sh
eng/scripts/test-extensions.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
eng/scripts/test-setup.sh
displayName: 'Install test python extension, dependencies and the worker'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- bash: |
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml pull
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml up -d
displayName: 'Install Azurite and Start EventHub Emulator'
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 --ignore=tests/emulator_tests/test_servicebus_functions.py tests/emulator_tests
env:
AzureWebJobsStorage: "UseDevelopmentStorage=true"
AzureWebJobsEventHubConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
displayName: "Running $(PYTHON_VERSION) Python Linux Emulator Tests"
- bash: |
# Stop and remove EventHub Emulator container to free up the port
docker stop eventhubs-emulator
docker container rm --force eventhubs-emulator
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml pull
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml up -d
env:
AzureWebJobsSQLPassword: $(AzureWebJobsSQLPassword)
displayName: 'Install Azurite and Start ServiceBus Emulator'
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 tests/emulator_tests/test_servicebus_functions.py
env:
AzureWebJobsStorage: "UseDevelopmentStorage=true"
AzureWebJobsServiceBusConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
displayName: "Running $(PYTHON_VERSION) Python ServiceBus Linux Emulator Tests"
21 changes: 20 additions & 1 deletion eng/templates/official/jobs/ci-e2e-tests.yml
Original file line number Diff line number Diff line change
@@ -115,6 +115,25 @@ jobs:
eng/scripts/test-setup.sh
displayName: 'Install test python extension, dependencies and the worker'
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
- powershell: |
$pipelineVarSet = "$(USETESTPYTHONSDK)"
Write-Host "pipelineVarSet: $pipelineVarSet"
$branch = "$(Build.SourceBranch)"
Write-Host "Branch: $branch"
if($branch.StartsWith("refs/heads/sdk/") -or $pipelineVarSet -eq "true")
{
Write-Host "##vso[task.setvariable variable=skipTest;]true"
}
else
{
Write-Host "##vso[task.setvariable variable=skipTest;]false"
}
displayName: 'Set skipTest variable'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- powershell: |
Write-Host "skipTest: $(skipTest)"
displayName: 'Display skipTest variable'
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
- bash: |
python -m pytest -q -n auto --dist loadfile --reruns 4 --cov=./azure_functions_worker --cov-report xml --cov-branch --cov-append tests/endtoend tests/extension_tests/deferred_bindings_tests tests/extension_tests/http_v2_tests
env:
@@ -125,5 +144,5 @@ jobs:
AzureWebJobsSqlConnectionString: $(SQL_CONNECTION)
AzureWebJobsEventGridTopicUri: $(EVENTGRID_URI)
AzureWebJobsEventGridConnectionKey: $(EVENTGRID_CONNECTION)
USETESTPYTHONSDK: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
skipTest: $(skipTest)
displayName: "Running $(PYTHON_VERSION) Python E2E Tests"
Loading

0 comments on commit c625306

Please sign in to comment.