Skip to content

Commit

Permalink
[Internal] Refine tracing e2e test pipeline (#2289)
Browse files Browse the repository at this point in the history
# Description

Refine tracing e2e test pipeline. Currently the pipeline didn't do auth
step, causing failure for 3p customers.

# All Promptflow Contribution checklist:
- [x] **The pull request does not introduce [breaking changes].**
- [ ] **CHANGELOG is updated for new features, bug fixes or other
significant changes.**
- [x] **I have read the [contribution guidelines](../CONTRIBUTING.md).**
- [ ] **Create an issue and link to the pull request to get dedicated
review from promptflow team. Learn more: [suggested
workflow](../CONTRIBUTING.md#suggested-workflow).**

## General Guidelines and Best Practices
- [x] Title of the pull request is clear and informative.
- [x] There are a small number of commits, each of which have an
informative message. This means that previously merged commits do not
appear in the history of the PR. For more information on cleaning up the
commits in your PR, [see this
page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md).

### Testing Guidelines
- [ ] Pull request includes test coverage for the included changes.

---------

Co-authored-by: Lina Tang <[email protected]>
  • Loading branch information
lumoslnt and Lina Tang authored Mar 11, 2024
1 parent b9b7269 commit 626bf39
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 22 deletions.
49 changes: 27 additions & 22 deletions .github/workflows/promptflow-tracing-e2e-test.yml
Original file line number Diff line number Diff line change
@@ -1,33 +1,41 @@
name: promptflow-tracing-e2e-test

on:
schedule:
- cron: "40 17 * * *" # Every day starting at 1:40 BJT

pull_request:
pull_request_target:
paths:
- src/promptflow/**
- scripts/building/**
- .github/workflows/promptflow-tracing-e2e-test.yml

workflow_dispatch:


env:
packageSetupType: promptflow_with_extra
testWorkingDirectory: ${{ github.workspace }}/src/promptflow
PYTHONPATH: ${{ github.workspace }}/src/promptflow
IS_IN_CI_PIPELINE: "true"


jobs:
authorize:
environment:
# forked prs from pull_request_target will be run in external environment, domain prs will be run in internal environment
${{ github.event_name == 'pull_request_target' &&
github.event.pull_request.head.repo.full_name != github.repository &&
'external' || 'internal' }}
runs-on: ubuntu-latest
steps:
- run: true
build:
needs: authorize
strategy:
fail-fast: false
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.ref }}
fetch-depth: 0
- name: merge main to current branch
uses: "./.github/actions/step_merge_main"
- name: Display and Set Environment Variables
run: |
env | sort >> $GITHUB_OUTPUT
Expand All @@ -50,7 +58,6 @@ jobs:
path: |
${{ github.workspace }}/src/promptflow/dist/*.whl
${{ github.workspace }}/src/promptflow-tools/dist/*.whl
tracing_tests:
needs: build
strategy:
Expand All @@ -62,24 +69,25 @@ jobs:
steps:
- name: checkout
uses: actions/checkout@v4

with:
ref: ${{ github.event.pull_request.head.sha || github.ref }}
fetch-depth: 0
- name: merge main to current branch
uses: "./.github/actions/step_merge_main"
- name: Display and Set Environment Variables
run: |
env | sort >> $GITHUB_OUTPUT
id: display_env
shell: bash -el {0}

- name: Python Setup - ${{ matrix.os }} - Python Version ${{ matrix.pythonVersion }}
uses: "./.github/actions/step_create_python_environment"
with:
pythonVersion: ${{ matrix.pythonVersion }}

- name: Download Artifacts
uses: actions/download-artifact@v3
with:
name: wheel
path: artifacts

- name: Install wheel
shell: pwsh
working-directory: artifacts
Expand All @@ -89,21 +97,17 @@ jobs:
gci ./promptflow -Recurse | % {if ($_.Name.Contains('.whl')) {python -m pip install "$($_.FullName)"}}
gci ./promptflow-tools -Recurse | % {if ($_.Name.Contains('.whl')) {python -m pip install $_.FullName}}
pip freeze
- name: Azure Login
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}

- name: Generate Configs
uses: "./.github/actions/step_generate_configs"
with:
targetFolder: ${{ env.testWorkingDirectory }}

- name: Get number of CPU cores
uses: SimenB/github-actions-cpu-cores@v1
id: cpu-cores

- name: run promptflow-tracing test
shell: pwsh
working-directory: ${{ env.testWorkingDirectory }}
Expand All @@ -116,7 +120,6 @@ jobs:
-n ${{ steps.cpu-cores.outputs.count }} `
--coverage-config ${{ github.workspace }}/src/promptflow/tests/tracing_test/.coveragerc `
-o "${{ env.testWorkingDirectory }}/test-results-tracing.xml"
- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v3
Expand All @@ -125,22 +128,24 @@ jobs:
path: |
${{ env.testWorkingDirectory }}/*.xml
${{ env.testWorkingDirectory }}/htmlcov/
publish-test-results-tracing-test:
name: "Publish Tests Results"
needs: tracing_tests
if: always()

runs-on: ubuntu-latest
permissions:
checks: write
pull-requests: write
contents: read
issues: read

steps:
- name: checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.ref }}
fetch-depth: 0
- name: merge main to current branch
uses: "./.github/actions/step_merge_main"
- name: Publish Test Results
uses: "./.github/actions/step_publish_test_results"
with:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/promptflow-tracing-unit-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ jobs:
publish-test-results-tracing-test:
name: "Publish Tests Results"
needs: tracing_tests
if: always()

Expand Down

0 comments on commit 626bf39

Please sign in to comment.