Skip to content

Commit

Permalink
use python 3.9.7 in windowai packaging pipeline (#14766) (#14813)
Browse files Browse the repository at this point in the history
### Description
Use python3.9.7 in windowsAI packaging pipeline.


### Motivation and Context
In WindowsAI packaging pipeline, cdpxwin1809 is deprecated and it will
no longer be cached from March 2023.

I used the recommended image

[onebranch.azurecr.io/windows/ltsc2019/vse2022:latest](https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.wiki/4587/Container-Images?anchor=recommendation-for-windows-container-image-userst)
But it always failed to pass arm32 jobs.
It's very likely a regression in VS2022. 
One user reported a similar issue #14190. 
I've submitted a bug to visual studio.

https://developercommunity.visualstudio.com/t/Compilation-Error-with-VS2022-ARM/10285309.

For `onebranch.azurecr.io/windows/ltsc2019/vse2019:latest`, there's an
exception` Error : init_sys_streams: can't initialize sys standard
streams`
It could be solved by updating from python3.7 to python3.9 in the
pipeline.


(https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=279301&view=results)
The root cause might be the conflicts between different python version.
The inherent python in onebranch image is python 3.9.7.


(https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.Code.Wiki/6766/manifest)


### Ref


https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.wiki/4587/Container-Images?anchor=windows-build-images

### Description
<!-- Describe your changes. -->



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Co-authored-by: Yi Zhang <[email protected]>
  • Loading branch information
PatriceVignola and mszhanyi authored Feb 24, 2023
1 parent 8f90066 commit 8a54bee
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 17 deletions.
2 changes: 1 addition & 1 deletion .pipelines/nuget_config/x64/packages.config
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="python" version="3.7.9" targetFramework="native" />
<package id="python" version="3.9.7" targetFramework="native" />
<package id="Microsoft.AI.DirectML" version="1.10.1" targetFramework="native" />
<package id="Microsoft.Windows.CppWinRT" version="2.0.201201.7" targetFramework="native" />
</packages>
2 changes: 1 addition & 1 deletion .pipelines/nuget_config/x86/packages.config
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="pythonx86" version="3.7.9" targetFramework="native" />
<package id="pythonx86" version="3.9.7" targetFramework="native" />
<package id="Microsoft.AI.DirectML" version="1.10.1" targetFramework="native" />
<package id="Microsoft.Windows.CppWinRT" version="2.0.201201.7" targetFramework="native" />
</packages>
30 changes: 15 additions & 15 deletions .pipelines/windowsai-steps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ parameters:
displayName: BuildArch
type: string
default: 'x64'

- name: Runtime
displayName: MSVC Runtime, should be 'dynamic' or 'static'.
type: string
default: 'dynamic'

- name: PythonPackageName
displayName: PythonPackageName on nuget.org to use
type: string
Expand All @@ -17,8 +17,8 @@ parameters:
jobs:
- job: Windows_Packaging_${{ parameters.BuildArch }}_${{ parameters.Runtime }}
pool:
type: windows
type: windows

variables:
ob_outputDirectory: '$(Build.ArtifactStagingDirectory)'
ob_sdl_binskim_break: true
Expand All @@ -40,7 +40,7 @@ jobs:
restoreSolution: $(Build.SourcesDirectory)\.pipelines\nuget_config\x64\packages.config
${{ if eq(parameters.BuildArch, 'arm64') }}:
restoreSolution: $(Build.SourcesDirectory)\.pipelines\nuget_config\x64\packages.config

- script: |
@echo off
set vswherepath="%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe"
Expand All @@ -50,10 +50,10 @@ jobs:
set vsdevcmd="%%i\Common7\Tools\vsdevcmd.bat"
)
)
@echo vslatest %vslatest%
@echo vsdevcmd %vsdevcmd%
@echo ##vso[task.setvariable variable=vslatest]%vslatest%
@echo ##vso[task.setvariable variable=vsdevcmd]%vsdevcmd% -arch=${{ parameters.BuildArch }}
displayName: 'locate vsdevcmd via vswhere'
Expand All @@ -78,9 +78,9 @@ jobs:
- script: |
curl -O -L https://github.com/Kitware/CMake/releases/download/v3.24.3/cmake-3.24.3-windows-x86_64.zip
7z x cmake-3.24.3-windows-x86_64.zip
set PYTHONHOME=$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.7.9\tools
set PYTHONPATH=$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.7.9\tools
$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.7.9\tools\python.exe "$(Build.SourcesDirectory)\tools\ci_build\build.py" --build_dir $(Build.BinariesDirectory) --build_shared_lib --enable_onnx_tests --ms_experimental --use_dml --use_winml --cmake_generator "Visual Studio 16 2019" --update --config RelWithDebInfo --enable_lto --use_telemetry --disable_rtti --enable_wcos $(BuildFlags) --cmake_extra_defines CMAKE_SYSTEM_VERSION=10.0.19041.0 --cmake_path $(Build.BinariesDirectory)\cmake-3.24.3-windows-x86_64\bin\cmake.exe --ctest_path $(Build.BinariesDirectory)\cmake-3.24.3-windows-x86_64\bin\ctest.exe
set PYTHONHOME=$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.9.7\tools
set PYTHONPATH=$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.9.7\tools
$(Build.BinariesDirectory)\${{ parameters.PythonPackageName }}.3.9.7\tools\python.exe "$(Build.SourcesDirectory)\tools\ci_build\build.py" --build_dir $(Build.BinariesDirectory) --build_shared_lib --enable_onnx_tests --ms_experimental --use_dml --use_winml --cmake_generator "Visual Studio 16 2019" --update --config RelWithDebInfo --enable_lto --use_telemetry --disable_rtti --enable_wcos $(BuildFlags) --cmake_extra_defines CMAKE_SYSTEM_VERSION=10.0.19041.0 --cmake_path $(Build.BinariesDirectory)\cmake-3.24.3-windows-x86_64\bin\cmake.exe --ctest_path $(Build.BinariesDirectory)\cmake-3.24.3-windows-x86_64\bin\ctest.exe
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Generate cmake config'
Expand All @@ -97,7 +97,7 @@ jobs:
maximumCpuCount: true
logProjectEvents: true
workingFolder: '$(Build.BinariesDirectory)\RelWithDebInfo'
createLogFile: true
createLogFile: true

- ${{ if eq(parameters.Runtime, 'dynamic') }}:
- script: |
Expand All @@ -114,8 +114,8 @@ jobs:
copy $(Build.SourcesDirectory)\onnxruntime\test\testdata\sequence_length.onnx $(Build.ArtifactStagingDirectory)\test_artifact\
copy $(Build.SourcesDirectory)\onnxruntime\test\testdata\sequence_construct.onnx $(Build.ArtifactStagingDirectory)\test_artifact\
displayName: 'Copy WinML test collateral to artifact directory'
- ${{ if eq(parameters.BuildArch, 'x64') }}:
- script: |
call $(vsdevcmd)
Expand All @@ -129,8 +129,8 @@ jobs:
signing_profile: 'external_distribution'
files_to_sign: '**/*.exe;**/*.dll'
search_root: '$(Build.BinariesDirectory)\RelWithDebInfo\RelWithDebInfo'
displayName: 'Sign runtime DLLs'
displayName: 'Sign runtime DLLs'

- ${{ if eq(parameters.BuildArch, 'x64') }}:
- script: |
call $(vsdevcmd)
Expand Down

0 comments on commit 8a54bee

Please sign in to comment.