Skip to content

Commit

Permalink
Split ondevice training cpu packaging pipeline to a separated pipeline (
Browse files Browse the repository at this point in the history
#21485)

### Description
Right now our "Zip-Nuget-Java-Nodejs Packaging Pipeline" is too big.
This OnDevice training part is independent of the others, so it can be
split out. Then our NPM Packaging pipeline will not depends on this
training stuff.

### Motivation and Context
Similar to #21235 

Also, this PR fixed a problem that: "NuGet_Test_Linux_Training_CPU" job
downloads artifacts from "onnxruntime-linux-x64" for getting customop
shared libs, but the job forget to declare it depends on the
"Linux_C_API_Packaging_CPU_x64" which produces the artifact. Such
problems can be hard to find when a pipeline goes big.
  • Loading branch information
snnn authored Jul 25, 2024
1 parent ebcb707 commit 4167b68
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,6 @@ stages:
SpecificArtifact: ${{ parameters.SpecificArtifact }}
BuildId: ${{ parameters.BuildId }}

- template: templates/ondevice-training-cpu-packaging-pipeline.yml
parameters:
RunOnnxRuntimeTests: ${{ parameters.RunOnnxRuntimeTests }}
DoCompliance: ${{ parameters.DoCompliance }}
DoEsrp: ${{ parameters.DoEsrp }}
IsReleaseBuild: ${{ parameters.IsReleaseBuild }}
OrtNugetPackageId: 'Microsoft.ML.OnnxRuntime.Training'
AdditionalBuildFlags: '--enable_training_apis'
AdditionalWinBuildFlags: '--enable_onnx_tests --enable_wcos'
BuildVariant: 'default'

- template: stages/java-cuda-packaging-stage.yml
parameters:
CudaVersion: 11.8
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
parameters:
- name: RunOnnxRuntimeTests
displayName: Run Tests?
type: boolean
default: true

- name: DoCompliance
displayName: Run Compliance Tasks?
type: boolean
default: true

- name: DoEsrp
displayName: Run code sign tasks? Must be true if you are doing an ONNX Runtime release
type: boolean
default: true

- name: IsReleaseBuild
displayName: Is a release build? Set it to true if you are doing an ONNX Runtime release.
type: boolean
default: false
- name: PreReleaseVersionSuffixString
displayName: Suffix added to pre-release package version. Only used if IsReleaseBuild is true. Denotes the type of pre-release package.
type: string
values:
- alpha
- beta
- rc
- none
default: none

- name: PreReleaseVersionSuffixNumber
displayName: Number added to pre-release package version. Only used if IsReleaseBuild is true. Denotes the sequence of a pre-release package.
type: number
default: 0

stages:
- template: stages/set_packaging_variables_stage.yml
parameters:
IsReleaseBuild: ${{ parameters.IsReleaseBuild }}
PreReleaseVersionSuffixString: ${{ parameters.PreReleaseVersionSuffixString }}
PreReleaseVersionSuffixNumber: ${{ parameters.PreReleaseVersionSuffixNumber }}
- template: templates/ondevice-training-cpu-packaging-pipeline.yml
parameters:
RunOnnxRuntimeTests: ${{ parameters.RunOnnxRuntimeTests }}
DoCompliance: ${{ parameters.DoCompliance }}
DoEsrp: ${{ parameters.DoEsrp }}
IsReleaseBuild: ${{ parameters.IsReleaseBuild }}
OrtNugetPackageId: 'Microsoft.ML.OnnxRuntime.Training'
AdditionalBuildFlags: '--enable_training_apis'
AdditionalWinBuildFlags: '--enable_onnx_tests --enable_wcos'
BuildVariant: 'default'
Original file line number Diff line number Diff line change
Expand Up @@ -317,3 +317,4 @@ stages:
ArtifactSuffix: 'Training-CPU'
StageSuffix: 'Training_CPU'
NativePackagePrefix: 'onnxruntime-training'
CustomOpArtifactName: 'onnxruntime-training-linux-x64'

0 comments on commit 4167b68

Please sign in to comment.