Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
/ corefx Public archive

[Arm64/Linux] Enable CI & Official builds #27656

Merged
merged 1 commit into from
Mar 20, 2018

Conversation

sdmaclea
Copy link
Contributor

@sdmaclea sdmaclea commented Mar 2, 2018

@jashook @janvorli

Needs coreclr linux-arm64 in nuget

PTAL let me know if there is anything else which I might have missed.

@@ -74,6 +74,20 @@
"Platform": "arm",
"Type": "build/product/"
}
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is for official builds, not for the CI. The CI is defined in the .groovy files in the same folder.
We cannot enable official builds of corefx until official build of coreclr produces nuget packages and uploads them to myget.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Creating PR keeps it in my queue..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janvorli Were you suggesting I split this PR? Can corefx CI be enabled before coreclr official builds?

@sdmaclea sdmaclea changed the title [Arm64/Linux] Enable CI WIP [Arm64/Linux] Enable CI Mar 7, 2018
__currentWorkingDirectory=`pwd`

# choose Docker image
__dockerImage=" hseok82/dotnet-buildtools-prereqs:ubuntu-16.04-crossx86-ef0ac75-20175511035548"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be using hseok82, instead we should be using official dotnet-builtools images.

/cc @MichaelSimons

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Dockerfile for the image can be checked into https://github.com/dotnet/dotnet-buildtools-prereqs-docker. Once this is done, it will get build and pushed to microsoft/dotnet-buildtools-prereqs

# choose Docker image
__dockerImage=" hseok82/dotnet-buildtools-prereqs:ubuntu-16.04-crossx86-ef0ac75-20175511035548"
__dockerEnvironmentVariables="-e ROOTFS_DIR=/crossrootfs/arm64"
__runtimeOS="ubuntu.16.04"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

--buildConfig=*)
__buildConfig="$(echo ${arg#*=} | awk '{print tolower($0)}')"
if [[ "$__buildConfig" != "debug" && "$__buildConfig" != "release" ]]; then
exit_with_error "--buildConfig can be only Debug or Release" true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the true for at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied code from x86 script had the same extra argument

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

esac
done

set -x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these set -x only for debugging? Or do we want them in the final code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept it the same as x86

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

@sdmaclea sdmaclea force-pushed the PR-ARM64-Enable-CI branch from 30fd822 to 298434b Compare March 9, 2018 22:54
@sdmaclea sdmaclea changed the title WIP [Arm64/Linux] Enable CI WIP [Arm64/Linux] Enable CI & Official builds Mar 9, 2018
@@ -345,6 +345,61 @@ def targetGroupOsMapInnerloop = ['netcoreapp': ['Windows_NT', 'Ubuntu14.04', 'Ub
} // targetGroup
} // isPR

// **************************
// Define Linux ARM64 cross builds. These jobs run on every merge.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of duplicating this whole section, would it make more sense to refactor it into the above section: "Define Linux ARM builds."?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was drafted from the simpler x86 section below. The arm version has more complications with its multiple flavors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eerhardt
There are subtle differences between x86, arm and arm64 which makes it complicated to merge these.
My groovy skills are immature at the moment. I would prefer to crawl before trying to walk.

If you disagree I can take a look tomorrow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, that's fine with me.

@sdmaclea sdmaclea force-pushed the PR-ARM64-Enable-CI branch from 298434b to 277f188 Compare March 16, 2018 02:07
@sdmaclea sdmaclea changed the title WIP [Arm64/Linux] Enable CI & Official builds [Arm64/Linux] Enable CI & Official builds Mar 18, 2018
@sdmaclea
Copy link
Contributor Author

#28112 has merged which should include arm64 offical CoreClr builds dotnet/coreclr#16981

We should be able to enable corefx official builds now. Changed title...

@sdmaclea
Copy link
Contributor Author

D:\j\workspace\windows-TGrou---33cbf18b>taskkill /F /IM VBCSCompiler.exe 
ERROR: The process "VBCSCompiler.exe" not found.
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done

test UWP CoreCLR x64 Debug Build

@sdmaclea
Copy link
Contributor Author

@dotnet-bot help

@dotnet-bot
Copy link

Welcome to the dotnet/corefx Repository

The following is a list of valid commands on this PR. To invoke a command, comment the indicated phrase on the PR

The following commands are valid for all PRs and repositories.

Click to expand
Comment Phrase Action
@dotnet-bot test this please Re-run all legs. Use sparingly
@dotnet-bot test ci please Generates (but does not run) jobs based on changes to the groovy job definitions in this branch
@dotnet-bot help Print this help message

The following jobs are launched by default for each PR against dotnet/corefx:master.

Click to expand
Comment Phrase Job Launched
@dotnet-bot test Alpine.3.6 x64 Debug Build Alpine.3.6 x64 Debug Build
@dotnet-bot test Linux x64 Release Build Linux x64 Release Build
@dotnet-bot test OSX x64 Debug Build OSX x64 Debug Build
@dotnet-bot test Packaging All Configurations x64 Debug Build Packaging All Configurations x64 Debug Build
@dotnet-bot test Windows x64 Debug Build Windows x64 Debug Build
@dotnet-bot test Windows x86 Release Build Windows x86 Release Build
@dotnet-bot test NETFX x86 Release Build NETFX x86 Release Build
@dotnet-bot test UWP CoreCLR x64 Debug Build UWP CoreCLR x64 Debug Build
@dotnet-bot test UWP NETNative x86 Release Build UWP NETNative x86 Release Build

The following optional jobs are available in PRs against dotnet/corefx:master.

Click to expand
Comment Phrase Job Launched
@dotnet-bot test Outerloop Alpine.3.6 x64 Debug Build Queues Outerloop Alpine.3.6 x64 Debug Build
@dotnet-bot test Alpine.3.6 x64 Release Build Queues Alpine.3.6 x64 Release Build
@dotnet-bot test Outerloop Alpine.3.6 x64 Release Build Queues Outerloop Alpine.3.6 x64 Release Build
@dotnet-bot test CentOS.6 x64 Debug Build Queues CentOS.6 x64 Debug Build
@dotnet-bot test Outerloop CentOS.6 x64 Debug Build Queues Outerloop CentOS.6 x64 Debug Build
@dotnet-bot test CentOS.6 x64 Release Build Queues CentOS.6 x64 Release Build
@dotnet-bot test Outerloop CentOS.6 x64 Release Build Queues Outerloop CentOS.6 x64 Release Build
@dotnet-bot test Linux x64 Debug Build Queues Linux x64 Debug Build
@dotnet-bot test Outerloop Linux x64 Debug Build Queues Outerloop Linux x64 Debug Build
@dotnet-bot test Outerloop Linux x64 Release Build Queues Outerloop Linux x64 Release Build
@dotnet-bot test Outerloop OSX x64 Debug Build Queues Outerloop OSX x64 Debug Build
@dotnet-bot test OSX x64 Release Build Queues OSX x64 Release Build
@dotnet-bot test Outerloop OSX x64 Release Build Queues Outerloop OSX x64 Release Build
@dotnet-bot test Outerloop Packaging All Configurations x64 Debug Build Queues Outerloop Packaging All Configurations x64 Debug Build
@dotnet-bot test Packaging All Configurations x86 Debug Build Queues Packaging All Configurations x86 Debug Build
@dotnet-bot test Outerloop Packaging All Configurations x86 Debug Build Queues Outerloop Packaging All Configurations x86 Debug Build
@dotnet-bot test Packaging All Configurations x64 Release Build Queues Packaging All Configurations x64 Release Build
@dotnet-bot test Outerloop Packaging All Configurations x64 Release Build Queues Outerloop Packaging All Configurations x64 Release Build
@dotnet-bot test Packaging All Configurations x86 Release Build Queues Packaging All Configurations x86 Release Build
@dotnet-bot test Outerloop Packaging All Configurations x86 Release Build Queues Outerloop Packaging All Configurations x86 Release Build
@dotnet-bot test Outerloop Windows x64 Debug Build Queues Outerloop Windows x64 Debug Build
@dotnet-bot test Windows x86 Debug Build Queues Windows x86 Debug Build
@dotnet-bot test Outerloop Windows x86 Debug Build Queues Outerloop Windows x86 Debug Build
@dotnet-bot test Windows x64 Release Build Queues Windows x64 Release Build
@dotnet-bot test Outerloop Windows x64 Release Build Queues Outerloop Windows x64 Release Build
@dotnet-bot test Outerloop Windows x86 Release Build Queues Outerloop Windows x86 Release Build
@dotnet-bot test NETFX x64 Debug Build Queues NETFX x64 Debug Build
@dotnet-bot test Outerloop NETFX x64 Debug Build Queues Outerloop NETFX x64 Debug Build
@dotnet-bot test NETFX x86 Debug Build Queues NETFX x86 Debug Build
@dotnet-bot test Outerloop NETFX x86 Debug Build Queues Outerloop NETFX x86 Debug Build
@dotnet-bot test NETFX x64 Release Build Queues NETFX x64 Release Build
@dotnet-bot test Outerloop NETFX x64 Release Build Queues Outerloop NETFX x64 Release Build
@dotnet-bot test Outerloop NETFX x86 Release Build Queues Outerloop NETFX x86 Release Build
@dotnet-bot test Outerloop UWP CoreCLR x64 Debug Build Queues Outerloop UWP CoreCLR x64 Debug Build
@dotnet-bot test UWP CoreCLR x86 Debug Build Queues UWP CoreCLR x86 Debug Build
@dotnet-bot test Outerloop UWP CoreCLR x86 Debug Build Queues Outerloop UWP CoreCLR x86 Debug Build
@dotnet-bot test UWP CoreCLR x64 Release Build Queues UWP CoreCLR x64 Release Build
@dotnet-bot test Outerloop UWP CoreCLR x64 Release Build Queues Outerloop UWP CoreCLR x64 Release Build
@dotnet-bot test UWP CoreCLR x86 Release Build Queues UWP CoreCLR x86 Release Build
@dotnet-bot test Outerloop UWP CoreCLR x86 Release Build Queues Outerloop UWP CoreCLR x86 Release Build
@dotnet-bot test UWP NETNative x64 Debug Build Queues UWP NETNative x64 Debug Build
@dotnet-bot test Outerloop UWP NETNative x64 Debug Build Queues Outerloop UWP NETNative x64 Debug Build
@dotnet-bot test UWP NETNative x86 Debug Build Queues UWP NETNative x86 Debug Build
@dotnet-bot test Outerloop UWP NETNative x86 Debug Build Queues Outerloop UWP NETNative x86 Debug Build
@dotnet-bot test UWP NETNative x64 Release Build Queues UWP NETNative x64 Release Build
@dotnet-bot test Outerloop UWP NETNative x64 Release Build Queues Outerloop UWP NETNative x64 Release Build
@dotnet-bot test Outerloop UWP NETNative x86 Release Build Queues Outerloop UWP NETNative x86 Release Build

Have a nice day!

@sdmaclea
Copy link
Contributor Author

@dotnet-bot test ci please

@dotnet-bot
Copy link

Welcome to the dotnet/corefx Repository

The following is a list of valid commands on this PR. To invoke a command, comment the indicated phrase on the PR

The following commands are valid for all PRs and repositories.

Click to expand
Comment Phrase Action
@dotnet-bot test this please Re-run all legs. Use sparingly
@dotnet-bot test ci please Generates (but does not run) jobs based on changes to the groovy job definitions in this branch
@dotnet-bot help Print this help message

The following jobs are launched by default for each PR against dotnet/corefx:master.

Click to expand
Comment Phrase Job Launched
@dotnet-bot test Linux arm Release Build Linux arm Release Build
@dotnet-bot test Tizen armel Debug Build Tizen armel Debug Build

The following optional jobs are available in PRs against dotnet/corefx:master.

Click to expand
Comment Phrase Job Launched
@dotnet-bot test innerloop CentOS7.1 Debug Queues Innerloop CentOS7.1 Debug x64 Build and Test
@dotnet-bot test innerloop CentOS7.1 Release Queues Innerloop CentOS7.1 Release x64 Build and Test
@dotnet-bot test code coverage Queues Code Coverage Windows Debug
@dotnet-bot test innerloop Debian8.4 Debug Queues Innerloop Debian8.4 Debug x64 Build and Test
@dotnet-bot test innerloop Debian8.4 Release Queues Innerloop Debian8.4 Release x64 Build and Test
@dotnet-bot test innerloop Fedora24 Debug Queues Innerloop Fedora24 Debug x64 Build and Test
@dotnet-bot test innerloop Fedora24 Release Queues Innerloop Fedora24 Release x64 Build and Test
@dotnet-bot test Linux arm Debug Queues Linux arm Debug Build
@dotnet-bot test code formatter check Queues Code Formatter Check
@dotnet-bot test innerloop OSX10.12 Debug Queues Innerloop OSX10.12 Debug x64 Build and Test
@dotnet-bot test innerloop OSX10.12 Release Queues Innerloop OSX10.12 Release x64 Build and Test
@dotnet-bot test outerloop netcoreapp CentOS7.1 Debug Queues OuterLoop netcoreapp CentOS7.1 Debug x64
@dotnet-bot test outerloop netcoreapp CentOS7.1 Release Queues OuterLoop netcoreapp CentOS7.1 Release x64
@dotnet-bot test outerloop netcoreapp Debian8.4 Debug Queues OuterLoop netcoreapp Debian8.4 Debug x64
@dotnet-bot test outerloop netcoreapp Debian8.4 Release Queues OuterLoop netcoreapp Debian8.4 Release x64
@dotnet-bot test outerloop netcoreapp Fedora24 Debug Queues OuterLoop netcoreapp Fedora24 Debug x64
@dotnet-bot test outerloop netcoreapp Fedora24 Release Queues OuterLoop netcoreapp Fedora24 Release x64
@dotnet-bot test outerloop netcoreapp OSX10.12 Debug Queues OuterLoop netcoreapp OSX10.12 Debug x64
@dotnet-bot test outerloop netcoreapp OSX10.12 Release Queues OuterLoop netcoreapp OSX10.12 Release x64
@dotnet-bot test outerloop netcoreapp PortableLinux Debug Queues OuterLoop netcoreapp PortableLinux Debug x64
@dotnet-bot test outerloop netcoreapp PortableLinux Release Queues OuterLoop netcoreapp PortableLinux Release x64
@dotnet-bot test outerloop netcoreapp RHEL7.2 Debug Queues OuterLoop netcoreapp RHEL7.2 Debug x64
@dotnet-bot test outerloop netcoreapp RHEL7.2 Release Queues OuterLoop netcoreapp RHEL7.2 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu14.04 Debug Queues OuterLoop netcoreapp Ubuntu14.04 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu14.04 Release Queues OuterLoop netcoreapp Ubuntu14.04 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.04 Debug Queues OuterLoop netcoreapp Ubuntu16.04 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.04 Release Queues OuterLoop netcoreapp Ubuntu16.04 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.10 Debug Queues OuterLoop netcoreapp Ubuntu16.10 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.10 Release Queues OuterLoop netcoreapp Ubuntu16.10 Release x64
@dotnet-bot test outerloop netcoreapp Windows 7 Debug Queues OuterLoop netcoreapp Windows 7 Debug x64
@dotnet-bot test outerloop netcoreapp Windows 7 Release Queues OuterLoop netcoreapp Windows 7 Release x64
@dotnet-bot test outerloop netcoreapp Windows_NT Debug Queues OuterLoop netcoreapp Windows_NT Debug x64
@dotnet-bot test outerloop netcoreapp Windows_NT Release Queues OuterLoop netcoreapp Windows_NT Release x64
@dotnet-bot test innerloop PortableLinux Debug Queues Innerloop PortableLinux Debug x64 Build and Test
@dotnet-bot test innerloop PortableLinux Release Queues Innerloop PortableLinux Release x64 Build and Test
@dotnet-bot test innerloop RHEL7.2 Debug Queues Innerloop RHEL7.2 Debug x64 Build and Test
@dotnet-bot test innerloop RHEL7.2 Release Queues Innerloop RHEL7.2 Release x64 Build and Test
@dotnet-bot test Tizen armel Release Queues Tizen armel Release Build
@dotnet-bot test innerloop Ubuntu14.04 Debug Queues Innerloop Ubuntu14.04 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu14.04 Release Queues Innerloop Ubuntu14.04 Release x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.04 Debug Queues Innerloop Ubuntu16.04 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.04 Release Queues Innerloop Ubuntu16.04 Release x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.10 Debug Queues Innerloop Ubuntu16.10 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.10 Release Queues Innerloop Ubuntu16.10 Release x64 Build and Test
@dotnet-bot test innerloop Windows_NT Debug Queues Innerloop Windows_NT Debug x86 Build and Test
@dotnet-bot test innerloop Windows_NT Release Queues Innerloop Windows_NT Release x64 Build and Test

Have a nice day!

@sdmaclea
Copy link
Contributor Author

I don't see any evidence the last trigger did anything. Trying again

@dotnet-bot test ci please

@sdmaclea
Copy link
Contributor Author

I see no evidence that either previous ci trigger did anything. Maybe I do not have permission to trigger?

Is there a way we can try the new CI triggers this will introduce before merging?

@sdmaclea
Copy link
Contributor Author

test OSX x64 Debug Build
test Windows x64 Debug Build

@eerhardt
Copy link
Member

@dotnet/dnceng - anyone know why @dotnet-bot test ci please isn't working?

@eerhardt
Copy link
Member

hmm, now it works...

@jonfortescue
Copy link
Contributor

@eerhardt Looks like a transient issue. Ping us again if it doesn't seem to be working.

@sdmaclea
Copy link
Contributor Author

CI build logs look OK to me, but I have never really looked at them

I don't think the x64 failures could possibly be related, but re-triggering
test Windows x64 Debug Build please
test OSX x64 Debug Build please

@sdmaclea
Copy link
Contributor Author

@weshaggard Do you need to review this?

@sdmaclea
Copy link
Contributor Author

Looks like all the OSX builds are failing right now so the error above can be ignored.

@eerhardt Looks like you are assigned to review this. Can this be approved/merged?

Copy link
Member

@janvorli janvorli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@janvorli
Copy link
Member

@DrewScoggins the OSX CI is failing due to an issue in the dumpling.py, can you please take a look?

15:49:38   Traceback (most recent call last):
15:49:38     File "/Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/Tools/DumplingHelper.py", line 131, in <module>
15:49:38       main(sys.argv)
15:49:38     File "/Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/Tools/DumplingHelper.py", line 113, in main
15:49:38       install_dumpling()
15:49:38     File "/Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/Tools/DumplingHelper.py", line 34, in install_dumpling
15:49:38       print("An unexpected error was encountered while installing dumpling.py: " + sys.exc_info()[0])
15:49:38   TypeError: cannot concatenate 'str' and 'type' objects

See the https://ci3.dot.net/job/dotnet_corefx/job/master/job/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/9995/ for the full log.

@janvorli janvorli merged commit 422dd24 into dotnet:master Mar 20, 2018
@janvorli
Copy link
Member

@DrewScoggins I have just found that our official builds are failing with the same issue on Linux and Alpine x64 too.

@karelz karelz added this to the 2.1.0 milestone Mar 27, 2018
@DrewScoggins
Copy link
Member

Sorry I was out of the office and just saw this. I should have all of the dumpling stuff wrapped up this week so we should be able to turn all of this back on.

ericstj pushed a commit to ericstj/corefx that referenced this pull request Mar 28, 2018
picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants