Migrate docker containers to latest tags

[michellemcdaniel]

This change moves all of the docker images to the latest tag as part of dotnet/dnceng#1294.

This change also moves the containers that no longer exist to replacements (for example, centos-8-rpmpkg no longer exists, and that was moved to centos-7-rpmpkg).

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

[michellemcdaniel]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[michellemcdaniel]

@wfurt @wtgodbe I think the change to how fpm is installed on the centos-7-rpmpkg container is causing the "FPM cannot be found" issue (https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/78d25937ee7c831b1c368ff0b0c3dce043cb98cb/src/centos/7/rpmpkg/Dockerfile#L11). On the centos-8-rpmpkg container (which no longer exists), we just ran

gem install --no-document fpm 

Do you know if the current way is installing fpm to a different place other than /usr/bin/env?

[wtgodbe]

I do not, I don't know much about how that works - I just copy/pasted the original workaround from aspnetcore 😢

[michellemcdaniel]

I'll create a docker container tomorrow and try to figure out where fpm is installed on it

[wfurt]

fpm is only needed to create msquic packages. Where so you see the error?
Also the quic/http3 failures looks suspicious. That was fixed by #76267 yesterday.
The images need at least msquic 2.1.2.

[wfurt]

cc: @rzikm

[michellemcdaniel]

fpm is only needed to create msquic packages. Where so you see the error?
Also the quic/http3 failures looks suspicious. That was fixed by #76267 yesterday.
The images need at least msquic 2.1.2.

The errors are in a bunch of the legs, for example: https://dev.azure.com/dnceng-public/public/_build/results?buildId=40304&view=logs&j=36f5df04-f069-50e9-49af-755a9549bd25&t=bdd8d2a1-63f8-59f1-8e0a-89cd06102b80&l=48
The msquic changes will roll out to production tomorrow, so I can rerun those legs post-rollout.

[wfurt]

ah, I was actually wrong @adiaaida. I add use of fpm recently for msquic, but that was mimicked after some runtime code.
It seems like the executable is truly missing in the docker image. We will probably need to investigate and rebuild the container(s)

[michellemcdaniel]

I think I have a working version of the docker image, after much hair pulling. I would like to build runtime/this scenario inside the container to make sure it works properly. Who is the right person to help me with the correct set of build commands?

FWIW, fpm is there, but it gets installed inside of the scl shell, and is in an out of the way place (/opt/rh/rh-ruby26/root/usr/local/bin/fpm), so my fix is to add things to the path/ld_library and a few other env variables so that things installed inside of scl are available outside. I want to make sure that adding those various things to the path doesn't break anything else.

[wfurt]

we may link it as well if needed. I don't think each stage/pipeline shows the commands at the beginning and they differ e.g. I don't think there is simple command set....

[michellemcdaniel]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[michellemcdaniel]

@dotnet/runtime-infrastructure I believe the failures in extra-platforms are all known failures. The base runtime pipeline is green. PTAL.

[wfurt]

Only one suspicious is AOT build failure

  Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
  Generating native code
EXEC : error : Failed to load assembly 'SQLitePCLRaw.core' [/__w/1/s/src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Unit.Tests/ComInterfaceGenerator.Unit.Tests.csproj]
##[error]EXEC(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Failed to load assembly 'SQLitePCLRaw.core'

Do you know @jkotas if that is normal/expected?

[michellemcdaniel]

The AOT failures are #77476. I believe there is a PR to fix it or disable a test, but it is unrelated to this change

[jkotas]

Only one suspicious is AOT build failure

This should be fixed by #77522 that was just merged.

[michellemcdaniel]

I might need to merge in main to get those changes. I think rerunning those legs will still run on the old commit sha which didn't have the fix.

[michellemcdaniel]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[michellemcdaniel]

Alrighty, I think this is good. I believe all of the failures currently in extra-platforms are on native runs, not on docker containers. PTAL

[michellemcdaniel]

@dotnet/runtime-infrastructure please take a look

[agocke]

Left a comment about the centos downgrade, but it looks like you investigated the fpm issues and fixed them. If so, LGTM