Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch Publishing Profiles from maccore to SecureFiles #22101

Merged
merged 55 commits into from
Feb 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
bb6d62e
Remove maccore
mcumming Feb 3, 2025
5fd566d
Add install-qa-provisioning-profiles template to replace the shell sc…
mcumming Feb 3, 2025
5fadf62
Attempt to fix yml
mcumming Feb 3, 2025
6602471
Only install certs once
mcumming Feb 4, 2025
57b1e6a
Remove all instances of install-qa-provisioning-profiles
mcumming Feb 4, 2025
c326b0a
Remove maccore checkouts
mcumming Feb 4, 2025
513de60
Remove clean_git_vsts.ps1
mcumming Feb 4, 2025
96951c4
Add build number to comment header for easier correlation
mcumming Feb 4, 2025
bd02e06
Clean CredScanSuppressions
mcumming Feb 4, 2025
a95647b
Misses maccore exclusion
mcumming Feb 4, 2025
2c659e1
Move compliance files from maccore
mcumming Feb 5, 2025
76fecbb
Enable verboseOutput
mcumming Feb 5, 2025
5145e4b
Fix pathing
mcumming Feb 5, 2025
24e4a28
re-add maccore to test
mcumming Feb 5, 2025
4e1290b
try removing again
mcumming Feb 5, 2025
39b3a12
Apply a path to the xamarin-macios repository
mcumming Feb 5, 2025
6273cea
more of the same
mcumming Feb 5, 2025
b32ef83
remove macios.ci repo
mcumming Feb 5, 2025
1761f10
Use commit hash in comment header
mcumming Feb 5, 2025
a906448
fix indentation
mcumming Feb 5, 2025
95d7e3a
Fix Substring usage
mcumming Feb 6, 2025
66c069a
Add logging
mcumming Feb 7, 2025
d2747d2
missing directory seperator
mcumming Feb 11, 2025
7400f3e
Fix yamllint.
mandel-macaque Feb 11, 2025
a133662
Copy in just one step
mcumming Feb 11, 2025
68cbaca
Remove old commented code
mcumming Feb 11, 2025
ad4d704
revert env var changes
mcumming Feb 12, 2025
491dca8
Fix passing pipeline object to powershell
mcumming Feb 12, 2025
e6a2776
Move shell script to file
mcumming Feb 12, 2025
2c769e2
quote the json
mcumming Feb 12, 2025
8195e10
Use other quotes
mcumming Feb 12, 2025
a1051c6
Fix variable names, appears bash script path are set in bash script
mcumming Feb 12, 2025
3e204d3
Make shellcheck happy
mcumming Feb 12, 2025
0f018b1
Try System.DefaultWorkingDirectory
mcumming Feb 13, 2025
bbf159d
again
mcumming Feb 13, 2025
0551224
use right filename
mcumming Feb 13, 2025
3fd29a5
Rename file to be consistent with other scripts
mcumming Feb 13, 2025
3827862
Run the script in the right folder, don't change directory in script
mcumming Feb 13, 2025
f5bda73
use right repo name
mcumming Feb 13, 2025
899e1de
Duh
mcumming Feb 13, 2025
c3b3467
Do not call provisionator
mcumming Feb 14, 2025
facf965
skip only provisionator
rolfbjarne Feb 14, 2025
7b22a5c
Look in the right directory
rolfbjarne Feb 14, 2025
88f2728
Use auth token from env to unlock certs
mcumming Feb 14, 2025
163661f
Clear the Repo title if the directory end with it
mcumming Feb 15, 2025
fb03e13
Remove extra directory
mcumming Feb 17, 2025
1927226
More directory changes
mcumming Feb 18, 2025
4e409b7
Add parameter for path to macios repository
mcumming Feb 18, 2025
b750c35
reference the parameter correctly
mcumming Feb 18, 2025
79b922d
Fix cd to change to the right subfolder
mcumming Feb 18, 2025
042ccbe
No, really
mcumming Feb 18, 2025
b214569
remove '-' in parameter name
mcumming Feb 18, 2025
20380b2
if you change it in the template, you have to change it where it's ca…
mcumming Feb 18, 2025
65b6479
Add REPO_PATH to coallese different path env vars
mcumming Feb 18, 2025
7ca4725
One more path issue
mcumming Feb 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions Make.config
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,6 @@ MONO_PATH=$(TOP)/external/mono
XAMARIN_MACDEV_PATH=$(TOP)/external/Xamarin.MacDev
MONOTOUCH_DIALOG_PATH=$(TOP)/external/MonoTouch.Dialog
API_TOOLS_PATH=$(TOP)/external/api-tools
MACCORE_PATH=$(TOP)/../maccore
ADR_PATH:=$(abspath $(TOP)/../macios-adr)

MONO_PREFIX ?= /Library/Frameworks/Mono.framework/Versions/Current
Expand All @@ -329,10 +328,6 @@ PRODUCT_KEY_PATH?=$(TOP)/product.snk

include $(TOP)/mk/quiet.mk

ifdef ENABLE_XAMARIN
-include $(MACCORE_PATH)/Make.config
endif

JENKINS_RESULTS_DIRECTORY ?= $(abspath $(TOP)/jenkins-results)

# Clone files instead of copying them on APFS file systems. Much faster.
Expand Down
6 changes: 0 additions & 6 deletions create-make-config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,4 @@ if which ccache > /dev/null 2>&1; then
echo "Found ccache on the system, enabling it"
fi

# Detect maccore / xamarin
if test -d ../maccore; then
printf "ENABLE_XAMARIN=1\n" >> "$OUTPUT_FILE"
echo "Detected the maccore repository, automatically enabled the Xamarin build"
fi

mv "$OUTPUT_FILE" "$OUTPUT"
12 changes: 0 additions & 12 deletions mk/xamarin.mk
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,6 @@ MONO_BRANCH := $(shell cd $(MONO_PATH) 2> /dev/null && git symbolic-ref --sho
endif

ifdef ENABLE_XAMARIN
NEEDED_MACCORE_VERSION := 5805eb9adacd8f2aeb2852ffbc78b24fb4a43866
NEEDED_MACCORE_BRANCH := main

MACCORE_DIRECTORY := maccore
MACCORE_MODULE := [email protected]:xamarin/maccore.git
MACCORE_VERSION := $(shell cd $(MACCORE_PATH) 2> /dev/null && git rev-parse HEAD 2> /dev/null)
MACCORE_BRANCH := $(shell cd $(MACCORE_PATH) 2> /dev/null && git symbolic-ref --short HEAD 2> /dev/null)


NEEDED_ADR_VERSION := adccd1c48a7c2540b0c37f2bf7add7d2c1398729
NEEDED_ADR_BRANCH := main

Expand Down Expand Up @@ -109,13 +100,10 @@ endif

ifdef ENABLE_XAMARIN
$(MACCORE_PATH):
$(Q) git clone --recursive $(MACCORE_MODULE) $(MACCORE_PATH)
$(Q) $(MAKE) reset-maccore

$(ADR_PATH):
$(Q) git clone --recursive $(ADR_MODULE) $(ADR_PATH)
$(Q) $(MAKE) reset-adr

$(eval $(call CheckVersionTemplate,maccore,MACCORE))
$(eval $(call CheckVersionTemplate,macios-adr,ADR))
endif
6 changes: 0 additions & 6 deletions tools/devops/automation/build-cronjob.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,6 @@ resources:
ref: refs/heads/main
endpoint: xamarin

- repository: maccore
type: github
name: xamarin/maccore
ref: refs/heads/main
endpoint: xamarin

trigger:
- main

Expand Down
6 changes: 0 additions & 6 deletions tools/devops/automation/build-lego.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,6 @@ resources:
ref: refs/heads/main
endpoint: xamarin

- repository: maccore
type: github
name: xamarin/maccore
ref: refs/heads/main
endpoint: xamarin

trigger:
branches:
include:
Expand Down
11 changes: 3 additions & 8 deletions tools/devops/automation/build-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,12 +129,6 @@ resources:
ref: refs/heads/main
endpoint: xamarin

- repository: maccore
type: github
name: xamarin/maccore
ref: refs/heads/main
endpoint: xamarin

- repository: macios-adr
type: git
name: macios-adr
Expand Down Expand Up @@ -205,10 +199,11 @@ extends:
sourceRepositoriesToScan:
runInSingleJob: true # run both maccore and macios in the same job
include:
- repository: maccore
- repository: self
path: xamarin-macios
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will change and many more once we go to dotnet/macios right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a variable maybe that we can use/create like repoName or something similar?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but lets do that in a separate PR for dotnet migration preparation?

exclude:
- repository: yaml-templates
- repository: macios-adr # macios-adr has it's own build pipeline and should not be included in the source analysis
- repository: macios-adr # macios-adr has it's own build pipeline and should not be included in the source analysis
stages:
- template: templates/main-stage.yml
parameters:
Expand Down
11 changes: 4 additions & 7 deletions tools/devops/automation/build-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,12 +120,6 @@ resources:
ref: refs/heads/main
endpoint: xamarin

- repository: maccore
type: github
name: xamarin/maccore
ref: refs/heads/main
endpoint: xamarin

- repository: macios-adr
type: git
name: macios-adr
Expand Down Expand Up @@ -191,12 +185,15 @@ extends:
enabled: false # we run our on sbom generation
credscan:
suppressionsFile: '$(System.DefaultWorkingDirectory)/$(BUILD_REPOSITORY_TITLE)/tools/devops/governance/CredScanSuppressions.json'
debugMode: true
verboseOutput: true
policheck:
exclusionsFile: '$(System.DefaultWorkingDirectory)/$(BUILD_REPOSITORY_TITLE)/tools/devops/governance/PoliCheckExclusions.xml'
sourceRepositoriesToScan:
runInSingleJob: true # run both maccore and macios in the same job
include:
- repository: maccore
- repository: self
path: xamarin-macios
exclude:
- repository: yaml-templates
- repository: macios-adr # macios-adr has it's own build pipeline and should not be included in the source analysis
Expand Down
12 changes: 0 additions & 12 deletions tools/devops/automation/publish-pr-html-results.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,6 @@ resources:
ref: refs/heads/main
endpoint: xamarin

- repository: maccore
type: github
name: xamarin/maccore
ref: refs/heads/main
endpoint: xamarin

- repository: macios.ci
type: github
name: xamarin/macios.ci
ref: refs/heads/main # get main and create the branch from main
endpoint: xamarin

# we need all stages to be completed, else we do not have the test results, this trigger is just for CI, because we have
# but because we have device issues, and it needs to be gree to trigger, we will deal with it later
pipelines:
Expand Down
3 changes: 2 additions & 1 deletion tools/devops/automation/run-post-ci-build-vs-insertion.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ extends:
sourceRepositoriesToScan:
runInSingleJob: true # run both maccore and macios in the same job
include:
- repository: maccore
- repository: self
path: xamarin-macios
exclude:
- repository: yaml-templates
- repository: macios-adr # macios-adr has it's own build pipeline and should not be included in the source analysis
Expand Down
4 changes: 2 additions & 2 deletions tools/devops/automation/scripts/GitHub.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -292,9 +292,9 @@ class GitHubComments {
}

if ([string]::IsNullOrEmpty($Env:PR_ID)) {
$prefix = "[CI Build]"
$prefix = "[CI Build #$($this.Hash.Substring(0,7))]"
} else {
$prefix = "[PR Build]"
$prefix = "[PR Build #$($this.Hash.Substring(0,7))]"
}

$stringBuilder.AppendLine("# $commentEmoji $prefix $commentTitle $commentEmoji")
Expand Down
21 changes: 14 additions & 7 deletions tools/devops/automation/scripts/bash/install-workloads.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,16 @@ fi
if test -z "${BUILD_REPOSITORY_TITLE:-}"; then
BUILD_REPOSITORY_TITLE="$(basename "$(git remote get-url --push origin)")"
fi

# Don't assume we're in the right directory (makes it easier to run the script
# locally).
cd "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE"
if [[ "$BUILD_SOURCESDIRECTORY" == *"$BUILD_REPOSITORY_TITLE" ]]; then
REPO_FOLDER=$BUILD_SOURCESDIRECTORY
else
REPO_FOLDER=$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE
fi
cd "${REPO_FOLDER}"


# Validate a few things
ARTIFACTS_PATH=$BUILD_SOURCESDIRECTORY/artifacts
Expand All @@ -47,15 +54,15 @@ make global.json

make -C builds dotnet

var=$(make -C "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE/tools/devops" print-variable VARIABLE=DOTNET)
var=$(make -C "${REPO_FOLDER}/tools/devops" print-variable VARIABLE=DOTNET)
DOTNET=${var#*=}
echo "Using dotnet found at $DOTNET"

var=$(make -C "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE/tools/devops" print-variable VARIABLE=DOTNET_PLATFORMS)
var=$(make -C "${REPO_FOLDER}/tools/devops" print-variable VARIABLE=DOTNET_PLATFORMS)
DOTNET_PLATFORMS=${var#*=}
echo "Dotnet platforms are '$DOTNET_PLATFORMS'"

var=$(make -C "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE/tools/devops" print-abspath-variable VARIABLE=DOTNET_NUPKG_DIR)
var=$(make -C "${REPO_FOLDER}/tools/devops" print-abspath-variable VARIABLE=DOTNET_NUPKG_DIR)
DOTNET_NUPKG_DIR=${var#*=}
echo "Using nuget dir $DOTNET_NUPKG_DIR"

Expand Down Expand Up @@ -85,9 +92,9 @@ done
for platform in $DOTNET_PLATFORMS; do
unzip -l "$DOTNET_NUPKG_DIR"/Microsoft."$platform".Bundle.*.zip
unzip "$DOTNET_NUPKG_DIR"/Microsoft."$platform".Bundle.*.zip -d tmpdir
rsync -av tmpdir/dotnet/sdk-manifests/* "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE"/builds/downloads/dotnet-*/sdk-manifests/
rsync -av tmpdir/dotnet/sdk-manifests/* "${REPO_FOLDER}"/builds/downloads/dotnet-*/sdk-manifests/
done
find "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE"/builds/downloads/dotnet-*
find "${REPO_FOLDER}"/builds/downloads/dotnet-*

# PLATFORMS may be empty/no values
set +u
Expand All @@ -96,6 +103,6 @@ if [ ${#PLATFORMS[@]} -gt 0 ]; then
fi
set -u

var=$(make -C "$BUILD_SOURCESDIRECTORY/$BUILD_REPOSITORY_TITLE/tools/devops" print-variable VARIABLE=DOTNET_DIR)
var=$(make -C "${REPO_FOLDER}/tools/devops" print-variable VARIABLE=DOTNET_DIR)
DOTNET_DIR=${var#*=}
ls -lR "$DOTNET_DIR"
15 changes: 0 additions & 15 deletions tools/devops/automation/scripts/clean_git_vsts.ps1

This file was deleted.

2 changes: 1 addition & 1 deletion tools/devops/automation/scripts/fetch-remote-binlogs.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Invoke-SshCommand `
-RemoteHost "$Env:MAC_AGENT_IP" `
-RemoteUserName "$Env:MAC_AGENT_USER" `
-- `
"$Env:MAC_AGENT_BUILD_SOURCESDIRECTORY/$Env:BUILD_REPOSITORY_TITLE/tests/dotnet/Windows/collect-binlogs.sh"
"$Env:MAC_AGENT_BUILD_SOURCESDIRECTORY/tests/dotnet/Windows/collect-binlogs.sh"

New-Item -Path "$Env:BUILD_ARTIFACTSTAGINGDIRECTORY" -Name "windows-binlogs" -ItemType "directory"
New-Item -Path "$Env:BUILD_ARTIFACTSTAGINGDIRECTORY\windows-binlogs" -Name "windows-remote-logs.zip" -ItemType "file" -Value "zip"
Expand Down
Loading