Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update CI to use kokoro and build for OSX, Windows, Linux #214

Closed
wants to merge 108 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
9873c73
Revert "fix: Update CI to use GitHub actions and build for osx, windo…
vishwarajanand May 15, 2024
45c3025
remove win py release
vishwarajanand May 15, 2024
6709f4f
remove password to prevent releases
vishwarajanand May 15, 2024
d91f68d
Upgrade python to 3.9
vishwarajanand May 15, 2024
04dcdd6
Remove duplicate publish-reporter-script
vishwarajanand May 17, 2024
977a01e
fixed osx venv path
vishwarajanand May 17, 2024
fe4dbbe
adding linux fix for libffi
vishwarajanand May 18, 2024
6133329
fix osc python path
vishwarajanand May 18, 2024
394e275
install python in osx
vishwarajanand May 22, 2024
38850d3
fix python paths in osx build
vishwarajanand May 22, 2024
8d862cd
fixed python path in release script
vishwarajanand May 22, 2024
84d9f77
fix python release script
vishwarajanand May 22, 2024
2552542
fix release script
vishwarajanand May 22, 2024
9f96157
add verbose debug on osx
vishwarajanand May 22, 2024
edbd3c8
fixed linux build
vishwarajanand May 22, 2024
e5c040b
added more python versions for 3.11 and 3.12
vishwarajanand May 23, 2024
7becf00
log commit
vishwarajanand May 23, 2024
9f87757
force win build to use 3.10 py instead of 3.9
vishwarajanand May 24, 2024
e59aa4a
fixing requirements.txt path
vishwarajanand May 29, 2024
1af3bc1
change hardcoded python path in release script
vishwarajanand May 29, 2024
e206d19
Added python version and safe directory
vishwarajanand May 29, 2024
9189412
windows build script
vishwarajanand May 29, 2024
a74f70b
fixing .kokoro/requirements.txt path
vishwarajanand May 29, 2024
9572942
fixing safe directory path
vishwarajanand May 29, 2024
546e714
Updated Visual studio to v17 2022 from v16 2019
vishwarajanand May 30, 2024
f2220af
fixed echoes
vishwarajanand May 30, 2024
c8e4786
Adding back the windows dll fix for python 3.8+
vishwarajanand Jun 2, 2024
ab5308c
fixing linux release
vishwarajanand Jun 2, 2024
34fdcb3
fix osx path
vishwarajanand Jun 2, 2024
4552486
osx fix path
vishwarajanand Jun 2, 2024
39e6852
fix release script path
vishwarajanand Jun 2, 2024
375dc72
fix release scripts
vishwarajanand Jun 2, 2024
232a739
fix build log for windows
vishwarajanand Jun 2, 2024
ffe08db
fix: publisher script path
vishwarajanand Jun 2, 2024
86370b3
fix REPO_ROOT
vishwarajanand Jun 2, 2024
b7dd5cf
chore: development build 1.6.0.dev0
parthea Jun 3, 2024
b6d8741
changelog
parthea Jun 3, 2024
800420e
build script updates
parthea Jun 3, 2024
fd8fc15
build script updates
parthea Jun 3, 2024
d759d2f
build script updates
parthea Jun 3, 2024
d37110b
build script updates
parthea Jun 3, 2024
3f9365f
fix build
parthea Jun 4, 2024
1051786
update check.sh
parthea Jun 4, 2024
458c721
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jun 6, 2024
86fcda2
fix check script python versions
vishwarajanand Jun 6, 2024
b8a92d4
Revert "🦉 Updates from OwlBot post-processor"
vishwarajanand Jun 6, 2024
4e70f5a
add pyenv required dependencies
vishwarajanand Jun 6, 2024
74b46c1
address PR comments
vishwarajanand Jun 6, 2024
bc1096d
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jun 6, 2024
9fe254c
Revert "🦉 Updates from OwlBot post-processor"
vishwarajanand Jun 7, 2024
3b8cdce
refactor check script to evaluate for every py version
vishwarajanand Jun 7, 2024
3e785ee
refactored check script for osx
vishwarajanand Jun 7, 2024
3c10af4
migrate pyenv local to pyenv shell
vishwarajanand Jun 7, 2024
b7b6a53
Move pip install statements
vishwarajanand Jun 7, 2024
58e8cff
fix pyenv init in osx check script
vishwarajanand Jun 9, 2024
30ba29a
fix pyenv init in linux check script
vishwarajanand Jun 9, 2024
3110688
fix wheels path osx
vishwarajanand Jun 9, 2024
ecc9bd6
fix pyenv path linux
vishwarajanand Jun 9, 2024
ca1030e
fix config import sys
vishwarajanand Jun 9, 2024
233cf6c
refactor osc build script
vishwarajanand Jun 9, 2024
a5575ff
Add dynamic wheel name based on SOABI_FLAG
vishwarajanand Jun 9, 2024
fb9b5c8
Move python check to a pyenv version check
vishwarajanand Jun 9, 2024
4247588
move pyenv init to build_python_wheel
vishwarajanand Jun 10, 2024
6db4d29
added missing c extension in win builds
vishwarajanand Jun 10, 2024
4bc12ad
added debugging logs
vishwarajanand Jun 10, 2024
f3d0c35
revert logs for osx build
vishwarajanand Jun 10, 2024
f1bfe2a
add presubmits on GH actions
vishwarajanand Jun 10, 2024
a0219dd
test upload osx wheels to gcs
vishwarajanand Jun 17, 2024
ac92e08
upload wheels to kokoro
vishwarajanand Jun 17, 2024
6351ce8
fix gcs upload path in windows
vishwarajanand Jun 18, 2024
745f181
ensure auth for gcs
vishwarajanand Jun 18, 2024
06f64fd
added linux wheels export to gcs
vishwarajanand Jun 18, 2024
452de57
move gcs wheels upload to build script
vishwarajanand Jun 18, 2024
0d1b06c
test service account email
vishwarajanand Jun 18, 2024
ed36c07
removed sa from win build
vishwarajanand Jun 18, 2024
1916d10
Added auth for gcs gsutil
vishwarajanand Jun 18, 2024
ef2c9ec
refactored gcs upload
vishwarajanand Jun 18, 2024
9094aba
fixing windows wheels upload
vishwarajanand Jun 18, 2024
793c535
added amd to osx build to support apple silicon
vishwarajanand Jun 18, 2024
19f706b
Enable pip install via --find-links
vishwarajanand Jun 18, 2024
e4c714f
Added error handling for gcloud authentication and wheel file upload …
vishwarajanand Jun 18, 2024
235a134
skip error in batch script while uploading wheels to gcs
vishwarajanand Jun 18, 2024
0788a35
experimental os x build changes
andrewsg Jun 26, 2024
c0cf574
further testing
andrewsg Jun 26, 2024
d210914
add presubmit for slcn
andrewsg Jun 27, 2024
87f8f2d
tweak build settings
andrewsg Jun 27, 2024
ed430e4
Fix build and checks
andrewsg Jul 3, 2024
3ba6b72
clean up versions and try to set windows to upload binaries properly
andrewsg Jul 4, 2024
c07b12f
update README, try another route on OSX check step
andrewsg Jul 4, 2024
0fa0399
update README again and enable verbosity on windows build_ext
andrewsg Jul 4, 2024
77d846f
remove copy_dll early return and readd debugging steps
andrewsg Jul 8, 2024
192a6ac
try to fix windows debugging issue
andrewsg Jul 8, 2024
0acd26a
try to add windows versions to scripts
andrewsg Jul 8, 2024
7ce2730
remove quotes
andrewsg Jul 9, 2024
d762b56
experimental windows spy version handling
andrewsg Jul 9, 2024
0fb517a
Try another tack for windows scripts
andrewsg Jul 9, 2024
520437f
Yet another attempt to fix the windows scripts
andrewsg Jul 9, 2024
0d9d7e5
try to fix py
andrewsg Jul 9, 2024
7c88a90
fix py usage again
andrewsg Jul 9, 2024
6721a30
install cmake to known dir outside of path
andrewsg Jul 9, 2024
342be57
a test failure should result in a red build
andrewsg Jul 9, 2024
b5d8711
fix build errors
andrewsg Jul 9, 2024
05795a3
test
andrewsg Jul 9, 2024
3279df1
fix
andrewsg Jul 9, 2024
7f01457
use delayed expansion
andrewsg Jul 9, 2024
58506ff
fix cmake path
andrewsg Jul 9, 2024
6d443cd
maybe we can just do without cmake.exe
andrewsg Jul 9, 2024
74d0599
call tests correctly
andrewsg Jul 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
250 changes: 0 additions & 250 deletions .github/workflows/python-publish.yml

This file was deleted.

34 changes: 34 additions & 0 deletions .kokoro/build-manylinux.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/bin/bash

# Copyright 2018 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Need enchant for spell check
sudo apt-get update
sudo apt-get -y install dictionaries-common aspell aspell-en \
hunspell-en-us libenchant1c2a enchant

set -eo pipefail

# Disable buffering, so that the logs stream through.
export PYTHONUNBUFFERED=1

# Debug: show build environment
env | grep KOKORO

cd github/python-crc32c

# Before running nox and such, build the extension.
./scripts/manylinux/build.sh
./scripts/manylinux/check.sh
29 changes: 29 additions & 0 deletions .kokoro/build-osx.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash

# Copyright 2018 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -eo pipefail

# Disable buffering, so that the logs stream through.
export PYTHONUNBUFFERED=1

# Debug: show build environment
env | grep KOKORO

cd github/python-crc32c

# Before running nox and such, build the extension.
./scripts/osx/build.sh
./scripts/osx/check.sh
11 changes: 11 additions & 0 deletions .kokoro/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,24 @@ if [[ -z "${PROJECT_ROOT:-}" ]]; then
fi

cd "${PROJECT_ROOT}"
# Need enchant for spell check
sudo apt-get update
sudo apt-get -y install dictionaries-common aspell aspell-en \
hunspell-en-us libenchant1c2a enchant

set -eo pipefail

cd github/python-crc32c

# Disable buffering, so that the logs stream through.
export PYTHONUNBUFFERED=1

# Debug: show build environment
env | grep KOKORO

# Setup firestore account credentials
export FIRESTORE_APPLICATION_CREDENTIALS=${KOKORO_GFILE_DIR}/firebase-credentials.json

# Setup service account credentials.
export GOOGLE_APPLICATION_CREDENTIALS=${KOKORO_GFILE_DIR}/service-account.json

Expand Down
7 changes: 7 additions & 0 deletions .kokoro/continuous/crc32c.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Format: //devtools/kokoro/config/proto/build.proto

# Tell the trampoline which build file to use.
env_vars: {
key: "PACKAGE"
value: "crc32c"
}
32 changes: 32 additions & 0 deletions .kokoro/presubmit.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
@rem Copyright 2019 Google LLC. All rights reserved.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.

@echo "Starting Windows build"

cd /d %~dp0
cd ..

@rem as this package uses submodules make sure we have all content
call git config --global --add safe.directory C:/tmpfs/src/github/python-crc32c
call git submodule update --recursive || goto :error

@echo "Build Wheels and Run Tests"
call scripts\windows\build.bat || goto :error

for /r %%a in (*.whl) do xcopy "%%a" %KOKORO_ARTIFACTS_DIR% /i

goto :EOF

:error
exit /b 1
Loading
Loading