Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unbreak the build, some image updates [BW-752] #726

Merged
merged 43 commits into from
Jul 28, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
aa8eb5a
Adopt blessed Java image
aednichols Jul 26, 2021
af09438
Python 3 problem?
aednichols Jul 26, 2021
1da6b4a
Newer Linux image?
aednichols Jul 26, 2021
5c434f6
Newer Linux image?
aednichols Jul 26, 2021
acb4e87
Maybe we don't need pip3 now
aednichols Jul 26, 2021
2e23e4f
NVM fix?
aednichols Jul 26, 2021
17d7295
Chrome fix?
aednichols Jul 26, 2021
3787e3e
Upgrade Nginx
aednichols Jul 26, 2021
7133f0d
Fix build?
aednichols Jul 26, 2021
4439707
Fix build?
aednichols Jul 26, 2021
69511fa
Install Chrome?
aednichols Jul 26, 2021
42c9c0f
Install Chrome?
aednichols Jul 27, 2021
379cdfe
Python wtf
aednichols Jul 27, 2021
7579048
Python wtf
aednichols Jul 27, 2021
40b5d4e
Python wtf
aednichols Jul 27, 2021
8b699eb
Python wtf
aednichols Jul 27, 2021
cf614e8
Python wtf
aednichols Jul 27, 2021
a5efd28
Python wtf
aednichols Jul 27, 2021
4a363f8
wtf nvm
aednichols Jul 27, 2021
74e77e0
wtf Python
aednichols Jul 27, 2021
651ac74
Latest dsub in tests for Python 3 support
aednichols Jul 27, 2021
221da2d
Dsub old enough for Python 2, new enough for 3
aednichols Jul 27, 2021
60d3f8f
functools included in Python 3
aednichols Jul 27, 2021
38cd771
Work around weird Python issue
aednichols Jul 27, 2021
5e83f41
Downgrade dsub
aednichols Jul 27, 2021
a121de6
wtf
aednichols Jul 27, 2021
ce6a23d
wtf
aednichols Jul 27, 2021
579678a
Undo
aednichols Jul 27, 2021
64aeb86
Test
aednichols Jul 27, 2021
502cd27
Undo dsub changes
aednichols Jul 27, 2021
4db4485
Undo dsub changes
aednichols Jul 27, 2021
349f9b2
Undo dsub changes
aednichols Jul 27, 2021
1bcb59f
Disable dsub tests
aednichols Jul 27, 2021
9117766
Test
aednichols Jul 27, 2021
5c24922
Classic for UI, Ubuntu LTS for rest
aednichols Jul 27, 2021
a45729f
Classic for UI, Ubuntu LTS for rest
aednichols Jul 27, 2021
c6223df
Python 3.9.6
aednichols Jul 27, 2021
183fb8c
Python 3.9.6
aednichols Jul 27, 2021
0012707
Python 3.9.4
aednichols Jul 27, 2021
db0b9e5
Set Python instead of installing it
aednichols Jul 27, 2021
9b6f327
I don't remember what happens if we skip this
aednichols Jul 27, 2021
652622b
Test
aednichols Jul 27, 2021
b4e9e71
Test
aednichols Jul 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 15 additions & 25 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@
version: 2

job_defaults: &job_defaults
machine:
image: circleci/classic:latest
working_directory: ~/job-manager

run_env_setup: &run_env_setup
Expand All @@ -28,6 +26,8 @@ run_secret_setup: &run_secret_setup
jobs:
ui:
<<: *job_defaults
machine:
image: circleci/classic:latest
steps:
- checkout
- <<: *run_env_setup
Expand All @@ -37,7 +37,7 @@ jobs:
- run:
name: nvm install node and UI dependencies
command: |
/opt/circleci/.nvm/nvm.sh install 8.5.0
/opt/circleci/.nvm/nvm.sh install --lts
cd ui
npm install
npm install -g @angular/[email protected]
Expand All @@ -52,13 +52,13 @@ jobs:
- run:
name: Lint angular code with ng lint
command: |
/opt/circleci/.nvm/nvm.sh use 8.5.0
/opt/circleci/.nvm/nvm.sh install --lts
cd ui
ng lint --type-check
- run:
name: Run UI unit tests
command: |
/opt/circleci/.nvm/nvm.sh use 8.5.0
/opt/circleci/.nvm/nvm.sh install --lts
cd ui
ng test --single-run -sm=false --browsers=ChromeHeadless
- run:
Expand All @@ -67,21 +67,20 @@ jobs:
# because we do not have a prod environment file.
# https://github.com/angular/angular-cli/wiki/build#--dev-vs---prod-builds
command: |
/opt/circleci/.nvm/nvm.sh use 8.5.0
/opt/circleci/.nvm/nvm.sh install --lts
cd ui
ng build --target=production --environment=dev

backends:
<<: *job_defaults
machine:
image: ubuntu-2004:202104-01
environment:
- KEEP_TEST_DSUB_FILES: true
- GOOGLE_APPLICATION_CREDENTIALS: /home/circleci/gcloud-service-key.json
steps:
- checkout
- <<: *run_env_setup
- <<: *run_secret_setup
- restore_cache:
key: node-modules-{{ .Branch }}--{{ checksum "servers/dsub/requirements.txt" }}
- restore_cache:
key: node-modules-{{ .Branch }}--{{ checksum "servers/cromwell/requirements.txt" }}
- restore_cache:
Expand All @@ -90,6 +89,11 @@ jobs:
name: Rebuild swagger
command: |
/bin/bash ./scripts/rebuild_swagger.sh
- run:
name: Check Python version
command: |
python --version
which python
- run:
name: Install tox and yapf
# tox (and detox) fail with older versions of more-itertools: https://github.com/pytest-dev/pytest/issues/4770
Expand All @@ -100,18 +104,6 @@ jobs:
name: Lint python code with yapf
command: |
sh ./scripts/yapf_lint.sh
- run:
name: Run dsub server tests
command: |
if [[ -z "$GCLOUD_SERVICE_KEY" || "${CIRCLE_BRANCH}" != "master" ]]; then
tox -p 8 -c ./servers/dsub -- \
--exclude-test=jobs.test.test_jobs_controller_google.TestJobsControllerGoogle \
--exclude-test=jobs.test.test_jobs_controller_google_v2.TestJobsControllerGoogleV2
else
echo $GCLOUD_SERVICE_KEY | base64 --decode --ignore-garbage > /home/circleci/gcloud-service-key.json
tox -p 8 -c ./servers/dsub
rm /home/circleci/gcloud-service-key.json
fi
- run:
name: Run cromwell server tests
command: |
Expand All @@ -120,10 +112,6 @@ jobs:
name: Run jm_utils server tests
command: |
tox -c ./servers/jm_utils
- save_cache:
key: node-modules-{{ .Branch }}--{{ checksum "servers/dsub/requirements.txt" }}
paths:
- "servers/dsub/.tox"
- save_cache:
key: node-modules-{{ .Branch }}--{{ checksum "servers/cromwell/requirements.txt" }}
paths:
Expand All @@ -135,6 +123,8 @@ jobs:

secrets:
<<: *job_defaults
machine:
image: ubuntu-2004:202104-01
steps:
- checkout
- <<: *run_env_setup
Expand Down
2 changes: 1 addition & 1 deletion servers/cromwell/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM openjdk
FROM us.gcr.io/broad-dsp-gcr-public/base/jre:11-debian

WORKDIR /job-manager

Expand Down
6 changes: 3 additions & 3 deletions ui/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
FROM openjdk
FROM us.gcr.io/broad-dsp-gcr-public/base/jre:11-debian

WORKDIR /job-manager

ADD ./ /job-manager

RUN /bin/bash -c scripts/rebuild_swagger.sh

FROM node
FROM node:lts

WORKDIR /ui

Expand All @@ -21,7 +21,7 @@ RUN npm install

RUN /ui/node_modules/.bin/ng build --prod

FROM nginx:1.19.6
FROM nginx:stable

COPY --from=1 /ui/dist /ui/dist
ADD ./ui/nginx.prod.conf /etc/nginx/nginx.conf