Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Integration Testing] PEcAn Docker Stack Run : Single Site, Single Model #3345

Open
wants to merge 48 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
6934bc9
added model run
meetagrawal09 Jul 27, 2024
652d81e
trial
Aug 1, 2024
8b36096
trial
Aug 1, 2024
c70a06e
Accepted Suggestions
meetagrawal09 Aug 3, 2024
2e39f1c
Added wait time
meetagrawal09 Aug 3, 2024
9db5d2e
Bug fix
meetagrawal09 Aug 3, 2024
d194e21
Shifted to executor
meetagrawal09 Aug 3, 2024
b94bda7
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 3, 2024
e8a359a
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 8, 2024
266f0c6
Trying run by avoiding creating a user
meetagrawal09 Aug 8, 2024
d3557ef
Create docker compose override
meetagrawal09 Aug 15, 2024
c8ba82e
Remove docker images
meetagrawal09 Aug 15, 2024
a389dd6
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 15, 2024
c78e999
Trying free-disk-space action
meetagrawal09 Aug 15, 2024
3c6d30a
typo
meetagrawal09 Aug 15, 2024
51060d7
trying rstudio container
meetagrawal09 Aug 16, 2024
d845de3
trial
meetagrawal09 Aug 16, 2024
3c51a46
trial
meetagrawal09 Aug 16, 2024
d3f835e
Removing entry for volume
meetagrawal09 Aug 22, 2024
363edd8
moved to executor
meetagrawal09 Aug 22, 2024
724160b
bug fix
meetagrawal09 Aug 22, 2024
76e5607
bug fix
meetagrawal09 Aug 22, 2024
d9ae803
Merge branch 'develop' into model-run-workflow
meetagrawal09 Aug 24, 2024
eb896d1
clean-up
meetagrawal09 Sep 5, 2024
0b6b71d
bug fix
meetagrawal09 Sep 5, 2024
87752b4
updated output dir
meetagrawal09 Sep 5, 2024
6ffefd2
Removed sleep
meetagrawal09 Sep 5, 2024
6771566
Keeping sleep with a comment
meetagrawal09 Sep 5, 2024
e4007f5
Update workflow name
meetagrawal09 Sep 5, 2024
998cce7
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 5, 2024
b4f6d09
updated settings
meetagrawal09 Sep 5, 2024
824bb65
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 8, 2024
a06e01b
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 12, 2024
5896395
Merge branch 'develop' into model-run-workflow
infotroph Sep 12, 2024
c6d38df
Removing sleep -> relying on healthchecks
meetagrawal09 Sep 14, 2024
8556f5b
Adding sleep for testing
meetagrawal09 Sep 14, 2024
0bc2c78
Using model name
meetagrawal09 Sep 15, 2024
a55143d
Running add models script
meetagrawal09 Sep 16, 2024
69c27d6
Run add.models.sh
meetagrawal09 Sep 21, 2024
a949401
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 21, 2024
2a3d512
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 23, 2024
3fa4084
Merge branch 'develop' into model-run-workflow
meetagrawal09 Sep 25, 2024
7391456
trying out monitor
meetagrawal09 Oct 5, 2024
41f76bc
Merge branch 'develop' into model-run-workflow
meetagrawal09 Dec 1, 2024
0236356
moving to guest queue
meetagrawal09 Dec 1, 2024
5062635
testing without queue
meetagrawal09 Dec 1, 2024
33e3393
run rabbitmq container
meetagrawal09 Dec 1, 2024
9e8fba5
updated settings
meetagrawal09 Dec 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/workflows/docker-stack-sipnet.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name : Docker Stack Model Test
on :
pull_request:

# allow manual triggering
workflow_dispatch:

schedule:
# run Thursday 4:30 AM UTC
- cron: '30 4 * * 4'
jobs:
sipnet:
runs-on: ubuntu-latest

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: false
swap-storage: true

- name: Checkout source code
uses: actions/checkout@v4

- name: Set up Docker Compose
run: |
sudo apt-get update
sudo apt-get -y install docker-compose

- name: Add Volumes to Executor
run: |
cp docker-compose.dev.yml docker-compose.override.yml
sed -i'' '/R_library/d' docker-compose.override.yml

- name: Build Containers
run: |
cp docker/env.example .env
echo "COMPOSE_PROJECT_NAME=pecan" >> .env
echo "PECAN_VERSION=develop" >> .env
echo "UID=$(id -u)" >> .env
echo "GID=$(id -g)" >> .env

docker compose up --wait -d postgres
docker run --rm --network pecan_pecan pecan/db:ci
docker run --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop
docker image rm pecan/db:ci pecan/data:develop

- name: Run required containers
run: |
docker compose up --wait -d sipnet executor monitor rabbitmq

- name: List running containers
run: |
sleep 50
docker ps

# - name: Add models
# run: |
# docker compose cp scripts/add.util.sh postgres:/tmp/add.util.sh
# docker compose cp scripts/add.models.sh postgres:/tmp/add.models.sh
# docker compose exec postgres /tmp/add.models.sh

- name: Run SIPNET
run: |
docker compose exec --workdir /pecan/tests executor R CMD ../web/workflow.R --settings docker-ghaction.sipnet.xml
66 changes: 66 additions & 0 deletions tests/docker-ghaction.sipnet.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
<?xml version="1.0"?><pecan>
<outdir>/tmp/docker_stack_tests/sipnet</outdir>
<database>
<bety>
<user>bety</user>
<password>bety</password>
<host>postgres</host>
<port>5432</port>
<dbname>bety</dbname>
<driver>PostgreSQL</driver>
<write>true</write>
</bety>
<dbfiles>/data/dbfiles</dbfiles>
</database>
<pfts>
<pft>
<name>semiarid.grassland</name>
</pft>
</pfts>
<meta.analysis>
<iter>3000</iter>
<random.effects>
<on>FALSE</on>
<use_ghs>TRUE</use_ghs>
</random.effects>
</meta.analysis>
<ensemble>
<size>1</size>
<variable>NPP</variable>
<samplingspace>
<parameters>
<method>uniform</method>
</parameters>
<met>
<method>sampling</method>
</met>
</samplingspace>
</ensemble>
<model>
<name>SIPNET</name>
<revision>git</revision>
</model>
<run>
<site>
<id>2000001297</id>
<met.start>2004/01/01</met.start>
<met.end>2004/12/31</met.end>
</site>
<inputs>
<met>
<source>CRUNCEP</source>
<output>SIPNET</output>
<product/>
</met>
</inputs>
<start.date>2004/01/01</start.date>
<end.date>2004/12/31</end.date>
</run>
<host>
<name>localhost</name>
<rabbitmq>
<uri>amqp://guest:guest@rabbitmq/%2F</uri>
<queue>SIPNET_git</queue>
</rabbitmq>
</host>
</pecan>
Loading