Accso delivery for pytroll image comparison tests #2

gerritholl · 2024-10-11T09:46:39Z

This PR adds pytroll image comparison tests as delivered by Accso, developed as part of a contract with DWD.

Details can be found in the README file that is part of this commit.

…sts repository

gerritholl · 2024-11-07T16:33:10Z

Corresponding PR in Satpy: pytroll/satpy#2912

gerritholl · 2024-11-18T09:18:25Z

behave/create_reference.py

+satellite = "GOES17"
+filenames = glob(f'{ext_data_path}/satellite_data/{satellite}/*.nc')
+
+scn = Scene(reader='abi_l1b', filenames=filenames)


We need to make this more flexible.

Shall we defer this to a later PR?

gerritholl · 2024-11-18T09:19:32Z

behave/features/steps/.gitkeep

Do we need this file?

gerritholl · 2024-11-18T09:21:01Z

behave/features/steps/image_comparison.py

+    filenames = glob(f'{ext_data_path}/satellite_data/{satellite}/*.nc')
+
+    scn = Scene(reader='abi_l1b', filenames=filenames)


We will need to make this more generic.

yes. if we add a third column in the table with the reader name, that should be fine

Translate comments and strings from German to English.

mraspaud

Not too bad!

ansible/files/systemd/bildabgleich.service

ansible/inventory.ini

ansible/roles/add_user/tasks/main.yml

ansible/roles/certbot_nginx/CODE_OF_CONDUCT.md

ansible/templates/nginx/sites-available/bildabgleich

mraspaud · 2024-11-18T09:33:23Z

behave/create_reference.py

+satellite = "GOES17"
+filenames = glob(f'{ext_data_path}/satellite_data/{satellite}/*.nc')
+
+scn = Scene(reader='abi_l1b', filenames=filenames)


mraspaud · 2024-11-18T09:34:30Z

behave/features/steps/image_comparison.py

+    filenames = glob(f'{ext_data_path}/satellite_data/{satellite}/*.nc')
+
+    scn = Scene(reader='abi_l1b', filenames=filenames)


yes. if we add a third column in the table with the reader name, that should be fine

serverLogic/server.py

serverLogic/start_server.sh

Update README to reflect recent changes in implementation

Closes #3.

gerritholl · 2024-11-20T09:04:24Z

A test with actual data fails on image-test:

Nov 20 08:56:32 image-test.pytroll.ewc start_server.sh[106787]: INFO:server:Starting to clone and test the repository pytroll/satpy
Nov 20 08:58:12 image-test.pytroll.ewc start_server.sh[106787]: Directory /home/imagetester/pull_request_feature-multifile-handler successfully emptied.
Nov 20 08:58:12 image-test.pytroll.ewc start_server.sh[106787]: Directory /home/imagetester/pull_request_feature-multifile-handler newly created.
Nov 20 08:58:12 image-test.pytroll.ewc systemd[1]: Stopping image-comparison-server...
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: State 'final-sigterm' timed out. Killing.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 106781 (gunicorn) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 106787 (gunicorn) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112208 (docker) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112187 (gunicorn) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112209 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112214 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112216 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112217 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112218 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Killing process 112219 (n/a) with signal SIGKILL.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Failed with result 'timeout'.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: Stopped image-comparison-server.
Nov 20 08:58:17 image-test.pytroll.ewc systemd[1]: image-comparison.service: Consumed 1.150s CPU time.

gerritholl · 2024-11-20T09:07:01Z

From the docker log:

$ tail /home/imagetester/pull_request_feature-multifile-handler/output.log
  Downloading zipp-3.21.0-py3-none-any.whl (9.6 kB)
Collecting locket
  Downloading locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
Installing collected packages: parse, zipp, toolz, tomli, six, pyyaml, pluggy, Pillow, packaging, numpy, locket, iniconfig, fsspec, exceptiongroup, cloudpickle, click, certifi, pytest, partd, parse-type, opencv-python, importlib-metadata, h5py, cftime, netcdf4, h5netcdf, dask, behave
Successfully installed Pillow-11.0.0 behave-1.2.6 certifi-2024.8.30 cftime-1.6.4.post1 click-8.1.7 cloudpickle-3.1.0 dask-2024.11.2 exceptiongroup-1.2.2 fsspec-2024.10.0 h5netcdf-1.4.1 h5py-3.12.1 importlib-metadata-8.5.0 iniconfig-2.0.0 locket-1.0.0 netcdf4-1.7.2 numpy-2.1.3 opencv-python-4.10.0.84 packaging-24.2 parse-1.20.2 parse-type-0.6.4 partd-1.4.2 pluggy-1.5.0 pytest-8.3.3 pyyaml-6.0.2 six-1.16.0 tomli-2.1.0 toolz-1.0.0 zipp-3.21.0

[notice] A new release of pip is available: 23.0.1 -> 24.3.1
[notice] To update, run: python3 -m pip install --upgrade pip
Cloning into '/app/repository'...
fatal: Remote branch feature-multifile-handler not found in upstream origin

gerritholl · 2024-11-20T09:08:15Z

Looks like it's searching for the branch in upstream and not in the fork where the pull request originates?

gerritholl · 2024-11-20T09:20:30Z

After increasing the logging amount for debugging purposes, rerunning fails with a new error message:

Nov 20 09:18:09 image-test.pytroll.ewc start_server.sh[118722]: INFO:server:Starting to clone and test the repository pytroll/satpy
Nov 20 09:18:10 image-test.pytroll.ewc start_server.sh[118769]: docker: Error response from daemon: Conflict. The container name "/clone-repo-image" is already in use by container "96379bfa2a82d5be90fdd895071f4a2491df31ecf323c714c8dade8b9194e969". You have to remove (or rename) that container to be able to reuse that name.
Nov 20 09:18:10 image-test.pytroll.ewc start_server.sh[118769]: See 'docker run --help'.
Nov 20 09:18:10 image-test.pytroll.ewc start_server.sh[118722]: ERROR:container_utils:Error while cloning the repository: Command '['docker', 'run', '--name', 'clone-repo-image', '-v', '/home/imagetester/pull_request_feature-multifile-handler:/app', '-v', '/home/imagetester/pytroll-image-comparison-tests/data:/app/ext_data', 'python:3.10-slim', 'bash', '-c', 'touch /app/output.log && apt-get update >> /app/output.log 2>&1 && apt-get install -y git libgl1-mesa-glx libglib2.0-0 python3-venv >> /app/output.log 2>&1 && python3 -m venv /app/venv >> /app/output.log 2>&1 && /app/venv/bin/pip install behave Pillow pytest numpy opencv-python dask netcdf4 h5netcdf >> /app/output.log 2>&1 && git clone https://[REDACTED]@github.com/pytroll/satpy.git --branch feature-multifile-handler /app/repository >> /app/output.log 2>&1 && source /app/venv/bin/activate && pip install -e /app/repository >> /app/output.log 2>&1 && source /app/venv/bin/activate && cd /app/repository/satpy/tests/behave && behave >> /app/output.log 2>&1 || true && chown -R 1005:1005 /app >> /app/output.log 2>&1']' returned non-zero exit status 125.
Nov 20 09:18:10 image-test.pytroll.ewc start_server.sh[118791]: Error response from daemon: Could not find the file /app/output.log in container clone-repo-image

Looks like there is an old hanging docker image from the previous failed job, that did not get cleaned up?

gerritholl · 2024-11-20T09:56:10Z

To test the hypothesis that it's only searching for the branch in upstream and not in the fork, I created a dummy PR in pytroll/satpy#2987 based on a branch in the upstream. However, now trying to trigger the behave tests fails with:

Nov 20 09:53:44 image-test.pytroll.ewc start_server.sh[118722]: ERROR:server:Signature verification failed.

gerritholl · 2024-11-20T10:10:04Z

Seems that for pytroll/satpy#2697 it did actually work in the end, despite the error message 'fatal: Remote branch feature-multifile-handler not found in upstream origin'.

It also works for pytroll/satpy#1136.

But for pytroll/satpy#2987 (where the branch is on the main fork, not the user's fork) it fails to verify the signature. Don't know why, but not critical as it works (despite the error message) for normal PRs.

gerritholl · 2024-11-20T10:44:31Z

Correction. It did not work. The linked test result refers to an old test from September. It cannot have worked, because there are no reference images on the server.

So probably the message fatal: Remote branch feature-multifile-handler not found in upstream origin is the problem after all. As a secondary problem, it shouldn't claim "The testing process was executed successfully." when it was not.

gerritholl · 2024-11-20T10:54:39Z

As a secondary problem, it shouldn't claim "The testing process was executed successfully." when it was not.

I guess docker run does not fail if the underlying git clone fails, and therefore subprocess.check_call does not fail either.

Clone from the fork, not from the original satpy repo.

Added pyproject.toml. Started some first unit tests. Did a bit of refactoring.

gerritholl · 2024-11-21T13:41:21Z

The code under behave/ is duplicated by pytroll/satpy#2912, for example:

https://github.com/pytroll/image-comparison-tests-dev/blob/106710d3534be8f326d67d3de358a9e210db80f1/behave/create_reference.py

and

https://github.com/KatrinPoepp/satpy/blob/ec4f99623642328f5eeab5691a035871501cd90f/satpy/tests/behave/create_reference.py

gerritholl

There are some duplications compared to pytroll/satpy#2912

behave/create_reference.py

behave/features/image_comparison.feature

behave/features/steps/image_comparison.py

behave/modify_image.py

Delete duplicate behave tests. Already present in Satpy PR at pytroll/satpy#2912

mraspaud

looks good, ready to merge?

Initial commit transferred from our Accso pytroll-image-comparison-te…

1354b57

…sts repository

gerritholl requested a review from mraspaud October 11, 2024 13:34

gerritholl mentioned this pull request Nov 7, 2024

Remove the application of Rayleigh correction above 0.8 micron on the Geo sats pytroll/satpy#2972

Merged

4 tasks

gerritholl mentioned this pull request Nov 7, 2024

add image comparison behave tests pytroll/satpy#2912

Closed

gerritholl self-assigned this Nov 8, 2024

Add license and authors files.

b405f91

gerritholl commented Nov 18, 2024

View reviewed changes

behave/features/steps/.gitkeep Outdated

Copy link

Member Author

gerritholl Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this file?

gerritholl commented Nov 18, 2024

View reviewed changes

Translate comments and strings to English

48d4fe1

Translate comments and strings from German to English.

mraspaud requested changes Nov 18, 2024

View reviewed changes

mraspaud and others added 4 commits November 18, 2024 17:03

Make playbook more flexible

106710d

Update AUTHORS.md

3ad39a3

Update README

f8b8db0

Update README to reflect recent changes in implementation

Verify that user is authorised.

61a4c89

Closes #3.

gerritholl added 2 commits November 20, 2024 12:20

Fix clone URL

9d28065

Clone from the fork, not from the original satpy repo.

pyproject, tests, refactoring

e1e6d1d

Added pyproject.toml. Started some first unit tests. Did a bit of refactoring.

gerritholl commented Nov 21, 2024

View reviewed changes

behave/create_reference.py Outdated Show resolved Hide resolved

behave/features/image_comparison.feature Outdated Show resolved Hide resolved

behave/features/steps/image_comparison.py Outdated Show resolved Hide resolved

behave/modify_image.py Outdated Show resolved Hide resolved

Deleted duplicate behave tests

070c9a9

Delete duplicate behave tests. Already present in Satpy PR at pytroll/satpy#2912

gerritholl mentioned this pull request Nov 22, 2024

Add Accsos image comparison tests pytroll/satpy#2999

Merged

4 tasks

mraspaud approved these changes Dec 5, 2024

View reviewed changes

gerritholl added 6 commits December 13, 2024 13:08

add hdf5plugin to dependency modules

8f34b56

use conda not pip/apt

a5c4b01

fix typo

16b3169

fix miniforge url and executable

b7cad58

add missing conda init

20f0a1a

Add note to README on expiring access tokens

de4a9d0

gerritholl merged commit 97d0b20 into main Jan 24, 2025

gerritholl deleted the pytroll-image-comparison-tests branch January 24, 2025 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Accso delivery for pytroll image comparison tests #2

Accso delivery for pytroll image comparison tests #2

gerritholl commented Oct 11, 2024 •

edited

Loading

gerritholl commented Nov 7, 2024

gerritholl Nov 18, 2024

mraspaud Nov 18, 2024

gerritholl Nov 19, 2024

gerritholl Nov 18, 2024

gerritholl Nov 18, 2024

mraspaud Nov 18, 2024

mraspaud left a comment

mraspaud Nov 18, 2024

mraspaud Nov 18, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024 •

edited

Loading

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 21, 2024

gerritholl left a comment

mraspaud left a comment

		filenames = glob(f'{ext_data_path}/satellite_data/{satellite}/*.nc')

		scn = Scene(reader='abi_l1b', filenames=filenames)

Accso delivery for pytroll image comparison tests #2

Accso delivery for pytroll image comparison tests #2

Conversation

gerritholl commented Oct 11, 2024 • edited Loading

gerritholl commented Nov 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024 • edited Loading

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 20, 2024

gerritholl commented Nov 21, 2024

gerritholl left a comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

gerritholl commented Oct 11, 2024 •

edited

Loading

gerritholl commented Nov 20, 2024 •

edited

Loading