Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failing tests from Pydantic v2 migration #558

Merged
merged 10 commits into from
Oct 10, 2023

Conversation

hrushikesh-s
Copy link
Collaborator

Summary

Checklist

Work-in-progress pull requests are encouraged, but please put [WIP] in the pull request
title.

Before a pull request can be merged, the following items must be checked:

  • Code is in the standard Python style.
    The easiest way to handle this is to run the following in the correct sequence on
    your local machine. Start with running black on your new code. This will
    automatically reformat your code to PEP8 conventions and removes most issues. Then run
    ruff.
  • Docstrings have been added in the Numpy docstring format.
    Run ruff on your code.
  • Type annotations are highly encouraged. Run mypy to
    type check your code.
  • Tests have been added for any new functionality or bug fixes.
  • All linting and tests pass.

Note that the CI system will run all the above checks. But it will be much more
efficient if you already fix most errors prior to submitting the PR. It is highly
recommended that you use the pre-commit hook provided in the repository. Simply run
pre-commit install and a check will be run prior to allowing commits.

@hrushikesh-s
Copy link
Collaborator Author

Hi @janosh ,
Since this PR is to pydantic branch of atomate2, the tests are not being run by github upon making the push action. Any idea of how to make github run the tests without explicitly changing the testing.yaml file ?

@hrushikesh-s hrushikesh-s changed the title [WIP] Porting Atomate2 to Pydanticv2 [WIP] Porting Atomate2 to Pydantic-v2 Oct 9, 2023
@janosh
Copy link
Member

janosh commented Oct 9, 2023

@hrushikesh-s Ah, that's a bit annoying. We should add workflow_dispatch as a trigger for manually running tests on select branches. But even that isn't ideal, so maybe we should get rid of the branches: [main] filter for PRs.

@janosh
Copy link
Member

janosh commented Oct 9, 2023

Try pulling 5fc8ad7 into your branch.

@validator("atomic_kind_info")
def remove_unnecessary(self, atomic_kind_info):
@field_validator("atomic_kind_info")
def remove_unnecessary(cls, atomic_kind_info):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self should stay as is.

hrushikesh-s and others added 4 commits October 9, 2023 10:45
Also, as per the pydantic documentation, @field_validator cannot be applied to an instance method and can only be applied to a class method. That's why changing the first argument of remove_unnecessary & cleanup_dft from self to cls
we are now down to 10 failed, 125 passed, 2 skipped, 31 warnings
@hrushikesh-s
Copy link
Collaborator Author

@janosh, I'm getting the following assertion error for test_m3gnet_static_maker & test_m3gnet_relax_maker

FAILED tests/forcefields/test_jobs.py::test_m3gnet_static_maker - assert -10.711267471313477 == -10.8454 ± 1.1e-03
FAILED tests/forcefields/test_jobs.py::test_m3gnet_relax_maker - assert -10.710837364196777 == -10.8441 ± 1.1e-03

@janosh
Copy link
Member

janosh commented Oct 9, 2023

Weird that the values are not consistent. Feel free to loosen for now:

assert output1.output.energy == approx(-10.8, abs=0.2)

Changing the assertion criterion from approx(-10.8454, rel=1e-4) to approx(-10.8, abs=0.2)
@hrushikesh-s
Copy link
Collaborator Author

@janosh , I'm getting the following errors while running test_nonrad_maker & test_formation_energy_maker function from tests/vasp/flows/test_defect.py

File "/.../atomate2/src/atomate2/common/jobs/defect.py", line 401, in check_charge_state
    raise ValueError(
ValueError: The charge of the output structure is 0, but expected charge state from the Defect object is -2.

File "/.../atomate2/src/atomate2/common/jobs/defect.py", line 401, in check_charge_state
    raise ValueError(
ValueError: The charge of the output structure is 0, but expected charge state from the Defect object is -1.

@janosh
Copy link
Member

janosh commented Oct 9, 2023

@hrushikesh-s Not sure what changed. Maybe @jmmshn has troubleshooting suggestions?

@hrushikesh-s
Copy link
Collaborator Author

@janosh, there's one last error that's coming up in test_formation_energy_maker function in test_defect.py:

File "/.../atomate2/src/atomate2/common/jobs/defect.py", line 416, in get_defect_entry
    energy=bulk_c_entry.energy,
AttributeError: 'NoneType' object has no attribute 'energy'

@jmmshn
Copy link
Contributor

jmmshn commented Oct 10, 2023

Hi @janosh, It looks like there might have been some upstream changes with parsing the net charge of structures from a VASP directory. The code here was just to do a sanity check of the structure charge before things enter the DB.
I'll take a look at this by Wednesday.

@jmmshn
Copy link
Contributor

jmmshn commented Oct 10, 2023

Actually, I just remembered. I think the tests still pass with this error message.
The problem is that we cannot actually put the POTCARs in the directories due to legal reasons.
So the parsed charges are not going to be correct, so in a sense, this is the expected behavior.
However, I recall the workflow is designed in a way that it still passes CI even if you see those errors.

@hrushikesh-s
Copy link
Collaborator Author

So, do we need to make any changes to deploy.yaml?

@janosh
Copy link
Member

janosh commented Oct 10, 2023

@jmmshn Thanks for digging into this so quickly! Re unusable POTCARs in CI, maybe something to simulate with from unittest.mock import patch? Either way, that's for a future PR.

@hrushikesh-s Thanks a lot for your help here! 👍 I'll merge this PR and take a look at the last error over in #548.

@janosh janosh changed the title [WIP] Porting Atomate2 to Pydantic-v2 Fix failing tests from Pydantic v2 migration Oct 10, 2023
@janosh janosh added testing Test all the things fix Bug fix PR version Package version-related issues labels Oct 10, 2023
@janosh janosh merged commit 2b00c1e into materialsproject:pydantic2 Oct 10, 2023
@jmmshn
Copy link
Contributor

jmmshn commented Oct 10, 2023

@hrushikesh-s I don't think so. If CI is actually breaking on this then we will just have to fix the code. I'll mark some time on Wednesday to make sure this all works.

@jmmshn
Copy link
Contributor

jmmshn commented Oct 10, 2023

@janosh, yeah I think I just ignored it because this failed job was passing CI and it's kind of nice to see the fact that the check is working. Either way I should probably:

  • make one of the defect charges parse properly by mocking
  • make the other one fail but process that exception better in the testing suite so it doesn't cause this exact confusion.

@janosh
Copy link
Member

janosh commented Oct 10, 2023

  • make one of the defect charges parse properly by mocking
  • make the other one fail but process that exception better in the testing suite so it doesn't cause this exact confusion.

That would be perfect!

@janosh janosh mentioned this pull request Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix PR testing Test all the things version Package version-related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants