Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping tests: restructure, refactor, decouple session and engine, improve #12765

Merged
merged 15 commits into from
Oct 22, 2021

Conversation

jdavcs
Copy link
Member

@jdavcs jdavcs commented Oct 22, 2021

Summary of changes:

  1. Do not use Galaxy's connection engine and session for testing mapping.
    The tests reused the session object defined in galaxy.model (which used an engine built for a functioning Galaxy). With that setup, in addition to testing the mapping, we also unintentionally tested Galaxy's session and engine setup (thus, "testing by coincidence"). Here we create a separate engine and session, ensuring that their setup does not interfere with the test. (see b0c291c)
  2. Restructure files; factor out common fixtures into conftest.py (which is why we need a separate directory) and common code into common.py. Address technical dept created in Convert install_model to declarative mapping #12756. (duplicate code in TS mapping tests still needs to be addressed; depends on how much we can/should isolate TS code base)
  3. Apply changes from Verify post-test cleanup for model mapping unit tests; fix a few errors #12714. (reviewed/approved)
  4. Improve test helper w.r.t. comparing collections of mapped objects using their primary keys to determine equality; add tests exposing missed cases. Justification in description of 43906e0. (Also see 518cc51, a834b54)

Note: This was supposed to get rid of code, not add! However, I used black with default settings on test_model_mapping.py to get rid of all the long lines that were a result of replacing a simple equality with a call to the new helper function. I don't love the formatting, but I love not having an opinion on how to format 7K lines 😄
(I don't mind reformatting it back if needed)

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.

License

  • I agree to license these contributions under Galaxy's current license.
  • I agree to allow the Galaxy committers to license these and all my past contributions to the core galaxy codebase under the MIT license. If this condition is an issue, uncheck and just let us know why with an e-mail to [email protected].

jdavcs added 13 commits October 21, 2021 18:37
It is possible (even likely, with a different/better/simpler session
setup) that the objects populating the collection attribute representing
a relationship on a stored object will NOT be the same Python objects
that were initially associated with the object via that relationship.

Thus, Python object identity comparison will fail (when we change the
session setup). Besides, we should be using the database primary key as
the comparison key, because that is the only correct measure of
database-mapped object equality.
Disable test verification fixture by default (too expensive!)
Add comment explaining fixture + how to enable it.
Here we are supposed to be testing mapping; however, by using Galaxy's
session and engine, we also end up testing session and engine setup
(i.e., "testing by coincidence"). Instead, we will use a plain engine
and session, thus limiting these tests to their expected scope.

Applied to model and install_model mapping tests.
@jdavcs jdavcs added kind/refactoring cleanup or refactoring of existing code, no functional changes area/testing area/database Galaxy's database or data access layer labels Oct 22, 2021
@jdavcs jdavcs added this to the 22.01 milestone Oct 22, 2021
@jmchilton
Copy link
Member

I feel like mapping.init("/tmp", db_uri, create_tables=True) is our interface to the mapping so I don't think I would skip that code for the unit tests. I would pragmatically chose to test what we use how we use it over a more pure unit test, but this may just a reflect an ignorance about the individual components on my part.

@jdavcs
Copy link
Member Author

jdavcs commented Oct 22, 2021

I feel like mapping.init("/tmp", db_uri, create_tables=True) is our interface to the mapping so I don't think I would skip that code for the unit tests. I would pragmatically chose to test what we use how we use it over a more pure unit test...

@jmchilton Thanks!
I deliberated over this. It was convenient: the mapping tests triggered our session and engine setup, so additional relevant warnings were reported. But I couldn't isolate them; as a result it was unclear whether a particular SQLAlchemy warning was triggered by our session setup (autocommit=True, binding SA's MetaData to a Connectable, and something else, I think), OR incorrect mapping - which is what the tests were meant to expose. Decoupling the session+engine setup from the mapping tests solved this issue.

That said, I will be adding additional unit tests covering our session and engine setup. I need that for fixing the SQLAlchemy removed-in-2.0 warnings; and also for modifying some of that code for Alembic.

@jdavcs jdavcs merged commit c3fa79b into galaxyproject:dev Oct 22, 2021
@jdavcs jdavcs deleted the dev_mapping_tests2 branch October 22, 2021 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/database Galaxy's database or data access layer area/testing kind/refactoring cleanup or refactoring of existing code, no functional changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants