Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update scheduling queries and unit tests to conform to Aerie v2.6 #129

Merged
merged 11 commits into from
Apr 8, 2024

Conversation

joshhaug
Copy link
Collaborator

The primary motivation for this PR is to update the scheduling module to use the new db schemas introduced in Aerie v2.6.0. More information on these changes is available on the Aerie upgrade docs.

Additionally, there are also some cleanup tasks that I've lumped in:

  • Replaced the default banananation 2.2 jar with the latest banananation 2.6 jar, and updates the .env file accordingly
  • Refactored to replace the deprecated datetime.utcnow() calls with the new timezone-aware datetime.now(timezone.utc). This is not a necessary change, but prevents us from throwing deprecation errors.

I'm happy to split these up into separate PRs if preferred.

@joshhaug joshhaug requested a review from cartermak March 26, 2024 20:06
@joshhaug joshhaug had a problem deploying to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug joshhaug had a problem deploying to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 20:06 — with GitHub Actions Inactive
@joshhaug
Copy link
Collaborator Author

Looks like this is failing integration tests because we no longer have the 2.2.0 banananation jar. I can add this file back in, or we can update the tests to use the newest jar. I'm leaning towards the latter.

@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
@joshhaug joshhaug had a problem deploying to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Error
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
@joshhaug joshhaug had a problem deploying to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 26, 2024 21:17 — with GitHub Actions Inactive
Copy link
Member

@cartermak cartermak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I suggest you update the integration test defaults to also use 2.6.0 here. (I forgot to put that in the integration test documentation!)
  2. I think what's breaking the integration tests now is that, since Aerie 2.4.0ish, Aerie-CLI is no longer able to tell if the Aerie host has auth enabled. This is the method that used to work and needs to be updated.

src/aerie_cli/aerie_client.py Outdated Show resolved Hide resolved
src/aerie_cli/aerie_client.py Outdated Show resolved Hide resolved
Comment on lines 1525 to 1532
# Note that as of Aerie v2.3.0, the metadata (incl. model_id and goal name) are stored in a separate table
formatted_upload_objects = []
for entry in upload_object:
new_entry = {}
new_entry["definition"] = entry["definition"]
new_entry["metadata"] = {"data": {"name": entry["name"], "models_using": {"data": {"model_id": entry["model_id"]}}}}
formatted_upload_objects.append(new_entry)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term maintainability, would it be better to break the format of upload_object than to hard-code this mapping? Who are the users of this method?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only user of this method is scheduling.upload. Is it preferable to deliver the upload_object to this method as it should be uploaded?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's preferable. I'd say we should avoid breaking changes to the API, but I don't know of anyone using the scheduling rule Python API.

@joshhaug joshhaug had a problem deploying to integration-test-workflow March 27, 2024 22:21 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 27, 2024 22:21 — with GitHub Actions Inactive
@joshhaug joshhaug had a problem deploying to integration-test-workflow March 27, 2024 22:21 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow March 27, 2024 22:21 — with GitHub Actions Inactive
@joshhaug joshhaug had a problem deploying to integration-test-workflow April 3, 2024 22:49 — with GitHub Actions Failure
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 3, 2024 22:57 — with GitHub Actions Inactive
@joshhaug
Copy link
Collaborator Author

joshhaug commented Apr 3, 2024

Update: I've modified this PR to also include updates to the constraint queries to bring them up to parity with the changes in v2.6.0. Also updated to fix the is_auth_enabled() function -- for some reason Aerie now presents the "Authentication is disabled" message on posting to the /auth/login endpoint instead of /auth/session. Updated this PR accordingly.

@joshhaug joshhaug requested a review from cartermak April 3, 2024 23:10
…sed and the same functionality is available with upload_scheduling_goals
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:48 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:48 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:48 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:49 — with GitHub Actions Inactive
@joshhaug joshhaug temporarily deployed to integration-test-workflow April 8, 2024 23:49 — with GitHub Actions Inactive
Copy link
Member

@cartermak cartermak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Feel free to merge once CI passes

@joshhaug joshhaug merged commit a9de995 into develop Apr 8, 2024
8 checks passed
@joshhaug joshhaug deleted the feature--unit-test-updates branch April 8, 2024 23:58
joshhaug added a commit that referenced this pull request Apr 11, 2024
* Update scheduling queries and unit tests to conform to Aerie v2.6 (#129)

* Updating scheduling module to work with Aerie 2.6

* Update to include a 2.6 banananation jar, and updated .env to use 2.6

* Refactor to replace deprecated datetime.utcnow() calls

* Removed banananation 2.2.0 jar

* Updated to use 2.6.0 instead of 2.2.0

* Updated aerie_client based on PR feedback

* Updating conftest to use v2.6.0

* Updated constraints queries to match v2.6.0 schema

* Updated aerie_host.is_auth_enabled()

* Updated constraints.py to remove unnecessary comments/add docstrings

* Removed upload_scheduling_goal wrapper function, as it is not being used and the same functionality is available with upload_scheduling_goals

* Documentation updates (#130)

* Updated pyproject.toml version to v2.4.0

---------

Co-authored-by: Carter Mak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants