-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Emit multiple xAPI events for a multi-question submission #325
Conversation
7dadfd5
to
8361da5
Compare
Thanks for the pull request, @pomegranited! Please note that it may take us up to several weeks or months to complete a review and merge your PR. Feel free to add as much of the following information to the ticket as you can:
All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here. Please let us know once your PR is ready for our review and all tests are green. |
869a9ce
to
452a84e
Compare
Expected fixtures record the current behavior
Test was failing locally because test4.com is a real URL that responded with a 200 when we posted events to it. Not sure why it was succeeding in CI?
Updates the event processor change to allow for the possibility that some event transformers may generate multiple events from a single event, and these events need to be carried down the processor chain.
…mation This reduces the side-effects of event transformation, which will allow us to re-use base event transformation methods when generating multiple events from a single source event. * Removes the BaseTransformerMixin.transformed_event instance variable in favor of passing an event through to base_transform() to be modified and returned. * Adds BaseTransformerMixin.get_object() so that caliper events don't need to reference self.transformed_event when updating the object data. * Adds BaseTransformerMixin.get_extensions() so that caliper events can don't have to hack in their transformerVersion during BaseTransformerMixin.transform()
Single-question problem_check events still only produce one xAPI event. Changes to the top-level multi-question problem_check event data: * object.type changed from Activity to GroupActivity * object.id shows the base problem usage_key * object.definition.interaction_type is "other" New events emitted for each child problem are identical the top-level event, except for: * object.type is Activity * object.id shows the base problem usage_key including the child usage string * object.definition.interaction_type is determined by the child problem response_type * result.score is omitted -- only relevant to the parent problem * result.response is provided, pulled from the child question submission * result.success is provided, pulled from the child question submission Related fixes to all problem_check events: * object.definition.name now shows the problem display_name * object.id now uses shows the problem usage_key * result.score max and raw are now always provided if present in the source event (bug fix)
412de79
to
6ad3492
Compare
6ad3492
to
675f77d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking really good! Couple of things:
- I think this should be a major version bump as it's a pretty big breaking change for downstream data handling
- The coverage failures just happen in this repo pretty frequently, we just retry failed jobs when they come up
- When testing with the demo course I ran into this error when submitting the "Multiple Choice Questions" problem (single answer problems worked great):
2023-08-07 10:37:23 2023-08-07 14:37:23,344 ERROR 19 [eventtracking.backends.routing] [user None] [ip None] routing.py:146 - Unable to send edx event "problem_check" to backend: xapi
2023-08-07 10:37:23 Traceback (most recent call last):
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/eventtracking/backends/routing.py", line 137, in send_to_backends
2023-08-07 10:37:23 backend.send(event)
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 158, in send
2023-08-07 10:37:23 event_routes = self.prepare_to_send([event])
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 82, in prepare_to_send
2023-08-07 10:37:23 processed_events = self.process_event(event)
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 191, in process_event
2023-08-07 10:37:23 events = processor(events)
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/processors/mixins/base_transformer_processor.py", line 37, in __call__
2023-08-07 10:37:23 transformed_event = self.transform_event(event)
2023-08-07 10:37:23 File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/processors/xapi/transformer_processor.py", line 48, in transform_event
2023-08-07 10:37:23 event_json = transformed_event.to_json()
2023-08-07 10:37:23 AttributeError: 'list' object has no attribute 'to_json'
* fixes "AttributeError: 'list' object has no attribute 'to_json'" during event processing for one-to-many problem_check events * adds test for ^ * reverts unneeded change to test_caliper to reinstate 100% test coverage
Thank you for testing this @bmtcril !
Done: ea3d27d
👍
Ach sorry.. I missed that during a refactor, since the tests were passing.. 40f7422 covers that error and adds a test. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost there, I think. What I see now is that the statements all get created with the same event id. I'd expect the child statements to each have their own id, but that the StatementRef would look up to the parent. As it stands, the child events get de-duplicated down to 1 event in the database because they have identical id/actor/object/verb/timestamps.
* stops mocking uuid5 during tests uuid5 generates the same UUID when provided with the same namespace + name, and so we can rely on this remaining the same in the expected fixture files. * updates test data to use the actual uuid5s generated for the given input events.
* pulls event ID generation into get_event_id(), taking care not to modify how parent event IDs are generated. * overrides get_event_id() for child events by using the child_id as part of the UUID namespace_key. * updates tests to check that child events and their parent event use different event IDs, and the affected problem_check multiple-question test data
@bmtcril Gotcha. I decided to do this in two separate commits:
|
Hiya @bmtcril , when you get a chance -- did I fix the de-duplication issue you noted? |
Hey there, I'll run again tomorrow. I've been flat out trying to get the asset import pr in shape |
No worries at all, thank you for your thorough testing @bmtcril ! |
Ran into some new issues here today, but haven't had time to sort them out. Ralph is failing insert on all three child events with:
This is what the child event looks like in the db:
|
Looks like it's trying to get the course id and failing since it's now in "grouping":
I think we'll need a migration to update the sql, it's trying to do:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is working for me from the xAPI side, but we need to update Aspects before we can upgrade do a new version!
@bmtcril what is needed to include this change in Aspects? |
@Ian2012 this will need a big rebase and testing again. We need to add a migration to |
I'm working to get this over the line in #352 |
@pomegranited Even though your pull request wasn’t merged, please take a moment to answer a two question survey so we can improve your experience in the future. |
1 similar comment
@pomegranited Even though your pull request wasn’t merged, please take a moment to answer a two question survey so we can improve your experience in the future. |
Description:
Updates the base event processor and tests to support transforming a single event into multiple events, and
modifies the server-side
problem_check
xAPI event transformer to transform multi-problem submissions into:Resolves #219
JIRA: FAL-3430 (OpenCraft internal link)
Testing instructions:
This change should be covered by tests, but to test it manually:
Merge checklist:
Post merge:
finished.
Author concerns:
object.definition.name
andinteractionType
.