Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add transformer to filter arguments. #374

Merged
merged 4 commits into from
Dec 6, 2023

Conversation

andrey-canon
Copy link
Contributor

Description:
This improves the filter usability by adding the transformer to the filter arguments, basically this allows the filter client to access to extra data given by the transformer instance.

Current implementation just pass as argument the result of the decorated method, for example, the activity returned by this method however that activity or another result has limited information, for this example the filler would have something like

   "object":{
      "id":"https://example.com/course/course-v1:edx+cs105+101_2023",
      "definition":{
         "name":{
            "en":" testing-course "
         },
         "type":"http://adlnet.gov/expapi/activities/course",
         "extensions":{
            "https://w3id.org/xapi/acrossx/extensions/type":"audit"
         }
      },
      "objectType":"Activity"
   }

In this case, if someone wants to get more data, for example, event_name, it's not possible, so adding the transformer instance allows to the person who is developing the filter to access instance methods and attributes like transformer.get_data() or transformer.event

Testing instructions:

This works like another openedx-filter https://github.com/openedx/openedx-filters/

  1. Define your pipeline, follow the openedx-filter instructions,finally the logic is up to you.A basic implementation could be the following
import logging
from openedx_filters import PipelineStep

LOGGER = logging.getLogger(__name__)

class Something(PipelineStep):

    def run_filter(self, transformer, result):
        LOGGER.info("I'm very happy with my first pipeline")
        return {
            "result": result
        }
  1. include the configuration in your settings
OPEN_EDX_FILTERS_CONFIG = {
    "event_routing_backends.processors.xapi.transformer.xapi_transformer.get_actor": {
        "pipeline": ["path.to.the.class.Something"],
        "fail_silently": False,
    }
}
  1. Raise any event and check that the pipeline logic was executed

Note

This is a breaking change since current filters won't work with this version

Merge checklist:

  • All reviewers approved
  • CI build is green
  • Version bumped
  • Changelog record added
  • Documentation updated (not only docstrings)
  • Commits are squashed

Post merge:

  • Create a tag
  • Check new version is pushed to PyPI after tag-triggered build is
    finished.

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Nov 22, 2023
@openedx-webhooks
Copy link

Thanks for the pull request, @andrey-canon! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

  • supporting documentation
  • Open edX discussion forum threads
  • timeline information ("this must be merged by XX date", and why that is)
  • partner information ("this is a course on edx.org")
  • any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

andrey-canon added a commit to eduNEXT/eox-nelp that referenced this pull request Nov 22, 2023
This version updates the filters signature in order to support transformer arguments
openedx/event-routing-backends#374
@mphilbrick211 mphilbrick211 requested a review from a team November 27, 2023 21:44
@mphilbrick211
Copy link

Hi @openedx/arch-bom! Is this something you could please review / merge for us?

@robrap
Copy link

robrap commented Nov 27, 2023

@andrey-canon: Is this just building on work from https://github.com/openedx/event-routing-backends/blob/master/CHANGELOG.rst#710, and should it have the same reviewers? Also, reminder to use 8.0.0 if this is a breaking change. Also, do you intend to fix anything using the older version if there are any?

@ziafazal ziafazal self-requested a review November 29, 2023 14:10
Copy link
Contributor

@ziafazal ziafazal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrey-canon Thanks for creating this PR. Overall looks good. Could you please update the docstrings to reflect addition of transformer in arguments e.g docstring of run_filter method.

Copy link

@robrap robrap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my earlier comment and update the CHANGELOG as a minimum, as well as the version if you plan on releasing this change. Thank you.

@robrap robrap added the waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. label Nov 29, 2023
This improves the filter usability by adding the transformer to the filter arguments,
basically this allows the filter client to access to extra data given by the transformer instance
@andrey-canon andrey-canon force-pushed the and/add_transformer_to_dymanic_filter branch from e71a470 to f59859a Compare November 29, 2023 22:45
@andrey-canon
Copy link
Contributor Author

@andrey-canon: Is this just building on work from https://github.com/openedx/event-routing-backends/blob/master/CHANGELOG.rst#710, and should it have the same reviewers? Also, reminder to use 8.0.0 if this is a breaking change. Also, do you intend to fix anything using the older version if there are any?

This doesn't fix anything this just complements the current implementation, since after using the current feature I realized that was very hard to get some basic information in the external filters e.g event name by adding this transformer the data can be extracted easily from the transformer

CHANGELOG.rst Outdated
[8.0.0]
~~~~~~~

* Add transformer argument to openedx dynamic filter.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Add transformer argument to openedx dynamic filter.
* **BREAKING CHANGE**: Add transformer argument to openedx dynamic filter.

@robrap
Copy link

robrap commented Nov 30, 2023

@andrey-canon: Thanks for the version and changelog updates.

This doesn't fix anything this just complements the current implementation, since after using the current feature I realized that was very hard to get some basic information in the external filters e.g event name by adding this transformer the data can be extracted easily from the transformer

In the description, you had written:

This is a breaking change since current filters won't work with this version

I was asking if there were current filters that will be broken and need to be updated to work with this version?

@andrey-canon
Copy link
Contributor Author

@andrey-canon: Thanks for the version and changelog updates.

This doesn't fix anything this just complements the current implementation, since after using the current feature I realized that was very hard to get some basic information in the external filters e.g event name by adding this transformer the data can be extracted easily from the transformer

In the description, you had written:

This is a breaking change since current filters won't work with this version

I was asking if there were current filters that will be broken and need to be updated to work with this version?

The filter is an extension point that anyone can create so I'm not aware if someone created that, for sure the community doesn't have any filters since the use the default behavior, since this is very recent probably I'm the only one that has implemented a filter for this library

@andrey-canon andrey-canon requested a review from robrap December 6, 2023 15:27
Copy link

@robrap robrap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @andrey-canon.

[inform] See https://discuss.openedx.org/t/readthedocs-build-failures-missing-config-or-invalid-build-os/11871

Is this something you can merge, or will you need help with that?

@robrap
Copy link

robrap commented Dec 6, 2023

@andrey-canon: See this comment on the cookiecutter update: openedx/edx-cookiecutters#419 (comment). It may have issues.

@andrey-canon andrey-canon force-pushed the and/add_transformer_to_dymanic_filter branch from f93e46a to 637812e Compare December 6, 2023 21:28
@andrey-canon
Copy link
Contributor Author

@andrey-canon: See this comment on the cookiecutter update: openedx/edx-cookiecutters#419 (comment). It may have issues.

@robrap I included the original fix however this repository has a lot of warnings that's why I had to remove the fail_on_warning: true and now all tests are passing

@robrap
Copy link

robrap commented Dec 6, 2023

Great. Thanks @andrey-canon. Who will be merging this? And releasing? I'll be out the rest of the week.

@andrey-canon
Copy link
Contributor Author

Great. Thanks @andrey-canon. Who will be merging this? And releasing? I'll be out the rest of the week.

@Ian2012 could you help us with this ?

@Ian2012 Ian2012 merged commit b8883f9 into openedx:master Dec 6, 2023
@openedx-webhooks
Copy link

@andrey-canon 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
open-source-contribution PR author is not from Axim or 2U waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants