Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: [FC-0031] Fix count items in pagination for api courses list. #33293

Conversation

oksana-slu
Copy link
Contributor

@oksana-slu oksana-slu commented Sep 20, 2023

Description

This PR is resolving an issue with incorrect count returned from pagination object for all pages except for the last page while searching for courses using API api/courses/v1/courses/ and providing search_term query parameter.

Supporting information

This contribution is done as a part of the project FC-0031

Testing instructions

GET api/courses/v1/courses/ specifying query_param search_term.
Check that count parameter in a response has a correct value.

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Sep 20, 2023
@openedx-webhooks
Copy link

openedx-webhooks commented Sep 20, 2023

Thanks for the pull request, @oksana-slu! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

  • supporting documentation
  • Open edX discussion forum threads
  • timeline information ("this must be merged by XX date", and why that is)
  • partner information ("this is a course on edx.org")
  • any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

@KyryloKireiev KyryloKireiev force-pushed the rg/fix/FC0031/fix_count_items_in_pagination_for_api_courses_list branch from bfb7bfc to 4b229ca Compare September 21, 2023 13:53
@oksana-slu oksana-slu force-pushed the rg/fix/FC0031/fix_count_items_in_pagination_for_api_courses_list branch from 4b229ca to 3bb2d49 Compare September 22, 2023 10:13
@oksana-slu oksana-slu marked this pull request as ready for review September 25, 2023 09:44
@mphilbrick211 mphilbrick211 added the FC Relates to an Axim Funded Contribution project label Oct 2, 2023
@jawad-khan jawad-khan self-requested a review October 5, 2023 07:57
@jawad-khan
Copy link
Contributor

Looks good to me. Can you please ask for a review in relevant squad i.e. team-aurora in this case.

@@ -100,13 +100,10 @@ def _filter_by_search(course_queryset, search_term):
)

search_courses_ids = {course['data']['id'] for course in search_courses['results']}

courses = [course for course in course_queryset if str(course.id) in search_courses_ids]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ormsbee is there gonna be a performance impact for changing the lazy sequence this way? I feel like getting an inaccurate count here is expected

https://github.com/openedx/edx-platform/blob/2033dcf6ace133719aaeb72dc5dd6ee521a7ac42/openedx/core/lib/api/view_utils.py#L246C1-L248C47

It is immutable, and accepts an estimated length in order to support __len__ without exhausting the underlying sequence.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@feanil: Almost certainly, yes. The whole point of LazySequence is to not have to iterate the entire queryset, which could potentially be 10K courses.

If this is related to the mobile effort, would it be possible to cap the number of results (e.g. pass in a parameter that will tell it to return no more than X results)? Doing something like the above with a cap of 100 courses or something should be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oksana-slu let me know your thoughts but given the potential performance problem, I'm reluctant to land the change as is,.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, that's definitely a miss.
We discussed the possible solution today with @volodymyr-chekyrta, and it's a fair tradeoff to limit the number of returned results for mobile to 100 or even to 50 for now.

@KyryloKireiev KyryloKireiev force-pushed the rg/fix/FC0031/fix_count_items_in_pagination_for_api_courses_list branch from 24fc358 to e120526 Compare November 1, 2023 13:40
@GlugovGrGlib GlugovGrGlib force-pushed the rg/fix/FC0031/fix_count_items_in_pagination_for_api_courses_list branch from e120526 to 258f3fc Compare November 1, 2023 15:07
@GlugovGrGlib
Copy link
Member

Hi @feanil @jawad-khan, we've updated the solution, kindly ask you to review the new implementation.

We have introduced mobile_search boolean query parameter to limit max search quantity to a specific value (default 100) as proposed by Dave and confirmed with @volodymyr-chekyrta

Copy link
Contributor

@feanil feanil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix all, this looks good to me. Do you need me to merge this as well?

@GlugovGrGlib
Copy link
Member

@feanil Yes, please merge it

@feanil feanil merged commit c4598c5 into openedx:master Nov 9, 2023
63 checks passed
@openedx-webhooks
Copy link

@oksana-slu 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

2U Release Notice: This PR has been deployed to the edX production environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FC Relates to an Axim Funded Contribution project open-source-contribution PR author is not from Axim or 2U
Projects
Archived in project
Status: Done
Development

Successfully merging this pull request may close these issues.

9 participants