Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix conflict with filtered dataset export and paginated results #484

Merged
merged 1 commit into from
Jan 25, 2024

Conversation

contolini
Copy link
Member

We use frm and search_after query params to handle paginated search results. But when a user exports a subset of our complaint database, the aforementioned pagination params are sent to the API causing it to throw an error because frm mod size != 0.

Instead of reworking the API logic it's better to just purge pagination params from export URLs because they're irrelevant. The whole point of exporting is to download all the search results and not just a single page of search results.

See DATAP-33 for more info and context.

Changes

  • Updates util function that generates the export link

Testing

  • yarn test should pass
  • yarn cypress run --browser chrome --headless --config baseUrl=http://localhost:8000/data-research/consumer-complaints/search/ should pass
  • Go to https://www.consumerfinance.gov/data-research/consumer-complaints/search/?tab=List
  • Scroll to bottom and click Next > to get to page 2 of results.
  • Scroll up and click Export data
  • Select Filtered dataset
  • Click Start export
  • You should be greeted by a 400 error
  • Do the above steps on localhost with this branch and it should correctly start a file download instead of 400ing

Checklist

  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows the standards laid out in the front end playbook
  • Passes all existing automated tests
  • New functions include new tests
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged
  • Visually tested in supported browsers and devices
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)

Our front-end uses `from` and `search_after` query params to handle paginated
search results. When a user exports a subset of our complaint database, the
aforementioned pagination params are sent to the API causing it to throw an
error because `from mod size != 0`.

Instead of reworking the API logic it's better to just purge pagination
params from export URLs because they're irrelevant. The whole point of exporting
is to download all the search results and not just a single page of search results.

See https://github.com/cfpb/ccdb5-api/blob/a63efc345e1a652f9c8ec4875e9247aef6f56128/complaint_search/serializer.py#L185-L189
See https://jira/browse/DATAP-33
Copy link
Contributor

@flacoman91 flacoman91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the api endpoint without those params.

I can't test it, but logically it seems solid 👍

@contolini contolini added this pull request to the merge queue Jan 25, 2024
Merged via the queue into main with commit 9bedfa7 Jan 25, 2024
2 checks passed
@contolini contolini deleted the datap-33 branch January 25, 2024 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants