Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use static caching to reduce lookups in import display #29059

Merged
merged 1 commit into from
Jan 22, 2024

Conversation

eileenmcnaughton
Copy link
Contributor

@eileenmcnaughton eileenmcnaughton commented Jan 21, 2024

Overview

Use static caching to reduce lookups in import display

Before

When loading an import display the modifiySpec function is called once per editable field * the number of lines. As each field is set to editable this means the number of columns in the csv * the number of rows. Commonly the number of columns is large because an export from civicrm has a large number of columns and the default number of rows displayed is 25 - so in the api call that is cached here is called hundreds of times (at least) to render the view

After

The use of the static cache removes this point of repetitiveness - and makes sense IMHO but the getEditableInfo function remains extremely slow as this is small part of that function

Technical Details

I used static caching rather than metadata cache as the information is quick to load and that way we can do it once per job. If we use the cache then we get into how we clear it & wind up putting all the jobs in one cache - which risks being slow to serialize & unserialize

Comments

Copy link

civibot bot commented Jan 21, 2024

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...
  • If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
  • A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
  • A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
  • If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
  • Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
  • A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
  • To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
  • For more information about contributing, see CONTRIBUTING.md.
Quick links for reviewers...

➡️ Online demo of this PR 🔗

@civibot civibot bot added the master label Jan 21, 2024
@colemanw colemanw merged commit 82b935d into civicrm:master Jan 22, 2024
3 checks passed
@colemanw colemanw deleted the import_cache branch January 22, 2024 01:13
wmfgerrit pushed a commit to wikimedia/wikimedia-fundraising-crm that referenced this pull request Jan 26, 2024
 Bug: T341902

Change-Id: I2687c58e30f40b219ebb20b52ec972fa3e3ec62d
wmfgerrit pushed a commit to wikimedia/wikimedia-fundraising-crm that referenced this pull request Jan 31, 2024
…undraising/crm into deployment

+ b0f0dee CiviCRM 5.70-rc
+ ab90337 Port civicrm/civicrm-core#29057
+ 1c810d9 Port civicrm/civicrm-core#29061
+ 36554c2 Port civicrm/civicrm-core#29059
+ d6acf5a Also update next_sched_contribution_date and cycle_day for for old paypal
+ 38ce06b Port civicrm/civicrm-core#29062
+ c30f7e3 Fix failure to calculate donor_status 90
+ 665f1b9 IDE formatting
+ 79b692c Apply IDE formatting for short-array syntax
+ 2f8e405 Use EntityTrait & createTestEntity creating contacts
+ 9b37027 Add test for unspecified contact ID bug
+ 6fa13ca Add missing use statement
+ 9b74cbc Set next_sched date when entering autoRescue
+ d649850 Fixes for recurring queue consumer and tests
+ ac3221b Add import_extensions
+ e286eef Add managed template for data axle import
+ 9f01676 Add failed recurring to fundraiseup imports
+ 5d9feb8 Pull in lastest 5.70rc updates
+ 96884f8 Fix recurring queue test
+ e60b161 Fix Deep Lapsed to 2-5 financial years

Change-Id: I86f92e6f5bc68debe16924aeaf4110c3eff39ba9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants