LG-15119 query optimization #11574
LG-15119 query optimization #11574
Conversation
Adjust query and add index
| partner_account_status: Agreements::PartnerAccountStatus.find_by(name: 'active'), | ||
| became_partner: ..report_date, | ||
| }). | ||
| distinct |
There was a problem hiding this comment.
this has a 2x speed improvement
| @facial_match_issuers ||= Profile.active.verified.facial_match. | ||
| where('verified_at <= ?', report_date.end_of_day). | ||
| distinct. | ||
| pluck(:initiating_service_provider_issuer) |
There was a problem hiding this comment.
I think this is now 6x faster
| def change | ||
| add_index :profiles, %i[active idv_level verified_at], algorithm: :concurrently, name: 'index_profiles_on_active_and_idv_level_and_verified_at' | ||
| end | ||
| end |
There was a problem hiding this comment.
This shouldn't have too many writes. It should significantly speed up the query above.
There was a problem hiding this comment.
Does the report run fast enough without the index?
There was a problem hiding this comment.
I think so. Are you concerned about the writes for this?
There was a problem hiding this comment.
I think in general we should avoid creating indexes if possible. The current query that would use this runs quickly enough, and we'll hopefully be able to avoid using Postgres for it soon.
Co-authored-by: Zach Margolis <[email protected]>
mitchellhenke
force-pushed
the
LG-15119/query-optimization
branch
from
ef9d53c to
49038c6
Compare
| require "rubygems" | ||
| require "bundler/setup" | ||
|
|
||
| load Gem.bin_path("rspec-core", "rspec") |
There was a problem hiding this comment.
helpful to not have to type 'bundle exec'
| Profile.where(active: true).where( | ||
| 'verified_at <= ?', | ||
| end_date, | ||
| Profile.active.where( |
There was a problem hiding this comment.
Updated most of this file to use the available class methods.
| pluck(:initiating_service_provider_issuer).uniq | ||
| @facial_match_issuers ||= Reports::BaseReport.transaction_with_timeout do | ||
| Profile.active.verified.facial_match_opt_in. | ||
| where('verified_at <= ?', report_date.end_of_day). |
There was a problem hiding this comment.
Is the verified scope on the prior line redundant with this?
There was a problem hiding this comment.
yeah, it can't be active without verified so idk why I am the way that I am.
Adjust queries and add index
Link to the relevant ticket:
LG-15119
🛠 Summary of changes
Noticed a timeout in the PG query. We were pulling all profiles and using ruby .uniq instead of .distinct. This needed wrapped with the read_replica time_out method.
I've also added a new IDV Facial Match level to be IAL2 opt-in and updated the queries with it.