-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
THREESCALE-11139 simplify application queries #3845
Changes from all commits
61e2c26
16f19f9
cef5079
9eba9d6
77ea9e7
63d063c
0f93b94
dcdcd9a
8377ecf
f368ddf
dc71b57
cd9acce
4073aec
c2ed9f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,7 +7,7 @@ class Admin::Api::ApplicationsController < Admin::Api::BaseController | |
# GET /admin/api/applications.xml | ||
def index | ||
apps = applications.scope_search(search) | ||
.serialization_preloading.paginate(:page => current_page, :per_page => per_page) | ||
.serialization_preloading(request.format).paginate(:page => current_page, :per_page => per_page) | ||
respond_with(apps) | ||
end | ||
|
||
|
@@ -24,28 +24,26 @@ def application_filter_params | |
end | ||
|
||
def applications | ||
@applications ||= begin | ||
cinstances = current_account.provided_cinstances.where(service: accessible_services) | ||
if (service_id = params[:service_id]) | ||
cinstances = cinstances.where(service_id: service_id) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is already being done by But I think it doesn't make sense for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the actual fix for the query in JIRA, right? The rest are just N+1 improvements? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I actually don't agree with this. We do have I tried these three tests (in
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks a lot for catching this! I added your tests and updated controller to add the service condition when searching by There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nope, it's actually the same for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 😵💫 ok
So it sounds like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, right,
searching by There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you have There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated, PTAL |
||
end | ||
cinstances | ||
end | ||
@applications ||= current_account.provided_cinstances.merge(accessible_services) | ||
end | ||
|
||
def application | ||
@application ||= case | ||
return @application if defined?(@application) | ||
|
||
scope = params[:service_id] ? applications.where(service_id: params[:service_id]) : applications | ||
|
||
@application = case | ||
|
||
when user_key = params[:user_key] | ||
when param_key = params[:user_key] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why rename There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. because underneath there is a baby_squeel expression |
||
# TODO: these scopes should be in model layer | ||
# but there is scope named by_user_key already | ||
applications.joins(:service).where("(services.backend_version = '1' AND cinstances.user_key = ?)", user_key).first! | ||
scope.where.has { (service.backend_version == '1') & (user_key == param_key) }.first! | ||
|
||
when app_id = params[:app_id] | ||
applications.joins(:service).where("(services.backend_version <> '1' AND cinstances.application_id = ?)", app_id).first! | ||
when app_id = params[:app_id] | ||
scope.where.has { (service.backend_version != '1') & (application_id == app_id) }.first! | ||
|
||
else | ||
applications.find(params[:application_id] || params[:id]) | ||
scope.find(params[:application_id] || params[:id]) | ||
end | ||
end | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -544,7 +544,7 @@ def on_trial? | |
# Grabs the support_email if defined, otherwise falls back to the email of first admin. Dog. | ||
def support_email | ||
se = self[:support_email] | ||
se.presence || admins.first&.email | ||
se.presence || first_admin&.email | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There are many occurrences of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needs to be evaluated. Maybe but I'm not sure it can be a simple I would rather leave this for further PRs. This is big enough already. |
||
end | ||
|
||
def finance_support_email | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -148,7 +148,9 @@ def validate_plan_is_unique? | |
} | ||
|
||
def self.provided_by(account) | ||
joins(:service).references(:service).merge(Service.of_account(account)).readonly(false) | ||
# we can access service through plan but also keep service.id in sync with plan.service.id | ||
# this is a simpler way to do the query used historically | ||
joins(:service).where.has { service.sift(:of_account, account) } | ||
Comment on lines
+151
to
+153
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the main fix point (UPDATED). But there are further optimizations in other files. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @akostadinov Do you have any idea why the ogiginal query had There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no idea, probably itdidn't work without it at some point. I think it shouldn't be needed anymore |
||
end | ||
|
||
scope :not_bought_by, ->(account) { where.has { user_account_id != account.id } } | ||
|
@@ -199,9 +201,17 @@ def self.by_service(service) | |
|
||
# maybe move both limit methods to their models? | ||
|
||
def self.serialization_preloading | ||
includes(:application_keys, :plan, :user_account, | ||
service: [:account, :default_application_plan]) | ||
def self.serialization_preloading(format = nil) | ||
# With Rails 6.1 trying to include plan->issuer without service results in | ||
# > Cannot eagerly load the polymorphic association :issuer | ||
# When both have the same sub-includes, cache takes care of the duplicate queries. | ||
service_includes = %i[proxy account] | ||
plan_includes = [{issuer: service_includes}] | ||
if format == "xml" | ||
service_includes << :default_application_plan | ||
plan_includes << :original | ||
end | ||
Comment on lines
+210
to
+213
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this only for XML responses?... JSON/UI don't need these fields? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For JSON it complained that we are eager loading stuff that is not later accessed. UI uses another controller. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't like this too much... but OK, if that makes the queries more efficient - let's keep it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Otherwise these are queried once per entry displayed in the XML. So we should have it. But if you have a better idea how to do, that would be cool. |
||
includes(:user_account, service: service_includes, plan: plan_includes) | ||
end | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -60,7 +60,11 @@ class Service < ApplicationRecord # rubocop:disable Metrics/ClassLength | |
service.has_many :api_docs_services, class_name: 'ApiDocs::Service' | ||
end | ||
|
||
scope :of_account, ->(account) { where.has { account_id == account.id } } | ||
sifter :of_account do |account| | ||
account_id == account.id | ||
end | ||
|
||
scope :of_account, ->(account) { where.has { sift(:of_account, account) } } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is using sifter any better than a plain scope?... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can use the sifter in a join query while I don't think you can use a scope like that. |
||
|
||
has_one :proxy, dependent: :destroy, inverse_of: :service, autosave: true | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -125,7 +125,7 @@ def raw_applications | |
def applications | ||
@applications ||= raw_applications.scope_search(search) | ||
.order_by(*sorting_params) | ||
.preload(:service, user_account: %i[admin_user], plan: %i[pricing_rules]) | ||
.includes(plan: %i[pricing_rules]) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry for my laziness - what is the main difference between There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. AFAIU In this case I don't want to mix them because IIRC there was some There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks a lot for the explanation! |
||
.paginate(pagination_params) | ||
.decorate | ||
end | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -26,7 +26,13 @@ | |
can :create, Account | ||
can :update, Account if account.provider_can_use?(:service_permissions) | ||
|
||
can %i[read show edit update], Cinstance, user.accessible_cinstances.where_values_hash | ||
# Using historical optimized way and leave canonical way (through plan) commented out below | ||
# The resulting hash presently is something like {"type"=>"Cinstance", "service_id"=>[ids..]} | ||
can %i[read show edit update], Cinstance, Cinstance.permitted_for(user).where_values_hash | ||
# can %i[read show edit update], Cinstance, user.accessible_cinstances do |cinstance| | ||
# cinstance.plan&.issuer_type == "Service" && cinstance.plan.issuer.account == user.account && | ||
# (!user.forbidden_some_services? || user.member_permission_service_ids.include?(cinstance.plan.issuer.id)) | ||
# end | ||
Comment on lines
+31
to
+35
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's the difference between the new one, the old one and the commented one? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the commented out uses a block which is not recommended. I believe more description is in the commit messages. But for the time being, I'm looking at other things. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
# abilities for buyer users | ||
can %i[read update update_role destroy suspend unsuspend], User, account: { provider_account_id: user.account_id } | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
if (Rails.env.development? || Rails.env.test?) && ENV["TRACE_SQL"] == "1" | ||
ActiveRecordQueryTrace.enabled = true | ||
ActiveRecordQueryTrace.level = :app | ||
Rails.application.config.log_level = :debug | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this method ever used for arrays? if not, maybe we should rename the argument as
account
to make it clear it's accepts and returns a single object?Otherwise (to make it more generic), it can be left as it is, but then I guess it should be
Array(accounts).flatten
in caseaccounts
is actually an array?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't need flatten.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh OK, nice
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's the first time I see
Preloader
, what's the advantage over.includes
? Why is this method supposed to receive "accounts" but instead it's called withbuyer_account
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there are no advantages over
#includes
. Issue is that#includes
only operates with Associations/queries that are not already performed. If your method receives a bunch of active record objects, then you can't call#includes
on them.