-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefetch descriptor functions at graph-handling time, not resource-handling time #10453
Comments
The last few bullet-points tacked on here don't provide enough value to work on.
[1] -- not very many queries |
Use count() for indexing progress bar re #10453
#10439 applied in two more places the trick from #10312 to skip queries for descriptor functions on the assumption that resources are generally grouped by graph:
arches/arches/app/utils/data_management/resources/formats/archesfile.py
Lines 393 to 395 in 1af6ea8
But we could get more savings by removing this "trick" and just prefetching the descriptor functions before we start iterating the resources.
That would go something like this:
resources
, use aPrefetch()
object to do the filtering for the graph's descriptor function and save it todescriptor_function
on each Resource.Then, with the hood popped on this (looking at callers of
save_descriptors()
), there are a few more wins to be had:index_resources_by_type()
refetches graph objects because the earlier call inindex_resources()
only gets a values arrayTile.delete()
refetches the resource instanceTile.delete()
saves the resource descriptor once per nodeimport_business_data_without_mapping()
fetches graphs one-by-oneThe text was updated successfully, but these errors were encountered: