[affiliation] Improve performance of affiliation jobs #836
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jjmerchante
requested changes
Dec 11, 2023
sortinghat/core/jobs.py
Outdated
Comment on lines
175
to
179
| try: | ||
| individual = find_individual_by_uuid(rec.key) | ||
| except NotFoundError: | ||
| logger.warning(f"Job {job.id} 'Individual {rec.key} not found'") | ||
| continue |
There was a problem hiding this comment.
I think we don't need this part. We have the individual.mk in rec.mk, and we can create the AffiliationRecommendation object using rec.mk, as we do in recommend_matches() with the following code:
try:
with transaction.atomic():
AffiliationRecommendation.objects.create(individual_id=rec.mk, organization=org)
except IntegrityError:
pass
We can also use rec.key because recommend_affiliations() yields the same value for both.
There was a problem hiding this comment.
I think we can only use rec.mk because if the list of uuids is not empty it will mean to create recommendations for those uuids only. In that case key is the uuid and not the mk
The current algorithm was running unnecessary calls to 'find_individual_by_uuid' when the generic affiliation job was called. In other words, when the list of individuals to affiliate was all the individuals in the database. To improve it, we have followed the same approach we did with the matching recommendations. We get the full list of individuals with a direct query using Django's API. Signed-off-by: Santiago Dueñas <[email protected]>
|
LGTM |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current algorithm was running unnecessary calls to 'find_individual_by_uuid' when the generic affiliation job was called. In other words, when the list of individuals to affiliate was all the individuals in the database.
To improve it, we have followed the same approach we did with the matching recommendations. We get the full list of individuals with a direct query using Django's API.