Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further improvement of Performance #1544

Merged
merged 3 commits into from
Apr 14, 2022
Merged

Conversation

vvmruder
Copy link
Collaborator

@vvmruder vvmruder commented Apr 6, 2022

@peterschaer
I did a quick profile of the extract generation (based on our small test data set). It reveals several points to poke around. But one easy catch is right in the beginning of the process:
Screenshot_20220406_114602

Is related to #1508

As you can see we lose almost 4% of process time between start and ExtractReder.read, I found that we had an old left over in the code. Instead of reading glossaries, disclaimers and municipalities for every extract they are read once on server start now.

Could you please try and tell what you think about this?

To use this patch it should be enough to install it. No config changes needed.

@codecov
Copy link

codecov bot commented Apr 6, 2022

Codecov Report

Merging #1544 (01cfd1b) into master (015009f) will increase coverage by 0.02%.
The diff coverage is 93.54%.

@@            Coverage Diff             @@
##           master    #1544      +/-   ##
==========================================
+ Coverage   70.78%   70.80%   +0.02%     
==========================================
  Files         121      121              
  Lines        4970     4988      +18     
==========================================
+ Hits         3518     3532      +14     
- Misses       1452     1456       +4     
Flag Coverage Δ
unittests 70.80% <93.54%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pyramid_oereb/core/views/webservice.py 87.26% <ø> (-0.07%) ⬇️
pyramid_oereb/core/config.py 79.78% <91.11%> (+0.72%) ⬆️
...ontrib/data_sources/standard/sources/disclaimer.py 100.00% <100.00%> (ø)
.../contrib/data_sources/standard/sources/glossary.py 100.00% <100.00%> (ø)
...trib/data_sources/standard/sources/municipality.py 100.00% <100.00%> (ø)
pyramid_oereb/core/processor.py 91.26% <100.00%> (-1.35%) ⬇️
pyramid_oereb/core/readers/disclaimer.py 100.00% <100.00%> (ø)
pyramid_oereb/core/readers/glossary.py 100.00% <100.00%> (ø)
pyramid_oereb/core/readers/municipality.py 100.00% <100.00%> (ø)
pyramid_oereb/core/sources/disclaimer.py 100.00% <100.00%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 015009f...01cfd1b. Read the comment docs.

@peterschaer
Copy link
Collaborator

Looks good to me, Thanks. It is certainly better to read those informations only once. Regarding the overall performance see my notes in #1508

@vvmruder
Copy link
Collaborator Author

@voisardf @kdeininger @michmuel I would be happy to see a quick review on that too :)

Copy link
Collaborator

@michmuel michmuel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deployed the branch in our test setting. Seems to work smoothly. Thanks.

@vvmruder vvmruder merged commit ce4a551 into master Apr 14, 2022
@vvmruder vvmruder deleted the bugfix/1508-improve-performance branch April 14, 2022 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants