Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid linecache unique file generation in case of linecache disabled #461

Merged
merged 3 commits into from
Nov 27, 2023

Conversation

sahu-sunil
Copy link
Contributor

When cattrs' converter/hooks or same classes are created too frequent or in large amount, generate_unique_filename (generation of unique file with uuid) becomes slow/inefficient even if linecache flag is disabled.
This PR executes this function only when cache is enabled. More optimisation can be done for compilation of same classes repeatably, as mentioned in below issue.
Issue discussion: #445

@Tinche
Copy link
Member

Tinche commented Nov 24, 2023

@sahu-sunil cool, I can merge this in. Does it fix your actual problem though? You can install from this branch and test.

If not I can try applying more optimizations.

@sahu-sunil
Copy link
Contributor Author

Yes it was bit helpful but in long duration CPU is still gradually increasing. I am confident that this is due to linecache left enabled somewhere hidden place. I am going to try cattrs with linecache False by default instead of True (current behaviour).

Yes, please check if you see some scope of optimisation on repeated compilation

@Tinche
Copy link
Member

Tinche commented Nov 27, 2023

@sahu-sunil Ok, let's merge this in and I can take a look at a bigger optimization.

Can you just add a short line to HISTORY.md?

@Tinche Tinche merged commit b47f446 into python-attrs:23.2 Nov 27, 2023
8 of 9 checks passed
@Tinche Tinche modified the milestone: 24.1 Nov 27, 2023
@Tinche
Copy link
Member

Tinche commented Nov 27, 2023

Thanks, I'll ping you when I have something more.

@sahu-sunil
Copy link
Contributor Author

sahu-sunil commented Nov 27, 2023

Yes it was bit helpful but in long duration CPU is still gradually increasing. I am confident that this is due to linecache left enabled somewhere hidden place. I am going to try cattrs with linecache False by default instead of True (current behaviour).

I tried with this locally and the cpu is stable now. Will see if I can find all the places where linecache can be disabled.

I know it won't be feasible to reverse on cattrs level, like I did locally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants