Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Measure memory usage during memtests #411

Merged
merged 5 commits into from
Nov 10, 2023
Merged

Measure memory usage during memtests #411

merged 5 commits into from
Nov 10, 2023

Conversation

almarklein
Copy link
Member

@almarklein almarklein commented Nov 9, 2023

Closes #406.

This modifies the tests in tests_mem/ to help investigate potential memory leaks.

In normal mode, runs the test twice for each object, with two different object-amounts. In memory-usage-mode, runs 40 iterations with an object-count of 8, track memory usage at each iteration, and report that.

  • Improve destruction of glfw and qt widgets.
  • Move some of the mem tests into their own test module (meta, gui).
  • Change the test so it can do multiple iterations.
  • By default it runs two iterations, one with 32 objects and another with 17.
  • By setting TEST_ITERS it will run that many iterations with 8 objects, printing diff memory usage at each iteration.

This work was originally about that last step, but in the end these memory measurement are pretty hard to interpret. They can sometimes be useful, and perhaps we can improve it in the future? But for now we only run those tests manually, not in CI.


Example:

test_release_adapter ...
 49.1KB      0B      0B      0B      0B      0B      0B      0B  16.3KB      0B
 16.3KB  16.3KB      0B      0B      0B      0B      0B      0B      0B      0B
     0B  16.3KB      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B  16.3KB      0B      0B  16.3KB      0B      0B  16.3KB      0B
>> SUSPICIOUS!
test_release_device ...
  skipped
test_release_bind_group ...
 49.1KB      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B  16.3KB      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B  16.3KB
>> SUSPICIOUS!
test_release_bind_group_layout ...
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B  32.7KB      0B
     0B      0B      0B      0B      0B      0B  16.3KB      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B  16.3KB
>> SUSPICIOUS!
test_release_buffer ...
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
test_release_command_buffer ...
 32.7KB      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B  16.3KB      0B      0B      0B      0B
>> SUSPICIOUS!
test_release_command_encoder ...
     0B      0B      0B -16.3KB      0B  16.3KB      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
test_release_compute_pass_encoder ...
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
     0B      0B      0B      0B      0B      0B      0B      0B      0B      0B
...

For the GUI backends, the memory usage seems to consistently increase. E.g. for glfw:

 18.6MB  2.49MB  2.94MB  3.08MB  1.04MB  1.37MB  1.14MB   966KB  1.34MB  2.14MB
  983KB   950KB  1.47MB  1.42MB  1.63MB  1.16MB  1.67MB  1.45MB  1.42MB   999KB
 2.06MB   671KB  1.04MB  1.50MB  1.03MB  1.34MB  1.44MB  1.42MB  1.29MB  1.44MB
 1.09MB  1.68MB  1.68MB  1.39MB  1.68MB  1.39MB  1.99MB  1.54MB  1.29MB  1.70MB

I'm not sure what to make of this, t.b.h. ... I wonder how useful these numbers are at all ...

For example, the glfw Canvas objects are deleted, so this is probably memory that glfw manages and may release at a later time? If I run with more iterations, I do see that memory is released (i.e. negative numbers in above tables), but not as much as is consumed.

@almarklein almarklein marked this pull request as ready for review November 9, 2023 13:24
@almarklein almarklein requested a review from Korijn as a code owner November 9, 2023 13:24
@almarklein
Copy link
Member Author

almarklein commented Nov 9, 2023

BTW, there are a couple of new files, this is just regrouping some of the unit tests in separate files. All the memory-usage stuff is inside testutils.py.

@almarklein almarklein mentioned this pull request Nov 10, 2023
30 tasks
@almarklein
Copy link
Member Author

I also improved the destruction of glfw and qt widgets. I simplified the mem-usage-measurement code so it gets out if the way, but can still be used. Added a summary of this PR in the top comment.

@almarklein almarklein merged commit 49b4ae4 into main Nov 10, 2023
17 checks passed
@almarklein almarklein deleted the mem branch November 10, 2023 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test against memory leaks by measuring process memory
2 participants