Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tunable to allow changing micro ZAP's max size #14292

Merged
merged 1 commit into from
Jan 10, 2023

Conversation

0mp
Copy link
Contributor

@0mp 0mp commented Dec 15, 2022

This change turns MZAP_MAX_BLKSZ into a ZFS_MODULE_PARAM() called zap_micro_max_size. As a result, we can experiment with different micro ZAP sizes to improve directory size scaling.

Co-authored-by: Mateusz Piotrowski [email protected]
Signed-off-by: Mateusz Piotrowski [email protected]
Sponsored-by: Wasabi Technology, Inc.

Motivation and Context

Having those additional tunables enables users to experiment with different micro ZAP sizes to improve directory size scaling.

Description

The patch introduces a variable in place of the MZAP_MAX_BLKSZ constant. Additionally, the new variable can be set as a ZFS module parameter (via ZFS_MODULE_PARAM).

How Has This Been Tested?

We've run this patch during some of Klara's recent benchmarks.

gmake cstyle is OK.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@0mp 0mp force-pushed the 0mp/klara/klara-mzap branch 2 times, most recently from a8c4045 to 1fd3e1a Compare December 21, 2022 10:44
@0mp
Copy link
Contributor Author

0mp commented Dec 21, 2022

I forced push to fix the author of the commit (it should be @tsoome first of all) and to resolve a merge conflict in man/man4/zfs.4.

@amotin
Copy link
Member

amotin commented Dec 21, 2022

For sizes >= 512KB it may worth to investigate increase the last argument of zfs_btree_create_custom() a bit to avoid additional level in the B-tree. It is a balance between memory usage for small directories, insert/remove cost and search cost.

And I think shrinking of the micro ZAP should be quite easy to implement and good for performance. The bigger you set the maximum size, the more you start wasting memory and CPU without ability to shrink it back.

@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Dec 22, 2022
This change turns `MZAP_MAX_BLKSZ` into a `ZFS_MODULE_PARAM()` called
`zap_micro_max_size`. As a result, we can experiment with different
micro ZAP sizes to improve directory size scaling.

Co-authored-by: Mateusz Piotrowski <[email protected]>
Signed-off-by: Mateusz Piotrowski <[email protected]>
Sponsored-by: Wasabi Technology, Inc.
@0mp 0mp force-pushed the 0mp/klara/klara-mzap branch from 1fd3e1a to fce7916 Compare January 9, 2023 22:25
@0mp
Copy link
Contributor Author

0mp commented Jan 9, 2023

Rebased and fixed a checkstyle issue (./module/zfs/zap_micro.c: 1730: comma or semicolon preceded by blank).

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jan 10, 2023
@behlendorf behlendorf merged commit a4b21ea into openzfs:master Jan 10, 2023
@0mp
Copy link
Contributor Author

0mp commented Jan 15, 2023

Thanks!

@0mp 0mp deleted the 0mp/klara/klara-mzap branch January 15, 2023 15:21
lundman pushed a commit to openzfsonwindows/openzfs that referenced this pull request Mar 3, 2023
This change turns `MZAP_MAX_BLKSZ` into a `ZFS_MODULE_PARAM()` called
`zap_micro_max_size`. As a result, we can experiment with different
micro ZAP sizes to improve directory size scaling.

Reviewed-by: Brian Behlendorf <[email protected]>
Co-authored-by: Mateusz Piotrowski <[email protected]>
Co-authored-by: Toomas Soome <[email protected]>
Signed-off-by: Mateusz Piotrowski <[email protected]>
Sponsored-by: Wasabi Technology, Inc.
Closes openzfs#14292
geoffamey pushed a commit to BlueArchive/storage-zfs-wasabi that referenced this pull request Jul 5, 2023
This change turns `MZAP_MAX_BLKSZ` into a `ZFS_MODULE_PARAM()` called
`zap_micro_max_size`. As a result, we can experiment with different
micro ZAP sizes to improve directory size scaling.

Reviewed-by: Brian Behlendorf <[email protected]>
Co-authored-by: Mateusz Piotrowski <[email protected]>
Co-authored-by: Toomas Soome <[email protected]>
Signed-off-by: Mateusz Piotrowski <[email protected]>
Sponsored-by: Wasabi Technology, Inc.
Closes openzfs#14292
(cherry picked from commit a4b21ea)
@robn robn mentioned this pull request Oct 2, 2024
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants