Combined writing for FSTree #2814

roman-khimov · 2024-04-18T21:23:30Z

We can think of replacing peapod with it.

roman-khimov · 2024-05-05T20:32:50Z

Notice that this scheme also helps with #1951 and #2107 (!) if we're to store meta separately.

carpawell

No changelog at all; the last commit was not DCOed.

So what is the peapod's fate?

pkg/local_object_storage/blobstor/fstree/fstree.go

pkg/local_object_storage/blobstor/fstree/fstree_write_linux.go

roman-khimov · 2024-05-06T19:47:08Z

So what is the peapod's fate?

TBD

codecov · 2024-05-06T20:04:49Z

Codecov Report

Attention: Patch coverage is 54.26357% with 118 lines in your changes missing coverage. Please review.

Project coverage is 23.62%. Comparing base (7baa16c) to head (1b40ec4).
Report is 13 commits behind head on master.

Files with missing lines	Patch %	Lines
...ject_storage/blobstor/fstree/fstree_write_linux.go	71.42%	22 Missing and 12 partials ⚠️
pkg/local_object_storage/blobstor/fstree/fstree.go	57.33%	27 Missing and 5 partials ⚠️
...node/config/engine/shard/blobstor/fstree/config.go	0.00%	18 Missing ⚠️
pkg/local_object_storage/blobstor/fstree/option.go	0.00%	12 Missing ⚠️
cmd/neofs-lens/internal/storage/root.go	0.00%	9 Missing ⚠️
cmd/neofs-node/storage.go	0.00%	5 Missing ⚠️
cmd/neofs-node/config.go	0.00%	4 Missing ⚠️
...ct_storage/blobstor/fstree/fstree_write_generic.go	40.00%	3 Missing ⚠️
cmd/neofs-lens/internal/storage/sanity.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2814      +/-   ##
==========================================
+ Coverage   23.51%   23.62%   +0.11%     
==========================================
  Files         776      775       -1     
  Lines       45325    45513     +188     
==========================================
+ Hits        10656    10754      +98     
- Misses      33822    33907      +85     
- Partials      847      852       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

carpawell · 2024-05-07T22:18:52Z

So what is the peapod's fate?

TBD

I am asking cause we have to rework some things if there is no "small" storage anymore.

Have you considered GET bench? Isn't it slower to read a combined object for only a part of it if there are a few parallel GET requests? Like reading 8MB for a 4KB object 128 times can be slower than reading 4KB 128 times from a peadod.

roman-khimov · 2024-05-13T13:47:45Z

Isn't it slower to read

For sure. But I doubt it's noticeable. I'll try to bench it.

Prevent panic on test error: --- FAIL: TestDumpIgnoreErrors (2.09s) dump_test.go:399: Error Trace: /home/rik/neofs-node/pkg/local_object_storage/shard/dump_test.go:399 Error: Not equal: expected: 10 actual : 11 Test: TestDumpIgnoreErrors panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa365a0] goroutine 1156 [running]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/shard.New.func1({0xbf05fb, 0x1e}, {0xd7ad80, 0xc000458bd0}) /home/rik/neofs-node/pkg/local_object_storage/shard/shard.go:138 +0x40 github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).reportFlushError(0xe6812a13a28266c3?, {0xbf05fb?, 0xb7e97c32721668a?}, {0xc0000a05a0?, 0x80008050838d2b0d?}, {0xd7ad80?, 0xc000458bd0?}) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:167 +0x25d github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).flushObject(0xc0004d62c0, 0xc0003a8480, {0x0, 0x0, 0x0}) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:272 +0x348 github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).flushWorker(0xc0004d62c0, 0x0?) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:241 +0x12d created by github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).runFlushLoop /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:35 +0x30 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa365a0] goroutine 1035 [running]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/shard.New.func1({0xbf05fb, 0x1e}, {0xd7ad80, 0xc0004d1500}) /home/rik/neofs-node/pkg/local_object_storage/shard/shard.go:138 +0x40 github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).reportFlushError(0xe6812a13a28266c3?, {0xbf05fb?, 0xb7e97c32721668a?}, {0xc00044c660?, 0x69f6f53b67fffa56?}, {0xd7ad80?, 0xc0004d1500?}) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:167 +0x25d github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).flushObject(0xc0004d62c0, 0xc0003a8420, {0x0, 0x0, 0x0}) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:272 +0x348 github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).flushWorker(0xc0004d62c0, 0xc0004d62c0?) /home/rik/neofs-node/pkg/local_object_storage/writecache/flush.go:241 +0x12d created by github.com/nspcc-dev/neofs-node/pkg/local_object_storage/writecache.(*cache).runFlushLoop Signed-off-by: Roman Khimov <[email protected]>

Writing small objects is a problem in that per-object overheads become comparable to the data write size. This is especially true for HDDs which prefer sequential writes as much as possible (but it's better for SSD block management as well). So for small objects we can try pushing them into a single file, but leaving hard links for each of them. This is what we have implemented here. The intention was to provide better results for 128K-1M range of objects since we already have Peapod for smaller ones and larger ones never were a problem. But in fact test results suggest that this approach works better than the original FSTree only in <128K range because at 256K it's slower. It at the same time outperforms Peapod in just about every case except 1B payloads which are purely synthetic (real objects have at least ~150 bytes overhead for various metadata). So the suggestion now it to replace Peapods with this mechanism eventually and simplify configuration (the only proper storage subsystem to configure). Caveats: * even though batching can be done in a more generic way we're heavily reliant on hard links, so it's at least Unix-only, but currently Linux is the primary target, so it's there only * decoder part is in the generic code since one can copy/paste (backup) shards to other systems and expect them to work * deletion is more fun now since the underlying file won't go away unless all hard links are deleted, this is expected to be OK for our use case * defaultTick is the same as for BoltDB for easy comparison * combinedSizeThresh was set to 8M for tests, but now lowered based on results * combinedSizeLimit/combinedCountLimit are a bit more empiric, but seem to be fine based on results Also tried: * O_DSYNC instead of fdatasync(), works fine for a single thread, but anything multithreaded slows down 10-20-fold, so not worth even an option * 32M batches packing more of 64K-1M objects, these just suck with 2-10x performance degradation * 2M batches packing less of 64K-1M objects, +30-80%% mostly with only one specific case improved by 16% (64K/20 threads) HDD results, original FSTree vs. combined: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 5 1600 Six-Core Processor │ hdd.fstree-original │ hdd.fstree-combined │ │ sec/op │ sec/op vs base │ Put/size=1,thread=1-12 32.16m ± 17% 42.07m ± 11% +30.83% (p=0.000 n=10) Put/size=1,thread=20-12 181.59m ± 21% 42.36m ± 6% -76.67% (p=0.000 n=10) Put/size=1,thread=100-12 261.72m ± 11% 52.95m ± 14% -79.77% (p=0.000 n=10) Put/size=1,thread=200-12 308.1m ± 13% 116.6m ± 9% -62.17% (p=0.000 n=10) Put/size=256,thread=1-12 39.63m ± 20% 45.50m ± 12% +14.83% (p=0.003 n=10) Put/size=256,thread=20-12 188.48m ± 17% 41.56m ± 7% -77.95% (p=0.000 n=10) Put/size=256,thread=100-12 254.06m ± 18% 51.91m ± 8% -79.57% (p=0.000 n=10) Put/size=256,thread=200-12 333.3m ± 18% 113.8m ± 8% -65.85% (p=0.000 n=10) Put/size=1024,thread=1-12 39.14m ± 20% 45.51m ± 9% +16.28% (p=0.000 n=10) Put/size=1024,thread=20-12 212.18m ± 14% 50.37m ± 15% -76.26% (p=0.000 n=10) Put/size=1024,thread=100-12 275.32m ± 9% 61.55m ± 13% -77.64% (p=0.000 n=10) Put/size=1024,thread=200-12 341.0m ± 18% 112.9m ± 18% -66.88% (p=0.000 n=10) Put/size=4096,thread=1-12 34.59m ± 15% 42.46m ± 17% +22.75% (p=0.000 n=10) Put/size=4096,thread=20-12 226.93m ± 14% 56.33m ± 10% -75.18% (p=0.000 n=10) Put/size=4096,thread=100-12 273.45m ± 8% 59.62m ± 9% -78.20% (p=0.000 n=10) Put/size=4096,thread=200-12 354.0m ± 16% 127.8m ± 9% -63.89% (p=0.000 n=10) Put/size=16384,thread=1-12 39.87m ± 3% 45.78m ± 19% +14.82% (p=0.000 n=10) Put/size=16384,thread=20-12 195.82m ± 30% 57.03m ± 8% -70.88% (p=0.000 n=10) Put/size=16384,thread=100-12 279.80m ± 15% 73.18m ± 5% -73.85% (p=0.000 n=10) Put/size=16384,thread=200-12 335.7m ± 20% 144.7m ± 11% -56.90% (p=0.000 n=10) Put/size=65536,thread=1-12 38.47m ± 8% 49.87m ± 7% +29.63% (p=0.000 n=10) Put/size=65536,thread=20-12 211.15m ± 12% 62.50m ± 8% -70.40% (p=0.000 n=10) Put/size=65536,thread=100-12 299.8m ± 14% 158.6m ± 7% -47.10% (p=0.000 n=10) Put/size=65536,thread=200-12 355.1m ± 117% 299.2m ± 8% -15.76% (p=0.000 n=10) Put/size=262144,thread=1-12 43.28m ± 20% 50.32m ± 11% +16.28% (p=0.001 n=10) Put/size=262144,thread=20-12 130.2m ± 11% 138.6m ± 9% ~ (p=0.123 n=10) Put/size=262144,thread=100-12 265.6m ± 18% 464.3m ± 11% +74.81% (p=0.000 n=10) Put/size=262144,thread=200-12 491.9m ± 10% 816.9m ± 12% +66.06% (p=0.000 n=10) Put/size=1048576,thread=1-12 42.59m ± 14% 50.79m ± 20% +19.25% (p=0.000 n=10) Put/size=1048576,thread=20-12 203.9m ± 9% 345.6m ± 14% +69.52% (p=0.000 n=10) Put/size=1048576,thread=100-12 855.1m ± 5% 1447.1m ± 4% +69.24% (p=0.000 n=10) Put/size=1048576,thread=200-12 1.650 ± 10% 2.726 ± 4% +65.23% (p=0.000 n=10) Put/size=4194304,thread=1-12 67.38m ± 12% 74.44m ± 9% +10.48% (p=0.035 n=10) Put/size=4194304,thread=20-12 745.9m ± 2% 1060.9m ± 2% +42.22% (p=0.000 n=10) Put/size=4194304,thread=100-12 3.794 ± 3% 4.952 ± 6% +30.52% (p=0.000 n=10) Put/size=4194304,thread=200-12 8.835 ± 30% 10.622 ± 9% ~ (p=0.123 n=10) geomean 218.6m 138.1m -36.84% │ hdd.fstree-original │ hdd.fstree-combined │ │ B/op │ B/op vs base │ Put/size=1,thread=1-12 2.430Ki ± 1% 2.812Ki ± 4% +15.71% (p=0.000 n=10) Put/size=1,thread=20-12 47.03Ki ± 7% 44.72Ki ± 1% -4.91% (p=0.000 n=10) Put/size=1,thread=100-12 224.7Ki ± 2% 206.1Ki ± 1% -8.28% (p=0.000 n=10) Put/size=1,thread=200-12 437.7Ki ± 2% 411.3Ki ± 1% -6.01% (p=0.000 n=10) Put/size=256,thread=1-12 2.433Ki ± 1% 2.813Ki ± 1% +15.63% (p=0.000 n=10) Put/size=256,thread=20-12 46.81Ki ± 1% 44.58Ki ± 1% -4.77% (p=0.000 n=10) Put/size=256,thread=100-12 226.5Ki ± 2% 207.1Ki ± 1% -8.55% (p=0.000 n=10) Put/size=256,thread=200-12 440.9Ki ± 2% 412.0Ki ± 2% -6.55% (p=0.000 n=10) Put/size=1024,thread=1-12 2.452Ki ± 1% 2.806Ki ± 2% +14.44% (p=0.000 n=10) Put/size=1024,thread=20-12 47.28Ki ± 1% 44.78Ki ± 1% -5.30% (p=0.000 n=10) Put/size=1024,thread=100-12 227.7Ki ± 1% 209.6Ki ± 0% -7.94% (p=0.000 n=10) Put/size=1024,thread=200-12 444.5Ki ± 1% 419.6Ki ± 1% -5.62% (p=0.000 n=10) Put/size=4096,thread=1-12 2.448Ki ± 1% 2.835Ki ± 0% +15.82% (p=0.000 n=10) Put/size=4096,thread=20-12 47.47Ki ± 1% 44.88Ki ± 0% -5.46% (p=0.000 n=10) Put/size=4096,thread=100-12 227.2Ki ± 3% 210.4Ki ± 0% -7.37% (p=0.000 n=10) Put/size=4096,thread=200-12 442.9Ki ± 4% 422.5Ki ± 1% -4.61% (p=0.000 n=10) Put/size=16384,thread=1-12 2.458Ki ± 1% 2.830Ki ± 2% +15.09% (p=0.000 n=10) Put/size=16384,thread=20-12 47.57Ki ± 1% 45.42Ki ± 1% -4.52% (p=0.000 n=10) Put/size=16384,thread=100-12 228.2Ki ± 2% 211.3Ki ± 2% -7.41% (p=0.000 n=10) Put/size=16384,thread=200-12 442.2Ki ± 4% 425.2Ki ± 1% -3.86% (p=0.000 n=10) Put/size=65536,thread=1-12 2.454Ki ± 2% 2.855Ki ± 3% +16.35% (p=0.000 n=10) Put/size=65536,thread=20-12 47.61Ki ± 1% 45.57Ki ± 1% -4.29% (p=0.000 n=10) Put/size=65536,thread=100-12 231.2Ki ± 2% 221.8Ki ± 1% -4.06% (p=0.000 n=10) Put/size=65536,thread=200-12 449.6Ki ± 2% 443.3Ki ± 1% -1.41% (p=0.000 n=10) Put/size=262144,thread=1-12 2.491Ki ± 2% 2.869Ki ± 2% +15.19% (p=0.000 n=10) Put/size=262144,thread=20-12 46.69Ki ± 1% 47.29Ki ± 1% +1.30% (p=0.000 n=10) Put/size=262144,thread=100-12 228.5Ki ± 1% 232.5Ki ± 1% +1.78% (p=0.002 n=10) Put/size=262144,thread=200-12 454.2Ki ± 2% 459.9Ki ± 1% ~ (p=0.052 n=10) Put/size=1048576,thread=1-12 2.506Ki ± 4% 2.911Ki ± 2% +16.17% (p=0.000 n=10) Put/size=1048576,thread=20-12 49.20Ki ± 5% 53.53Ki ± 6% +8.79% (p=0.000 n=10) Put/size=1048576,thread=100-12 245.1Ki ± 3% 249.3Ki ± 12% +1.72% (p=0.043 n=10) Put/size=1048576,thread=200-12 528.8Ki ± 5% 484.2Ki ± 1% -8.43% (p=0.002 n=10) Put/size=4194304,thread=1-12 2.582Ki ± 1% 2.913Ki ± 1% +12.82% (p=0.000 n=10) Put/size=4194304,thread=20-12 51.03Ki ± 1% 57.15Ki ± 1% +12.00% (p=0.000 n=10) Put/size=4194304,thread=100-12 248.3Ki ± 4% 259.1Ki ± 4% +4.35% (p=0.005 n=10) Put/size=4194304,thread=200-12 505.3Ki ± 2% 509.2Ki ± 1% ~ (p=0.165 n=10) geomean 59.58Ki 60.34Ki +1.27% │ hdd.fstree-original │ hdd.fstree-combined │ │ allocs/op │ allocs/op vs base │ Put/size=1,thread=1-12 27.00 ± 4% 33.50 ± 1% +24.07% (p=0.000 n=10) Put/size=1,thread=20-12 518.5 ± 1% 496.5 ± 1% -4.24% (p=0.000 n=10) Put/size=1,thread=100-12 2.537k ± 1% 2.284k ± 1% -9.97% (p=0.000 n=10) Put/size=1,thread=200-12 4.979k ± 1% 4.564k ± 1% -8.35% (p=0.000 n=10) Put/size=256,thread=1-12 27.00 ± 0% 33.50 ± 1% +24.07% (p=0.000 n=10) Put/size=256,thread=20-12 514.5 ± 1% 496.0 ± 1% -3.60% (p=0.000 n=10) Put/size=256,thread=100-12 2.547k ± 1% 2.289k ± 0% -10.13% (p=0.000 n=10) Put/size=256,thread=200-12 4.985k ± 2% 4.555k ± 2% -8.63% (p=0.000 n=10) Put/size=1024,thread=1-12 27.00 ± 4% 33.00 ± 3% +22.22% (p=0.000 n=10) Put/size=1024,thread=20-12 517.0 ± 1% 496.0 ± 1% -4.06% (p=0.000 n=10) Put/size=1024,thread=100-12 2.545k ± 0% 2.303k ± 1% -9.51% (p=0.000 n=10) Put/size=1024,thread=200-12 5.006k ± 1% 4.606k ± 1% -8.00% (p=0.000 n=10) Put/size=4096,thread=1-12 27.00 ± 0% 34.00 ± 0% +25.93% (p=0.000 n=10) Put/size=4096,thread=20-12 519.0 ± 1% 496.5 ± 1% -4.34% (p=0.000 n=10) Put/size=4096,thread=100-12 2.538k ± 3% 2.307k ± 0% -9.12% (p=0.000 n=10) Put/size=4096,thread=200-12 4.971k ± 4% 4.631k ± 1% -6.84% (p=0.000 n=10) Put/size=16384,thread=1-12 27.00 ± 4% 34.00 ± 3% +25.93% (p=0.000 n=10) Put/size=16384,thread=20-12 517.5 ± 1% 499.0 ± 1% -3.57% (p=0.000 n=10) Put/size=16384,thread=100-12 2.540k ± 2% 2.316k ± 2% -8.84% (p=0.000 n=10) Put/size=16384,thread=200-12 4.963k ± 3% 4.656k ± 1% -6.19% (p=0.000 n=10) Put/size=65536,thread=1-12 27.00 ± 4% 34.00 ± 3% +25.93% (p=0.000 n=10) Put/size=65536,thread=20-12 518.0 ± 1% 501.0 ± 1% -3.28% (p=0.000 n=10) Put/size=65536,thread=100-12 2.571k ± 2% 2.429k ± 1% -5.54% (p=0.000 n=10) Put/size=65536,thread=200-12 5.042k ± 2% 4.857k ± 0% -3.67% (p=0.000 n=10) Put/size=262144,thread=1-12 27.00 ± 4% 34.00 ± 3% +25.93% (p=0.000 n=10) Put/size=262144,thread=20-12 507.0 ± 1% 520.5 ± 1% +2.66% (p=0.000 n=10) Put/size=262144,thread=100-12 2.542k ± 1% 2.551k ± 1% ~ (p=0.171 n=10) Put/size=262144,thread=200-12 5.090k ± 2% 5.048k ± 1% ~ (p=0.105 n=10) Put/size=1048576,thread=1-12 28.00 ± 4% 34.00 ± 0% +21.43% (p=0.000 n=10) Put/size=1048576,thread=20-12 522.5 ± 1% 570.0 ± 2% +9.09% (p=0.000 n=10) Put/size=1048576,thread=100-12 2.701k ± 2% 2.754k ± 2% +1.96% (p=0.001 n=10) Put/size=1048576,thread=200-12 5.814k ± 4% 5.340k ± 1% -8.16% (p=0.001 n=10) Put/size=4194304,thread=1-12 28.50 ± 2% 34.00 ± 0% +19.30% (p=0.000 n=10) Put/size=4194304,thread=20-12 554.0 ± 1% 644.5 ± 1% +16.34% (p=0.000 n=10) Put/size=4194304,thread=100-12 2.755k ± 4% 2.938k ± 2% +6.62% (p=0.000 n=10) Put/size=4194304,thread=200-12 5.628k ± 2% 5.781k ± 0% +2.73% (p=0.002 n=10) geomean 659.6 677.9 +2.77% HDD results, Peapod vs. combined FSTree: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 5 1600 Six-Core Processor │ hdd.peapod │ hdd.fstree-combined │ │ sec/op │ sec/op vs base │ Put/size=1,thread=1-12 37.24m ± 13% 42.07m ± 11% +12.98% (p=0.015 n=10) Put/size=1,thread=20-12 81.66m ± 10% 42.36m ± 6% -48.12% (p=0.000 n=10) Put/size=1,thread=100-12 85.56m ± 16% 52.95m ± 14% -38.12% (p=0.000 n=10) Put/size=1,thread=200-12 85.10m ± 10% 116.56m ± 9% +36.96% (p=0.000 n=10) Put/size=256,thread=1-12 51.45m ± 9% 45.50m ± 12% -11.55% (p=0.001 n=10) Put/size=256,thread=20-12 87.50m ± 12% 41.56m ± 7% -52.51% (p=0.000 n=10) Put/size=256,thread=100-12 100.30m ± 12% 51.91m ± 8% -48.24% (p=0.000 n=10) Put/size=256,thread=200-12 108.0m ± 11% 113.8m ± 8% ~ (p=0.315 n=10) Put/size=1024,thread=1-12 63.41m ± 11% 45.51m ± 9% -28.23% (p=0.000 n=10) Put/size=1024,thread=20-12 88.01m ± 9% 50.37m ± 15% -42.77% (p=0.000 n=10) Put/size=1024,thread=100-12 107.36m ± 9% 61.55m ± 13% -42.67% (p=0.000 n=10) Put/size=1024,thread=200-12 120.9m ± 11% 112.9m ± 18% ~ (p=0.165 n=10) Put/size=4096,thread=1-12 70.93m ± 12% 42.46m ± 17% -40.14% (p=0.000 n=10) Put/size=4096,thread=20-12 100.68m ± 15% 56.33m ± 10% -44.06% (p=0.000 n=10) Put/size=4096,thread=100-12 129.19m ± 7% 59.62m ± 9% -53.85% (p=0.000 n=10) Put/size=4096,thread=200-12 158.4m ± 2% 127.8m ± 9% -19.28% (p=0.000 n=10) Put/size=16384,thread=1-12 74.06m ± 18% 45.78m ± 19% -38.19% (p=0.000 n=10) Put/size=16384,thread=20-12 117.52m ± 9% 57.03m ± 8% -51.47% (p=0.000 n=10) Put/size=16384,thread=100-12 188.78m ± 9% 73.18m ± 5% -61.24% (p=0.000 n=10) Put/size=16384,thread=200-12 242.2m ± 13% 144.7m ± 11% -40.27% (p=0.000 n=10) Put/size=65536,thread=1-12 83.27m ± 5% 49.87m ± 7% -40.10% (p=0.000 n=10) Put/size=65536,thread=20-12 164.54m ± 12% 62.50m ± 8% -62.01% (p=0.000 n=10) Put/size=65536,thread=100-12 342.2m ± 11% 158.6m ± 7% -53.66% (p=0.000 n=10) Put/size=65536,thread=200-12 392.7m ± 14% 299.2m ± 8% -23.81% (p=0.000 n=10) Put/size=262144,thread=1-12 79.08m ± 19% 50.32m ± 11% -36.36% (p=0.000 n=10) Put/size=262144,thread=20-12 226.6m ± 11% 138.6m ± 9% -38.83% (p=0.000 n=10) Put/size=262144,thread=100-12 492.3m ± 5% 464.3m ± 11% -5.68% (p=0.015 n=10) Put/size=262144,thread=200-12 707.5m ± 6% 816.9m ± 12% +15.45% (p=0.000 n=10) Put/size=1048576,thread=1-12 50.79m ± 20% Put/size=1048576,thread=20-12 345.6m ± 14% Put/size=1048576,thread=100-12 1.447 ± 4% Put/size=1048576,thread=200-12 2.726 ± 4% Put/size=4194304,thread=1-12 74.44m ± 9% Put/size=4194304,thread=20-12 1.061 ± 2% Put/size=4194304,thread=100-12 4.952 ± 6% Put/size=4194304,thread=200-12 10.62 ± 9% geomean 125.0m 138.1m -34.43% │ hdd.peapod │ hdd.fstree-combined │ │ B/op │ B/op vs base │ Put/size=1,thread=1-12 11.890Ki ± 1% 2.812Ki ± 4% -76.35% (p=0.000 n=10) Put/size=1,thread=20-12 51.05Ki ± 22% 44.72Ki ± 1% ~ (p=0.143 n=10) Put/size=1,thread=100-12 213.6Ki ± 3% 206.1Ki ± 1% -3.52% (p=0.004 n=10) Put/size=1,thread=200-12 440.7Ki ± 13% 411.3Ki ± 1% ~ (p=0.143 n=10) Put/size=256,thread=1-12 9.207Ki ± 1% 2.813Ki ± 1% -69.44% (p=0.000 n=10) Put/size=256,thread=20-12 43.71Ki ± 13% 44.58Ki ± 1% ~ (p=0.481 n=10) Put/size=256,thread=100-12 239.7Ki ± 8% 207.1Ki ± 1% -13.62% (p=0.000 n=10) Put/size=256,thread=200-12 450.8Ki ± 4% 412.0Ki ± 2% -8.60% (p=0.002 n=10) Put/size=1024,thread=1-12 12.169Ki ± 2% 2.806Ki ± 2% -76.94% (p=0.000 n=10) Put/size=1024,thread=20-12 114.50Ki ± 4% 44.78Ki ± 1% -60.89% (p=0.000 n=10) Put/size=1024,thread=100-12 570.2Ki ± 3% 209.6Ki ± 0% -63.25% (p=0.000 n=10) Put/size=1024,thread=200-12 1088.3Ki ± 6% 419.6Ki ± 1% -61.45% (p=0.000 n=10) Put/size=4096,thread=1-12 32.011Ki ± 4% 2.835Ki ± 0% -91.14% (p=0.000 n=10) Put/size=4096,thread=20-12 463.28Ki ± 6% 44.88Ki ± 0% -90.31% (p=0.000 n=10) Put/size=4096,thread=100-12 2155.6Ki ± 3% 210.4Ki ± 0% -90.24% (p=0.000 n=10) Put/size=4096,thread=200-12 4317.9Ki ± 2% 422.5Ki ± 1% -90.21% (p=0.000 n=10) Put/size=16384,thread=1-12 90.904Ki ± 2% 2.830Ki ± 2% -96.89% (p=0.000 n=10) Put/size=16384,thread=20-12 1528.41Ki ± 5% 45.42Ki ± 1% -97.03% (p=0.000 n=10) Put/size=16384,thread=100-12 7272.0Ki ± 2% 211.3Ki ± 2% -97.09% (p=0.000 n=10) Put/size=16384,thread=200-12 14619.0Ki ± 4% 425.2Ki ± 1% -97.09% (p=0.000 n=10) Put/size=65536,thread=1-12 327.574Ki ± 2% 2.855Ki ± 3% -99.13% (p=0.000 n=10) Put/size=65536,thread=20-12 5368.91Ki ± 4% 45.57Ki ± 1% -99.15% (p=0.000 n=10) Put/size=65536,thread=100-12 27878.2Ki ± 3% 221.8Ki ± 1% -99.20% (p=0.000 n=10) Put/size=65536,thread=200-12 61247.9Ki ± 4% 443.3Ki ± 1% -99.28% (p=0.000 n=10) Put/size=262144,thread=1-12 1244.865Ki ± 1% 2.869Ki ± 2% -99.77% (p=0.000 n=10) Put/size=262144,thread=20-12 20783.90Ki ± 8% 47.29Ki ± 1% -99.77% (p=0.000 n=10) Put/size=262144,thread=100-12 125171.4Ki ± 1% 232.5Ki ± 1% -99.81% (p=0.000 n=10) Put/size=262144,thread=200-12 319065.4Ki ± 1% 459.9Ki ± 1% -99.86% (p=0.000 n=10) Put/size=1048576,thread=1-12 2.911Ki ± 2% Put/size=1048576,thread=20-12 53.53Ki ± 6% Put/size=1048576,thread=100-12 249.3Ki ± 12% Put/size=1048576,thread=200-12 484.2Ki ± 1% Put/size=4194304,thread=1-12 2.913Ki ± 1% Put/size=4194304,thread=20-12 57.15Ki ± 1% Put/size=4194304,thread=100-12 259.1Ki ± 4% Put/size=4194304,thread=200-12 509.2Ki ± 1% geomean 857.7Ki 60.34Ki -93.18% │ hdd.peapod │ hdd.fstree-combined │ │ allocs/op │ allocs/op vs base │ Put/size=1,thread=1-12 51.00 ± 2% 33.50 ± 1% -34.31% (p=0.000 n=10) Put/size=1,thread=20-12 188.0 ± 4% 496.5 ± 1% +164.10% (p=0.000 n=10) Put/size=1,thread=100-12 942.5 ± 17% 2284.0 ± 1% +142.33% (p=0.000 n=10) Put/size=1,thread=200-12 2.150k ± 10% 4.564k ± 1% +112.26% (p=0.000 n=10) Put/size=256,thread=1-12 57.00 ± 0% 33.50 ± 1% -41.23% (p=0.000 n=10) Put/size=256,thread=20-12 233.5 ± 12% 496.0 ± 1% +112.42% (p=0.000 n=10) Put/size=256,thread=100-12 1.278k ± 8% 2.289k ± 0% +79.11% (p=0.000 n=10) Put/size=256,thread=200-12 2.565k ± 6% 4.555k ± 2% +77.58% (p=0.000 n=10) Put/size=1024,thread=1-12 61.00 ± 0% 33.00 ± 3% -45.90% (p=0.000 n=10) Put/size=1024,thread=20-12 331.0 ± 4% 496.0 ± 1% +49.85% (p=0.000 n=10) Put/size=1024,thread=100-12 1.823k ± 4% 2.303k ± 1% +26.30% (p=0.000 n=10) Put/size=1024,thread=200-12 3.783k ± 2% 4.606k ± 1% +21.77% (p=0.000 n=10) Put/size=4096,thread=1-12 63.00 ± 2% 34.00 ± 0% -46.03% (p=0.000 n=10) Put/size=4096,thread=20-12 369.0 ± 5% 496.5 ± 1% +34.55% (p=0.000 n=10) Put/size=4096,thread=100-12 1.819k ± 6% 2.307k ± 0% +26.83% (p=0.000 n=10) Put/size=4096,thread=200-12 3.825k ± 4% 4.631k ± 1% +21.07% (p=0.000 n=10) Put/size=16384,thread=1-12 66.00 ± 2% 34.00 ± 3% -48.48% (p=0.000 n=10) Put/size=16384,thread=20-12 419.0 ± 4% 499.0 ± 1% +19.09% (p=0.000 n=10) Put/size=16384,thread=100-12 1.954k ± 1% 2.316k ± 2% +18.47% (p=0.000 n=10) Put/size=16384,thread=200-12 3.827k ± 3% 4.656k ± 1% +21.66% (p=0.000 n=10) Put/size=65536,thread=1-12 73.00 ± 1% 34.00 ± 3% -53.42% (p=0.000 n=10) Put/size=65536,thread=20-12 452.0 ± 4% 501.0 ± 1% +10.84% (p=0.000 n=10) Put/size=65536,thread=100-12 2.035k ± 2% 2.429k ± 1% +19.36% (p=0.000 n=10) Put/size=65536,thread=200-12 3.904k ± 1% 4.857k ± 0% +24.43% (p=0.000 n=10) Put/size=262144,thread=1-12 84.00 ± 1% 34.00 ± 3% -59.52% (p=0.000 n=10) Put/size=262144,thread=20-12 459.0 ± 4% 520.5 ± 1% +13.40% (p=0.000 n=10) Put/size=262144,thread=100-12 1.964k ± 2% 2.551k ± 1% +29.91% (p=0.000 n=10) Put/size=262144,thread=200-12 4.268k ± 2% 5.048k ± 1% +18.29% (p=0.000 n=10) Put/size=1048576,thread=1-12 34.00 ± 0% Put/size=1048576,thread=20-12 570.0 ± 2% Put/size=1048576,thread=100-12 2.754k ± 2% Put/size=1048576,thread=200-12 5.340k ± 1% Put/size=4194304,thread=1-12 34.00 ± 0% Put/size=4194304,thread=20-12 644.5 ± 1% Put/size=4194304,thread=100-12 2.938k ± 2% Put/size=4194304,thread=200-12 5.781k ± 0% geomean 587.1 677.9 +11.94% SSD results, Peapod vs. combined FSTree: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics │ ssd.peapod │ ssd.fstree-combined │ │ sec/op │ sec/op vs base │ Put/size=1,thread=1-16 10.65m ± 1% 14.04m ± 1% +31.77% (p=0.000 n=10) Put/size=1,thread=20-16 10.59m ± 0% 15.09m ± 2% +42.49% (p=0.000 n=10) Put/size=1,thread=100-16 10.46m ± 1% 17.31m ± 3% +65.42% (p=0.000 n=10) Put/size=1,thread=200-16 11.62m ± 85% 33.77m ± 2% +190.61% (p=0.000 n=10) Put/size=256,thread=1-16 10.95m ± 20% 14.02m ± 1% +27.97% (p=0.000 n=10) Put/size=256,thread=20-16 11.72m ± 6% 15.58m ± 2% +32.99% (p=0.000 n=10) Put/size=256,thread=100-16 22.24m ± 12% 16.87m ± 6% -24.15% (p=0.000 n=10) Put/size=256,thread=200-16 26.05m ± 2% 29.55m ± 2% +13.44% (p=0.000 n=10) Put/size=1024,thread=1-16 10.72m ± 10% 12.30m ± 0% +14.76% (p=0.000 n=10) Put/size=1024,thread=20-16 19.92m ± 47% 13.86m ± 2% ~ (p=0.481 n=10) Put/size=1024,thread=100-16 26.05m ± 1% 15.69m ± 2% -39.75% (p=0.000 n=10) Put/size=1024,thread=200-16 28.87m ± 2% 30.10m ± 1% +4.28% (p=0.000 n=10) Put/size=4096,thread=1-16 12.49m ± 20% 12.86m ± 1% ~ (p=0.481 n=10) Put/size=4096,thread=20-16 21.47m ± 4% 14.14m ± 2% -34.15% (p=0.000 n=10) Put/size=4096,thread=100-16 19.41m ± 42% 16.10m ± 2% -17.07% (p=0.000 n=10) Put/size=4096,thread=200-16 33.15m ± 1% 31.72m ± 2% -4.30% (p=0.002 n=10) Put/size=16384,thread=1-16 12.07m ± 17% 12.89m ± 1% ~ (p=0.143 n=10) Put/size=16384,thread=20-16 24.64m ± 2% 14.59m ± 1% -40.78% (p=0.000 n=10) Put/size=16384,thread=100-16 33.66m ± 3% 17.60m ± 2% -47.72% (p=0.000 n=10) Put/size=16384,thread=200-16 43.27m ± 2% 35.42m ± 1% -18.15% (p=0.000 n=10) Put/size=65536,thread=1-16 10.84m ± 84% 13.11m ± 1% ~ (p=0.353 n=10) Put/size=65536,thread=20-16 28.73m ± 3% 16.55m ± 2% -42.39% (p=0.000 n=10) Put/size=65536,thread=100-16 50.44m ± 3% 23.58m ± 7% -53.25% (p=0.000 n=10) Put/size=65536,thread=200-16 73.42m ± 4% 42.20m ± 2% -42.51% (p=0.000 n=10) Put/size=262144,thread=1-16 23.70m ± 5% 13.67m ± 4% -42.30% (p=0.000 n=10) Put/size=262144,thread=20-16 39.23m ± 3% 18.69m ± 4% -52.36% (p=0.000 n=10) Put/size=262144,thread=100-16 112.04m ± 5% 68.14m ± 2% -39.19% (p=0.000 n=10) Put/size=262144,thread=200-16 192.9m ± 7% 113.5m ± 23% -41.17% (p=0.000 n=10) Put/size=1048576,thread=1-16 17.17m ± 1% Put/size=1048576,thread=20-16 40.61m ± 18% Put/size=1048576,thread=100-16 197.9m ± 3% Put/size=1048576,thread=200-16 353.1m ± 4% Put/size=4194304,thread=1-16 20.74m ± 1% Put/size=4194304,thread=20-16 143.2m ± 9% Put/size=4194304,thread=100-16 700.5m ± 4% Put/size=4194304,thread=200-16 1.356 ± 10% geomean 23.62m 31.30m -13.41% │ ssd.peapod │ ssd.fstree-combined │ │ B/op │ B/op vs base │ Put/size=1,thread=1-16 13.366Ki ± 0% 2.678Ki ± 1% -79.96% (p=0.000 n=10) Put/size=1,thread=20-16 119.02Ki ± 2% 42.40Ki ± 1% -64.38% (p=0.000 n=10) Put/size=1,thread=100-16 580.7Ki ± 1% 194.3Ki ± 0% -66.54% (p=0.000 n=10) Put/size=1,thread=200-16 1153.1Ki ± 27% 389.5Ki ± 0% -66.22% (p=0.000 n=10) Put/size=256,thread=1-16 9.384Ki ± 1% 2.703Ki ± 0% -71.20% (p=0.000 n=10) Put/size=256,thread=20-16 72.47Ki ± 3% 42.44Ki ± 0% -41.45% (p=0.000 n=10) Put/size=256,thread=100-16 305.2Ki ± 2% 195.0Ki ± 1% -36.11% (p=0.000 n=10) Put/size=256,thread=200-16 577.8Ki ± 1% 389.8Ki ± 1% -32.54% (p=0.000 n=10) Put/size=1024,thread=1-16 13.377Ki ± 1% 2.699Ki ± 0% -79.83% (p=0.000 n=10) Put/size=1024,thread=20-16 147.66Ki ± 8% 42.43Ki ± 1% -71.26% (p=0.000 n=10) Put/size=1024,thread=100-16 639.4Ki ± 1% 196.5Ki ± 1% -69.27% (p=0.000 n=10) Put/size=1024,thread=200-16 1250.3Ki ± 1% 392.4Ki ± 1% -68.61% (p=0.000 n=10) Put/size=4096,thread=1-16 32.826Ki ± 2% 2.696Ki ± 1% -91.79% (p=0.000 n=10) Put/size=4096,thread=20-16 484.87Ki ± 2% 42.28Ki ± 1% -91.28% (p=0.000 n=10) Put/size=4096,thread=100-16 2416.2Ki ± 3% 196.8Ki ± 0% -91.85% (p=0.000 n=10) Put/size=4096,thread=200-16 4659.8Ki ± 1% 392.8Ki ± 0% -91.57% (p=0.000 n=10) Put/size=16384,thread=1-16 90.928Ki ± 1% 2.707Ki ± 0% -97.02% (p=0.000 n=10) Put/size=16384,thread=20-16 1539.51Ki ± 1% 42.83Ki ± 1% -97.22% (p=0.000 n=10) Put/size=16384,thread=100-16 7535.5Ki ± 1% 198.0Ki ± 0% -97.37% (p=0.000 n=10) Put/size=16384,thread=200-16 14926.7Ki ± 1% 395.3Ki ± 0% -97.35% (p=0.000 n=10) Put/size=65536,thread=1-16 302.934Ki ± 1% 2.715Ki ± 0% -99.10% (p=0.000 n=10) Put/size=65536,thread=20-16 5398.51Ki ± 1% 43.22Ki ± 0% -99.20% (p=0.000 n=10) Put/size=65536,thread=100-16 26256.7Ki ± 1% 200.1Ki ± 1% -99.24% (p=0.000 n=10) Put/size=65536,thread=200-16 52366.5Ki ± 1% 400.0Ki ± 0% -99.24% (p=0.000 n=10) Put/size=262144,thread=1-16 1169.852Ki ± 2% 2.725Ki ± 1% -99.77% (p=0.000 n=10) Put/size=262144,thread=20-16 20540.44Ki ± 2% 43.49Ki ± 0% -99.79% (p=0.000 n=10) Put/size=262144,thread=100-16 99619.5Ki ± 2% 214.4Ki ± 1% -99.78% (p=0.000 n=10) Put/size=262144,thread=200-16 207670.7Ki ± 3% 422.7Ki ± 2% -99.80% (p=0.000 n=10) Put/size=1048576,thread=1-16 2.770Ki ± 1% Put/size=1048576,thread=20-16 46.23Ki ± 5% Put/size=1048576,thread=100-16 229.8Ki ± 2% Put/size=1048576,thread=200-16 460.2Ki ± 1% Put/size=4194304,thread=1-16 2.771Ki ± 1% Put/size=4194304,thread=20-16 50.57Ki ± 1% Put/size=4194304,thread=100-16 257.8Ki ± 3% Put/size=4194304,thread=200-16 509.5Ki ± 11% geomean 984.0Ki 56.68Ki -94.41% │ ssd.peapod │ ssd.fstree-combined │ │ allocs/op │ allocs/op vs base │ Put/size=1,thread=1-16 55.00 ± 0% 32.00 ± 3% -41.82% (p=0.000 n=10) Put/size=1,thread=20-16 241.0 ± 1% 473.5 ± 1% +96.47% (p=0.000 n=10) Put/size=1,thread=100-16 1.214k ± 3% 2.167k ± 0% +78.39% (p=0.000 n=10) Put/size=1,thread=200-16 2.806k ± 20% 4.338k ± 0% +54.62% (p=0.000 n=10) Put/size=256,thread=1-16 58.00 ± 0% 32.00 ± 0% -44.83% (p=0.000 n=10) Put/size=256,thread=20-16 312.5 ± 3% 473.5 ± 0% +51.52% (p=0.000 n=10) Put/size=256,thread=100-16 1.607k ± 3% 2.163k ± 0% +34.57% (p=0.000 n=10) Put/size=256,thread=200-16 3.374k ± 2% 4.319k ± 0% +28.01% (p=0.000 n=10) Put/size=1024,thread=1-16 61.00 ± 0% 32.00 ± 0% -47.54% (p=0.000 n=10) Put/size=1024,thread=20-16 495.5 ± 14% 470.0 ± 1% ~ (p=0.467 n=10) Put/size=1024,thread=100-16 2.308k ± 2% 2.160k ± 0% -6.41% (p=0.000 n=10) Put/size=1024,thread=200-16 4.651k ± 3% 4.319k ± 0% -7.12% (p=0.000 n=10) Put/size=4096,thread=1-16 64.00 ± 0% 32.00 ± 0% -50.00% (p=0.000 n=10) Put/size=4096,thread=20-16 463.0 ± 3% 469.0 ± 0% +1.30% (p=0.004 n=10) Put/size=4096,thread=100-16 2.469k ± 7% 2.162k ± 0% -12.42% (p=0.000 n=10) Put/size=4096,thread=200-16 4.593k ± 1% 4.319k ± 0% -5.98% (p=0.000 n=10) Put/size=16384,thread=1-16 68.00 ± 1% 32.00 ± 0% -52.94% (p=0.000 n=10) Put/size=16384,thread=20-16 497.5 ± 1% 471.0 ± 0% -5.33% (p=0.000 n=10) Put/size=16384,thread=100-16 2.301k ± 1% 2.173k ± 0% -5.58% (p=0.000 n=10) Put/size=16384,thread=200-16 4.476k ± 2% 4.340k ± 0% -3.04% (p=0.000 n=10) Put/size=65536,thread=1-16 77.00 ± 1% 32.00 ± 0% -58.44% (p=0.000 n=10) Put/size=65536,thread=20-16 514.5 ± 1% 476.5 ± 1% -7.39% (p=0.000 n=10) Put/size=65536,thread=100-16 2.261k ± 2% 2.197k ± 1% -2.81% (p=0.000 n=10) Put/size=65536,thread=200-16 4.367k ± 2% 4.389k ± 0% ~ (p=0.342 n=10) Put/size=262144,thread=1-16 86.00 ± 1% 32.00 ± 0% -62.79% (p=0.000 n=10) Put/size=262144,thread=20-16 556.5 ± 4% 478.5 ± 0% -14.02% (p=0.000 n=10) Put/size=262144,thread=100-16 2.231k ± 2% 2.354k ± 1% +5.54% (p=0.000 n=10) Put/size=262144,thread=200-16 4.123k ± 2% 4.641k ± 2% +12.54% (p=0.000 n=10) Put/size=1048576,thread=1-16 33.00 ± 3% Put/size=1048576,thread=20-16 509.5 ± 2% Put/size=1048576,thread=100-16 2.535k ± 1% Put/size=1048576,thread=200-16 5.062k ± 1% Put/size=4194304,thread=1-16 33.00 ± 0% Put/size=4194304,thread=20-16 573.0 ± 1% Put/size=4194304,thread=100-16 2.914k ± 3% Put/size=4194304,thread=200-16 5.767k ± 10% geomean 688.6 638.2 -10.30% Signed-off-by: Roman Khimov <[email protected]>

Open() (and ReadFile()) will return the same exact error, we don't need to Stat(). Signed-off-by: Roman Khimov <[email protected]>

We use ReadFile() for Get and it works just fine there, no reasons for it to not work here as well. Large files can overflow us anyway (but we're not supposed to have them in a proper storage). Signed-off-by: Roman Khimov <[email protected]>

IsCompressed() is handled already internally in Decompress(), duplicating it doesn't make our code better. Signed-off-by: Roman Khimov <[email protected]>

Signed-off-by: Roman Khimov <[email protected]>

Peapod has no depth/width, FSTree has no width/size/opened_cache_capacity. Signed-off-by: Roman Khimov <[email protected]>

Signed-off-by: Roman Khimov <[email protected]>

roman-khimov · 2024-08-27T14:36:08Z

Ironically, it doesn't have any conflicts with current master even without a rebase. So my proposal is to merge this as is and then deal with peapod. @cthulhu-rider, @carpawell?

carpawell · 2024-08-27T15:47:17Z

Have you considered GET bench? Isn't it slower to read a combined object for only a part of it if there are a few parallel GET requests? Like reading 8MB for a 4KB object 128 times can be slower than reading 4KB 128 times from a peadod.

@roman-khimov, my questions are the same. I am afraid there (at least this PR) is not enough info about if it is really a better one than we had before, from any perspective (i consider it a worse solution for GETs). If there is a single slower thing in results, we should discuss it and find out (at least write it somewhere) what is better for us.

roman-khimov · 2024-08-27T16:10:37Z

But you've seen the test results. GETs are the same, PUTs are ~15% better for the target cases. GETs do incur some additional overhead, but it's totally acceptable, there is no real difference in practice.

We don't need to combine for SSDs: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics │ ssd.fstree-original │ ssd.fstree-combined │ │ sec/op │ sec/op vs base │ Put/size=1,thread=1-16 91.98µ ± 2% 14040.16µ ± 1% +15164.94% (p=0.000 n=10) Put/size=1,thread=20-16 1.010m ± 1% 15.095m ± 2% +1393.89% (p=0.000 n=10) Put/size=1,thread=100-16 5.272m ± 1% 17.306m ± 3% +228.27% (p=0.000 n=10) Put/size=1,thread=200-16 10.77m ± 1% 33.77m ± 2% +213.52% (p=0.000 n=10) Put/size=256,thread=1-16 91.37µ ± 2% 14018.07µ ± 1% +15242.09% (p=0.000 n=10) Put/size=256,thread=20-16 1.028m ± 4% 15.583m ± 2% +1415.81% (p=0.000 n=10) Put/size=256,thread=100-16 5.420m ± 1% 16.869m ± 6% +211.22% (p=0.000 n=10) Put/size=256,thread=200-16 12.42m ± 11% 29.55m ± 2% +137.89% (p=0.000 n=10) Put/size=1024,thread=1-16 85.38µ ± 7% 12301.06µ ± 0% +14306.75% (p=0.000 n=10) Put/size=1024,thread=20-16 1.053m ± 4% 13.855m ± 2% +1216.07% (p=0.000 n=10) Put/size=1024,thread=100-16 5.629m ± 1% 15.694m ± 2% +178.82% (p=0.000 n=10) Put/size=1024,thread=200-16 12.55m ± 9% 30.10m ± 1% +139.83% (p=0.000 n=10) Put/size=4096,thread=1-16 150.6µ ± 6% 12864.7µ ± 1% +8441.32% (p=0.000 n=10) Put/size=4096,thread=20-16 1.308m ± 1% 14.136m ± 2% +980.81% (p=0.000 n=10) Put/size=4096,thread=100-16 6.670m ± 2% 16.101m ± 2% +141.40% (p=0.000 n=10) Put/size=4096,thread=200-16 13.45m ± 1% 31.72m ± 2% +135.88% (p=0.000 n=10) Put/size=16384,thread=1-16 181.7µ ± 5% 12891.6µ ± 1% +6994.13% (p=0.000 n=18+10) Put/size=16384,thread=20-16 1.131m ± 4% 14.592m ± 1% +1189.98% (p=0.000 n=10) Put/size=16384,thread=100-16 6.060m ± 31% 17.599m ± 2% +190.41% (p=0.000 n=10) Put/size=16384,thread=200-16 12.38m ± 2% 35.42m ± 1% +185.99% (p=0.000 n=10) Put/size=65536,thread=1-16 282.8µ ± 5% 13109.3µ ± 1% +4535.47% (p=0.000 n=10) Put/size=65536,thread=20-16 1.599m ± 10% 16.549m ± 2% +935.19% (p=0.000 n=10) Put/size=65536,thread=100-16 8.472m ± 4% 23.581m ± 7% +178.34% (p=0.000 n=10) Put/size=65536,thread=200-16 16.60m ± 2% 42.20m ± 2% +154.30% (p=0.000 n=10) Put/size=262144,thread=1-16 577.9µ ± 4% 13673.9µ ± 4% +2266.11% (p=0.000 n=10) Put/size=262144,thread=20-16 3.961m ± 30% 18.688m ± 4% +371.76% (p=0.000 n=10) Put/size=262144,thread=100-16 20.09m ± 36% 68.14m ± 2% +239.16% (p=0.000 n=10) Put/size=262144,thread=200-16 38.53m ± 33% 113.45m ± 23% +194.48% (p=0.000 n=10) Put/size=1048576,thread=1-16 1.766m ± 5% 17.167m ± 1% +871.86% (p=0.000 n=10) Put/size=1048576,thread=20-16 15.71m ± 21% 40.61m ± 18% +158.54% (p=0.000 n=10) Put/size=1048576,thread=100-16 61.86m ± 25% 197.93m ± 3% +219.98% (p=0.000 n=10) Put/size=1048576,thread=200-16 123.7m ± 18% 353.1m ± 4% +185.47% (p=0.000 n=10) Put/size=4194304,thread=1-16 5.485m ± 2% 20.737m ± 1% +278.07% (p=0.000 n=10) Put/size=4194304,thread=20-16 51.55m ± 16% 143.20m ± 9% +177.79% (p=0.000 n=10) Put/size=4194304,thread=100-16 260.4m ± 20% 700.5m ± 4% +169.04% (p=0.000 n=10) Put/size=4194304,thread=200-16 521.9m ± 11% 1356.2m ± 10% +159.86% (p=0.000 n=10) geomean 4.278m 31.30m +631.70% │ ssd.fstree-original │ ssd.fstree-combined │ │ B/op │ B/op vs base │ Put/size=1,thread=1-16 1.904Ki ± 0% 2.678Ki ± 1% +40.68% (p=0.000 n=10) Put/size=1,thread=20-16 36.94Ki ± 0% 42.40Ki ± 1% +14.76% (p=0.000 n=10) Put/size=1,thread=100-16 186.9Ki ± 0% 194.3Ki ± 0% +3.99% (p=0.000 n=10) Put/size=1,thread=200-16 370.9Ki ± 1% 389.5Ki ± 0% +5.01% (p=0.000 n=10) Put/size=256,thread=1-16 1.917Ki ± 0% 2.703Ki ± 0% +40.98% (p=0.000 n=10) Put/size=256,thread=20-16 37.51Ki ± 0% 42.44Ki ± 0% +13.14% (p=0.000 n=10) Put/size=256,thread=100-16 188.3Ki ± 1% 195.0Ki ± 1% +3.53% (p=0.000 n=10) Put/size=256,thread=200-16 378.5Ki ± 1% 389.8Ki ± 1% +3.00% (p=0.000 n=10) Put/size=1024,thread=1-16 1.914Ki ± 0% 2.699Ki ± 0% +40.99% (p=0.000 n=10) Put/size=1024,thread=20-16 37.57Ki ± 1% 42.43Ki ± 1% +12.95% (p=0.000 n=10) Put/size=1024,thread=100-16 190.3Ki ± 1% 196.5Ki ± 1% +3.28% (p=0.000 n=10) Put/size=1024,thread=200-16 380.9Ki ± 0% 392.4Ki ± 1% +3.02% (p=0.000 n=10) Put/size=4096,thread=1-16 1.952Ki ± 0% 2.696Ki ± 1% +38.12% (p=0.000 n=10) Put/size=4096,thread=20-16 38.12Ki ± 1% 42.28Ki ± 1% +10.91% (p=0.000 n=10) Put/size=4096,thread=100-16 191.5Ki ± 1% 196.8Ki ± 0% +2.81% (p=0.000 n=10) Put/size=4096,thread=200-16 382.1Ki ± 1% 392.8Ki ± 0% +2.80% (p=0.000 n=10) Put/size=16384,thread=1-16 1.977Ki ± 1% 2.707Ki ± 0% +36.90% (p=0.000 n=18+10) Put/size=16384,thread=20-16 38.00Ki ± 0% 42.83Ki ± 1% +12.71% (p=0.000 n=10) Put/size=16384,thread=100-16 191.1Ki ± 0% 198.0Ki ± 0% +3.63% (p=0.000 n=10) Put/size=16384,thread=200-16 382.2Ki ± 0% 395.3Ki ± 0% +3.43% (p=0.000 n=10) Put/size=65536,thread=1-16 2.020Ki ± 1% 2.715Ki ± 0% +34.40% (p=0.000 n=10) Put/size=65536,thread=20-16 38.42Ki ± 1% 43.22Ki ± 0% +12.51% (p=0.000 n=10) Put/size=65536,thread=100-16 193.1Ki ± 0% 200.1Ki ± 1% +3.64% (p=0.000 n=10) Put/size=65536,thread=200-16 386.1Ki ± 0% 400.0Ki ± 0% +3.61% (p=0.000 n=10) Put/size=262144,thread=1-16 2.119Ki ± 0% 2.725Ki ± 1% +28.57% (p=0.000 n=10) Put/size=262144,thread=20-16 39.47Ki ± 1% 43.49Ki ± 0% +10.18% (p=0.000 n=10) Put/size=262144,thread=100-16 197.9Ki ± 1% 214.4Ki ± 1% +8.31% (p=0.000 n=10) Put/size=262144,thread=200-16 395.9Ki ± 1% 422.7Ki ± 2% +6.77% (p=0.000 n=10) Put/size=1048576,thread=1-16 2.243Ki ± 0% 2.770Ki ± 1% +23.51% (p=0.000 n=10) Put/size=1048576,thread=20-16 42.91Ki ± 3% 46.23Ki ± 5% +7.73% (p=0.000 n=10) Put/size=1048576,thread=100-16 210.3Ki ± 2% 229.8Ki ± 2% +9.27% (p=0.000 n=10) Put/size=1048576,thread=200-16 427.9Ki ± 2% 460.2Ki ± 1% +7.55% (p=0.000 n=10) Put/size=4194304,thread=1-16 2.325Ki ± 1% 2.771Ki ± 1% +19.17% (p=0.000 n=10) Put/size=4194304,thread=20-16 45.24Ki ± 2% 50.57Ki ± 1% +11.79% (p=0.000 n=10) Put/size=4194304,thread=100-16 231.7Ki ± 2% 257.8Ki ± 3% +11.26% (p=0.000 n=10) Put/size=4194304,thread=200-16 474.7Ki ± 4% 509.5Ki ± 11% +7.33% (p=0.000 n=10) geomean 50.02Ki 56.68Ki +13.32% │ ssd.fstree-original │ ssd.fstree-combined │ │ allocs/op │ allocs/op vs base │ Put/size=1,thread=1-16 22.00 ± 0% 32.00 ± 3% +45.45% (p=0.000 n=10) Put/size=1,thread=20-16 413.0 ± 0% 473.5 ± 1% +14.65% (p=0.000 n=10) Put/size=1,thread=100-16 2.069k ± 0% 2.167k ± 0% +4.71% (p=0.000 n=10) Put/size=1,thread=200-16 4.137k ± 0% 4.338k ± 0% +4.87% (p=0.000 n=10) Put/size=256,thread=1-16 22.00 ± 0% 32.00 ± 0% +45.45% (p=0.000 n=10) Put/size=256,thread=20-16 415.0 ± 0% 473.5 ± 0% +14.10% (p=0.000 n=10) Put/size=256,thread=100-16 2.070k ± 0% 2.163k ± 0% +4.47% (p=0.000 n=10) Put/size=256,thread=200-16 4.160k ± 0% 4.319k ± 0% +3.83% (p=0.000 n=10) Put/size=1024,thread=1-16 22.00 ± 5% 32.00 ± 0% +45.45% (p=0.000 n=10) Put/size=1024,thread=20-16 413.0 ± 1% 470.0 ± 1% +13.80% (p=0.000 n=10) Put/size=1024,thread=100-16 2.074k ± 0% 2.160k ± 0% +4.15% (p=0.000 n=10) Put/size=1024,thread=200-16 4.155k ± 0% 4.319k ± 0% +3.96% (p=0.000 n=10) Put/size=4096,thread=1-16 22.00 ± 0% 32.00 ± 0% +45.45% (p=0.000 n=10) Put/size=4096,thread=20-16 419.0 ± 0% 469.0 ± 0% +11.93% (p=0.000 n=10) Put/size=4096,thread=100-16 2.088k ± 0% 2.162k ± 0% +3.54% (p=0.000 n=10) Put/size=4096,thread=200-16 4.173k ± 0% 4.319k ± 0% +3.50% (p=0.000 n=10) Put/size=16384,thread=1-16 22.00 ± 0% 32.00 ± 0% +45.45% (p=0.000 n=18+10) Put/size=16384,thread=20-16 414.0 ± 0% 471.0 ± 0% +13.77% (p=0.000 n=10) Put/size=16384,thread=100-16 2.082k ± 1% 2.173k ± 0% +4.37% (p=0.000 n=10) Put/size=16384,thread=200-16 4.162k ± 0% 4.340k ± 0% +4.29% (p=0.000 n=10) Put/size=65536,thread=1-16 23.00 ± 4% 32.00 ± 0% +39.13% (p=0.000 n=10) Put/size=65536,thread=20-16 420.0 ± 0% 476.5 ± 1% +13.45% (p=0.000 n=10) Put/size=65536,thread=100-16 2.103k ± 0% 2.197k ± 1% +4.49% (p=0.000 n=10) Put/size=65536,thread=200-16 4.204k ± 0% 4.389k ± 0% +4.41% (p=0.000 n=10) Put/size=262144,thread=1-16 23.00 ± 0% 32.00 ± 0% +39.13% (p=0.000 n=10) Put/size=262144,thread=20-16 430.0 ± 1% 478.5 ± 0% +11.28% (p=0.000 n=10) Put/size=262144,thread=100-16 2.162k ± 2% 2.354k ± 1% +8.91% (p=0.000 n=10) Put/size=262144,thread=200-16 4.362k ± 2% 4.641k ± 2% +6.38% (p=0.000 n=10) Put/size=1048576,thread=1-16 25.00 ± 0% 33.00 ± 3% +32.00% (p=0.000 n=10) Put/size=1048576,thread=20-16 464.5 ± 3% 509.5 ± 2% +9.69% (p=0.000 n=10) Put/size=1048576,thread=100-16 2.322k ± 2% 2.535k ± 1% +9.17% (p=0.000 n=10) Put/size=1048576,thread=200-16 4.760k ± 1% 5.062k ± 1% +6.36% (p=0.000 n=10) Put/size=4194304,thread=1-16 26.00 ± 4% 33.00 ± 0% +26.92% (p=0.000 n=10) Put/size=4194304,thread=20-16 491.0 ± 2% 573.0 ± 1% +16.70% (p=0.000 n=10) Put/size=4194304,thread=100-16 2.550k ± 2% 2.914k ± 3% +14.29% (p=0.000 n=10) Put/size=4194304,thread=200-16 5.254k ± 4% 5.767k ± 10% +9.76% (p=0.000 n=10) geomean 552.5 638.2 +15.50% This makes flush_interval common for peapod and fstree since they're almost the same in meaning and then there are no peapod-specific configurations left. Signed-off-by: Roman Khimov <[email protected]>

It tries to push data with 100 threads which is close to the default maximum combining count. Then reads are done with as many threads as needed trying to read various objects. Unfortunately, size is not a real object size here, it can't be because one can push any garbage to the store with Put, but Get decodes input data and we need something that looks like an object, so size is a payload size here. Signed-off-by: Roman Khimov <[email protected]>

It does the same thing. Signed-off-by: Roman Khimov <[email protected]>

roman-khimov · 2024-08-28T14:38:43Z

Rebased, otherwise lens failed to build after the merge (even though the merge itself is clean).

Keeping whole combined file in memory is excessive. Tests with the default write combining enabled. HDD, before the patch: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 5 3600 6-Core Processor BenchmarkGet/size=1,thread=1/fstree-12 33870 35109 ns/op 14054 B/op 26 allocs/op BenchmarkGet/size=1,thread=1/peapod-12 104599 11292 ns/op 1496 B/op 27 allocs/op BenchmarkGet/size=1,thread=20/peapod-12 9690 123428 ns/op 30027 B/op 591 allocs/op BenchmarkGet/size=1,thread=20/fstree-12 6931 170086 ns/op 280829 B/op 501 allocs/op BenchmarkGet/size=1,thread=100/peapod-12 1978 586932 ns/op 148056 B/op 2683 allocs/op BenchmarkGet/size=1,thread=100/fstree-12 1828 656545 ns/op 1404125 B/op 2501 allocs/op BenchmarkGet/size=1024,thread=1/peapod-12 95288 12732 ns/op 3664 B/op 38 allocs/op BenchmarkGet/size=1024,thread=1/fstree-12 16780 71580 ns/op 117472 B/op 26 allocs/op BenchmarkGet/size=1024,thread=20/peapod-12 7676 147320 ns/op 73223 B/op 770 allocs/op BenchmarkGet/size=1024,thread=20/fstree-12 2733 440093 ns/op 2349173 B/op 501 allocs/op BenchmarkGet/size=1024,thread=100/peapod-12 1666 711513 ns/op 366425 B/op 3877 allocs/op BenchmarkGet/size=1024,thread=100/fstree-12 439 2773494 ns/op 11746542 B/op 2510 allocs/op BenchmarkGet/size=102400,thread=1/peapod-12 15387 82956 ns/op 214422 B/op 30 allocs/op BenchmarkGet/size=102400,thread=1/fstree-12 898 1494701 ns/op 6583348 B/op 26 allocs/op BenchmarkGet/size=102400,thread=20/peapod-12 1015 1056701 ns/op 4287961 B/op 564 allocs/op BenchmarkGet/size=102400,thread=20/fstree-12 57 18260165 ns/op 124299921 B/op 501 allocs/op BenchmarkGet/size=102400,thread=100/peapod-12 199 6087298 ns/op 21442817 B/op 3066 allocs/op BenchmarkGet/size=102400,thread=100/fstree-12 12 91612104 ns/op 630612319 B/op 2501 allocs/op PASS ok github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor 282.428s HDD, after the patch: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 5 3600 6-Core Processor BenchmarkGet/size=1,thread=1/fstree-12 13465 91189 ns/op 1687 B/op 26 allocs/op BenchmarkGet/size=1,thread=1/peapod-12 107473 11447 ns/op 1516 B/op 30 allocs/op BenchmarkGet/size=1,thread=20/peapod-12 9819 122212 ns/op 29637 B/op 542 allocs/op BenchmarkGet/size=1,thread=20/fstree-12 4321 276909 ns/op 33456 B/op 501 allocs/op BenchmarkGet/size=1,thread=100/peapod-12 2032 582728 ns/op 147916 B/op 2666 allocs/op BenchmarkGet/size=1,thread=100/fstree-12 1224 983660 ns/op 167230 B/op 2501 allocs/op BenchmarkGet/size=1024,thread=1/peapod-12 93832 12741 ns/op 3676 B/op 39 allocs/op BenchmarkGet/size=1024,thread=1/fstree-12 11612 97948 ns/op 3776 B/op 26 allocs/op BenchmarkGet/size=1024,thread=20/peapod-12 8413 145797 ns/op 73244 B/op 772 allocs/op BenchmarkGet/size=1024,thread=20/fstree-12 4113 288727 ns/op 75216 B/op 501 allocs/op BenchmarkGet/size=1024,thread=100/peapod-12 1677 688383 ns/op 366354 B/op 3866 allocs/op BenchmarkGet/size=1024,thread=100/fstree-12 1131 1054335 ns/op 376035 B/op 2501 allocs/op BenchmarkGet/size=102400,thread=1/peapod-12 16148 79871 ns/op 214414 B/op 29 allocs/op BenchmarkGet/size=102400,thread=1/fstree-12 6381 183071 ns/op 214592 B/op 26 allocs/op BenchmarkGet/size=102400,thread=20/peapod-12 1029 1163020 ns/op 4288312 B/op 608 allocs/op BenchmarkGet/size=102400,thread=20/fstree-12 1545 774230 ns/op 4291601 B/op 501 allocs/op BenchmarkGet/size=102400,thread=100/peapod-12 204 5881155 ns/op 21448378 B/op 3629 allocs/op BenchmarkGet/size=102400,thread=100/fstree-12 286 4097583 ns/op 21458835 B/op 2513 allocs/op SSD, before the patch: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics BenchmarkGet/size=1,thread=1/peapod-16 206061 5598 ns/op 1550 B/op 34 allocs/op BenchmarkGet/size=1,thread=1/fstree-16 43969 24736 ns/op 14056 B/op 26 allocs/op BenchmarkGet/size=1,thread=20/peapod-16 8701 130963 ns/op 30784 B/op 684 allocs/op BenchmarkGet/size=1,thread=20/fstree-16 10915 109432 ns/op 280837 B/op 501 allocs/op BenchmarkGet/size=1,thread=100/peapod-16 1615 697317 ns/op 153837 B/op 3395 allocs/op BenchmarkGet/size=1,thread=100/fstree-16 2515 459902 ns/op 1404191 B/op 2502 allocs/op BenchmarkGet/size=1024,thread=1/peapod-16 135268 7757 ns/op 3658 B/op 37 allocs/op BenchmarkGet/size=1024,thread=1/fstree-16 24434 47873 ns/op 117472 B/op 26 allocs/op BenchmarkGet/size=1024,thread=20/peapod-16 8172 148323 ns/op 73263 B/op 773 allocs/op BenchmarkGet/size=1024,thread=20/fstree-16 4244 243463 ns/op 2349182 B/op 501 allocs/op BenchmarkGet/size=1024,thread=100/peapod-16 1370 843700 ns/op 366623 B/op 3873 allocs/op BenchmarkGet/size=1024,thread=100/fstree-16 606 1917891 ns/op 11746920 B/op 2514 allocs/op BenchmarkGet/size=102400,thread=1/fstree-16 769 1667689 ns/op 8439914 B/op 26 allocs/op BenchmarkGet/size=102400,thread=1/peapod-16 26174 41144 ns/op 214476 B/op 36 allocs/op BenchmarkGet/size=102400,thread=20/fstree-16 49 21872222 ns/op 169093205 B/op 501 allocs/op BenchmarkGet/size=102400,thread=20/peapod-16 1674 636594 ns/op 4289612 B/op 758 allocs/op BenchmarkGet/size=102400,thread=100/peapod-16 296 3582319 ns/op 21447448 B/op 3535 allocs/op BenchmarkGet/size=102400,thread=100/fstree-16 10 110982517 ns/op 841263811 B/op 2505 allocs/op PASS ok github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor 203.115s SSD, after the patch: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics BenchmarkGet/size=1,thread=1/fstree-16 26210 44577 ns/op 1687 B/op 26 allocs/op BenchmarkGet/size=1,thread=1/peapod-16 203316 5551 ns/op 1550 B/op 34 allocs/op BenchmarkGet/size=1,thread=20/peapod-16 8142 127892 ns/op 30745 B/op 679 allocs/op BenchmarkGet/size=1,thread=20/fstree-16 8155 141522 ns/op 33457 B/op 501 allocs/op BenchmarkGet/size=1,thread=100/fstree-16 2160 526853 ns/op 167233 B/op 2501 allocs/op BenchmarkGet/size=1,thread=100/peapod-16 1633 695693 ns/op 153747 B/op 3383 allocs/op BenchmarkGet/size=1024,thread=1/fstree-16 24176 51102 ns/op 3776 B/op 26 allocs/op BenchmarkGet/size=1024,thread=1/peapod-16 139054 7587 ns/op 3675 B/op 39 allocs/op BenchmarkGet/size=1024,thread=20/peapod-16 8098 145081 ns/op 72656 B/op 697 allocs/op BenchmarkGet/size=1024,thread=20/fstree-16 7908 152477 ns/op 75216 B/op 501 allocs/op BenchmarkGet/size=1024,thread=100/peapod-16 1339 807344 ns/op 365303 B/op 3704 allocs/op BenchmarkGet/size=1024,thread=100/fstree-16 1930 578930 ns/op 376032 B/op 2501 allocs/op BenchmarkGet/size=102400,thread=1/peapod-16 22363 49137 ns/op 214402 B/op 28 allocs/op BenchmarkGet/size=102400,thread=1/fstree-16 8257 143084 ns/op 214592 B/op 26 allocs/op BenchmarkGet/size=102400,thread=20/peapod-16 1596 654267 ns/op 4289593 B/op 753 allocs/op BenchmarkGet/size=102400,thread=20/fstree-16 2222 507259 ns/op 4291623 B/op 501 allocs/op BenchmarkGet/size=102400,thread=100/peapod-16 286 3655491 ns/op 21447877 B/op 3592 allocs/op BenchmarkGet/size=102400,thread=100/fstree-16 415 2938832 ns/op 21459193 B/op 2517 allocs/op PASS ok github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor 213.832s Notice that peapod degrades more when multi-threaded. It still wins for super-small objects with low thread number, but that's the simplest scenario and this case can be improved for FSTree as well. Also, SSDs work better without write combining as we know from previous results, non-combined reads work like this for SSDs: goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics BenchmarkGet/size=1,thread=1/peapod-16 150696 6881 ns/op 1552 B/op 35 allocs/op BenchmarkGet/size=1,thread=1/fstree-16 60586 17133 ns/op 2279 B/op 26 allocs/op BenchmarkGet/size=1,thread=20/peapod-16 9412 127615 ns/op 30750 B/op 679 allocs/op BenchmarkGet/size=1,thread=20/fstree-16 19616 60384 ns/op 45298 B/op 501 allocs/op BenchmarkGet/size=1,thread=100/fstree-16 5278 203212 ns/op 226452 B/op 2501 allocs/op BenchmarkGet/size=1,thread=100/peapod-16 1600 691861 ns/op 153755 B/op 3385 allocs/op BenchmarkGet/size=1024,thread=1/peapod-16 148694 8222 ns/op 3677 B/op 39 allocs/op BenchmarkGet/size=1024,thread=1/fstree-16 55369 19654 ns/op 3936 B/op 26 allocs/op BenchmarkGet/size=1024,thread=20/peapod-16 7117 157147 ns/op 73252 B/op 772 allocs/op BenchmarkGet/size=1024,thread=20/fstree-16 17496 71552 ns/op 78417 B/op 501 allocs/op BenchmarkGet/size=1024,thread=100/peapod-16 1346 815546 ns/op 364150 B/op 3564 allocs/op BenchmarkGet/size=1024,thread=100/fstree-16 4941 242115 ns/op 392055 B/op 2501 allocs/op BenchmarkGet/size=102400,thread=1/peapod-16 23816 43648 ns/op 214439 B/op 31 allocs/op BenchmarkGet/size=102400,thread=1/fstree-16 16125 72096 ns/op 214752 B/op 26 allocs/op BenchmarkGet/size=102400,thread=20/peapod-16 1748 658507 ns/op 4288086 B/op 580 allocs/op BenchmarkGet/size=102400,thread=20/fstree-16 1838 617794 ns/op 4294815 B/op 501 allocs/op BenchmarkGet/size=102400,thread=100/peapod-16 284 3732700 ns/op 21441371 B/op 2795 allocs/op BenchmarkGet/size=102400,thread=100/fstree-16 438 2749439 ns/op 21475282 B/op 2518 allocs/op PASS ok github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor 172.643s Signed-off-by: Roman Khimov <[email protected]>

roman-khimov · 2024-08-28T14:58:39Z

There is potential for improvement still left, oid-offset mapping can be stored at the end of file minimizing seeks.

roman-khimov added the performance More of something per second label Apr 18, 2024

roman-khimov added this to the v0.42.0 milestone Apr 18, 2024

roman-khimov force-pushed the fstree-perf branch from ae1eb83 to 8a94c71 Compare April 18, 2024 21:26

This was referenced Apr 22, 2024

Panic on deletion from peapod #2813

Open

neofs-adm: Add container estimations inspector #2826

Merged

roman-khimov marked this pull request as ready for review April 26, 2024 11:56

roman-khimov requested review from carpawell and cthulhu-rider as code owners April 26, 2024 11:56

carpawell reviewed May 6, 2024

View reviewed changes

roman-khimov force-pushed the fstree-perf branch from 4af6699 to fa3b83f Compare May 6, 2024 20:02

roman-khimov force-pushed the fstree-perf branch 2 times, most recently from 1e0b560 to a8c75b2 Compare May 6, 2024 20:19

carpawell mentioned this pull request May 7, 2024

node/objects: Distribute objects in container via replication protocol #2802

Merged

roman-khimov modified the milestones: v0.42.0, v0.43.0 May 22, 2024

roman-khimov force-pushed the fstree-perf branch 2 times, most recently from eb46e27 to 97f570f Compare June 24, 2024 18:59

roman-khimov modified the milestones: v0.43.0, v0.44.0 Aug 5, 2024

roman-khimov added 6 commits August 27, 2024 17:33

fstree: don't do useless Stat() for Get

9fc8cb4

Open() (and ReadFile()) will return the same exact error, we don't need to Stat(). Signed-off-by: Roman Khimov <[email protected]>

fstree: simplify GetBytes

d99e490

We use ReadFile() for Get and it works just fine there, no reasons for it to not work here as well. Large files can overflow us anyway (but we're not supposed to have them in a proper storage). Signed-off-by: Roman Khimov <[email protected]>

fstree: simplify compression handling in GetBytes()

56ab649

IsCompressed() is handled already internally in Decompress(), duplicating it doesn't make our code better. Signed-off-by: Roman Khimov <[email protected]>

fstree: deduplicate Get/GetBytes/Iterate code

0d9ac77

Signed-off-by: Roman Khimov <[email protected]>

roman-khimov added 2 commits August 27, 2024 17:33

docs: drop obsolete parameters from blobstor config example

68dfb30

Peapod has no depth/width, FSTree has no width/size/opened_cache_capacity. Signed-off-by: Roman Khimov <[email protected]>

docs: add no_sync doc for fstree

56ace14

Signed-off-by: Roman Khimov <[email protected]>

cthulhu-rider approved these changes Aug 27, 2024

View reviewed changes

roman-khimov mentioned this pull request Aug 27, 2024

Deprecate peapod, provide migration to fstree #2924

Closed

roman-khimov mentioned this pull request Aug 28, 2024

Store objects in "header+payload" format #2925

Open

roman-khimov added 3 commits August 28, 2024 17:30

fstree: deduplicate iterate code further

391cfb4

It does the same thing. Signed-off-by: Roman Khimov <[email protected]>

roman-khimov force-pushed the fstree-perf branch from 97f570f to e86567e Compare August 28, 2024 14:38

roman-khimov force-pushed the fstree-perf branch from e86567e to 1b40ec4 Compare August 28, 2024 14:49

carpawell approved these changes Aug 28, 2024

View reviewed changes

roman-khimov merged commit 9975aa4 into master Aug 28, 2024
22 checks passed

roman-khimov deleted the fstree-perf branch August 28, 2024 18:39

roman-khimov mentioned this pull request Nov 20, 2024

Revise write-cache role, functionality, etc #2337

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Combined writing for FSTree #2814

Combined writing for FSTree #2814

roman-khimov commented Apr 18, 2024 •

edited

Loading

roman-khimov commented May 5, 2024

carpawell left a comment

roman-khimov commented May 6, 2024

codecov bot commented May 6, 2024 •

edited

Loading

carpawell commented May 7, 2024 •

edited

Loading

roman-khimov commented May 13, 2024

roman-khimov commented Aug 27, 2024

carpawell commented Aug 27, 2024

roman-khimov commented Aug 27, 2024

roman-khimov commented Aug 28, 2024

roman-khimov commented Aug 28, 2024

Combined writing for FSTree #2814

Combined writing for FSTree #2814

Conversation

roman-khimov commented Apr 18, 2024 • edited Loading

roman-khimov commented May 5, 2024

carpawell left a comment

Choose a reason for hiding this comment

roman-khimov commented May 6, 2024

codecov bot commented May 6, 2024 • edited Loading

Codecov Report

carpawell commented May 7, 2024 • edited Loading

roman-khimov commented May 13, 2024

roman-khimov commented Aug 27, 2024

carpawell commented Aug 27, 2024

roman-khimov commented Aug 27, 2024

roman-khimov commented Aug 28, 2024

roman-khimov commented Aug 28, 2024

roman-khimov commented Apr 18, 2024 •

edited

Loading

codecov bot commented May 6, 2024 •

edited

Loading

carpawell commented May 7, 2024 •

edited

Loading