write blocks retrieved from the exchange to the blockstore #92
Conversation
|
cc @Jorropo |
blockservice.go
Outdated
| @@ -326,6 +331,15 @@ func getBlocks(ctx context.Context, ks []cid.Cid, bs blockstore.Blockstore, fget | |||
|
|
|||
| for b := range rblocks { | |||
| logger.Debugf("BlockService.BlockFetched %s", b.Cid()) | |||
|
|
|||
| // also write in the blockstore for caching | |||
| // TODO: for performance reason, it might make sense to batch those writes | |||
There was a problem hiding this comment.
This probably needs to happen before we merge (it has significant performance implications). A simple version would be to read from rblocks with a default select statement (i.e., drain it), then write a batch. That's not perfect, but it'll help as long as rblocks is buffered (IIRC, it is).
There was a problem hiding this comment.
That's not perfect, but it'll help as long as rblocks is buffered (IIRC, it is).
Turns out, it's not buffered: https://github.com/ipfs/go-bitswap/blob/b18a91d6023b83821c72253bbe5e37190db64d63/internal/getter/getter.go#L93
This follows the change in bitswap where that responsibility was removed.
MichaelMure
force-pushed
the
bitswap-no-add-bs
branch
from
c519ec9
to
4a60416
Compare
|
I added the Put batching, tests to make sure that blockservice does write blocks, and followed the interface changes. Again, we need to release and bubble up the dependency. I think the last remaining point is possibly to buffer the output channel in bitswap: #92 (comment) |
MichaelMure
force-pushed
the
bitswap-no-add-bs
branch
from
195d8b0
to
f2a4f4f
Compare
…rvice into bitswap-no-add-bs
|
Blocking in MichaelMure#1 |
Jorropo
force-pushed
the
bitswap-no-add-bs
branch
from
06f850f
to
7374061
Compare
|
The batch loop was too confusing, I rewrote it (hopefully) slightly better. I would like a merge review on 30ca6aa pls. |
LGTM |
This follows the change in bitswap where that responsibility was removed (ipfs/go-bitswap#571).