feat: add floyd #2347
Merged
feat: add floyd #2347
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
added 12 commits
January 24, 2024 18:30
AlexStocks
reviewed
Jan 27, 2024
| std::unique_ptr<RedisZSets> zsets_db_; | ||
| std::unique_ptr<RedisLists> lists_db_; | ||
| std::atomic<bool> is_opened_ = false; | ||
| std::vector<std::shared_ptr<Redis>> insts_; |
There was a problem hiding this comment.
容器里面一般使用 unique_ptr 就 ok 了
There was a problem hiding this comment.
个人觉得这个地方还是用shared_ptr比较好。insts_中的元素多个线程都会用到，unique_ptr不能拷贝，所以都是在使用unique_ptr的引用。如果坚持使用unique_ptr，那用户线程使用的inst建议直接用unique_ptr管理的裸指针。
longfar-ncy
reviewed
Jan 28, 2024
| SlotIndexer(uint32_t inst_num) : inst_num_(inst_num) {} | ||
| ~SlotIndexer() {} | ||
| uint32_t GetInstanceID(int32_t slot_id) {return slot_id % inst_num_; } | ||
| void ReshardSlots(const std::vector<uint32_t>& slots) {} |
There was a problem hiding this comment.
请问这个函数的作用是什么？
目前来看这个SlotIndexer只有GetInstanceID方法有用。个人感觉这个类有点鸡肋，保存的分片数量在storage类里也有（inst_.size()），完全可以放在storage里做一个方法。
There was a problem hiding this comment.
算哪个slot应该存在哪个rocskdb实例里。
拆出来原因：1. 当前的实现是直接求模，可能会导致每个rocksdb实例里的slot不均匀，后期期望有人提供其他的slot和rocskdb映射关系管理的实现。
2. storage类整体已经比较庞大了，独立的功能最好是拆个单独的类出来实现。
dingxiaoshuai123
approved these changes
Jan 31, 2024
AlexStocks
approved these changes
Jan 31, 2024
wangshao1
added a commit
to wangshao1/pika
that referenced
this pull request
Feb 19, 2024
* floyd: refactor storage to support multi rocksdb instance && todis key format * fix compile error after rebase * fix compile error after rebase * fix by review comments * fix by review comments * fix by review comments * fix by lqx and panlei's review comments * fix by review comments * fix by review comments * fix by review comments * fix by review comments * rebase upstream code * rebase upstream code * fix storage construct error * change uint32_t ttl to uint64_t * change int() to static_cast * fix by xiaoshuai's review comments * fix compile error in debug mode * fix slot_indexer by review comments * fix by xiaoshuai's review comments * fix dynamic change ttl and periodic_seconds * define CRCU32 type * change uint64_t to int64_t to support nagative time ttl * fix cache list error in floyd * fix by yuge's review comments * change shared_ptr to unique_ptr to be consisten with pikiwidb --------- Co-authored-by: wangshaoyi <[email protected]>
AlexStocks
pushed a commit
that referenced
this pull request
Feb 23, 2024
* feat: add floyd (#2347) * floyd: refactor storage to support multi rocksdb instance && todis key format --------- Co-authored-by: wangshaoyi <[email protected]> * fix floyd keys (#2364) Co-authored-by: wuxianrong <[email protected]> * floyd rebase unstable branch * fix by review comments --------- Co-authored-by: wangshaoyi <[email protected]> Co-authored-by: Mixficsol <[email protected]> Co-authored-by: wuxianrong <[email protected]>
bigdaronlee163
pushed a commit
to bigdaronlee163/pika
that referenced
this pull request
Jun 8, 2024
* feat: add floyd (OpenAtomFoundation#2347) * floyd: refactor storage to support multi rocksdb instance && todis key format --------- Co-authored-by: wangshaoyi <[email protected]> * fix floyd keys (OpenAtomFoundation#2364) Co-authored-by: wuxianrong <[email protected]> * floyd rebase unstable branch * fix by review comments --------- Co-authored-by: wangshaoyi <[email protected]> Co-authored-by: Mixficsol <[email protected]> Co-authored-by: wuxianrong <[email protected]>
cheniujh
pushed a commit
to cheniujh/pika
that referenced
this pull request
Sep 24, 2024
* feat: add floyd (OpenAtomFoundation#2347) * floyd: refactor storage to support multi rocksdb instance && todis key format --------- Co-authored-by: wangshaoyi <[email protected]> * fix floyd keys (OpenAtomFoundation#2364) Co-authored-by: wuxianrong <[email protected]> * floyd rebase unstable branch * fix by review comments --------- Co-authored-by: wangshaoyi <[email protected]> Co-authored-by: Mixficsol <[email protected]> Co-authored-by: wuxianrong <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
数据在rocksdb存储格式参考todis实现，增加了保留字段用于后续扩展。
与数据类型相关的，Compactionfilter，compactrange，comparator，key/value encode/decode逻辑的修改。
单个pika进程拆分多个rocksdb实例。
修改pika层代码，兼容storage层代码修改。