feat: floyd supports "Within the same Redis database, a single key name can only have one type of data structure" #2609

Mixficsol · 2024-04-17T08:50:42Z

背景

当前 Pika 的一个 Key 可以对应多种数据结构，和 Redis 不一致

解决方案

在 Floyd 现有的设计之上，将之前 String 类型所在的Column-Family 用于存放所有的 Key 的 Meta 信息

修改前：
enum ColumnFamilyIndex {
    kStringsCF = 0,
    kHashesMetaCF = 1,
    kHashesDataCF = 2,
    kSetsMetaCF = 3,
    kSetsDataCF = 4,
    kListsMetaCF = 5,
    kListsDataCF = 6,
    kZsetsMetaCF = 7,
    kZsetsDataCF = 8,
    kZsetsScoreCF = 9,
    kStreamsMetaCF = 10,
    kStreamsDataCF = 11,
};

修改后：
enum ColumnFamilyIndex {
    kMetaCF = 0,
    kHashesDataCF = 1,
    kSetsDataCF = 2,
    kListsDataCF = 3,
    kZsetsDataCF = 4,
    kZsetsScoreCF = 5,
    kStreamsDataCF = 6,
};

kMetaCF 字段设计

修改前

String

key格式
| reserve1 | key | reserve2 | 
|    8B    |     |   16B    |

value格式
| value | reserve | cdate | timestamp | 
|       |   16B   |   8B  |    8B     |

Hash

key格式
| reserve1 | key | reserve2 | 
|    8B    |     |    16B   |

value格式
| hash_size | version |  reserve  | cdate | timestamp | 
|    4B     |    8B   |    16B    |   8B  |     8B    |

List

key格式
| reserve1 | key | reserve2 |
|    8B    |     |   16B    |

value格式
|list_size| version | left index | right index | reserve |  cdate | timestamp | 
|   8B    |   8B    |     8B     |     8B      |   16B   |   8B   |    8B     |

set

key格式
| reserve1 | key | reserve2 |
|    8B    |     |    16B    |

value格式
| set_size | version | reserve | cdate | timestamp | 
|    4B    |    8B   |   16B   |   8B  |    8B     |

zset

key格式
| reserve1 | key | reserve2 |
|    8B    |     |    16B   |  

value格式
| zset_size | version | reserve  | cdate | timestamp | 
|    4B     |    8B   |    16B   |   8B  |    8B     |

Stream

key格式
| reserve1 | key | reserve2 |
|    8B    |     |    16B   |  

value格式
|  group_id_ | entries_added_ | first_id_ms | first_id_seq | last_id_ms | last_id_seq | max_deleted_entry_ms | max_deleted_entry_seq | length | version |
|     4B     |       8B       |     8B      |      8B      |     8B     |      8B     |         8B           |          8B           |   4B   |    4B   |

修改后

我们对每一种数据类型的 Meta 的 value 前增加一个字段 Type 用于区别每个 Key 对应的数据结构

以 Set 类型举例

key格式
| reserve1 | key | reserve2 |
|    8B    |     |    16B   |

value格式
| type | set_size | version | reserve | cdate | timestamp | 
|  1B  |    4B    |    8B   |   16B   |   8B  |    8B     |

String 的 Meta 格式

key格式
| reserve1 | key | reserve2 | 
|    8B    |     |   16B    |

value格式
| Type | value | reserve | cdate | timestamp | 
|  1B  |       |   16B   |  8B   |    8B     |

在解析的时候，先解析头部的第一个字节，然后根据类型判断是否需要继续解析下去

Mixficsol · 2024-04-25T10:13:14Z

Cmd 层多加了一次对返回值的判断(用于判断多 key 的错误信息)

else if (s_.ToString() == ErrTypeMessage) {
  res_.SetRes(CmdRes::kMultiKey);
}

Cache 层删除了 Redis 前缀

// 修改前：
std::string CachePrefixKeyK = PCacheKeyPrefixK + key_;
db_->cache()->SetBitIfKeyExist(CachePrefixKeyK, bit_offset_, on_);

// 修改后
db_->cache()->SetBitIfKeyExist(key_, bit_offset_, on_);

Storage 层对 Expire, Del, Exists, Expireat, Persist, TTL, GetType 这几个之前需要遍历每种数据类型的接口修改成指定操作某一个具体类型的接口

修改前:
// Keys Commands
int32_t Storage::Expire(const Slice& key, int64_t ttl, std::map<DataType, Status>* type_status) {
  type_status->clear();
  int32_t ret = 0;
  bool is_corruption = false;

  auto& inst = GetDBInstance(key);
  // Strings
  Status s = inst->StringsExpire(key, ttl);
  if (s.ok()) {
    ret++;
  } else if (!s.IsNotFound()) {
    is_corruption = true;
    (*type_status)[DataType::kStrings] = s;
  }

  // Hash
  s = inst->HashesExpire(key, ttl);
  if (s.ok()) {
    ret++;
  } else if (!s.IsNotFound()) {
    is_corruption = true;
    (*type_status)[DataType::kHashes] = s;
  }

  // Sets
  s = inst->SetsExpire(key, ttl);
  if (s.ok()) {
    ret++;
  } else if (!s.IsNotFound()) {
    is_corruption = true;
    (*type_status)[DataType::kSets] = s;
  }

  // Lists
  s = inst->ListsExpire(key, ttl);
  if (s.ok()) {
    ret++;
  } else if (!s.IsNotFound()) {
    is_corruption = true;
    (*type_status)[DataType::kLists] = s;
  }

  // Zsets
  s = inst->ZsetsExpire(key, ttl);
  if (s.ok()) {
    ret++;
  } else if (!s.IsNotFound()) {
    is_corruption = true;
    (*type_status)[DataType::kZSets] = s;
  }

  if (is_corruption) {
    return -1;
  } else {
    return ret;
  }
}

修改后
// Keys Commands
int32_t Storage::Expire(const Slice& key, int64_t ttl) {
  auto& inst = GetDBInstance(key);
  Status s = inst->Expire(key, ttl);
  if (s.ok()) {
    return 1;
  } else if (!s.IsNotFound()) {
    return -1;
  }
  return 1;
}

Redis 层每次需要对 Key 进行修改时，新增了对 meta-key 类型的判断

if (!ExpectedMetaValue(Type::kSet, meta_value)) {
  return Status::InvalidArgument("WRONGTYPE Operation against a key holding the wrong kind of value");
}

chejinge · 2024-05-06T07:50:58Z

src/pika_migrate_thread.cc

@@ -71,17 +71,15 @@ static int migrateKeyTTl(net::NetCli *cli, const std::string& key, storage::Data
                         const std::shared_ptr<DB>& db) {
  net::RedisCmdArgsType argv;
  std::string send_str;
-  std::map<storage::DataType, int64_t> type_timestamp;


这里为啥把timestape弄没了

改成了 int64_t type_timestamp = db->storage()->TTL(key); 因为 key 的类型是唯一的

wangshao1 · 2024-05-07T09:00:05Z

src/storage/src/redis.cc

@@ -8,7 +8,6 @@
 #include "rocksdb/env.h"

 #include "src/redis.h"
-#include "src/strings_filter.h"


strings_filter.h这个文件现在用不到了吧，删掉吧。

wangshao1 · 2024-05-07T09:06:37Z

src/storage/src/redis_strings.cc

 Status Redis::Append(const Slice& key, const Slice& value, int32_t* ret) {
  std::string old_value;
  *ret = 0;
  ScopeRecordLock l(lock_mgr_, key);

  BaseKey base_key(key);
  Status s = db_->Get(default_read_options_, base_key.Encode(), &old_value);
+  if (s.ok() && !ExpectedMetaValue(Type::kString, old_value)) {


ScanStringsKeyNum这个函数在遍历的时候没有判断类型，应该得加上。

wangshao1 · 2024-05-07T09:12:30Z

src/storage/src/redis_zsets.cc

@@ -1038,7 +1106,14 @@ Status Redis::ZScore(const Slice& key, const Slice& member, double* score) {


  BaseMetaKey base_meta_key(key);
-  Status s = db_->Get(read_options, handles_[kZsetsMetaCF], base_meta_key.Encode(), &meta_value);
+  Status s = db_->Get(read_options, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok() && !ExpectedMetaValue(Type::kZset, meta_value) && !ExpectedMetaValue(Type::kSet, meta_value)) {


如果s.ok()但是metavalue类型是kset，也是不符合预期的吧。

这个是因为当时遇到一个这样的TCL样例：

test {ZINTERSTORE regression with two sets, intset+hashtable} { r del seta setb setc r sadd set1 a r sadd set2 10 r zinterstore set3 2 set1 set2 } {0}

简单理解就是 zinterstore 是 Zset 命令但是操作 Set 类型也是可以的

wangshao1 · 2024-05-07T09:17:29Z

src/storage/src/lists_filter.h

+/*
+ * Because the meta data filtering strategy for list
+ * is integrated into base_filter.h, we delete it here
+ */


ZSetsScoreFilter，ListsDataFilter等复杂数据类型的data columnfamily 的compactionfilter也需要改下，从meta_index读出数据之后，判断类型是否匹配。

AlexStocks · 2024-05-07T09:21:46Z

src/pika_migrate_thread.cc

    key_type = 'z';
-  } else if (type_str[0] == "none") {
+  } else if (type_str == "none") {


floyd 底层不是也支持存储 stream 数据吗？这里不考虑 stream 类型？

AlexStocks · 2024-05-07T09:24:48Z

src/pika_slot_command.cc

-      return -3;
-  }
+  int64_t type_timestamp = db->storage()->TTL(key);
+  return type_timestamp;


return db->storage()->TTL(key);

AlexStocks · 2024-05-07T09:29:53Z

src/storage/src/base_value_format.h

-      }
-  virtual ~InternalValue() {
+ explicit InternalValue(Type type, const rocksdb::Slice& user_value) : type_(type), user_value_(user_value) {
+   ctime_ = pstd::NowMicros() / 1000000;


1000000 改为 1e6

AlexStocks · 2024-05-07T09:39:36Z

src/storage/src/lists_meta_value_format.h

  }

  // Use this constructor in rocksdb::CompactionFilter::Filter();
  explicit ParsedListsMetaValue(const rocksdb::Slice& internal_value_slice)
      : ParsedInternalValue(internal_value_slice) {
    assert(internal_value_slice.size() >= kListsMetaValueSuffixLength);
+    // TODO Why need This judgment logic; Mixficsol


搞清楚，给个回复

解析的字段大小必须要大于最小设置的字段值才开始做解析，不然可能会访问到未定义的内存

AlexStocks · 2024-05-07T10:25:38Z

src/storage/src/redis_sets.cc

+      if (ExpectedStale(meta_value)) {
+        s = Status::NotFound();
+      } else {
+        return Status::InvalidArgument("WRONGTYPE Operation against a key holding the wrong kind of value");


是酱紫，你把类似的代码中的错误信息都改下，后面补充进：1 期望的类型；2 实际 get 到的类型

这样方便排查问题

AlexStocks · 2024-05-07T10:27:32Z

src/storage/src/redis_sets.cc

+    if (ExpectedStale(meta_value)) {
+      s = Status::NotFound();
+    } else {
+      return Status::InvalidArgument("WRONGTYPE Operation against a key holding the wrong kind of value");


补全下错误信息

src/storage/src/redis_strings.cc

AlexStocks · 2024-05-07T10:49:45Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


改成 switch

AlexStocks · 2024-05-07T10:52:12Z

src/storage/src/redis_strings.cc

+  while (iter->Valid()) {
+    auto meta_type = static_cast<enum Type>(static_cast<uint8_t>(iter->value()[0]));
+    ParsedBaseMetaKey parsed_meta_key(iter->key().ToString());
+    if (meta_type == Type::kString) {


改成 switch

AlexStocks · 2024-05-07T10:54:26Z

src/storage/src/redis_zsets.cc

+    if (ExpectedStale(meta_value)) {
+      s = Status::NotFound();
+    } else {
+      return Status::InvalidArgument("WRONGTYPE Operation against a key holding the wrong kind of value");


1 上层比较时，只比较 “WRONGTYPE”；
2 constexpr string InvalidType = "WRONGTYPE xxxxx "
3 return Status::InvalidArgument(InvalidType + ", key %s, get type %s, wanna type %s", base_meta_key0, string(metaValue), string(Type::kSet))

AlexStocks · 2024-05-07T10:56:47Z

src/storage/src/type_iterator.h

+  bool ShouldSkip() override {
+    std::string user_value;
+    auto type = static_cast<Type>(static_cast<uint8_t>(raw_iter_->value()[0]));
+    if (type == Type::kZset || type == Type::kSet || type == Type::kHash || type == Type::kStream) {


改为 switch

AlexStocks · 2024-05-07T11:08:08Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


AlexStocks · 2024-05-07T11:08:14Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


AlexStocks · 2024-05-07T11:08:34Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


AlexStocks · 2024-05-07T11:08:41Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


AlexStocks · 2024-05-07T11:08:49Z

src/storage/src/redis_strings.cc

+  rocksdb::Status s = db_->Get(default_read_options_, handles_[kMetaCF], base_meta_key.Encode(), &meta_value);
+  if (s.ok()) {
+    auto type = static_cast<Type>(static_cast<uint8_t>(meta_value[0]));
+    if (type == Type::kSet) {


AlexStocks · 2024-05-07T11:11:29Z

src/storage/src/base_filter.h

+     */
+    auto type = static_cast<enum Type>(static_cast<uint8_t>(value[0]));
+    DEBUG("==========================START==========================");
+    if (type == Type::kString) {


改成 switch

AlexStocks · 2024-05-13T06:19:46Z

src/pika_kv.cc

@@ -516,7 +517,7 @@ void MgetCmd::DoInitial() {

 void MgetCmd::Do() {
  db_value_status_array_.clear();
-  s_ = db_->storage()->MGetWithTTL(keys_, &db_value_status_array_);
+  s_ = db_->storage()->MGet(keys_, &db_value_status_array_);


这里原始接口调用的是 MGetWithTTL，为何改了接口

因为在之前在Redis层的Mgetwithtll接口会对非法的多key有报错返回值，但是mget命令如果对于没有类型的key值的话，不会有报错，只会返回nil，所以自己重新实现了一个MGet接口去兼容Redis语义

wangshao1 · 2024-05-07T09:30:52Z

src/pika_slot_command.cc

    key_type = "s";
-  } else if (type_str[0] == "zset") {
+  } else if (type_str == "zset") {


少了一个stream吧？

wangshao1 · 2024-05-15T12:48:39Z

src/storage/src/base_filter.h

+        auto type = static_cast<enum RedisType>(static_cast<uint8_t>(meta_value[0]));
+        if (type != type_) {
+          return true;
+        } else if (type == RedisType::kHash || type == RedisType::kSet || type == RedisType::kStream) {


是不是还有zset data类型？

wangshao1 · 2024-05-15T12:57:57Z

src/storage/src/lists_filter.h

+        auto type = static_cast<enum RedisType>(static_cast<uint8_t>(meta_value[0]));
+        if (type != type_) {
+          return true;
+        } else if (type == RedisType::kList) {


list的datafilter只有一种类型，所以我理解这里只判断type是否等于type_，其他的else不需要了。

wangshao1 · 2024-05-15T13:07:43Z

src/storage/src/lists_filter.h

-        ParsedListsMetaValue parsed_lists_meta_value(&meta_value);
-        cur_meta_version_ = parsed_lists_meta_value.Version();
-        cur_meta_etime_ = parsed_lists_meta_value.Etime();
+        auto type = static_cast<enum RedisType>(static_cast<uint8_t>(meta_value[0]));


有一种情况是：

一个list data类型的key记为l1，从meta里获取元信息，发现类型不对，然后直接返回true了。

compaction iterator继续迭代有发现了一个l2，同属于同一个list。此时它不会去读meta元信息，因为cur_key_等于meta_key_enc，这时它会继续走后边比对时间戳和version的逻辑，但此时的时间戳和version值是不准确的。

src/pika_slot_command.cc

src/pika_migrate_thread.cc

src/pika_slot_command.cc

src/storage/tests/lists_filter_test.cc

tests/unit/type/string.tcl

src/storage/src/type_iterator.h

wangshao1 · 2024-05-20T03:21:55Z

src/storage/src/storage.cc

@@ -1242,7 +1242,7 @@ int64_t Storage::Scan(const DataType& dtype, int64_t cursor, const std::string&
      LOG(WARNING) << "Invalid key_type: " << key_type;
      return 0;
    }
-    std::copy(pos, iter_end, std::back_inserter(types));
+    std::copy(pos, iter_end - 2, std::back_inserter(types));


这个地方为什么减2？

这个是因为之前DataType类型只有all+五种数据类型，然后现在我复用了这个DataType需要加一个none类型，然后-2是因为之前可以直接到迭代器尾端，但是现在得在末端前两个

chejinge · 2024-05-20T06:16:06Z

src/pika_kv.cc

@@ -224,19 +222,15 @@ void DelCmd::DoUpdateCache() {
  if (s_.ok()) {
    std::vector<std::string> v;


这里既然只存在一种类型，那直接delete key就好了，没必要用v转换一下

chejinge · 2024-05-20T06:34:17Z

src/pika_list.cc

@@ -663,6 +676,10 @@ void BRPopCmd::Do() {
      return;
    } else if (s_.IsNotFound()) {
      continue;
+    } else if (s_.IsInvalidArgument()) {
+      // TODO use return or continue; Mixficsol


这里确认了就把注释删了吧

…me can only have one type of data structure" (OpenAtomFoundation#2609) * floyd supports one key for one data structure * Modified the criteria for determining multiple keys

github-actions bot added the ✏️ Feature New feature or request label Apr 17, 2024

Mixficsol added 5 commits April 18, 2024 18:42

floyd supports one key for one data structure

65e5118

fix tcl test

25b900a

fix conflix

f998b05

fix ctest

4234335

fix review shaoyi

685335d

Mixficsol added 3 commits April 26, 2024 00:48

Delete the redis prefix

a51ff8b

change compactrange and compact

aa17919

Modified the criteria for determining multiple keys

28220fa

Mixficsol marked this pull request as ready for review April 26, 2024 12:04

chejinge reviewed May 6, 2024

View reviewed changes

wangshao1 reviewed May 7, 2024

View reviewed changes

AlexStocks reviewed May 7, 2024

View reviewed changes

src/storage/src/redis_strings.cc Outdated Show resolved Hide resolved

AlexStocks reviewed May 7, 2024

View reviewed changes

AlexStocks reviewed May 13, 2024

View reviewed changes

change if-else to switch

a6eacea

AlexStocks changed the title ~~feat: floyd supports one key for one data structure~~ feat: floyd supports "one key should just have one data type" May 14, 2024

AlexStocks changed the title ~~feat: floyd supports "one key should just have one data type"~~ feat: floyd supports "Within the same Redis database, a single key name can only have one type of data structure" May 14, 2024

wangshao1 reviewed May 15, 2024

View reviewed changes

review change

02e0a2d

Mixficsol commented May 16, 2024

View reviewed changes

src/pika_slot_command.cc Outdated Show resolved Hide resolved

change shaoyi review compaction-filter

8e0376b

AlexStocks reviewed May 16, 2024

View reviewed changes

src/pika_migrate_thread.cc Outdated Show resolved Hide resolved

AlexStocks reviewed May 16, 2024

View reviewed changes

src/pika_slot_command.cc Outdated Show resolved Hide resolved

AlexStocks reviewed May 16, 2024

View reviewed changes

src/pika_slot_command.cc Outdated Show resolved Hide resolved

AlexStocks reviewed May 16, 2024

View reviewed changes

src/storage/tests/lists_filter_test.cc Outdated Show resolved Hide resolved

AlexStocks reviewed May 16, 2024

View reviewed changes

tests/unit/type/string.tcl Show resolved Hide resolved

AlexStocks changed the base branch from unstable to floyd_unique_type May 16, 2024 07:00

AlexStocks changed the base branch from floyd_unique_type to unstable May 16, 2024 10:06

Mixficsol added 2 commits May 17, 2024 10:11

RedisType to DataType

4334994

fix scan bug

528c6b8

AlexStocks reviewed May 20, 2024

View reviewed changes

src/storage/src/type_iterator.h Outdated Show resolved Hide resolved

wangshao1 reviewed May 20, 2024

View reviewed changes

AlexStocks approved these changes May 20, 2024

View reviewed changes

wangshao1 approved these changes May 20, 2024

View reviewed changes

chejinge reviewed May 20, 2024

View reviewed changes

chejinge approved these changes May 20, 2024

View reviewed changes

fix scan bug

bf332df

AlexStocks merged commit e38e255 into OpenAtomFoundation:unstable May 20, 2024
14 checks passed

		@@ -224,19 +222,15 @@ void DelCmd::DoUpdateCache() {
		if (s_.ok()) {
		std::vector<std::string> v;

feat: floyd supports "Within the same Redis database, a single key name can only have one type of data structure" #2609

feat: floyd supports "Within the same Redis database, a single key name can only have one type of data structure" #2609

Conversation

Mixficsol commented Apr 17, 2024 • edited Loading

背景

解决方案

kMetaCF 字段设计

Mixficsol commented Apr 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mixficsol May 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexStocks May 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexStocks May 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mixficsol commented Apr 17, 2024 •

edited

Loading

Mixficsol commented Apr 25, 2024 •

edited

Loading

Mixficsol May 7, 2024 •

edited

Loading

AlexStocks May 7, 2024 •

edited

Loading

AlexStocks May 7, 2024 •

edited

Loading