Skip to content

Storing metadata

Bernardo Ramos edited this page Sep 21, 2018 · 7 revisions

Informing the branch and/or commit:

PRAGMA set_meta [{key}] {value}
PRAGMA get_meta [{key}] [{format}]

Where key is [{branch}][.][{commit}]

Example:

Setting the value:

PRAGMA set_meta master {value}
PRAGMA set_meta master.5 {value}
PRAGMA set_meta 5 {value}
PRAGMA set_meta {value}

Getting the value:

PRAGMA get_meta master
PRAGMA get_meta master.5
PRAGMA get_meta 5
PRAGMA get_meta

The branch name and commit id must be valid, when supplied.

When not supplied, it uses the current branch and commit.

The value can be formatted as:

Netstring (null characters can be in the buffer):

3:abc,

Using quotes:

""

In hex:

x'010203ABCDEF…'

Retrieving binary values

In Netstring format:

PRAGMA get_meta {key} --netstring

In hex format:

PRAGMA get_meta {key} --hex

Optional / Experimental: use of pointers

The advantage of this is to avoid encoding and decoding the values

Setting the value:

PRAGMA set_meta {key} {ptr} {size}

Example:

PRAGMA set_meta master.5 0xdeadbeef 125

Getting the value:

PRAGMA get_meta {key} --pointer

Returns:

{ptr} {size}

Example:

PRAGMA get_meta master.5 --pointer

Returns:

0x900df00d 125

For later

The ideas below can be omited

Another proposal

In the above examples, when storing and retrieving values in the same "key" (branch and/or commit) we must serialize and deserialize the data.

Example

Storing these 3 values in the commit master.5:

id: 730f6d1
author: John
datetime: 2018-09-01 01:35

Using the above approach we would need to serialize this. We can do it in JSON or Binn or any other format.

{
"id": "730f6d1",
"author": "John",
"datetime": "2018-09-01 01:35"
}

And then store the serialized data:

PRAGMA set_meta master.5 '{"id":"730f6d1","author":"John","datetime":"2018-09-01 01:35"}'

When reading just one piece of data from the commit we would need to read the entire data, deserialize it and retrieve the value.

Proposal

A good option would be to specify each sub-value as a sub-key:

To store only the id:

PRAGMA set_meta master.5:datetime '2018-09-01 01:35'

To read it:

PRAGMA get_meta master.5:datetime

So the key could be: [{branch}][.][{commit}][:{sub-key}]

Important: Notice that the sub-values stored in this way cannot be retrieved by the main key


So we have these 3 options:

  1. Use only the first format. The app must serialize and deserialize the data if using more than one sub-value.
  2. Use only the second format, with sub-values.
  3. Support both methods, knowing that the sub-values cannot be retrieved bundled in a single time

Option 4

Store the values in a binary container (Binn) and then store it in the database

Advantage: when reading just the branch.commit it would return all the sub-values (in which format?)

To write:

PRAGMA set_meta {branch.commit:key} {value}

or:

PRAGMA set_meta {branch.commit} {value}

To read:

PRAGMA get_meta {branch.commit:key} [{format}]

or:

PRAGMA get_meta {branch.commit} [{format}]

con: setting each sub-value may not happen in a single txn...

Option 5

Use both storage with and without sub-keys (not affecting each other) and use this pragma:

PRAGMA list_meta {branch.commit}

To retrieve the list of keys for that given branch and/or commit. It can return each key in a single line.

So we can read each value with:

PRAGMA get_meta {branch.commit:key} [{format}]

eg:

PRAGMA get_meta master.5:datetime

To write:

PRAGMA set_meta {branch.commit:key} {value}