[v2] Blob Metadata Store #818
Conversation
| return &dynamodb.CreateTableInput{ | ||
| AttributeDefinitions: []types.AttributeDefinition{ | ||
| { | ||
| AttributeName: aws.String("PK"), |
There was a problem hiding this comment.
Could you document the tables that are abbreviations? It's not obvious to me what PK and SK stand for.
|
|
||
| blobKeyPrefix = "BlobKey#" | ||
| dispersalIDPrefix = "DispersalID#" | ||
| blobMetadataSK = "BlobMetadata" |
There was a problem hiding this comment.
So we're not using immutable design where blob metadata updates create a new unique sortable item via compound SK like BlobMetadata#UUID7 or BlobMetadata#EpochSeconds?
There was a problem hiding this comment.
yeah as discussed offline, some fields (BlobStatus) will be updated
| return &dynamodb.CreateTableInput{ | ||
| AttributeDefinitions: []types.AttributeDefinition{ | ||
| { | ||
| AttributeName: aws.String("PK"), |
There was a problem hiding this comment.
Could you document the tables that are abbreviations? It's not obvious to me what PK and SK stand for.
ian-shim
force-pushed
the
v2-blob-metadata
branch
from
75c03ab to
a4b23f5
Compare
| data = append(data, paymentBytes...) | ||
|
|
||
| return crypto.Keccak256(data) | ||
| func (pm *PaymentMetadata) Hash() ([32]byte, error) { |
There was a problem hiding this comment.
updated this hash function to make it compatible with solidity
cc @hopeyen
ian-shim
force-pushed
the
v2-blob-metadata
branch
from
a4b23f5 to
2837f2f
Compare
| OperatorResponseIndexName = "OperatorResponseIndex" | ||
|
|
||
| blobKeyPrefix = "BlobKey#" | ||
| dispersalIDPrefix = "DispersalID#" |
There was a problem hiding this comment.
It's used in a follow up PR, removed for now
|
|
||
| // GetBlobMetadataByStatus returns all the metadata with the given status | ||
| // Because this function scans the entire index, it should only be used for status with a limited number of items. | ||
| func (s *blobMetadataStore) GetBlobMetadataByStatus(ctx context.Context, status v2.BlobStatus) ([]*v2.BlobMetadata, error) { |
There was a problem hiding this comment.
Would there ever be a need for a batched version of this?
There was a problem hiding this comment.
What do you mean by a batched version?
| // Existing fields | ||
| AccountID string | ||
| // AccountID is the ETH account address for the payer | ||
| AccountID string `json:"account_id"` |
There was a problem hiding this comment.
Where is the json serialization used?
There was a problem hiding this comment.
It's used when we marshal/unmarshal the struct to/from json, i.e. in dataapi.
I've removed these for now
| return hash, nil | ||
| } | ||
|
|
||
| func (pm *PaymentMetadata) MarshalDynamoDBAttributeValue() (types.AttributeValue, error) { |
There was a problem hiding this comment.
Does this belong in core? Is there any way to have this code in the blobstore?
There was a problem hiding this comment.
They need to belong in the same package as where the type is defined. We can wrap and clone these types in blobstore but not sure if that's worth it
| @@ -59,7 +81,153 @@ func (b *BlobHeader) GetEncodingParams() (encoding.EncodingParams, error) { | |||
| NumChunks: uint64(params.NumChunks), | |||
| ChunkLength: uint64(length), | |||
| }, nil | |||
| } | |||
|
|
|||
| func (b *BlobHeader) BlobKey() (BlobKey, error) { | |||
ian-shim
force-pushed
the
v2-blob-metadata
branch
2 times, most recently
from
0dbb4c9 to
1b63aee
Compare
ian-shim
force-pushed
the
v2-blob-metadata
branch
from
1b63aee to
3b4a2eb
Compare
Why are these changes needed?
Set up v2 blob metadata store. Only implements few basic methods for now.
Checks