-
-
Notifications
You must be signed in to change notification settings - Fork 540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow primary keys to contain address columns sorted by their resolved values. #8870
Conversation
@nicktobey DOLT
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -669,3 +670,53 @@ func (td TupleDesc) Equals(other TupleDesc) bool { | |||
} | |||
return true | |||
} | |||
|
|||
type AddressTypeHandler struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use a type comment
"github.com/dolthub/dolt/go/store/hash" | ||
) | ||
|
||
type ValueStore interface { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comments for this type and its methods
} | ||
} | ||
|
||
func (handler AddressTypeHandler) SerializedCompare(ctx context.Context, v1 []byte, v2 []byte) (int, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really a criticism, but I'm not sure this is in the spirit of this method. I.e. if you have to deref an address to get the value to compare, this is probably going to be very slow, and the point of this method is to be faster than deserializing the values just to compare them.
Not really sure what to do about it though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose that for very large values we only need to load a single chunk at a time: they might differ in the very first chunk. But that would only work if the child handler has certain properties that we can't guarantee. I added a TODO.
@nicktobey DOLT
|
@coffeegoddd DOLT
|
This is the Dolt part of this change.
GMS PR: dolthub/go-mysql-server#2854
Doltgres PR: dolthub/doltgresql#1214
The goal of the change is to allow for indexes to use an out-of-line variable-length type (like TEXT or BLOB) as a primary key while still storing just the address in the index (instead of being forced to store a prefix of the value).
As a result of this change, any tuple comparison operation may need to resolve a hash in the NodeStore. This poses two complications:
The tuple logic exists at a much lower level than the node store and can't depend on it without creating a dependency cycle. We get around this with a new ValueStore interface that can store and retrieve variable-length bytestrings by their content hash. NodeStore is the only implementation of this interface, but decoupling the interface from the implementation allows us to not depend on NodeStore's internals when passing it to lower-level code.
Tuple comparison operations can now end up doing disk IO, which means they need a context parameter.