changefeedccl: support remaining simple types in Avro #34417
Labels
A-cdc
Change Data Capture
Comments
This was referenced Jan 30, 2019
danhhz
added a commit
to danhhz/cockroach
that referenced
this issue
Feb 14, 2019
This adds support for sql DATE, TIME, UUID, INET, and JSONB columns with the `experimental_avro` CHANGEFEED format. It also adds a set of "golden" tests for what avro schema each of our sql column types map to. INTERVAL has a corresponding avro logical type, but is left unimplemented for now because our INTERVALs have a larger domain of values than avro's: 8 bytes vs 4. ARRAY, BIT, and COLLATEDSTRING are a little tricker, so we'll wait for customer demand. These are expected to be safe enough to backport if we need them. Closes cockroachdb#32472 Closes cockroachdb#34421 Closes cockroachdb#34417 Release note (enterprise change): the `CHANGEFEED` `experimental_avro` format now supports sql columns of type DATE, TIME, UUID, INET, and JSONB.
danhhz
added a commit
to danhhz/cockroach
that referenced
this issue
Feb 15, 2019
This adds support for sql DATE, TIME, UUID, INET, and JSONB columns with the `experimental_avro` CHANGEFEED format. It also adds a set of "golden" tests for what avro schema each of our sql column types map to. Ditto for some example values for each type. INTERVAL has a corresponding avro logical type, but is left unimplemented for now because our INTERVALs have a larger domain of values than avro's: 8 bytes vs 4. ARRAY, BIT, and COLLATEDSTRING are a little tricker, so we'll wait for customer demand. These are expected to be safe enough to backport if we need them. Closes cockroachdb#32472 Closes cockroachdb#34421 Closes cockroachdb#34417 Release note (enterprise change): the `CHANGEFEED` `experimental_avro` format now supports sql columns of type DATE, TIME, UUID, INET, and JSONB.
craig bot
pushed a commit
that referenced
this issue
Feb 15, 2019
34918: changefeedccl: support more columns types with avro r=nvanbenschoten a=danhhz This adds support for sql DATE, TIME, UUID, INET, and JSONB columns with the `experimental_avro` CHANGEFEED format. It also adds a set of "golden" tests for what avro schema each of our sql column types map to. INTERVAL has a corresponding avro logical type, but is left unimplemented for now because our INTERVALs have a larger domain of values than avro's: 8 bytes vs 4. ARRAY, BIT, and COLLATEDSTRING are a little tricker, so we'll wait for customer demand. These are expected to be safe enough to backport if we need them. Closes #32472 Closes #34421 Closes #34417 Release note (enterprise change): the `CHANGEFEED` `experimental_avro` format now supports sql columns of type DATE, TIME, UUID, INET, and JSONB. Co-authored-by: Daniel Harrison <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
#32472 tracks the work to support time-related types (timestamp/timestamptz, date, time, and interval) in Avro. I'm going to open separate issues for Avro support of JSONB and ARRAY. That just leaves the following types as unsupported:
uuid
The Avro docs don't reference an official convention for UUID values, but according to this issue, UUIDs are actually implemented in Avro as a string type with a logicalType of uuid. We should do the same.
inet
The CockroachDB/Postgres
inettype stores an IPv4 or IPv6 address, optionally including a netmask. Absent an Avro convention for this sort of value, I recommend just representing these as strings in Avro.The text was updated successfully, but these errors were encountered: