Annotate design document in preparation for update #9634

spencerkimball · 2016-09-30T03:47:48Z

@mjibson @petermattis @mrtracy @tschottdorf @arjunravinarayan @andreimatei @jordanlewis @jseldess @sploiselle

All, you've been called to a higher purpose. Our design doc is in parlous condition. It's time to give it a miracle makeover. For those of you who feel the call of duty, we're going to organize a designdoc hackathon tomorrow at 1:30p to split design.md into areas of concern, parcel them out amongst ourselves, knock out the asbestos and water-stained drywall, and erect a modern, glass and steel structure in its place.

I've yet to delineate the missing sections (too late already tonight); we can spend some time on that tomorrow. But if we just get as far as dumping no-longer-applicable sections and cleaning up what remains, we'll be in significantly better shape.

This change is

petermattis · 2016-09-30T13:22:17Z

I'm happy to take on some of the work, though 1:30pm on Friday's is rough for me due to various shuttling of kids to activities.

spencerkimball · 2016-09-30T14:06:26Z

Why don't you comment in this PR for sections you'd like to take on.

petermattis · 2016-09-30T15:36:02Z

docs/design.md

 In order to support diverse client usage, Cockroach clients connect to
 any node via HTTPS using protocol buffers or JSON. The connected node
 proxies involved client work including key lookups and write buffering.

 # Keys

+[Update referencing concepts in keys/constants.go required here]


I can tackle this section. Will probably rename it to Internal Keys to delineate it from SQL.

petermattis · 2016-09-30T15:40:49Z

docs/design.md

@@ -633,6 +663,7 @@ A really good reference on tuning Linux installations with RocksDB is

 # Range Metadata

+[Peter: did you actually have numbers for the size of meta2 records?]


Yes. Easy enough to recompute, though it was below 256 bytes and that upper bound still seems reasonable. I can take this section as well.

spencerkimball · 2016-09-30T16:27:34Z

+cc @knz

andreimatei · 2016-09-30T17:42:03Z

docs/design.md

+
+
+
+![Distributed SQL]


I can take this

andreimatei · 2016-09-30T17:42:57Z

docs/design.md

 such as schemas, tables, columns, and indexes. The structured data API
 in turn depends on the [distributed key value store](#key-value-api),
 which handles the details of range addressing to provide the abstraction
 of a single, monolithic key value store. The distributed KV store
 communicates with any number of physical cockroach nodes. Each node
 contains one or more stores, one per physical device.

+![SQL]


@knz was volunteered

tbg · 2016-09-30T17:43:45Z

I'm happy to do whatever I'm being volunteered for.

andreimatei · 2016-09-30T17:45:37Z

docs/design.md

@@ -64,15 +74,25 @@ grained data on the level of entity groups.

 Cockroach implements a layered architecture. The highest level of
 abstraction is the SQL layer (currently unspecified in this document).
+[FIX]


I'll fix this

spencerkimball · 2016-09-30T17:45:39Z

docs/design.md

@@ -83,6 +103,7 @@ raft. The color coding shows associated range replicas.

 ![Ranges](media/ranges.png)

+[Update RoachNode concept]


I'll fix references to RoachNode.

spencerkimball · 2016-09-30T17:46:09Z

docs/design.md

@@ -101,12 +122,14 @@ Up to `F` failures can be tolerated, where the total number of replicas `N = 2F

 # Cockroach Client


I'll delete this whole section.

tbg · 2016-09-30T17:51:40Z

I'll get to the Linearizability section.

rjnn · 2016-09-30T17:52:10Z

I'll take Raft (+ Quiescing + Coalescing).

spencerkimball · 2016-09-30T17:53:48Z

docs/design.md

@@ -377,6 +401,7 @@ An exploration of retries with contention and abort times with abandoned
 transaction is
 [here](https://docs.google.com/document/d/1kBCu4sdGAnvLqpT-_2vaTbomNmX3_saayWEGYu1j7mQ/edit?usp=sharing).

+[Update (conceptually) s/table/records/g]


@tamird could you address this comment?

knz · 2016-09-30T17:54:16Z

docs/design.md

@@ -4,6 +4,9 @@ by Spencer Kimball from early 2014.

 # Overview

+[Spencer to edit overview. This stuff is so ripe it's starting to
+stink: "structured data? WTF is that?]
+
 Cockroach is a distributed key:value datastore (SQL and structured


I'm rewriting the SQL reference here.

knz · 2016-09-30T17:56:06Z

docs/design.md

@@ -14,6 +17,10 @@ disruption and **no manual intervention**. Cockroach nodes are
 symmetric; a design goal is **homogeneous deployment** (one binary) with
 minimal configuration.

+[Start at top of the new stack here: SQL, mention of distributed SQL,


Taking this over

knz · 2016-09-30T17:56:22Z

docs/design.md

@@ -64,15 +74,25 @@ grained data on the level of entity groups.

 Cockroach implements a layered architecture. The highest level of
 abstraction is the SQL layer (currently unspecified in this document).
+[FIX]


knz · 2016-09-30T17:57:30Z

docs/design.md

@@ -1305,6 +1356,7 @@ write-optimized (HBase, Cassandra, SQLite3/LSM, CockroachDB).

 ## Architecture

+[Keep this diagram; update if necessary]
 CockroachDB implements a layered architecture, with various
 subdirectories implementing layers as appropriate. The highest level of
 abstraction is the [SQL layer][5], which depends


Rewriting this to clean up the SQL references

knz · 2016-09-30T17:58:28Z

docs/design.md

@@ -1331,6 +1383,7 @@ replicas.

 ## Client Architecture

+[Update this diagram]


Taking this over to explain SQL connections

rjnn · 2016-09-30T17:59:18Z

Review status: 0 of 1 files reviewed at latest revision, 14 unresolved discussions, all commit checks successful.

docs/design.md, line 812 at r1 (raw file):

# Raft - Consistency of Range Replicas

[Raft experts review this nonsense below]

Taking this.

docs/design.md, line 878 at r1 (raw file):

## Relationship to Raft leadership

[Is this all still accurate?]

Taking this.

Comments from Reviewable

BramGruneir · 2016-09-30T17:59:43Z

docs/design.md

@@ -609,6 +638,7 @@ for further details.

 # Node Storage

+[Updates necessary below]


I'll clean this section up a bit. Remove the range tree.

spencerkimball · 2016-09-30T18:02:22Z

docs/design.md

@@ -660,6 +691,7 @@ is sparse, the successor key is defined as the next key which is present. The
 found using the same process. The *meta2* record identifies the range
 containing `key1`, which is again found the same way (see examples below).

+[Updates required below]


@tamird could you update here as well?

BramGruneir · 2016-09-30T18:03:06Z

docs/design.md

 # Range-Spanning Binary Tree

+[Remove this section I think.]


BramGruneir · 2016-09-30T18:03:19Z

docs/design.md

@@ -903,6 +941,7 @@ expired, the command will be rejected by the replica.

 # Splitting / Merging Ranges

+[Accurate still? Needs review.]


Should we still discuss merges if we don't do them yet?

spencerkimball · 2016-09-30T18:04:34Z

docs/design.md

@@ -805,6 +839,7 @@ Future optimizations may include two-phase elections and quiescent ranges

 # Range Leases

+[Cleanup, add section on epoch-based range leases and motivation & link to range lease RFC]


@spencerkimball to do range leases

spencerkimball · 2016-09-30T18:05:23Z

docs/design.md

@@ -840,6 +875,7 @@ offset.

 ## Relationship to Raft leadership

+[Is this all still accurate?]


@arjunravinarayan to do this raft leadership section

spencerkimball · 2016-09-30T18:07:02Z

docs/design.md

@@ -1091,6 +1136,8 @@ The gossip protocol itself contains two primary components:

 # Node Accounting

+[Replace with description of MVCC, store-level aggregation, recorder,


@mrtracy would be great if you could replace Node Accounting section with an Accounting...

Sounds like @tschottdorf is handling this bit.

spencerkimball · 2016-09-30T18:08:27Z

docs/design.md

@@ -1225,6 +1275,7 @@ it discovers differences, it reconfigures ranges in the same way
 that it rebalances away from busy nodes, via special-case 1:1
 split to a duplicate range comprising the new configuration.

+[Rest of this document is tossable]


@spencerkimball to delete tail of document.

petermattis · 2016-09-30T18:11:18Z

If everyone is making edits to this doc, the merge might be painful. What is the plan?

Part of cockroachdb#9634.

spencerkimball · 2016-09-30T18:30:23Z

@petermattis just stay in your assigned section.

This adds a little bit on repair and rebalancing as well. Part of cockroachdb#9634.

BramGruneir · 2016-09-30T19:46:30Z

docs/design.md

@@ -609,6 +638,7 @@ for further details.

 # Node Storage

+[Updates necessary below]


BramGruneir · 2016-10-05T18:00:32Z

docs/design.md

@@ -609,6 +638,7 @@ for further details.

 # Node Storage

+[Updates necessary below]


Annotate design document update

4b112c0

petermattis reviewed Sep 30, 2016

View reviewed changes

andreimatei reviewed Sep 30, 2016

View reviewed changes

docs/design.md

![Distributed SQL]

Copy link

Contributor

andreimatei Sep 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can take this

andreimatei reviewed Sep 30, 2016

View reviewed changes

spencerkimball commented Sep 30, 2016

View reviewed changes

knz reviewed Sep 30, 2016

View reviewed changes

BramGruneir reviewed Sep 30, 2016

View reviewed changes

spencerkimball commented Sep 30, 2016

View reviewed changes

BramGruneir reviewed Sep 30, 2016

View reviewed changes

spencerkimball commented Sep 30, 2016

View reviewed changes

BramGruneir added a commit to BramGruneir/cockroach that referenced this pull request Sep 30, 2016

docs: remove references to the deleted rangetree

067734f

Part of cockroachdb#9634.

BramGruneir mentioned this pull request Sep 30, 2016

docs: remove references to the deleted rangetree #9643

Merged

BramGruneir added a commit to BramGruneir/cockroach that referenced this pull request Sep 30, 2016

docs: rework section on node storage

2c0f3f1

This adds a little bit on repair and rebalancing as well. Part of cockroachdb#9634.

BramGruneir mentioned this pull request Sep 30, 2016

docs: rework section on node storage #9652

Merged

spencerkimball closed this Oct 3, 2016

spencerkimball deleted the spencerkimball/design-doc branch October 3, 2016 16:34

BramGruneir added a commit to BramGruneir/cockroach that referenced this pull request Oct 4, 2016

docs: rework section on node storage

0a0d79a

This adds a little bit on repair and rebalancing as well. Part of cockroachdb#9634.

BramGruneir reviewed Oct 5, 2016

View reviewed changes

		@@ -633,6 +663,7 @@ A really good reference on tuning Linux installations with RocksDB is

		# Range Metadata

		[Peter: did you actually have numbers for the size of meta2 records?]

		@@ -83,6 +103,7 @@ raft. The color coding shows associated range replicas.

		![Ranges](media/ranges.png)

		[Update RoachNode concept]

		@@ -101,12 +122,14 @@ Up to `F` failures can be tolerated, where the total number of replicas `N = 2F

		# Cockroach Client

		@@ -1331,6 +1383,7 @@ replicas.

		## Client Architecture

		[Update this diagram]

		@@ -609,6 +638,7 @@ for further details.

		# Node Storage

		[Updates necessary below]

		@@ -903,6 +941,7 @@ expired, the command will be rejected by the replica.

		# Splitting / Merging Ranges

		[Accurate still? Needs review.]

		@@ -805,6 +839,7 @@ Future optimizations may include two-phase elections and quiescent ranges

		# Range Leases

		[Cleanup, add section on epoch-based range leases and motivation & link to range lease RFC]

		@@ -840,6 +875,7 @@ offset.

		## Relationship to Raft leadership

		[Is this all still accurate?]

		@@ -1091,6 +1136,8 @@ The gossip protocol itself contains two primary components:

		# Node Accounting

		[Replace with description of MVCC, store-level aggregation, recorder,

Annotate design document in preparation for update #9634

Annotate design document in preparation for update #9634

Conversation

spencerkimball commented Sep 30, 2016 • edited by rjnn Loading

petermattis commented Sep 30, 2016

spencerkimball commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spencerkimball commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tbg commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tbg commented Sep 30, 2016

rjnn commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rjnn commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

petermattis commented Sep 30, 2016

spencerkimball commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spencerkimball commented Sep 30, 2016 •

edited by rjnn

Loading