Add APIs for M3Aggregator placement #1030

richardartoul · 2018-10-06T22:15:39Z

Add M3Agg placement endpoints and separate them out by path from the M3DB placement endpoints
Run all tests for both services
Update documentation for new and old endpoints
Refactor service to accept options instead of parsing headers directly

codecov · 2018-10-06T22:45:05Z

Codecov Report

Merging #1030 into master will decrease coverage by 0.13%.
The diff coverage is 46.81%.

@@            Coverage Diff            @@
##           master   #1030      +/-   ##
=========================================
- Coverage   77.74%   77.6%   -0.14%     
=========================================
  Files         532     532              
  Lines       45866   45566     -300     
=========================================
- Hits        35659   35362     -297     
- Misses       7950    7962      +12     
+ Partials     2257    2242      -15

Flag	Coverage Δ
#aggregator	`81.69% <ø> (+0.02%)`	⬆️
#collector	`96.47% <ø> (ø)`	⬆️
#dbnode	`81.18% <ø> (-0.12%)`	⬇️
#m3em	`73.21% <ø> (ø)`	⬆️
#m3ninx	`75.25% <ø> (ø)`	⬆️
#m3nsch	`51.19% <ø> (ø)`	⬆️
#query	`63.32% <46.81%> (-1.09%)`	⬇️
#x	`76.2% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2ffc763...570fc35. Read the comment docs.

cw9 · 2018-10-06T23:14:15Z

src/query/api/v1/handler/placement/common.go

 	// DefaultServiceEnvironment is the default service ID environment
 	DefaultServiceEnvironment = "default_env"
 	// DefaultServiceZone is the default service ID zone
 	DefaultServiceZone = "embedded"
-	// HeaderClusterServiceName is the header used to specify the service name.
-	HeaderClusterServiceName = "Cluster-Service-Name"


Why remove these headers?

I'm moving the service to the URL itself. I'm gonna add back the zone/environment headers.

cw9 · 2018-10-06T23:21:25Z

src/query/api/v1/handler/placement/common.go

+	errUnableToParseService         = errors.New("unable to parse service")
+
+	// AllowedServiceNames is the list of allowed service names
+	AllowedServiceNames = []string{M3DBServiceName, M3AggServiceName}


can this be private? also maybe just use a map?

cw9 · 2018-10-06T23:25:02Z

src/query/api/v1/handler/placement/delete_all.go

-	DeleteAllURL = handler.RoutePrefixV1 + "/placement"
+	// OldM3DBDeleteAllURL is the old url for the handler to delete all placements, maintained
+	// for backwards compatibility.
+	OldM3DBDeleteAllURL = handler.RoutePrefixV1 + "/placement"


hmm, this does not contain delete in it?

nope, just DELETE method

nit: DeprecatedM3DBDeleteAllURL

cw9 · 2018-10-06T23:27:08Z

src/query/generated/proto/admin/placement.proto

@@ -21,3 +21,9 @@ message PlacementAddRequest {
  // are AVAILABLE for all their shards. force overrides that.
  bool force = 2;
 }
+
+message AggPlacementInitRequest {


Just reuse PlacementInitRequest for this? They should always be the same

richardartoul · 2018-10-07T00:13:10Z

src/query/api/v1/handler/placement/init_test.go

+	assert.Equal(t, initTestInvalidRequestResponse, string(body))
+}
+
+func TestAggPlacementInitHandler(t *testing.T) {


TODO: Fix this test

richardartoul · 2018-10-07T00:13:46Z

src/query/util/logging/log.go

@@ -123,3 +123,22 @@ func WithResponseTimeLogging(next http.Handler) http.Handler {
 		}
 	})
 }
+
+// WithServiceNameResponseTimeLogging wraps around the given handler, providing response time logging
+func WithServiceNameResponseTimeLogging(


TODO: Rename this

richardartoul · 2018-10-07T18:52:11Z

src/query/api/v1/handler/placement/common.go

+		)
+		pOpts = pOpts.
+			SetIsMirrored(true).
+			// TODO(rartoul): Do we need to set placement cutover time? Seems like that would


Using immediate for placement cutover is fine, but I'd rather we copy all the logic from r2admin here to keep them in sync.
For example, the placementCutoff time is calculated as

now. Add(h.cfg.Placement.Clock.MaxPositiveSkew). Add(h.cfg.Placement.Clock.MaxNegativeSkew). Add(h.cfg.PropagationDelay)

in r2admin, we could just set all those three values as 0 to achieve immediate placement cutover.
Keeping the logic the same makes debugging much simpler and I could see us migrating r2admin/m3admin to call these endpoints here if we just run them with production configs.

robskillington · 2018-10-07T20:11:59Z

src/query/api/v1/handler/placement/add.go

+
+	// M3DBAddURL is the url for the placement add handler (with the POST method)
+	// for the M3DB service.
+	M3DBAddURL = handler.RoutePrefixV1 + "/services/m3db/placement"


Hm, are we worried about breaking all our guides/etc? I guess it's ok, at some point we'll probably have to version our API docs similar to ElasticSearch, etc so that we can go and see what the APIs were at a specific version (for users still running an older version).

Oh I see, you register the old one too. Sounds good.

Do you want to call OldM3DBAddURL perhaps DeprecatedM3DBAddURL? Just as it's a bit more clear that it's active but deprecated, rather than old which is ever so slightly not as clear.

Is it better to write:
M3DBAddURL = handler.RoutePrefixV1 + "/services/" + M3DBServiceName + "/placement"
in case you update the service in one place and forgot the others.

Same goes to the m3agg url as well. Also nit: use m3aggregator instead of m3agg everywhere?

Also consider making a const for "/services/" + M3DBServiceName + "/placement", I see that string being reused in a lot of places.

Agreed can we break up the consts?

robskillington · 2018-10-07T20:12:30Z

src/query/api/v1/handler/placement/add.go

+
+	// M3AggAddURL is the url for the placement add handler (with the POST method)
+	// for the M3Agg service.
+	M3AggAddURL = handler.RoutePrefixV1 + "/services/m3db/placement"


Should this perhaps be /services/m3aggregator/placement or similar?

robskillington · 2018-10-07T20:14:02Z

src/query/api/v1/handler/placement/add.go

@@ -115,7 +131,17 @@ func (h *AddHandler) Add(
 		return nil, err
 	}

-	service, algo, err := ServiceWithAlgo(h.client, httpReq.Header)
+	serviceOpts := NewServiceOptionsFromHeaders(serviceName, httpReq.Header)
+	if serviceName == M3AggServiceName {


nit: Maybe use a switch here so when we add further specialization later it's straightforward where to make the change (and we don't use an if/else if)?

robskillington · 2018-10-07T20:14:32Z

src/query/api/v1/handler/placement/add.go

-	service, algo, err := ServiceWithAlgo(h.client, httpReq.Header)
+	serviceOpts := NewServiceOptionsFromHeaders(serviceName, httpReq.Header)
+	if serviceName == M3AggServiceName {
+		if req.MaxAggregationWindowSizeNanos == 0 || req.WarmupDurationNanos == 0 {


Perhaps use <= 0? Just because a negative value is invalid too I assume.

robskillington · 2018-10-07T20:16:05Z

src/query/api/v1/handler/placement/add.go

+		}
+		serviceOpts.M3Agg = &M3AggServiceOptions{
+			MaxAggregationWindowSize: time.Duration(req.MaxAggregationWindowSizeNanos),
+			WarmupDuration:           time.Duration(req.WarmupDurationNanos),


Could we not have a default warmup duration and max aggregation window size if they don't specify any? I wouldn't have a good idea of what to choose for either of these without reading something first, and most users could probably change this later/shouldn't need to worry too much about these values perhaps?

Could these be configured instead of request based? I don't think they should change per request

We can probably have a sane default for warmupDuration, but maxAggregationWindowSize is very difficult to pick a sane default value for because if it’s too short they could lose data and if it’s too long their deploy takes forever

robskillington · 2018-10-07T20:18:30Z

src/query/api/v1/handler/placement/add_test.go

+			req *http.Request
+		)
+		if serviceName == M3AggServiceName {
+			req = httptest.NewRequest(AddHTTPMethod, M3DBAddURL, strings.NewReader("{\"instances\":[], \"max_aggregation_window_size_nanos\": 1000000, \"warmup_duration_nanos\": 1000000}"))


Do you want to use back ticks to avoid having to escape the quotes? Looks much more readable, e.g.:

strings.NewReader(`{"instances": [], "max_aggregation_window_size_nanos": 1000000, "warmup_duration_nanos": 1000000}"`)

robskillington · 2018-10-07T20:23:56Z

src/query/api/v1/handler/placement/common.go

+	return func(s shard.Shard) error {
+		switch s.State() {
+		case shard.Initializing:
+			if s.CutoverNanos() > now.UnixNano() {


Hm @cw9 is this what's expected? I though that cutover times always had to be in the future (inverse of this check)?

This is expected, this lambda is called before marking an intializing shard as available, so we don't allow users to mark the shard if now is before its CutoverNanos

robskillington · 2018-10-07T20:25:00Z

src/query/api/v1/handler/placement/common.go

+		case shard.Leaving:
+			// TODO(rartoul): This seems overly cautious, basically it requires an entire maxAggregationWindowSize
+			// to elapse before "leaving" shards can be cleaned up.
+			if s.CutoffNanos() > now.UnixNano()-maxAggregationWindowSize.Nanoseconds() {


Another query here, should it not be cutoff has to be greater than now+maxAggregationWindowSize rather than equal or less?

similar to cutover logic, this is called to when a leaving shard is being removed(when marking its corresponding initializing shard), this logic checks if the time has passed this shard's CutoffNanos

To answer Richie's TODO question, yes it's overly cautious, but let's just keep the logic the same as r2admin for now.

robskillington · 2018-10-07T20:26:01Z

src/query/api/v1/handler/placement/common.go

+	}
+}
+
+func parseServiceFromRequest(r *http.Request) (string, error) {


Hm, seems like a regexp might be slightly cleaner here? shrug

robskillington · 2018-10-07T20:26:32Z

src/query/api/v1/handler/placement/delete.go

-	DeleteURL = fmt.Sprintf("%s/placement/{%s}", handler.RoutePrefixV1, placementIDVar)
+	// OldM3DBDeleteURL is the old url for the placement delete handler, maintained
+	// for backwards compatibility.
+	OldM3DBDeleteURL = fmt.Sprintf("%s/placement/{%s}", handler.RoutePrefixV1, placementIDVar)


nit: DeprecatedM3DBDeleteURL?

robskillington · 2018-10-07T20:27:51Z

src/query/api/v1/handler/placement/delete_test.go

+		// require.Equal(t, http.StatusOK, resp.StatusCode)
+		// if serviceName == M3AggServiceName {
+
+		// } else {


Still need the if/else commented out code?
Also: perhaps use backticks here for these literal requests?

robskillington · 2018-10-07T20:28:02Z

src/query/api/v1/handler/placement/get.go

-	GetURL = handler.RoutePrefixV1 + "/placement"
+	// OldM3DBGetURL is the old url for the placement get handler, maintained for
+	// backwards compatibility.
+	OldM3DBGetURL = handler.RoutePrefixV1 + "/placement"


nit: DeprecatedM3DBGetURL?

robskillington · 2018-10-07T20:28:16Z

src/query/api/v1/handler/placement/get_test.go


-	const placementJSON = "{\"placement\":{\"instances\":{\"host1\":{\"id\":\"host1\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host1:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host1\",\"port\":1234},\"host2\":{\"id\":\"host2\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host2:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host2\",\"port\":1234}},\"replicaFactor\":0,\"numShards\":0,\"isSharded\":false,\"cutoverTime\":\"0\",\"isMirrored\":false,\"maxShardSetId\":0},\"version\":%d}"
+		const placementJSON = "{\"placement\":{\"instances\":{\"host1\":{\"id\":\"host1\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host1:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host1\",\"port\":1234},\"host2\":{\"id\":\"host2\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host2:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host2\",\"port\":1234}},\"replicaFactor\":0,\"numShards\":0,\"isSharded\":false,\"cutoverTime\":\"0\",\"isMirrored\":false,\"maxShardSetId\":0},\"version\":%d}"


nit: Backticks for literal JSON.

robskillington · 2018-10-07T20:28:28Z

src/query/api/v1/handler/placement/init.go

-	InitURL = handler.RoutePrefixV1 + "/placement/init"
+	// OldM3DBInitURL is the old url for the placement init handler, maintained for backwards
+	// compatibility. (with the POST method).
+	OldM3DBInitURL = handler.RoutePrefixV1 + "/placement/init"


nit: DeprecatedM3DBInitURL?

robskillington · 2018-10-07T20:28:59Z

src/query/api/v1/handler/placement/init.go

+	serviceOpts := NewServiceOptionsFromHeaders(serviceName, httpReq.Header)
+	if serviceName == M3AggServiceName {
+		if req.MaxAggregationWindowSizeNanos == 0 || req.WarmupDurationNanos == 0 {
+			return nil, errAggWindowAndWarmupMustBeSet


Hm seems this code is duplicated elsewhere? Can we reuse? Also same question about whether we can have defaults for these.

robskillington · 2018-10-07T20:31:18Z

src/query/generated/proto/admin/placement.proto

@@ -8,6 +8,10 @@ message PlacementInitRequest {
  repeated placementpb.Instance instances = 1;
  int32 num_shards = 2;
  int32 replication_factor = 3;
+
+  // M3Agg placements only


Maybe we should place these together, i.e.

message M3AggregatorPlacementOptions { uint64 max_aggregation_window_size_nanos = 1; uint64 warmup_duration_nanos = 2; } message PlacementInitRequest { repeated placementpb.Instance instances = 1; int32 num_shards = 2; int32 replication_factor = 3; M3AggregatorPlacementOptions aggregator_options = 4; } message PlacementAddRequest { repeated placementpb.Instance instances = 1; bool force = 2; M3AggregatorPlacementOptions aggregator_options = 4; }

Any way we can avoid passing M3AggregatorPlacementOptions in request? I think it can be configured in the handler

The problem is that we can only guess at the correct value. They may not have all the namespaces configured on this coordinator, or the endpoints may be embedded in a different service that doesn't have the info....lets talk offline

cw9 · 2018-10-07T22:42:25Z

src/query/api/v1/handler/placement/add.go

+		}
+		serviceOpts.M3Agg = &M3AggServiceOptions{
+			MaxAggregationWindowSize: time.Duration(req.MaxAggregationWindowSizeNanos),
+			WarmupDuration:           time.Duration(req.WarmupDurationNanos),


Could these be configured instead of request based? I don't think they should change per request

cw9 · 2018-10-07T22:44:58Z

src/query/api/v1/handler/placement/common.go

+		SetValidZone(opts.ServiceZone).
+		SetIsSharded(true).
+		// Can use goal-based placement for both M3DB and M3Agg
+		SetIsStaged(false).


We can't use goal-based for m3agg yet, m3agg expect the protobuf to be stored as staged, so it parses the proto differently than m3db

cw9 · 2018-10-07T22:45:52Z

src/query/api/v1/handler/placement/common.go

+		)
+		pOpts = pOpts.
+			SetIsMirrored(true).
+			// TODO(rartoul): Do we need to set placement cutover time? Seems like that would


Using immediate for placement cutover is fine, but I'd rather we copy all the logic from r2admin here to keep them in sync.
For example, the placementCutoff time is calculated as

now. Add(h.cfg.Placement.Clock.MaxPositiveSkew). Add(h.cfg.Placement.Clock.MaxNegativeSkew). Add(h.cfg.PropagationDelay)

in r2admin, we could just set all those three values as 0 to achieve immediate placement cutover.
Keeping the logic the same makes debugging much simpler and I could see us migrating r2admin/m3admin to call these endpoints here if we just run them with production configs.

cw9 · 2018-10-07T23:02:37Z

src/query/api/v1/handler/placement/common.go

+	return func(s shard.Shard) error {
+		switch s.State() {
+		case shard.Initializing:
+			if s.CutoverNanos() > now.UnixNano() {


This is expected, this lambda is called before marking an intializing shard as available, so we don't allow users to mark the shard if now is before its CutoverNanos

cw9 · 2018-10-07T23:04:02Z

src/query/api/v1/handler/placement/common.go

+		case shard.Leaving:
+			// TODO(rartoul): This seems overly cautious, basically it requires an entire maxAggregationWindowSize
+			// to elapse before "leaving" shards can be cleaned up.
+			if s.CutoffNanos() > now.UnixNano()-maxAggregationWindowSize.Nanoseconds() {


similar to cutover logic, this is called to when a leaving shard is being removed(when marking its corresponding initializing shard), this logic checks if the time has passed this shard's CutoffNanos

cw9 · 2018-10-07T23:04:42Z

src/query/api/v1/handler/placement/common.go

+		case shard.Leaving:
+			// TODO(rartoul): This seems overly cautious, basically it requires an entire maxAggregationWindowSize
+			// to elapse before "leaving" shards can be cleaned up.
+			if s.CutoffNanos() > now.UnixNano()-maxAggregationWindowSize.Nanoseconds() {


To answer Richie's TODO question, yes it's overly cautious, but let's just keep the logic the same as r2admin for now.

cw9 · 2018-10-07T23:05:56Z

src/query/generated/proto/admin/placement.proto

@@ -8,6 +8,10 @@ message PlacementInitRequest {
  repeated placementpb.Instance instances = 1;
  int32 num_shards = 2;
  int32 replication_factor = 3;
+
+  // M3Agg placements only


Any way we can avoid passing M3AggregatorPlacementOptions in request? I think it can be configured in the handler

cw9 · 2018-10-08T19:09:17Z

src/query/api/v1/handler/placement/add.go

+
+	// M3DBAddURL is the url for the placement add handler (with the POST method)
+	// for the M3DB service.
+	M3DBAddURL = handler.RoutePrefixV1 + "/services/m3db/placement"


Is it better to write:
M3DBAddURL = handler.RoutePrefixV1 + "/services/" + M3DBServiceName + "/placement"
in case you update the service in one place and forgot the others.

Same goes to the m3agg url as well. Also nit: use m3aggregator instead of m3agg everywhere?

cw9 · 2018-10-08T19:14:49Z

src/query/api/v1/handler/placement/common.go

+		SetValidZone(opts.ServiceZone).
+		SetIsSharded(true).
+		// M3Agg expects a staged placement.
+		SetIsStaged(true).


Is this shared by m3db as well? should use isStaged=false for m3db and isStaged=true for m3agg?

not shared, this is inside a if m3agg switch statement

oh no its not you're right, will fix

cw9 · 2018-10-08T19:17:19Z

src/query/api/v1/handler/placement/common.go

-	r.HandleFunc(DeleteAllURL, logged(NewDeleteAllHandler(client, cfg)).ServeHTTP).Methods(DeleteAllHTTPMethod)
-	r.HandleFunc(AddURL, logged(NewAddHandler(client, cfg)).ServeHTTP).Methods(AddHTTPMethod)
-	r.HandleFunc(DeleteURL, logged(NewDeleteHandler(client, cfg)).ServeHTTP).Methods(DeleteHTTPMethod)
+type placementCutoverOpts struct {


nit: rename to m3aggregatorPlacementOpts?

cw9 · 2018-10-08T19:20:08Z

src/query/api/v1/handler/placement/common.go

+// this by adding the warmup time and the max aggregation window size to the current time, and then truncating
+// to the max aggregation window size. This ensure that we always return a time that is at the beginning of an
+// aggregation window size, but is also later than now.Add(warmup).
+func newShardCutOffNanosFn(now time.Time, maxAggregationWindowSize, warmup time.Duration) placement.TimeNanosFn {


just call shardCutoverNanos like r2admin?

cw9 · 2018-10-08T19:22:12Z

src/query/api/v1/handler/placement/add.go

+
+	// M3DBAddURL is the url for the placement add handler (with the POST method)
+	// for the M3DB service.
+	M3DBAddURL = handler.RoutePrefixV1 + "/services/m3db/placement"


Also consider making a const for "/services/" + M3DBServiceName + "/placement", I see that string being reused in a lot of places.

robskillington · 2018-10-09T14:44:01Z

src/query/api/v1/handler/placement/delete_all.go

+
+	// M3DBDeleteAllURL is the url for the handler to delete all placements (with the DELETE method)
+	// for the M3DB service.
+	M3DBDeleteAllURL = handler.RoutePrefixV1 + "/services/m3db/placement"


This could be reused from somewhere yeah? "/services/m3db/placement" is rewritten a fair bit.

richardartoul changed the title ~~Add APIs for M3Aggregator placement~~ [WIP - Dont Review] Add APIs for M3Aggregator placement Oct 6, 2018

cw9 reviewed Oct 6, 2018

View reviewed changes

richardartoul changed the title ~~[WIP - Dont Review] Add APIs for M3Aggregator placement~~ [WIP] Add APIs for M3Aggregator placement Oct 7, 2018

richardartoul commented Oct 7, 2018

View reviewed changes

richardartoul requested review from robskillington and prateek October 7, 2018 18:48

richardartoul changed the title ~~[WIP] Add APIs for M3Aggregator placement~~ Add APIs for M3Aggregator placement Oct 7, 2018

richardartoul commented Oct 7, 2018

View reviewed changes

robskillington reviewed Oct 7, 2018

View reviewed changes

cw9 reviewed Oct 7, 2018

View reviewed changes

richardartoul force-pushed the ra/agg-placement branch from acb4174 to 06f779c Compare October 8, 2018 17:08

cw9 reviewed Oct 8, 2018

View reviewed changes

robskillington reviewed Oct 9, 2018

View reviewed changes

Richard Artoul added 27 commits October 9, 2018 13:12

use headers

3ecd412

more refactoring

98c376a

Refactor all tests to run for all services

300b3d5

Allow setting Max aggregation window size and warmup

0731b8f

More tests

0725299

Update spec.yml

aa24e27

regen assets

a6017a8

Add dry run header

d9664eb

run common tests for all services

cf7a834

remove unnecessary conversions

84e51b2

address review feedback

faa788f

refactor

10f7709

Regen assets

a14033a

lowercase var name

1018ef6

More refactoring;

abd2401

more refactoring

781f120

Regen assets

5bef3ae

fix query

7de2606

update comment

ed2645d

remove unused interface

b282808

Fix broken tests

1c6dd4d

Refactor based on feedback

3cd74f8

Refactor based on feedback

45b6077

Fix test

29c5de0

Fix lint issues

1f342f1

Add nil check

06dafdb

Fix URL init

570fc35

richardartoul force-pushed the ra/agg-placement branch from d4093d6 to 570fc35 Compare October 9, 2018 17:13

richardartoul merged commit 1fc18ec into master Oct 9, 2018

justinjc deleted the ra/agg-placement branch January 22, 2019 15:50


		const placementJSON = "{\"placement\":{\"instances\":{\"host1\":{\"id\":\"host1\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host1:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host1\",\"port\":1234},\"host2\":{\"id\":\"host2\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host2:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host2\",\"port\":1234}},\"replicaFactor\":0,\"numShards\":0,\"isSharded\":false,\"cutoverTime\":\"0\",\"isMirrored\":false,\"maxShardSetId\":0},\"version\":%d}"
		const placementJSON = "{\"placement\":{\"instances\":{\"host1\":{\"id\":\"host1\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host1:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host1\",\"port\":1234},\"host2\":{\"id\":\"host2\",\"isolationGroup\":\"rack1\",\"zone\":\"test\",\"weight\":1,\"endpoint\":\"http://host2:1234\",\"shards\":[],\"shardSetId\":0,\"hostname\":\"host2\",\"port\":1234}},\"replicaFactor\":0,\"numShards\":0,\"isSharded\":false,\"cutoverTime\":\"0\",\"isMirrored\":false,\"maxShardSetId\":0},\"version\":%d}"

Add APIs for M3Aggregator placement #1030

Add APIs for M3Aggregator placement #1030

Conversation

richardartoul commented Oct 6, 2018 • edited Loading

codecov bot commented Oct 6, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardartoul commented Oct 6, 2018 •

edited

Loading

codecov bot commented Oct 6, 2018 •

edited

Loading