ft: ZENKO-583 get CRR status #331

philipyoo · 2018-06-25T21:19:37Z

Check if CRR is enabled for a given destination replication location via the API.
Save CRR status state within zookeeper. If pod goes down, read and resume from previous state

ironman-machine · 2018-06-25T21:22:53Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-26T01:35:05Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-26T23:32:09Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T00:25:27Z

PR has been updated. Reviewers, please be cautious.

jonathan-gramain · 2018-06-27T00:04:06Z

docs/crr-pause-resume.md

-When the paused Consumer is resumed, it will again resume consuming entries
-from its last offset.
+QueueProcessors will subscribe to this channel, and on receiving a request
+to pause or resume CRR, will notify all of its BackbeatConsumers to perform the


all of their BackbeatConsumers

jonathan-gramain · 2018-06-27T00:08:42Z

docs/crr-pause-resume.md

+    all sites configured as destination replication endpoints.
+
+    Response:
+    ```sh


I think you should use json not sh.

jonathan-gramain · 2018-06-27T00:11:11Z

docs/crr-pause-resume.md

+    all locations configured as destination replication endpoints.
+
+    Response:
+    ```sh


I think you should use json not sh.

jonathan-gramain · 2018-06-27T00:11:30Z

docs/crr-pause-resume.md

+    Response:
+    ```sh
+    {
+        location1: "disabled",


Since the output shall be JSON, let's add double-quotes to the keys like "location1"

jonathan-gramain · 2018-06-27T00:12:15Z

docs/crr-pause-resume.md

+    Response:
+    ```sh
+    {
+        location: "enabled",


May be better to stick with the name that appears in the syntax line: "<location-name>": "enabled"

jonathan-gramain · 2018-06-27T00:45:11Z

extensions/replication/queueProcessor/task.js

+        if (err && err.name === 'NODE_EXISTS') {
+            zkClient.getData(path, (err, data) => {
+                if (err) {
+                    log.fatal('could not check check site status in zookeeper',


double check check

jonathan-gramain · 2018-06-27T00:46:02Z

extensions/replication/queueProcessor/task.js

+        zkClient.connect();
+        zkClient.once('error', err => {
+            log.fatal('error connecting to zookeeper', {
+                error: err,


error: err.message

jonathan-gramain · 2018-06-27T00:46:18Z

extensions/replication/queueProcessor/task.js

+                    log.fatal('could not create path in zookeeper', {
+                        method: 'QueueProcessor:task',
+                        zookeeperPath: path,
+                        error: err,


jonathan-gramain · 2018-06-27T00:49:38Z

lib/BackbeatConsumer.js

+        if (!disableConsumer) {
+            this._consumer.subscribe([this._topic]);
+        } else {
+            this._log.info(`consumer not subcribed to topic ${this._topic}`);


At this level I would rather log in debug mode, as the message is likely obscure for a normal user. Such message would better fit at the service level (stating that it's paused, for example).

jonathan-gramain · 2018-06-27T00:51:22Z

lib/api/BackbeatAPI.js

@@ -236,7 +239,9 @@ class BackbeatAPI {
    getHealthcheck(details, cb) {
        return this._healthcheck.getHealthcheck((err, data) => {
            if (err) {
-                this._logger.error('error getting healthcheck', err);
+                this._logger.error('error getting healthcheck', {
+                    error: err,


error: err.message

oops, this is my mistake, I want to pass err here as is because its a unique object returned from the Healthcheck class. I'll look into figuring out a better way of implementing this in a future refactor.
Change is reverted

ironman-machine · 2018-06-27T00:55:32Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T14:38:14Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T14:38:14Z

CONFLICT (add/add): Merge conflict in docs/crr-pause-resume.md

ironman-machine · 2018-06-27T15:36:25Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T15:52:47Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T15:58:31Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T16:11:47Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-27T16:11:48Z

CONFLICT (add/add): Merge conflict in docs/crr-pause-resume.md

ironman-machine · 2018-06-27T16:19:26Z

PR has been updated. Reviewers, please be cautious.

bert-e · 2018-06-27T18:27:48Z

Hello philipyoo,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get
information on this process.

Status report is not available.

bert-e · 2018-06-27T18:27:50Z

Conflict

There is a conflict between your branch feature/ZENKO-583-getCRRStatus and the
destination branch development/8.0.

Please resolve the conflict on the feature branch (feature/ZENKO-583-getCRRStatus).

#!bash
 $ git fetch
 $ git checkout origin/feature/ZENKO-583-getCRRStatus
 $ git merge origin/development/8.0
 $ # <intense conflict resolution>
 $ git push origin HEAD:feature/ZENKO-583-getCRRStatus

ironman-machine · 2018-06-28T00:24:10Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-28T00:30:14Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-28T17:50:52Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-28T22:12:31Z

PR has been updated. Reviewers, please be cautious.

bert-e · 2018-06-29T23:42:59Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/8.0

The following branches will NOT be impacted:

development/7.4

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of Release Engineering will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of Release Engineering now.

philipyoo · 2018-06-30T01:32:34Z

bert-e help

philipyoo · 2018-06-30T01:32:43Z

@bert-e help

bert-e · 2018-06-30T01:32:44Z

Help page

The following options and commands are available at this time.

Options

name	description	privileged
➡️ bypass_jira_check	Bypass the Jira issue check	⭐
➡️ after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
➡️ bypass_author_approval	Bypass the pull request author's approval	⭐
➡️ no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
➡️ unanimity	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
➡️ bypass_tester_approval	Bypass the pull request testers' approval	⭐
➡️ bypass_peer_approval	Bypass the pull request peers' approval	⭐
➡️ bypass_commit_size	Bypass the check on the size of the changeset `TBA`	⭐
➡️ bypass_build_status	Bypass the build and test status	⭐
➡️ bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
➡️ create_pull_requests	Allow the creation of integration pull requests.
➡️ wait	Instruct Bert-E not to run until further notice.

Commands

name	description	privileged
➡️ help	Print Bert-E's manual in the pull request.
➡️ build	Re-start a fresh build `TBA`
➡️ retry	Re-start a fresh build `TBA`
➡️ clear	Remove all comments from Bert-E from the history `TBA`
➡️ reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.
➡️ status	Print Bert-E's current status in the pull request `TBA`
➡️ force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.

Do it again human slave!:point_right: :runner: (Oh and the pull request has been updated, by the way.)

ironman-machine · 2018-06-30T02:19:49Z

PR has been updated. Reviewers, please be cautious.

ironman-machine · 2018-06-30T02:19:49Z

CONFLICT (add/add): Merge conflict in extensions/lifecycle/management.js
CONFLICT (add/add): Merge conflict in docs/crr-pause-resume.md

ironman-machine · 2018-06-30T02:27:09Z

PR has been updated. Reviewers, please be cautious.

rahulreddy · 2018-06-30T08:36:54Z

@bert-e reset

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 0da2585 to 61c09d3 Compare June 25, 2018 21:22

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 61c09d3 to 8d14038 Compare June 26, 2018 01:35

philipyoo force-pushed the feature/ZENKO-235-manualCRRPauseResume branch from 9dc16cf to cd8ca97 Compare June 26, 2018 23:31

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 8d14038 to ccb0c63 Compare June 26, 2018 23:32

philipyoo force-pushed the feature/ZENKO-235-manualCRRPauseResume branch 2 times, most recently from 5a75051 to b8502e2 Compare June 27, 2018 00:14

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from ccb0c63 to c817a92 Compare June 27, 2018 00:25

philipyoo force-pushed the feature/ZENKO-235-manualCRRPauseResume branch 2 times, most recently from 58fbbe4 to f288387 Compare June 27, 2018 00:51

jonathan-gramain requested changes Jun 27, 2018

View reviewed changes

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from c817a92 to cdcc885 Compare June 27, 2018 00:55

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from ee01f6d to 0bb1482 Compare June 27, 2018 15:52

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 0bb1482 to 51155a7 Compare June 27, 2018 15:58

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from c3f8525 to 6896d6b Compare June 27, 2018 16:19

philipyoo changed the base branch from feature/ZENKO-235-manualCRRPauseResume to development/8.0 June 27, 2018 18:27

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from d381aa7 to 5a3349e Compare June 28, 2018 00:24

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 5a3349e to 2155857 Compare June 28, 2018 00:30

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 2155857 to 2d3a30f Compare June 28, 2018 17:50

ft: ZENKO-583 add crr status check docs

3f2ab8f

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from 2d3a30f to 74d91ae Compare June 28, 2018 22:12

alexanderchan-scality previously approved these changes Jun 29, 2018

View reviewed changes

rahulreddy added the high priority label Jun 29, 2018

jonathan-gramain previously approved these changes Jun 29, 2018

View reviewed changes

ft: ZENKO-583 add status check in processor, api

dcd1b14

philipyoo force-pushed the feature/ZENKO-583-getCRRStatus branch from d007e52 to dcd1b14 Compare June 30, 2018 02:27

jonathan-gramain approved these changes Jun 30, 2018

View reviewed changes

alexanderchan-scality approved these changes Jun 30, 2018

View reviewed changes

jonathan-gramain merged commit 0b2cc21 into development/8.0 Jun 30, 2018

rahulreddy deleted the feature/ZENKO-583-getCRRStatus branch June 30, 2018 08:36

rahulreddy restored the feature/ZENKO-583-getCRRStatus branch June 30, 2018 08:36

rahulreddy deleted the feature/ZENKO-583-getCRRStatus branch June 30, 2018 08:39

ft: ZENKO-583 get CRR status #331

ft: ZENKO-583 get CRR status #331

Conversation

philipyoo commented Jun 25, 2018

ironman-machine commented Jun 25, 2018

ironman-machine commented Jun 26, 2018

ironman-machine commented Jun 26, 2018

ironman-machine commented Jun 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philipyoo Jun 27, 2018 • edited Loading

Choose a reason for hiding this comment

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

ironman-machine commented Jun 27, 2018

bert-e commented Jun 27, 2018

Hello philipyoo,

bert-e commented Jun 27, 2018

Conflict

ironman-machine commented Jun 28, 2018

ironman-machine commented Jun 28, 2018

ironman-machine commented Jun 28, 2018

ironman-machine commented Jun 28, 2018

bert-e commented Jun 29, 2018

In the queue

philipyoo commented Jun 30, 2018

philipyoo commented Jun 30, 2018

bert-e commented Jun 30, 2018

Help page

Options

Commands

ironman-machine commented Jun 30, 2018

ironman-machine commented Jun 30, 2018

ironman-machine commented Jun 30, 2018

rahulreddy commented Jun 30, 2018

philipyoo Jun 27, 2018 •

edited

Loading