Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools, toc: reorganize TiDB Binlog documents #1264

Merged
merged 113 commits into from
Apr 29, 2019
Merged

tools, toc: reorganize TiDB Binlog documents #1264

merged 113 commits into from
Apr 29, 2019

Conversation

ericsyh
Copy link
Contributor

@ericsyh ericsyh commented Apr 23, 2019

What's PR background.

The current tidb-binlog put everything in one page, which is not so friendly to users.

What had done in this PR.

I split the previous documents into the overview, deploy, upgrade, operation and monitor 5 parts.

To-do List

  • Add FAQ troubleshooting documents for Binlog
  • Binlog cluster scale up/down documents

@ericsyh ericsyh requested a review from lilin90 April 23, 2019 08:47
@ericsyh ericsyh changed the title Tidb binlog documents reorg TiDB Binlog documents reorg Apr 23, 2019
@lilin90 lilin90 changed the title TiDB Binlog documents reorg tools, toc: reorganize TiDB Binlog documents Apr 23, 2019
@lilin90 lilin90 added the type/refactor The issue belongs to a refactor work. label Apr 23, 2019
@lilin90 lilin90 requested a review from dcalvin April 23, 2019 11:34
@ericsyh ericsyh mentioned this pull request Apr 24, 2019
Merged
IANTHEREAL
IANTHEREAL reviewed Apr 24, 2019
View reviewed changes
tools/binlog/deploy.md Outdated Show resolved Hide resolved
lilin90
lilin90 reviewed Apr 25, 2019
View reviewed changes
tools/binlog/overview.md Outdated Show resolved Hide resolved
tools/binlog/overview.md Outdated Show resolved Hide resolved
tools/binlog/overview.md Show resolved Hide resolved
tools/binlog/deploy.md Outdated Show resolved Hide resolved
tools/binlog/deploy.md Outdated Show resolved Hide resolved
tools/binlog/deploy.md Show resolved Hide resolved
TOC.md Show resolved Hide resolved
TOC.md Outdated Show resolved Hide resolved
lilin90 and others added 2 commits April 25, 2019 13:10
@ericsyh
Copy link
Contributor Author

ericsyh commented Apr 29, 2019

@GregoryIan @WangXiangUSTC TiDB-Binlog docs-cn all updated.

lilin90
lilin90 reviewed Apr 29, 2019
View reviewed changes
tools/binlog/overview.md Outdated Show resolved Hide resolved
@IANTHEREAL
Copy link
Contributor

LGTM

lilin90
lilin90 reviewed Apr 29, 2019
View reviewed changes
tools/binlog/overview.md Outdated Show resolved Hide resolved
tools/binlog/overview.md Outdated Show resolved Hide resolved
tools/binlog/tidb-binlog-local.md Outdated Show resolved Hide resolved
tools/binlog/tidb-binlog-local.md Outdated Show resolved Hide resolved
tools/binlog/tidb-binlog-kafka.md Outdated Show resolved Hide resolved
tools/binlog/tidb-binlog-kafka.md Outdated Show resolved Hide resolved
tools/binlog/operation.md Outdated Show resolved Hide resolved
tools/binlog/operation.md Outdated Show resolved Hide resolved
tools/binlog/deploy.md Outdated Show resolved Hide resolved
tools/binlog/upgrade.md Outdated Show resolved Hide resolved
lilin90 and others added 11 commits April 29, 2019 13:57
@lilin90 @ericsyh
Update tools/binlog/overview.md
2e22175 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/overview.md
4c0b75a 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/tidb-binlog-local.md
b716fe8 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/tidb-binlog-local.md
27df2b2 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/tidb-binlog-kafka.md
fb8d50e 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/tidb-binlog-kafka.md
b68e3d0 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/operation.md
14c61dd 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/deploy.md
e43cdf0 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@lilin90 @ericsyh
Update tools/binlog/upgrade.md
628a023 
LGTM

Co-Authored-By: ericsyh <[email protected]>
@WangXiangUSTC
Update operation.md
d46a483
@lilin90 @ericsyh
Update tools/binlog/operation.md
ca04d33 
LGTM

Co-Authored-By: ericsyh <[email protected]>
lilin90
lilin90 approved these changes Apr 29, 2019
View reviewed changes
Copy link
Member

@lilin90 lilin90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ericsyh ericsyh merged commit d972560 into pingcap:master Apr 29, 2019
dcalvin
dcalvin reviewed Apr 29, 2019
View reviewed changes
tools/binlog/tidb-binlog-local.md Outdated

## TiDB-Binlog-local 简介

TiDB-Binlog 用于收集 TiDB 的 Binlog，并提供实时备份和同步功能的商业工具。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TiDB-Binlog 用于收集 TiDB 的 Binlog，并提供实时备份和同步功能的商业工具。
TiDB-Binlog 是用于收集 TiDB 的 Binlog，并提供实时备份和同步功能的商业工具。

tools/binlog/tidb-binlog-local.md

## TiDB-Binlog-local 架构

首先介绍 TiDB-Binlog 的整体架构。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
首先介绍 TiDB-Binlog 的整体架构。
下图为 TiDB-Binlog 的整体架构。

tools/binlog/tidb-binlog-local.md

TiDB-Binlog 集群主要分为两个组件：

#### Pump
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Pump

tools/binlog/upgrade.md
category: tools
---

## TiDB-Binlog Cluster 版本升级方法
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always start a doc with 1st level header

Suggested change
## TiDB-Binlog Cluster 版本升级方法
# TiDB-Binlog Cluster 版本升级方法

tools/binlog/upgrade.md
## TiDB-Binlog Cluster 版本升级方法

新版本的 TiDB（v2.0.8-binlog、v2.1.0-rc.5 及以上版本）不兼容 [Kafka 版本](../binlog/tidb-binlog-kafka.md)以及 [Local 版本](../binlog/tidb-binlog-local.md)的 TiDB-Binlog，集群升级到新版本后只能使用 Cluster 版本的 TiDB-Binlog。如果在升级前已经使用了 Kafka／Local 版本的 TiDB-Binlog，必须将其升级到 Cluster 版本。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## TiDB-Binlog 与 TiDB 版本对应关系

tools/binlog/upgrade.md
升级流程：

* 如果能接受重新导全量数据，则可以直接废弃老版本，按本文档部署。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

重新导入全量数据的升级过程？

tools/binlog/upgrade.md Outdated

如果返回的 `Synced` 为 true，则可以认为 Binlog 数据已经全部同步到了下游。
6. 启动新版本 Drainer；
7. 下线无用的老版本的 Pump、Drainer 以及依赖的 Kafka 和 Zookeeper。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
7. 下线无用的老版本的 Pump、Drainer 以及依赖的 Kafka 和 Zookeeper。
7. 下线老版本的 Pump、Drainer 以及依赖的 Kafka 和 Zookeeper。

tools/binlog/tidb-binlog-local.md

#### Pump

Pump 是一个守护进程，在每个 TiDB 的主机上后台运行。他的主要功能是实时记录 TiDB 产生的 Binlog 并顺序写入磁盘文件
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Pump 是一个守护进程，在每个 TiDB 的主机上后台运行。他的主要功能是实时记录 TiDB 产生的 Binlog 并顺序写入磁盘文件
- **Pump** 是一个守护进程，在每个 TiDB 的主机上后台运行。他的主要功能是实时记录 TiDB 产生的 Binlog 并顺序写入磁盘文件

tools/binlog/tidb-binlog-local.md

Pump 是一个守护进程，在每个 TiDB 的主机上后台运行。他的主要功能是实时记录 TiDB 产生的 Binlog 并顺序写入磁盘文件

#### Drainer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Drainer

tools/binlog/tidb-binlog-local.md

#### Drainer

Drainer 从各个 Pump 节点收集 Binlog，并按照在 TiDB 中事务的提交顺序转化为指定数据库兼容的 SQL 语句，最后同步到目的数据库或者写到顺序文件
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Drainer 从各个 Pump 节点收集 Binlog，并按照在 TiDB 中事务的提交顺序转化为指定数据库兼容的 SQL 语句，最后同步到目的数据库或者写到顺序文件
- **Drainer** 从各个 Pump 节点收集 Binlog，并按照在 TiDB 中事务的提交顺序转化为指定数据库兼容的 SQL 语句，最后同步到目的数据库或者写到顺序文件。

@ericsyh ericsyh deleted the tidb-binlog-reorg branch April 29, 2019 08:26
lilin90 added a commit that referenced this pull request May 13, 2019
@lilin90
*: test renaming deploy/configure/secure/monitor files (#1322)
863c552 
* sql: update tidb slow log operation

* update log

* update log

* modify the translation: requirements vs recommendations

* improve the translation

* dm: add description for task overview monitor

* dm: add description for instances' monitor

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* dm: address comments

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* v2.1/tools, v2.1/tools: update TiDB-Binlog (#1245)

v2.1/tools: update TiDB-Binlog

* fix incorrect IP address parsing and other inline code formatting

* *: move overview and features (#1253)

Via: pingcap/docs#1045

* TOC: update the quickstart link (#1255)

* sql: remove redundant content in privilege && user, tiny clean up

* Apply suggestions from code review

Co-Authored-By: dcalvin <[email protected]>

* Apply suggestions from code review

Co-Authored-By: dcalvin <[email protected]>

* Update tools/dm/monitor.md

Co-Authored-By: csuzhangxc <[email protected]>

* dm: address comments

* add digest declare

* update eg

* add stats eg

* update eg

* address comment and add more field

* TOC: covert docs-cn to DITA TOC (#1242)

* TOC: covert docs-cn to DITA TOC

* TOC: update wording

* TOC: refine wording

* toc: update description and a link

* toc: update wording

* toc: update a file name

* toc: remove the K8s link

* toc: fix a link about incremental replication

* toc: update a blog post link to Chinese version

* toc: fix a link and update wording

* toc: update key features and overview

* toc: update About TiDB

* add declare for v2.1.8

* address comment

* change the link of back-restore in tidb-binlog document

* address comment

* *: remove doc redundant space

* sql/slow_query: add pt-digest-query version

* tools: update a lightning download link

* address comment

* sql, TOC: update doc title (#1262)

* benchmark: update the Note format

* op-guide: update the Note format

* benchmark, op-guide: Update the Note format (#1273)

benchmark, op-guide: update the Note format

* *: update the Note format

* address comment

* *: update the Note format

* benchmark, v2.1/benchmark: remove an invalid link (#1275)

* benchmark: remove an invalid link

* v2.1/benchmark: remove an invalid link

* update condition (#1265)

* Update sql/slow-query.md

Co-Authored-By: crazycs520 <[email protected]>

* Update sql/slow-query.md

Co-Authored-By: crazycs520 <[email protected]>

* sql: update the USE command explanation in util document (#1269)

* docs-cn #1112 Update the USE command explanation in util document

* Update sql/util.md

LGTM

Co-Authored-By: ericsyh <[email protected]>

* Update v1.0/sql/util.md

LGTM

Co-Authored-By: ericsyh <[email protected]>

* Update v2.0/sql/util.md

LGTM

Co-Authored-By: ericsyh <[email protected]>

* Update v2.1/sql/util.md

LGTM

Co-Authored-By: ericsyh <[email protected]>

* update the loader pprof-addr default port (#1256)

* tools: clarify meaning of "N/A" (#1267)

* releases: correct the TiDB-Ansible version (#1280)

* sql: fix some format mistakes (#1277)

* Sql: fix some format mistakes

* Update sql/slow-query.md

Co-Authored-By: IzabelWang <[email protected]>

* Update sql/slow-query.md

Co-Authored-By: IzabelWang <[email protected]>

* Update sql/slow-query.md

Co-Authored-By: IzabelWang <[email protected]>

* Update op-guide/tidb-config-file.md

Co-Authored-By: IzabelWang <[email protected]>

* Update sql/slow-query.md

Co-Authored-By: IzabelWang <[email protected]>

* Remove unnecessary space

* Update slow-query.md

* Update slow-query.md

* sql: update generated columns docs

* address comment and add issue link

* *: rename how-to get started (#1281)

* *: rename how-to get started

Related PR: pingcap/docs#1046

* *: fix aliases link path

* sql: update content structure and wording (#1259)

* TOC: update a doc title (#1285)

* tools/lightning: fix a broken link (#1287)

* explain for update statement (#1284)

* tools, toc: reorganize TiDB Binlog documents (#1264)

* reorg the binlog documents

* update TOC format

* change the pd to file in overview

* update the ansible-deployment link

* rename the previous version of tidb-binlog to tidb-binlog-local

* Make the explaination of pausing and paused more clear

* change the order in TOC

* uncomment some necessary configs in kafka version

* add more explaination of paused and offline in overview

* fix the typo

* Add the Topic config in Kafka driver

* update the gc intro in deployment

* remove unnecessary step in deployment

* remove the compression in deploy document

* remove sql in the file dainer

* update the download link in deployment

* Time and size limits delete

* abandon enable-tolerant config

* move the reparo document into binlog folder

* update the rapero download link

* update drainer_pd to drainer_file

* trouble-shooting: update the issue report link (#1289)

* trouble-shooting: update the issue report link

* Update trouble-shooting.md

Co-Authored-By: CaitinChen <[email protected]>

* Update trouble-shooting.md

Co-Authored-By: CaitinChen <[email protected]>

* Update trouble-shooting.md

Co-Authored-By: CaitinChen <[email protected]>

* tools/binlog: optimize the TiDB-Binlog documents (#1291)

* fix some problems with suggestions from Lilian and Calvin

* tools/binlog: fix and update format (#1292)

* tools/binlog: fix and update format

* tools/binlog: update wording

* tools/binlog: update a link

* Update tools/binlog/upgrade.md

Co-Authored-By: lilin90 <[email protected]>

* tools: update syncer batch size (#1288)

* tools/dm: fix the error in valus from ansible command (#1293)

* sql: fix punctuation and format

* circle.yml: [WIP] comment publish pdf cmd

* TOC, releases: add 2.1.9 release notes (#1298)

* TOC, releases: add 2.1.9 release notes

PTAL

* Update 2.1.9.md

* releases: fix format

* tools: update loader status addr (#1299)

* Add charset check in Syncer precheck (#1300)

* Add charset check in Syncer precheck

* Update tools/syncer.md

Co-Authored-By: ericsyh <[email protected]>

* circle.yml: remove IP address (#1301)

* *: change "登陆" to “登录” (#1302)

* sql: fix format (#1303)

* sql: update a comma (#1305)

* sql: replace "jeffrey" with "test" (#1309)

* op-guide/monitor: update the PD API DOC link (#1311)

* Update the link

* Update the PD API DOC link

* tools: fix the format error in Syncer (#1306)

* Fix the format error in Syncer

* Update tools/syncer.md

Co-Authored-By: ericsyh <[email protected]>

* update a link (#1313)

* op-guide: add a guide about upgrading to 3.0 (#1258)

* Add rolling update TiDB 3.0 guide.

* Add rolling update TiDB 3.0 guide.

* Apply suggestions from code review

Co-Authored-By: superlzs0476 <[email protected]>

* Apply suggestions from code review

* Apply suggestions from code review

Co-Authored-By: liubo <[email protected]>

* op-guide: update a note format

* op-guide: update wording

* op-guide: update format

* circle.yml: add a new IP for qiniu (#1315)

* sql: fix typo (#1286)

* media, tools: update TiDB-Lightning architecture

Via: pingcap/docs#1128

* Delete useless instruction (#1308)

* TOC, releases: add TiDB 3.0.0 RC1 release notes (#1316)

* TOC, releases: add TiDB 3.0.0 RC1 release notes

PTAL

* Fix format

* add new updates

* Update TOC.md

* Update rn.md

* op-guide, toc: fix upgrade wording and update TOC (#1318)

* op-guide, tools: fix a link and remove duplicated file (#1319)

* *: update version limit for DM/Syncer (#1312)

* toc, support: reorganize TOC and update a link (#1320)

* toc, support: reorganize TOC and update a link

Related PR: #1279

* toc: udpate description

* toc: update wording

* *: rename how-to deploy/configure/secure/monitor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@IANTHEREAL IANTHEREAL IANTHEREAL left review comments

@WangXiangUSTC WangXiangUSTC WangXiangUSTC left review comments

@csuzhangxc csuzhangxc csuzhangxc left review comments

@dcalvin dcalvin dcalvin left review comments

@july2993 july2993 july2993 left review comments

@lilin90 lilin90 lilin90 approved these changes

Assignees
No one assigned
Labels
type/refactor The issue belongs to a refactor work.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@ericsyh @IANTHEREAL @csuzhangxc @july2993 @dcalvin @WangXiangUSTC @lilin90