zh: add document about how to do performance test #85

WangXiangUSTC · 2020-05-20T05:03:17Z

What is changed, added or deleted? (Required)

add document about how to do the performance test, most content is copy from https://pingcap.com/docs-cn/tidb-data-migration/stable/benchmark-v1.0-ga/ and then update

Which TiDB Data Migration (DM) version(s) do your changes apply to? (Required)

master (the latest development version, including v2.0 changes for now)
v1.0 (TiDB DM 1.0 versions)
Other reference link(s):

csuzhangxc · 2020-05-20T06:27:36Z

most content is copy from https://pingcap.com/docs-cn/tidb-data-migration/stable/benchmark-v1.0-ga/

@WangXiangUSTC 我们更新一下 benchmark-v1.0-ga 吗？（比如删除里面一部分内容、然后 link 到这个文档之类的？）

WangXiangUSTC · 2020-05-20T07:54:17Z

most content is copy from https://pingcap.com/docs-cn/tidb-data-migration/stable/benchmark-v1.0-ga/

@WangXiangUSTC 我们更新一下 benchmark-v1.0-ga 吗？（比如删除里面一部分内容、然后 link 到这个文档之类的？）

可以的，我在这个 pr 里更新下

lichunzhu · 2020-05-20T11:24:10Z

zh/performance-test.md

+
+## 部署测试环境
+
+- 使用 TiDB Ansible 部署 TiDB 测试集群，所有配置使用 TiDB Ansible 提供的默认配置。


是不是让用户拿 tiup 部署比较好，ansible 4.0 后已经不是官方文档推荐的方式了，贴上 tiup 部署的 link

tiup 部署 dm 的文档现在还没有，等后面有了再改下这部分内容吧

这句话不是让用户用 Ansible 部署 TiDB 集群吗？TiUP 已经支持 TiDB 了吧

对，改成 tiup 了

lichunzhu · 2020-05-20T11:26:01Z

zh/performance-test.md

+
+### 获取测试结果
+
+通过 `query-status` 命令观测 DM 的同步状态，通过 Grafana 观测 DM 的监控指标。主要包括同步延迟 `replicate lag`，单位时间内完成的 job 数量 `finished sqls jobs` 等。


需要说一下怎么弄 Grafana 吗？

replicate lag 在这个任务配置里应该看不到，因为没有启用 heartbeat。

更新了配置，加上了 enable-heartbeat

需要说一下怎么弄 Grafana 吗？

暂时还没有专门介绍怎么部署 grafana 的文档，都是 ansible 自动部署上的，后面有的话再加个链接吧

zh/performance-test.md

…ocs-dm into xiang/performance_test

zh/benchmark-v1.0-ga.md

zh/performance-test.md

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

csuzhangxc

rest LGTM

zh/performance-test.md

miaoqingli · 2020-05-22T05:53:50Z

zh/performance-test.md

+
+#### 创建数据同步任务
+
+1. 创建上游 MySQL 的 source，将 `source-id` 配置为 `source-1`。


感觉可以增加一个新增 source 方法的链接

加了链接

miaoqingli · 2020-05-22T05:55:13Z

zh/performance-test.md

+
+#### 获取测试结果
+
+通过 `query-status` 命令观测 DM 的同步状态，通过 Grafana 观测 DM 的监控指标。主要包括同步延迟 `replicate lag`，单位时间内完成的 job 数量 `finished sqls jobs` 等。


增加性能定位的文章链接

加了链接

miaoqingli · 2020-05-22T08:06:09Z

LGTM

toutdesuite

Rest LGTM

zh/benchmark-v1.0-ga.md

toutdesuite · 2020-05-22T09:27:24Z

zh/performance-test.md

+
+#### 创建数据同步任务
+
+1. 创建上游 MySQL 的 source，将 `source-id` 配置为 `source-1`。详细操作方法参考：[加载数据源配置](manage-replication-tasks.md#加载数据源配置)。


加载数据源配置这篇文档已经删除了，可以看看链接到这篇是否合适

好的，等那个 pr 合并了我再更新这里

@WangXiangUSTC #90 has been merged, please update links and resolve conflict~

链接改了

toutdesuite · 2020-05-22T09:30:28Z

zh/performance-test.md

+      threads: 32
+  ```
+
+创建数据同步任务的详细操作参考[创建数据同步任务]((manage-replication-tasks.md#创建数据同步任务)。


这里链接到这篇是否合适

链接改了

toutdesuite · 2020-05-22T09:32:34Z

zh/performance-test.md

+
+#### 创建数据同步任务
+
+1. 创建上游 MySQL 的 source, source-id 配置为 `source-1`。（如果在全量同步性能测试中已经创建，则不需要再次创建）。详细操作方法参考：[加载数据源配置](manage-replication-tasks.md#加载数据源配置)。


这里链接也要改，同理下面涉及到的相关链接也要改~

链接改了

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

WangXiangUSTC · 2020-05-25T13:07:59Z

@toutdesuite PTAL again

zh/performance-test.md

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

zh/performance-test.md

* scripts: add a script to run markdownlint check locally (#74) * Update ci.yaml (#84) * zh: update TOC (#75) * update TOC * Update TOC.md * Update TOC.md * Update zh/TOC.md Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> * Update zh/TOC.md Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> * address comments * fix dead link and address comment * Apply suggestions from code review * fix levels * Update links in TOC * fix CI * remove link for 测试验证 Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> Co-authored-by: yikeke <yikeke@pingcap.com> * zh: add the requirement of software and hardware (#76) * add requirement * Apply suggestions from code review Co-authored-by: toutdesuite <guizhiluo2014@163.com> * update word * address comemnt * Apply suggestions from code review Co-authored-by: Lilian Lee <lilin@pingcap.com> * update disk requirement * Apply suggestions from code review Co-authored-by: toutdesuite <guizhiluo2014@163.com> * rename file name * Apply suggestions from code review Co-authored-by: Lilian Lee <lilin@pingcap.com> * address comment Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: Lilian Lee <lilin@pingcap.com> Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> * zh: update documents about error handling (#78) * zh: add a document about performance handling (#81) * zh: add document about creating task and verify (#77) * zh: add quick start doc for dm deployment (#80) * zh: update docs about key features (#86) * zh: add a document about how to optimize config (#89) * add config optimize * rename file * Apply suggestions from code review Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com> * format * address comment * Apply suggestions from code review Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Apply suggestions from code review Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/tune-configuration.md * Apply suggestions from code review Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/tune-configuration.md Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com> Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: Lilian Lee <lilin@pingcap.com> * zh: Further update TOC.md (#92) * Update TOC.md * add omitted links * update format * Update TOC.md * Update TOC.md * Update zh/TOC.md Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> * update file name * add alias * fix dead links * Revert "fix dead links" This reverts commit 9cf7b75. * fix dead links in zh Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> * zh: update dm-upgrade (#87) * zh: add a document about scenarios (#88) * zh: update daily-check.md (#91) * update daily-check.md * address comment * address comment * Update zh/daily-check.md * Update zh/daily-check.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/daily-check.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/daily-check.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: WangXiangUSTC <wx347249478@gmail.com> * zh: update docs about alert rules and handling (#93) * zh: add a doc about command-line-flags and update TOC (#94) * zh: add a doc about command-line-flags and update TOC * zh: update TOC * command-line-flags: address comments * Update zh/command-line-flags.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/command-line-flags.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * Update zh/command-line-flags.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * command-line-flags: address comments * remove empty files Co-authored-by: toutdesuite <guizhiluo2014@163.com> * zh: add files about manage task (#90) * add task manage for special week * minor change * rename file * rename file * minor change * minor change * fix CI * fix CI * address comment * minor change * Update skip-or-replace-abnormal-sql-statements.md * Update zh/create-task.md Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com> * address comment * address comment * update TOC * address comment * Update TOC.md * add query-status and query-error in TOC * fix toc * Update zh/query-error.md Co-authored-by: Lilian Lee <lilin@pingcap.com> * Update zh/query-status.md Co-authored-by: Lilian Lee <lilin@pingcap.com> * rm manage-replication-tasks.md * fix CI * fix CI * address comment * fix bug * update TOC * add dmctl-intro.md * Update zh/dmctl-intro.md Co-authored-by: toutdesuite <guizhiluo2014@163.com> * rename file * Update TOC.md * address comment Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com> Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: Lilian Lee <lilin@pingcap.com> Co-authored-by: WangXiangUSTC <wx347249478@gmail.com> * zh: add document about how to do performance test (#85) * zh: update TOC and remove empty files (#95) * update TOC and remove empty files * Update TOC.md Co-authored-by: yikeke <yikeke@pingcap.com> * zh: update release notes in toc * zh: update wording * *: make image name unique at the repo level (#96) * *: make image name unique at repo level * en: fix one link Co-authored-by: Ran <huangran@pingcap.com> Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> Co-authored-by: toutdesuite <guizhiluo2014@163.com> Co-authored-by: yikeke <yikeke@pingcap.com> Co-authored-by: WangXiangUSTC <wx347249478@gmail.com> Co-authored-by: Xuecheng Zhang <csuzhangxc@gmail.com> Co-authored-by: Chunzhu Li <lichunzhu@stu.xjtu.edu.cn> Co-authored-by: GMHDBJD <35025882+GMHDBJD@users.noreply.github.com> Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com>

WangXiangUSTC added 2 commits May 19, 2020 17:57

add performance test file

2900df6

update

5115714

WangXiangUSTC added the special-week PR from Document Special Week. label May 20, 2020

TomShawn requested review from csuzhangxc, IANTHEREAL, kissmydb and lilin90 May 20, 2020 05:10

minor update

97fe70b

TomShawn added the translation/welcome Waits for a contributor to translate this PR to docs-dm/zh or docs-dm/en. label May 20, 2020

WangXiangUSTC and others added 2 commits May 20, 2020 15:54

Merge branch 'docs-special-week' into xiang/performance_test

db05130

update benchmark test

45bb59f

lichunzhu reviewed May 20, 2020

View reviewed changes

yikeke reviewed May 20, 2020

View reviewed changes

zh/performance-test.md Show resolved Hide resolved

WangXiangUSTC and others added 4 commits May 21, 2020 11:16

Merge branch 'xiang/performance_test' of https://github.com/pingcap/d…

75a9a84

…ocs-dm into xiang/performance_test

update task config

c594ac5

Merge branch 'docs-special-week' into xiang/performance_test

0d15918

update toc

fcd670f

sre-bot added the status/PTAL This PR is ready for reviewing. label May 21, 2020

yikeke requested review from miaoqingli and removed request for kissmydb May 21, 2020 03:59

update ansible to tiup

2c36940

lilin90 requested review from toutdesuite and removed request for lilin90 May 21, 2020 07:27

toutdesuite reviewed May 21, 2020

View reviewed changes

WangXiangUSTC and others added 3 commits May 21, 2020 18:03

Apply suggestions from code review

bd1ab19

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

format

aad8ec4

minor fix

26a6e48

csuzhangxc reviewed May 22, 2020

View reviewed changes

miaoqingli reviewed May 22, 2020

View reviewed changes

toutdesuite added the status/require-change The author needs to address comments. label May 22, 2020

WangXiangUSTC added 2 commits May 22, 2020 15:45

add some link

ba47534

merge docs-special-week

4ebbc0e

miaoqingli approved these changes May 22, 2020

View reviewed changes

yikeke added translation/doing This PR’s assignee is translating this PR. and removed translation/welcome Waits for a contributor to translate this PR to docs-dm/zh or docs-dm/en. labels May 22, 2020

yikeke assigned toutdesuite May 22, 2020

toutdesuite reviewed May 22, 2020

View reviewed changes

WangXiangUSTC and others added 2 commits May 23, 2020 23:52

Apply suggestions from code review

c23fd25

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

merge docs-special-week and resolve conflicts

6776dc8

WangXiangUSTC added status/LGT2 Indicates that a PR has LGTM 2. and removed status/PTAL This PR is ready for reviewing. labels May 25, 2020

WangXiangUSTC added 3 commits May 25, 2020 20:50

merge and resolve conflicts

b9cf691

update link

50fb9f6

minor fix

aeb4fd5

WangXiangUSTC mentioned this pull request May 25, 2020

zh: update TOC and remove empty files #95

Merged

2 tasks

toutdesuite reviewed May 25, 2020

View reviewed changes

zh/performance-test.md Show resolved Hide resolved

zh/performance-test.md Outdated Show resolved Hide resolved

zh/performance-test.md Show resolved Hide resolved

Update zh/performance-test.md

3367cc7

Co-authored-by: toutdesuite <guizhiluo2014@163.com>

toutdesuite approved these changes May 26, 2020

View reviewed changes

zh/performance-test.md Outdated Show resolved Hide resolved

Update zh/performance-test.md

15ef5ef

WangXiangUSTC merged commit d451066 into docs-special-week May 26, 2020

WangXiangUSTC deleted the xiang/performance_test branch May 26, 2020 06:25

toutdesuite mentioned this pull request Jul 10, 2020

en: add document about how to do performance test #131

Merged

2 tasks

toutdesuite added translation/done This PR has been translated. and removed translation/doing This PR’s assignee is translating this PR. labels Jul 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

zh: add document about how to do performance test #85

zh: add document about how to do performance test #85

WangXiangUSTC commented May 20, 2020 •

edited

Loading

csuzhangxc commented May 20, 2020

WangXiangUSTC commented May 20, 2020

lichunzhu May 20, 2020 •

edited

Loading

WangXiangUSTC May 21, 2020

lichunzhu May 21, 2020

WangXiangUSTC May 21, 2020

lichunzhu May 20, 2020

csuzhangxc May 21, 2020 •

edited

Loading

WangXiangUSTC May 21, 2020

WangXiangUSTC May 21, 2020

csuzhangxc left a comment

miaoqingli May 22, 2020 •

edited

Loading

WangXiangUSTC May 22, 2020

miaoqingli May 22, 2020

WangXiangUSTC May 22, 2020

miaoqingli commented May 22, 2020

toutdesuite left a comment

toutdesuite May 22, 2020

WangXiangUSTC May 23, 2020

toutdesuite May 25, 2020

WangXiangUSTC May 25, 2020

toutdesuite May 22, 2020

WangXiangUSTC May 25, 2020

toutdesuite May 22, 2020

WangXiangUSTC May 25, 2020

WangXiangUSTC commented May 25, 2020


		## 部署测试环境

		- 使用 TiDB Ansible 部署 TiDB 测试集群，所有配置使用 TiDB Ansible 提供的默认配置。


		### 获取测试结果

		通过 `query-status` 命令观测 DM 的同步状态，通过 Grafana 观测 DM 的监控指标。主要包括同步延迟 `replicate lag`，单位时间内完成的 job 数量 `finished sqls jobs` 等。


		#### 创建数据同步任务

		1. 创建上游 MySQL 的 source，将 `source-id` 配置为 `source-1`。


		#### 获取测试结果

		通过 `query-status` 命令观测 DM 的同步状态，通过 Grafana 观测 DM 的监控指标。主要包括同步延迟 `replicate lag`，单位时间内完成的 job 数量 `finished sqls jobs` 等。


		#### 创建数据同步任务

		1. 创建上游 MySQL 的 source, source-id 配置为 `source-1`。（如果在全量同步性能测试中已经创建，则不需要再次创建）。详细操作方法参考：[加载数据源配置](manage-replication-tasks.md#加载数据源配置)。

zh: add document about how to do performance test #85

zh: add document about how to do performance test #85

Conversation

WangXiangUSTC commented May 20, 2020 • edited Loading

What is changed, added or deleted? (Required)

Which TiDB Data Migration (DM) version(s) do your changes apply to? (Required)

csuzhangxc commented May 20, 2020

WangXiangUSTC commented May 20, 2020

lichunzhu May 20, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csuzhangxc May 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csuzhangxc left a comment

Choose a reason for hiding this comment

miaoqingli May 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miaoqingli commented May 22, 2020

toutdesuite left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WangXiangUSTC commented May 25, 2020

WangXiangUSTC commented May 20, 2020 •

edited

Loading

lichunzhu May 20, 2020 •

edited

Loading

csuzhangxc May 21, 2020 •

edited

Loading

miaoqingli May 22, 2020 •

edited

Loading