test: refactor real cluster test and implement cluster id case #8685

okJiang · 2024-10-10T08:51:06Z

What problem does this PR solve?

Issue Number: Close #8683, close #8684

What is changed and how does it work?

Using Golang to deploy cluster. In this way, we can deploy and destroy cluster depend on our test demand
Using tag to manage different cluster between different test case. Different clusters do not affect each other
Implement cluster id test case Implement cluster id test case in real cluster test #8684

Check List

Tests

Integration test

make test-real-cluster
# testing with the real cluster...
cd /root/code/okjiang/pd/tests/integrations/realcluster && make check
make[1]: Entering directory `/root/code/okjiang/pd/tests/integrations/realcluster'
CGO_ENABLED=1 go test ./... -v -tags deadlock -race -cover || (\
echo "follow is pd-0 log\n" ; \
cat ~/.tiup/data/pd_real_cluster_test/pd-0/pd.log ; \
echo "follow is pd-1 log\n" ; \
cat ~/.tiup/data/pd_real_cluster_test/pd-1/pd.log ; \
echo "follow is pd-2 log\n" ; \
cat ~/.tiup/data/pd_real_cluster_test/pd-2/pd.log ; \
exit 1)
=== RUN   TestClusterID
=== RUN   TestClusterID/TestClientClusterID
[2024/10/10 16:47:18.115 +08:00] [INFO] [real_cluster.go:63] ["start to deploy a real cluster"]
[2024/10/10 16:47:18.116 +08:00] [INFO] [real_cluster.go:113] [/root/code/okjiang/pd/tests/integrations/realcluster]
[2024/10/10 16:47:28.124 +08:00] [INFO] [real_cluster.go:146] ["start to wait TiUP ready"] [tag=pd_real_cluster_test_cluster_id_0]
Error: Post "http://127.0.0.1:33032/command": dial tcp 127.0.0.1:33032: connect: connection refused
[2024/10/10 16:47:28.500 +08:00] [INFO] [real_cluster.go:154] ["TiUP is not ready, will retry"] ["retry times"=0] [tag=pd_real_cluster_test_cluster_id_0] [error="exit status 1"]
Pid   Role     Uptime
---   ----     ------
2946  pd       14.960070143s
2954  pd       14.912455918s
2961  pd       14.867212079s
2979  tikv     14.814994144s
2990  tikv     14.773555858s
3032  tikv     14.72004818s
3040  tidb     14.678945795s
3730  tiflash  3.642770313s
[2024/10/10 16:47:33.764 +08:00] [INFO] [real_cluster.go:150] ["TiUP is ready"] [tag=pd_real_cluster_test_cluster_id_0]
[2024/10/10 16:47:33.764 +08:00] [INFO] [real_cluster.go:63] ["start to deploy a real cluster"]
[2024/10/10 16:47:33.764 +08:00] [INFO] [real_cluster.go:113] [/root/code/okjiang/pd/tests/integrations/realcluster]
[2024/10/10 16:47:43.772 +08:00] [INFO] [real_cluster.go:146] ["start to wait TiUP ready"] [tag=pd_real_cluster_test_cluster_id_1]
Error: Post "http://127.0.0.1:36151/command": dial tcp 127.0.0.1:36151: connect: connection refused
[2024/10/10 16:47:44.082 +08:00] [INFO] [real_cluster.go:154] ["TiUP is not ready, will retry"] ["retry times"=0] [tag=pd_real_cluster_test_cluster_id_1] [error="exit status 1"]
Pid   Role     Uptime
---   ----     ------
5102  pd       14.919982227s
5111  pd       14.896268836s
5112  pd       14.876884134s
5122  tikv     14.860474382s
5128  tikv     14.84466765s
5151  tikv     14.82536701s
5175  tidb     14.808416266s
5843  tiflash  4.756919588s
[2024/10/10 16:47:49.369 +08:00] [INFO] [real_cluster.go:150] ["TiUP is ready"] [tag=pd_real_cluster_test_cluster_id_1]
[2024/10/10 16:47:50.409 +08:00] [INFO] [pd_service_discovery.go:638] ["[pd] close pd service discovery client"]
[2024/10/10 16:47:50.411 +08:00] [INFO] [real_cluster.go:72] ["start to destroy a real cluster"] [tag=pd_real_cluster_test_cluster_id_1]
[2024/10/10 16:47:50.448 +08:00] [INFO] [real_cluster.go:106] ["destroy success"] [pid="5063\n"]
[2024/10/10 16:47:55.452 +08:00] [INFO] [real_cluster.go:72] ["start to destroy a real cluster"] [tag=pd_real_cluster_test_cluster_id_1]
sh: line 0: kill: (7173) - No such process
[2024/10/10 16:47:55.498 +08:00] [INFO] [real_cluster.go:106] ["destroy success"] [pid="7173\n"]
[2024/10/10 16:48:00.503 +08:00] [INFO] [real_cluster.go:72] ["start to destroy a real cluster"] [tag=pd_real_cluster_test_cluster_id_0]
[2024/10/10 16:48:00.548 +08:00] [INFO] [real_cluster.go:106] ["destroy success"] [pid="2909\n"]
--- PASS: TestClusterID (47.47s)
    --- PASS: TestClusterID/TestClientClusterID (47.46s)
PASS
coverage: 56.2% of statements
ok      github.com/tikv/pd/tests/integrations/realcluster       48.686s coverage: 56.2% of statements

Release note

None.

Signed-off-by: okJiang <819421878@qq.com>

okJiang · 2024-10-10T08:52:50Z

tests/integrations/realcluster/reboot_pd_test.go

-	restartTiUP()
-	checkLabelsAreEqual()
-}
+// func TestReloadLabel(t *testing.T) {


these cases will be updated with new style in the next pr

what do you think? Do I need update them in this pr? @rleungx

Both are ok to me.

codecov · 2024-10-10T08:58:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.08%. Comparing base (f7c8e43) to head (247eeb9).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8685      +/-   ##
==========================================
+ Coverage   70.03%   70.08%   +0.05%     
==========================================
  Files         517      517              
  Lines       79947    79947              
==========================================
+ Hits        55989    56033      +44     
+ Misses      20382    20346      -36     
+ Partials     3576     3568       -8

Flag	Coverage Δ
unittests	`70.08% <ø> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

okJiang · 2024-10-10T08:59:11Z

pull-integration-realcluster-test should be passed after PingCAP-QE/ci#3168 merge

tests/integrations/realcluster/download_integration_test_binaries.sh

Signed-off-by: okJiang <819421878@qq.com>

…ster-test

Signed-off-by: okJiang <819421878@qq.com>

okJiang · 2024-10-11T08:29:14Z

tests/integrations/realcluster/real_cluster.go

+		// --kv.binpath ./third_bin/tikv-server \
+		// --db.binpath ./third_bin/tidb-server --tiflash.binpath ./third_bin/tiflash \
+		--kv.binpath ./bin/tikv-server \
+		--db.binpath ./bin/tidb-server --tiflash.binpath ./bin/tiflash \


We will use third_bin after PingCAP-QE/ci#3168 merge

rleungx · 2024-10-12T08:59:43Z

tests/integrations/realcluster/real_cluster.go

+}
+
+func (s *realClusterSuite) tag() string {
+	return fmt.Sprintf("pd_real_cluster_test_%s_%d", s.suiteName, s.clusterCnt)


How about using a random way?

It is not needed at the moment. If using random numbers, we need to maintain an additional array of random numbers.

rleungx · 2024-10-12T09:00:51Z

tests/integrations/realcluster/real_cluster.go

+		--pd.config ./tests/integrations/realcluster/pd.toml \
+		> `+filepath.Join(curPath, "playground", tag+".log")+` 2>&1 & `)
+
+	time.Sleep(10 * time.Second)


What the purpose of it compared with waitTiupReady?

To avoid to change the dir before execute tiup playground. I will add some comments for it.

rleungx · 2024-10-12T09:02:25Z

tests/integrations/realcluster/real_cluster.go

+func (s *realClusterSuite) SetupSuite() {
+	t := s.T()
+
+	dataDir := filepath.Join(os.Getenv("HOME"), ".tiup", "data", "pd_real_cluster_test_"+s.suiteName+"_*")


Should we use a tmp dir?

This directory is the default dir of tiup.

https://docs.pingcap.com/tidb/stable/tiup-playground#specify-a-tag-when-starting-the-tidb-cluster

Signed-off-by: okJiang <819421878@qq.com>

okJiang · 2024-10-16T07:05:26Z

ping~ @rleungx

ti-chi-bot · 2024-10-18T08:25:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JmPotato, rleungx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [JmPotato,rleungx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-10-18T08:25:13Z

[LGTM Timeline notifier]

Timeline:

2024-10-15 06:43:48.508482056 +0000 UTC m=+339825.657391880: ☑️ agreed by JmPotato.
2024-10-18 08:25:12.001284279 +0000 UTC m=+605109.150194132: ☑️ agreed by rleungx.

implement cluster id test case

1b45fac

Signed-off-by: okJiang <819421878@qq.com>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 10, 2024

okJiang commented Oct 10, 2024

View reviewed changes

rleungx reviewed Oct 10, 2024

View reviewed changes

tests/integrations/realcluster/download_integration_test_binaries.sh Outdated Show resolved Hide resolved

okJiang added 10 commits October 10, 2024 17:08

fix static && add comment

c71b3ef

Signed-off-by: okJiang <819421878@qq.com>

remove useless code

6653d42

Signed-off-by: okJiang <819421878@qq.com>

trigger ci

d69f5c5

Signed-off-by: okJiang <819421878@qq.com>

trigger ci

9176f7d

Signed-off-by: okJiang <819421878@qq.com>

trigger ci

b4b1b12

Signed-off-by: okJiang <819421878@qq.com>

Merge branch 'master' of https://github.com/tikv/pd into add-real-clu…

447716f

…ster-test

trigger ci

348db7b

Signed-off-by: okJiang <819421878@qq.com>

trigger ci

546707d

Signed-off-by: okJiang <819421878@qq.com>

install tiup

c2844b5

Signed-off-by: okJiang <819421878@qq.com>

fix

d1f9143

Signed-off-by: okJiang <819421878@qq.com>

okJiang commented Oct 11, 2024

View reviewed changes

rleungx reviewed Oct 12, 2024

View reviewed changes

add some comment

a5ca52e

Signed-off-by: okJiang <819421878@qq.com>

JmPotato approved these changes Oct 15, 2024

View reviewed changes

ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 15, 2024

rleungx approved these changes Oct 18, 2024

View reviewed changes

ti-chi-bot bot added the lgtm label Oct 18, 2024

ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Oct 18, 2024

Merge branch 'master' into add-real-cluster-test

247eeb9

ti-chi-bot bot merged commit e01a7dd into tikv:master Oct 18, 2024
24 of 25 checks passed

okJiang deleted the add-real-cluster-test branch October 18, 2024 09:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: refactor real cluster test and implement cluster id case #8685

test: refactor real cluster test and implement cluster id case #8685

okJiang commented Oct 10, 2024 •

edited

Loading

okJiang Oct 10, 2024

okJiang Oct 10, 2024

rleungx Oct 10, 2024

codecov bot commented Oct 10, 2024 •

edited

Loading

okJiang commented Oct 10, 2024

okJiang Oct 11, 2024

rleungx Oct 12, 2024

okJiang Oct 12, 2024

rleungx Oct 12, 2024

okJiang Oct 12, 2024

rleungx Oct 12, 2024

okJiang Oct 12, 2024

okJiang Oct 12, 2024

okJiang commented Oct 16, 2024

ti-chi-bot bot commented Oct 18, 2024

ti-chi-bot bot commented Oct 18, 2024

test: refactor real cluster test and implement cluster id case #8685

test: refactor real cluster test and implement cluster id case #8685

Conversation

okJiang commented Oct 10, 2024 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 10, 2024 • edited Loading

Codecov Report

okJiang commented Oct 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

okJiang commented Oct 16, 2024

ti-chi-bot bot commented Oct 18, 2024

ti-chi-bot bot commented Oct 18, 2024

[LGTM Timeline notifier]

okJiang commented Oct 10, 2024 •

edited

Loading

codecov bot commented Oct 10, 2024 •

edited

Loading