downstream tidb add index from syncer with retry will block dmctl operate-source show

[River2000i]

What did you do?

1. upstream mysql exec create index
2. run dmctl operate-source show
3. admin ddl job cancel create index
4. run dmctl operate-source show

What did you expect to see?

query-status show run success

What did you see instead?

[root@dm-dm-master-0 log]# time /dmctl --master-addr=[127.0.0.1:8261](http://127.0.0.1:8261/) operate-source  show
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": false,
            "msg": "[code=38045:class=dm-master:scope=internal:level=medium], Message: fail to get expected result",
            "source": "source-0",
            "worker": "dm-dm-worker-2"
        }
    ]
}

tidb:

[2025/02/24 22:57:06.711 +00:00] [WARN] [local.go:1255] ["meet retryable error when writing to TiKV"] [category=ddl-ingest] [error="peer 1993, store 1, region 1992, epoch conf_ver:1 version:585 , when close write stream: rpc error: code = Unknown desc = ResourceNotEnough(\"Memory usage too high: 7163785216 bytes\")"] ["job stage"=needRescan]

dm:

[2025/02/24 00:20:16.812 +00:00] [INFO] [ddl.go:418] ["start to handle ddls in normal mode"] [task=dm_migrate_incremental_ddl] [unit="binlog replication"] [component=ddl] [mode=normal] [event=query] [queryEventContext="{schema: , originSQL: create index idx0 on dm_migrate_incremental_ddl.sbtest1 (k), startLocation: position: (mysql-bin.000001, 190958), gtid-set: d60bcd02-f22a-11ef-8d9f-a6eeb70df738:1-3, endLocation: position: (mysql-bin.000001, 191109), gtid-set: d60bcd02-f22a-11ef-8d9f-a6eeb70df738:1-4, lastLocation: position: (mysql-bin.000001, 191109), gtid-set: d60bcd02-f22a-11ef-8d9f-a6eeb70df738:1-4, re-sync: , needHandleDDLs: CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`), trackInfos: {originDDL: CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`), routedDDL: CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`), sourceTables: `dm_migrate_incremental_ddl`.`sbtest1`, targetTables: `dm_migrate_incremental_ddl`.`sbtest1`}}"]
[2025/02/24 00:20:16.812 +00:00] [INFO] [tracker.go:460] ["Remove downstream schema tracker"] [task=dm_migrate_incremental_ddl] [unit="binlog replication"] [tableID=`dm_migrate_incremental_ddl`.`sbtest1`]
[2025/02/24 00:25:16.815 +00:00] [ERROR] [baseconn.go:206] ["execute statement failed"] [task=dm_migrate_incremental_ddl] [unit="binlog replication"] [query="CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`)"] [argument="[]"] [error="invalid connection"]
[2025/02/24 00:25:16.815 +00:00] [ERROR] [baseconn.go:213] ["rollback failed"] [task=dm_migrate_incremental_ddl] [unit="binlog replication"] [query="CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`)"] [argument="[]"] [error="invalid connection"]
[2025/02/24 00:25:16.815 +00:00] [ERROR] [db.go:204] ["execute statements failed after retry"] [task=dm_migrate_incremental_ddl] [unit="binlog replication"] [queries="[SET TIMESTAMP = 1740356416 CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`) SET TIMESTAMP = DEFAULT]"] [arguments="[]"] [error="[code=10006:class=database:scope=not-set:level=high], Message: execute statement failed: CREATE INDEX `idx0` ON `dm_migrate_incremental_ddl`.`sbtest1` (`k`), RawCause: invalid connection"]

dmctl operate-source show success after cancel ddl job

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

(paste DM version here, and you must ensure versions of dmctl, DM-worker and DM-master are same)

Upstream MySQL/MariaDB server version:

(paste upstream MySQL/MariaDB server version here)

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

How did you deploy DM: tiup or manually?

(leave TiUP or manually here)

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

(paste current status of DM cluster here)