Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(backend): refine system account #15530

Merged
merged 49 commits into from
Mar 2, 2025
Merged

enhance(backend): refine system account #15530

merged 49 commits into from
Mar 2, 2025

Conversation

syuilo
Copy link
Member

@syuilo syuilo commented Feb 21, 2025

What

Fix #15525
Fix #14857

  • 柔軟性確保のため、システムアカウントをユーザー名ハードコードではなく別テーブルで管理するようにした
  • システムアカウントを消せないようにした
  • proxyアカウントのシステムアカウント化
  • isRootなアカウントかどうかをユーザーテーブルのカラムではなくmetaに持つようにしユーザーテーブルの効率化

Why

Additional info (optional)

Fix #15011

Checklist

  • Read the contribution guide
  • Test working in a local environment
  • (If needed) Add story of storybook
  • (If needed) Update CHANGELOG.md
  • (If possible) Add tests

@github-actions github-actions bot added the packages/backend Server side specific issue/PR label Feb 21, 2025
Copy link

codecov bot commented Feb 21, 2025

Codecov Report

Attention: Patch coverage is 62.68116% with 206 lines in your changes missing coverage. Please review.

Project coverage is 44.15%. Comparing base (5d68372) to head (c6def84).
Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
packages/backend/src/core/SystemAccountService.ts 79.65% 35 Missing ⚠️
packages/frontend/src/pages/admin-user.vue 0.00% 28 Missing ⚠️
.../backend/src/core/activitypub/ApRendererService.ts 33.33% 26 Missing ⚠️
...server/api/endpoints/admin/update-proxy-account.ts 59.67% 25 Missing ⚠️
...kages/backend/src/server/api/endpoints/reset-db.ts 16.66% 15 Missing ⚠️
packages/frontend/src/pages/admin/settings.vue 0.00% 15 Missing ⚠️
packages/backend/src/core/MetaService.ts 7.69% 12 Missing ⚠️
packages/backend/src/core/DeleteAccountService.ts 50.00% 6 Missing ⚠️
packages/backend/src/core/SignupService.ts 44.44% 5 Missing ⚠️
.../src/server/api/endpoints/admin/accounts/create.ts 16.66% 5 Missing ⚠️
... and 12 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #15530      +/-   ##
===========================================
+ Coverage    42.14%   44.15%   +2.00%     
===========================================
  Files         1609     1613       +4     
  Lines       162370   168364    +5994     
  Branches      4044     4098      +54     
===========================================
+ Hits         68425    74333    +5908     
- Misses       93472    93558      +86     
  Partials       473      473              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

たった1レコードのために全userレコードにフラグ持たせるの無駄すぎるからisRoot消してmetaに移動させた

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

proxyアカウントのアイコンをサーバーアイコンにしたいけどサーバーアイコンはdriveFileとして存在していないからめんどいわね

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

システムドライブ機能を先に実装する必要がある可能性が出てきた

@github-actions github-actions bot added packages/frontend Client side specific issue/PR packages/misskey-js labels Feb 21, 2025
Copy link
Contributor

github-actions bot commented Feb 21, 2025

このPRによるapi.jsonの差分

差分はこちら
--- base
+++ head
@@ -9666,10 +9666,7 @@
                       "type": "boolean"
                     },
                     "proxyAccountId": {
-                      "type": [
-                        "string",
-                        "null"
-                      ],
+                      "type": "string",
                       "format": "id"
                     },
                     "email": {
@@ -17397,13 +17394,6 @@
                   "enableSensitiveMediaDetectionForVideos": {
                     "type": "boolean"
                   },
-                  "proxyAccountId": {
-                    "type": [
-                      "string",
-                      "null"
-                    ],
-                    "format": "misskey:id"
-                  },
                   "maintainerName": {
                     "type": [
                       "string",
@@ -17746,6 +17736,163 @@
           },
           "400": {
             "description": "Client error",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "INVALID_PARAM": {
+                    "value": {
+                      "error": {
+                        "message": "Invalid param.",
+                        "code": "INVALID_PARAM",
+                        "id": "3d81ceae-475f-4600-b2a8-2bc116157532"
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "401": {
+            "description": "Authentication error",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "CREDENTIAL_REQUIRED": {
+                    "value": {
+                      "error": {
+                        "message": "Credential required.",
+                        "code": "CREDENTIAL_REQUIRED",
+                        "id": "1384574d-a912-4b81-8601-c7b1c4085df1"
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "403": {
+            "description": "Forbidden error",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "AUTHENTICATION_FAILED": {
+                    "value": {
+                      "error": {
+                        "message": "Authentication failed. Please ensure your token is correct.",
+                        "code": "AUTHENTICATION_FAILED",
+                        "id": "b0a7f5f8-dc2f-4171-b91f-de88ad238e14"
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "418": {
+            "description": "I'm Ai",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "I_AM_AI": {
+                    "value": {
+                      "error": {
+                        "message": "You sent a request to Ai-chan, Misskey's showgirl, instead of the server.",
+                        "code": "I_AM_AI",
+                        "id": "60c46cd1-f23a-46b1-bebe-5d2b73951a84"
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "500": {
+            "description": "Internal server error",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "INTERNAL_ERROR": {
+                    "value": {
+                      "error": {
+                        "message": "Internal error occurred. Please contact us if the error persists.",
+                        "code": "INTERNAL_ERROR",
+                        "id": "5d37dbcb-891e-41ca-a3d6-e690c97775ac"
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/admin/update-proxy-account": {
+      "post": {
+        "operationId": "post___admin___update-proxy-account",
+        "summary": "admin/update-proxy-account",
+        "description": "No description provided.\n\n**Credential required**: *Yes* / **Permission**: *write:admin:account*",
+        "externalDocs": {
+          "description": "Source code",
+          "url": "https://github.com/misskey-dev/misskey/blob/develop/packages/backend/src/server/api/endpoints/admin/update-proxy-account.ts"
+        },
+        "tags": [
+          "admin"
+        ],
+        "security": [
+          {
+            "bearerAuth": []
+          }
+        ],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "properties": {
+                  "description": {
+                    "type": [
+                      "string",
+                      "null"
+                    ],
+                    "minLength": 1,
+                    "maxLength": 1500
+                  }
+                }
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "OK (with results)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "$ref": "#/components/schemas/UserDetailed"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Client error",
             "content": {
               "application/json": {
                 "schema": {

Get diff files from Workflow Page

@samunohito
Copy link
Member

オッ

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

ほぼ完成

@syuilo syuilo marked this pull request as ready for review February 21, 2025 12:07
@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

proxyアカウントのアイコンをサーバーアイコンにしたいけどサーバーアイコンはdriveFileとして存在していないからめんどいわね

これどうするかなぁ

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

urlだけ設定するのでもいいけどurlが設定されているのにavatarIdがnullになっていると不具合起きそう

@syuilo
Copy link
Member Author

syuilo commented Feb 21, 2025

Userをpackするときに上書きしてもいいけどいちいちプロキシかどうかチェックするのもな

@Sayamame-beans
Copy link
Member

ファイル用のシステムアカウントを用意しても良いのかも? みたいな話はおさむさんがされてましたね…(しかし、本人とファイルの所有者が異なるとマズい? 良い感じに共用するみたいなことをしないといけない…?)

@zyoshoka
Copy link
Contributor

zyoshoka commented Mar 1, 2025

ログ見ると deadlock detected になっているので削除できていないかもです(なおローカルでは再現しない)

2025-02-28T14:35:43.7321395Z misskey.b.test-1  | ERR  1	[queue db]	failed(QueryFailedError: deadlock detected) id=1 {
2025-02-28T14:35:43.7321507Z misskey.b.test-1  |   job: {
2025-02-28T14:35:43.7321670Z misskey.b.test-1  |     name: 'deleteAccount',
2025-02-28T14:35:43.7321882Z misskey.b.test-1  |     info: 'id=1 attempts=1/0 age=1013ms',
2025-02-28T14:35:43.7322085Z misskey.b.test-1  |     failedReason: 'deadlock detected',
2025-02-28T14:35:43.7322257Z misskey.b.test-1  |     data: { user: [Object] }
2025-02-28T14:35:43.7322358Z misskey.b.test-1  |   },
2025-02-28T14:35:43.7322585Z misskey.b.test-1  |   e: {
2025-02-28T14:35:43.7322832Z misskey.b.test-1  |     stack: 'QueryFailedError: deadlock detected\n' +
2025-02-28T14:35:43.7323597Z misskey.b.test-1  |       '    at PostgresQueryRunner.query (/misskey/node_modules/.pnpm/typeorm@0.3.20_ioredis@5.5.0_pg@8.13.3/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)\n' +
2025-02-28T14:35:43.7324007Z misskey.b.test-1  |       '    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
2025-02-28T14:35:43.7324763Z misskey.b.test-1  |       '    at async DeleteQueryBuilder.execute (/misskey/node_modules/.pnpm/typeorm@0.3.20_ioredis@5.5.0_pg@8.13.3/node_modules/typeorm/query-builder/DeleteQueryBuilder.js:52:33)\n' +
2025-02-28T14:35:43.7325613Z misskey.b.test-1  |       '    at async DeleteAccountProcessorService.process (file:///misskey/packages/backend/built/queue/processors/DeleteAccountProcessorService.js:118:13)\n' +
2025-02-28T14:35:43.7326132Z misskey.b.test-1  |       '    at async /misskey/node_modules/.pnpm/bullmq@5.41.1/node_modules/bullmq/dist/cjs/classes/worker.js:517:32\n' +
2025-02-28T14:35:43.7326800Z misskey.b.test-1  |       '    at async Worker.retryIfFailed (/misskey/node_modules/.pnpm/bullmq@5.41.1/node_modules/bullmq/dist/cjs/classes/worker.js:741:24)',
2025-02-28T14:35:43.7326999Z misskey.b.test-1  |     message: 'deadlock detected',
2025-02-28T14:35:43.7327170Z misskey.b.test-1  |     name: 'QueryFailedError'
2025-02-28T14:35:43.7327281Z misskey.b.test-1  |   }
2025-02-28T14:35:43.7327379Z misskey.b.test-1  | }
2025-02-28T14:35:43.7327793Z misskey.b.test-1  | query is slow: DELETE FROM "drive_file" WHERE "id" IN ($1) -- PARAMETERS: ["a4sfuv9clm1i001f"]
2025-02-28T14:35:43.7327939Z misskey.b.test-1  | execution time: 1009

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

🙏🏻 🙏🏻 🙏🏻 🙏🏻 🙏🏻

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

うーんデッドロック

@zyoshoka
Copy link
Contributor

zyoshoka commented Mar 1, 2025

#15005 かも

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

あっいけた

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

このPRで直すことではないのでrevert

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

#15574

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

再発した
image

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

TASUKETE

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

またデッドロックしてるっぽい

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

#15574 では不十分だったか

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

てかそもそもトランザクション使ってないのになんでデッドロックするんだ

@syuilo
Copy link
Member Author

syuilo commented Mar 1, 2025

と思ったらまた通ったしこのPRが関係しているわけではなさそうだからマージする

@kakkokari-gtyih
Copy link
Contributor

system_accountテーブルからmeta.proxyAccountIdを復元するクエリがあったほうが良いように思えます。
#15530 (comment)

ここら辺の変更はこれで良いのかしら(rootという名前だけど実際には通常ユーザーそう)
#15530 (comment)

これ確認した?

const _user = await this.usersRepository.findOneByOrFail({ id: user.id });
if (_user.isRoot) throw new Error('cannot delete a root account');

if (user.host === null && _user.username.includes('.')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

メモ:host === null で username.includes('.') なので、外部サーバーの . つきアカウント(brid.gyなど)に対する削除処理がうまくいかないとかは起こらない

@syuilo
Copy link
Member Author

syuilo commented Mar 2, 2025

ひとまずマージ

@syuilo syuilo merged commit 616cccf into develop Mar 2, 2025
35 of 37 checks passed
@syuilo syuilo deleted the system-accounts branch March 2, 2025 11:06
@syuilo
Copy link
Member Author

syuilo commented Mar 2, 2025

🙏🏻 🙏🏻 🙏🏻

harumaki2000 pushed a commit to harumaki2000/misskey-layp that referenced this pull request Mar 2, 2025
* wip

* wip

* wip

* Update SystemAccountService.ts

* Update 1740121393164-system-accounts.js

* Update DeleteAccountService.ts

* wip

* wip

* wip

* wip

* Update 1740121393164-system-accounts.js

* Update RepositoryModule.ts

* wip

* wip

* wip

* Update ApRendererService.ts

* wip

* wip

* Update SystemAccountService.ts

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* add print logs

* ログが長すぎて出てないかもしれない

* fix migration

* refactor

* fix fed-tests

* Update RelayService.ts

* merge

* Update user.test.ts

* chore: emit log

* fix: tweak sleep duration

* fix: exit 1

* fix: wait for misskey processes to become healthy

* fix: longer sleep for user deletion

* fix: make sleep longer again

* デッドロック解消の試み

misskey-dev#15005

* Revert "デッドロック解消の試み"

This reverts commit 266141f.

* wip

* Update SystemAccountService.ts

---------

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
Co-authored-by: zyoshoka <107108195+zyoshoka@users.noreply.github.com>
DA-TENSHI pushed a commit to SHINANOSKEY-Projekt/SHINANOSKEY that referenced this pull request Mar 6, 2025
* wip

* wip

* wip

* Update SystemAccountService.ts

* Update 1740121393164-system-accounts.js

* Update DeleteAccountService.ts

* wip

* wip

* wip

* wip

* Update 1740121393164-system-accounts.js

* Update RepositoryModule.ts

* wip

* wip

* wip

* Update ApRendererService.ts

* wip

* wip

* Update SystemAccountService.ts

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* add print logs

* ログが長すぎて出てないかもしれない

* fix migration

* refactor

* fix fed-tests

* Update RelayService.ts

* merge

* Update user.test.ts

* chore: emit log

* fix: tweak sleep duration

* fix: exit 1

* fix: wait for misskey processes to become healthy

* fix: longer sleep for user deletion

* fix: make sleep longer again

* デッドロック解消の試み

misskey-dev#15005

* Revert "デッドロック解消の試み"

This reverts commit 266141f.

* wip

* Update SystemAccountService.ts

---------

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
Co-authored-by: zyoshoka <107108195+zyoshoka@users.noreply.github.com>
@kakkokari-gtyih kakkokari-gtyih removed this from the v2025.3.3 milestone Mar 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment