Skip to content

Commit

Permalink
feat: add experimental HTTP APIs to control sync (#2111)
Browse files Browse the repository at this point in the history 
## Motivation

Add HTTP APIs to force sync with a particular peer id and to interrupt
current sync.

NOTE: This is experimental, these endpoints may be removed or
significantly altered in the future.

## Change Summary

### GET /v1/syncStatus

```
{
  "isSyncing": true,
  "syncStatus": [
    {
      "peerId": "12D3KooWDgi3EZGdsghsuyk17QUJUm81WdypjUfUpQFMv1KHqFPg",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452062847,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWBeHgpW1YJer1exohYJmBugcCxw6nPivDWPinc7GzZwtW",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452193687,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWBM99sMoBfWZdA8eDMbU6z2awAKvnpwPWfsrJEkK6gE82",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452066054,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWAd4CS8JrFzhavbX8VYoqZgYvixq92SKoMwkKSmEYn3ji",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 451854112,
      "ourMessages": 449263973,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWAceovMknjAkFVxsazdq1zFsBA1FdQcd9quq8szhmi4YG",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452123129,
      "ourMessages": 449263973,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooW9zRXheJk13bPxxGzsDqK9WjienqoeQTpZq1kh3hbi8vs",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452386192,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWAiWjWtt29EgQtANUPtZLmXrwermBf1PFvHtSkrkfWuVN",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 451970452,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWET8RWKH5nRGmGn7FWMkSP5p8E48ipSoqC8CdCrCKekhh",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452125619,
      "ourMessages": 449263973,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWEFY9y65Su3LFd68BuJ3GLD37KxczMxxJ2h3QX3FsrTMw",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452009411,
      "ourMessages": 449263973,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWAg56TGduGdNjCtuXEdLkiyuTQykjBsKLmnBhKx2WvMgU",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452250942,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWCw6zzXy2zeXgHj1jhPruUaqjfbNxK8Gp6hGoaA17ReYf",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452077312,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWELYvHd5MkDYaT1BXCEVn61PuznAj15mUmQn1ioLVQC1h",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452160134,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWDvzJMYFAzdJ4NPtHfm4Ygoz9JSWXgi346q5VAE6stnSm",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452012326,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWBCbzwWTYkU7hzQHo8dN2PTYtoom8Eo3heP1QtuuaYbPy",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452241704,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWC4LJ62SHpTVoSvyzA63jBHXeTNm1vboKUygR83d8EH5X",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452097420,
      "ourMessages": 449266923,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWBuS8TUiqeabs9YKTz17vQRD2csVA4PDUqFFmKDyCGepq",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 451996803,
      "ourMessages": 449264141,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWMRrDr4JmWFnxusb5S9nsrkxBAXqWf7BymJtgQRsJMVQ2",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452267236,
      "ourMessages": 449264141,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWB8uy9VxTpipGoRjEc2BFoswXKgyGS6rhNTd1ofD9JXLG",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452081457,
      "ourMessages": 449267092,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWDrDvLLvRNaUiqmrA9YvRp41gZaEmoatqVAdJwzopdgBN",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 452128777,
      "ourMessages": 449264141,
      "lastBadSync": -1,
      "score": 0
    },
    {
      "peerId": "12D3KooWBfPafShMrDwmVwM1qGqajyyYnfsr4K2MznKRfnxx6giU",
      "inSync": "false",
      "shouldSync": false,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 451763652,
      "ourMessages": 449264184,
      "lastBadSync": -1,
      "score": 0
    }
  ],
  "engineStarted": true
}
```

### POST /v1/stopSync
```
{
  "isSyncing": false,
  "syncStatus": [],
  "engineStarted": true
}
```

### POST /v1/forceSync?peer_id=${PEER_ID_HERE}
```
{
  "isSyncing": true,
  "syncStatus": [
    {
      "peerId": "12D3KooWRnSZUxjVJjbSHhVKpXtvibMarSfLSKDBeMpfVaNm1Joo",
      "inSync": "false",
      "shouldSync": true,
      "divergencePrefix": "",
      "divergenceSecondsAgo": 0,
      "theirMessages": 448940578,
      "ourMessages": 449276683,
      "lastBadSync": -1,
      "score": 0
    }
  ],
  "engineStarted": true
}
```

## Merge Checklist

_Choose all relevant options below by adding an `x` now or at any time
before submitting for review_

- [x] PR title adheres to the [conventional
commits](https://www.conventionalcommits.org/en/v1.0.0/) standard
- [x] PR has a
[changeset](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#35-adding-changesets)
- [x] PR has been tagged with a change label(s) (i.e. documentation,
feature, bugfix, or chore)
- [x] PR includes
[documentation](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#32-writing-docs)
if necessary.
- [x] All [commits have been
signed](https://github.com/farcasterxyz/hub-monorepo/blob/main/CONTRIBUTING.md#22-signing-commits)

## Additional Context

If this is a relatively large or complex change, provide more details
here that will help reviewers


<!-- start pr-codex -->

---

## PR-Codex overview
The focus of this PR is to add experimental HTTP APIs for controlling
synchronization in the `@farcaster/hubble` app.

### Detailed summary
- Added experimental HTTP APIs for controlling sync: `stopSync`,
`syncStatus`, `forceSync`
- Implemented corresponding API endpoints in `httpServer.ts`
- Updated documentation with examples and responses for each API

> ✨ Ask PR-Codex anything about this PR by commenting with `/codex {your
question}`

<!-- end pr-codex -->
  • Loading branch information
@Wazzymandias
Wazzymandias authored Jun 29, 2024
1 parent c723f65 commit 667a5b3
Show file tree
Hide file tree
Showing 4 changed files with 149 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .changeset/tiny-kangaroos-refuse.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@farcaster/hubble": patch
---

feat: add experimental HTTP APIs to control sync
23 changes: 23 additions & 0 deletions apps/hubble/src/rpc/httpServer.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,29 @@ export class HttpAPIServer {
this.grpcImpl.getCurrentPeers(call, handleResponse(reply, ContactInfoResponse));
});

//================stopSync================
// @doc-tag: /stopSync
this.app.post("/v1/stopSync", (request, reply) => {
const call = getCallObject("stopSync", {}, request);
this.grpcImpl.stopSync(call, handleResponse(reply, SyncStatusResponse));
});

//================syncStatus================
// @doc-tag: /syncStatus?peer_id=...
this.app.get<{ Querystring: { peer_id: string } }>("/v1/syncStatus", (request, reply) => {
const { peer_id } = request.query;
const call = getCallObject("getSyncStatus", { peerId: peer_id }, request);
this.grpcImpl.getSyncStatus(call, handleResponse(reply, SyncStatusResponse));
});

//================forceSync================
// @doc-tag: /forceSync?peer_id=...
this.app.post<{ Querystring: { peer_id: string } }>("/v1/forceSync", (request, reply) => {
const { peer_id } = request.query;
const call = getCallObject("forceSync", { peerId: peer_id }, request);
this.grpcImpl.forceSync(call, handleResponse(reply, SyncStatusResponse));
});

//================Casts================
// @doc-tag: /castById?fid=...&hash=...
this.app.get<{ Querystring: { fid: string; hash: string } }>("/v1/castById", (request, reply) => {
Expand Down
19 changes: 19 additions & 0 deletions apps/hubble/src/rpc/test/httpServer.test.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -839,6 +839,25 @@ describe("httpServer", () => {
});
});

describe("sync APIs", () => {
test("stopSync", async () => {
const url = getFullUrl("/v1/stopSync");
const response = await axios.post(url, {});

expect(response.status).toBe(200);
expect(response.data).toEqual(
protoToJSON(
SyncStatusResponse.create({
isSyncing: false,
engineStarted: true,
syncStatus: [],
}),
SyncStatusResponse,
),
);
});
});

async function axiosGet(url: string) {
try {
return await axios.get(url);
Expand Down
102 changes: 102 additions & 0 deletions apps/hubble/www/docs/docs/httpapi/sync.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Sync API
> ⚠️ **WARNING:**
> These APIs are experimental and should not be relied on for general use. They may change without notice or be removed in future versions.
## stopSync

Stop the synchronization process

**Query Parameters**
| Parameter | Description | Example |
| --------- | ----------- | ------- |
| | This endpoint accepts no parameters | |

- **Example**

```bash
curl -X POST http://127.0.0.1:2281/v1/stopSync
```

**Response**

```json
{
"isSyncing": false,
"syncStatus": [],
"engineStarted": true
}
```

## syncStatus

Get the current synchronization status

**Query Parameters**
| Parameter | Description | Example |
| --------- | ----------- | ------- |
| peer_id | ID of the peer to get sync status for | 12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o |

- **Example**

```bash
curl http://127.0.0.1:2281/v1/syncStatus?peer_id=12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o
```

**Response**

```json
{
"isSyncing": true,
"syncStatus": [
{
"peerId": "12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o",
"inSync": "true",
"shouldSync": true,
"divergencePrefix": "0x1234567890abcdef",
"divergenceSecondsAgo": 300,
"theirMessages": 1000000,
"ourMessages": 999950,
"lastBadSync": 1705796040744,
"score": 0.95
}
],
"engineStarted": true
}
```

## forceSync

Force synchronization with a specific peer

**Query Parameters**
| Parameter | Description | Example |
| --------- | ----------- | ------- |
| peer_id | ID of the peer to force sync with | 12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o |

- **Example**

```bash
curl -X POST http://127.0.0.1:2281/v1/forceSync?peer_id=12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o
```

**Response**

```json
{
"isSyncing": true,
"syncStatus": [
{
"peerId": "12D3KooWJJ9h4XVrVKgMr8ZgF6FKasEBiFEGtL7bmE8RQgzhKq1o",
"inSync": "false",
"shouldSync": true,
"divergencePrefix": "0x1234567890abcdef",
"divergenceSecondsAgo": 0,
"theirMessages": 1000000,
"ourMessages": 999950,
"lastBadSync": 0,
"score": 1.0
}
],
"engineStarted": true
}
```

0 comments on commit 667a5b3

Please sign in to comment.