Skip to content

Commit 736c19e

Browse files
committed
docs: update readme
1 parent 9a129eb commit 736c19e

File tree

2 files changed

+11
-9
lines changed

2 files changed

+11
-9
lines changed

README.MD

+4
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ ChatLuna 核心相关库。
1515

1616
ChatLuna 对话存储相关库,提供相关 API。基于 [minato](https://github.com/cordiverse/minato) 实现与数据库的交互。
1717

18+
- [@chatluna/utils](./packages/utils/README.MD)
19+
20+
ChatLuna 辅助库,提供一些简单的 API 如 SSE 解析,等待锁等。
21+
1822
- [@chatluna/koishi](./packages/koishi/README.MD)
1923

2024
ChatLuna 运行在 Koishi 上的核心支持库。

packages/core/src/utils/tiktoken.ts

+7-9
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
import { Request } from '@chatluna/core/service'
2+
import {
3+
ChatLunaError,
4+
ChatLunaErrorCode,
5+
withResolver
6+
} from '@chatluna/core/utils'
17
import { Context } from '@cordisjs/core'
28
import {
39
getEncodingNameForModel,
@@ -6,13 +12,7 @@ import {
612
TiktokenEncoding,
713
TiktokenModel
814
} from 'js-tiktoken/lite'
9-
import { Request } from '@chatluna/core/service'
1015
import { fetch } from 'undici'
11-
import {
12-
ChatLunaError,
13-
ChatLunaErrorCode,
14-
withResolver
15-
} from '@chatluna/core/utils'
1616

1717
globalThis.chatluna_tiktoken_cache = globalThis.chatluna_tiktoken_cache ?? {}
1818

@@ -50,12 +50,10 @@ export async function getEncoding(
5050
throw e
5151
})
5252

53-
const tiktoken = new Tiktoken(
53+
cache[encoding] = new Tiktoken(
5454
tiktokenBPE,
5555
options?.extendedSpecialTokens
5656
)
57-
58-
cache[encoding] = tiktoken
5957
}
6058

6159
return cache[encoding]

0 commit comments

Comments
 (0)