feat: added local TTS support using Kokoro #401
Conversation
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
❌ Changes requested. Reviewed everything up to cd2f7ff in 1 minute and 34 seconds
More details
- Looked at
378lines of code in3files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. gptme/tools/tts.py:155
- Draft comment:
Consider adding a timeout to therequests.getcall to handle cases where the TTS server is unreachable. - Reason this comment was not posted:
Marked as duplicate.
Workflow ID: wflow_7wCANXf1iIAyU5DO
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 5f67371 in 31 seconds
More details
- Looked at
274lines of code in3files - Skipped
1files when reviewing. - Skipped posting
5drafted comments based on config settings.
1. gptme/tools/tts.py:198
- Draft comment:
Consider making the TTS server URL configurable instead of hardcoding it. This allows flexibility in server location or port. - Reason this comment was not posted:
Confidence changes required:50%
Thespeakfunction ingptme/tools/tts.pyuses a hardcoded URL for the TTS server. This could be made configurable to allow flexibility in server location or port.
2. gptme/tools/tts.py:201
- Draft comment:
Consider adding exception handling for network-related errors when making requests to the TTS server to prevent unhandled exceptions. - Reason this comment was not posted:
Confidence changes required:50%
Thespeakfunction ingptme/tools/tts.pydoes not handle the case where therequests.getcall fails due to network issues or server unavailability. This could lead to unhandled exceptions.
3. gptme/tools/tts.py:211
- Draft comment:
Ensure that the response content is a valid WAV file before attempting to read it withwavfile.readto avoid potential errors. - Reason this comment was not posted:
Confidence changes required:50%
Thespeakfunction ingptme/tools/tts.pyuseswavfile.readwhich might not handle all audio formats correctly. It assumes the response content is always a valid WAV file.
4. pyproject.toml:105
- Draft comment:
Thettsextra is correctly defined with necessary dependencies:sounddevice,scipy, andnumpy. - Reason this comment was not posted:
Confidence changes required:0%
Thettsextra inpyproject.tomlincludesnumpy,scipy, andsounddevice, which are necessary for TTS functionality. This is correctly set up.
5. scripts/tts_server.py:104
- Draft comment:
Consider validating thespeedparameter to ensure it falls within an acceptable range (e.g., 0.5 to 2.0) to prevent unexpected behavior. - Reason this comment was not posted:
Confidence changes required:50%
Thetext_to_speechendpoint inscripts/tts_server.pyuses a default speed of 1.0, which is reasonable. However, it should validate thespeedparameter to ensure it falls within an acceptable range.
Workflow ID: wflow_CrMfSPYC0JQ63v0J
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 1e7ca05 in 37 seconds
More details
- Looked at
36lines of code in1files - Skipped
0files when reviewing. - Skipped posting
2drafted comments based on config settings.
1. scripts/tts_server.py:43
- Draft comment:
The# fmt: skipcomment is unnecessary here unless there's a specific reason to skip formatting. Consider removing it for cleaner code. - Reason this comment was not posted:
Confidence changes required:20%
The import statements forgenerateandbuild_modelare using# fmt: skip, which is unnecessary unless there is a specific formatting issue. It's better to remove it if not needed.
2. scripts/tts_server.py:80
- Draft comment:
torch.loaddoes not have aweights_onlyargument. This will cause a runtime error. Please remove or correct this argument. - Reason this comment was not posted:
Comment was on unchanged code.
Workflow ID: wflow_a4SJ8E7bmq8YjJLZ
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on e006783 in 21 seconds
More details
- Looked at
433lines of code in3files - Skipped
0files when reviewing. - Skipped posting
3drafted comments based on config settings.
1. scripts/tts_server.py:113
- Draft comment:
Consider adding more specific exception handling or logging the stack trace to help diagnose initialization issues. - Reason this comment was not posted:
Confidence changes required:50%
The PR introduces a new feature for TTS using Kokoro, but there are some issues with exception handling and logging that need to be addressed.
2. gptme/tools/tts.py:309
- Draft comment:
Consider logging the stack trace for exceptions to aid in debugging. - Reason this comment was not posted:
Confidence changes required:50%
The PR introduces a new feature for TTS using Kokoro, but there are some issues with exception handling and logging that need to be addressed.
3. gptme/chat.py:252
- Draft comment:
Ensure that thespeakfunction handles exceptions gracefully to prevent crashes if TTS fails. - Reason this comment was not posted:
Confidence changes required:50%
The PR introduces a new feature for TTS using Kokoro, but there are some issues with exception handling and logging that need to be addressed.
Workflow ID: wflow_6N9qV5e15GOYzIWR
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 066f38a in 47 seconds
More details
- Looked at
249lines of code in4files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. gptme/tools/tts.py:314
- Draft comment:
Consider adding a timeout to therequests.getcall to prevent it from hanging indefinitely if the TTS server is unresponsive. Also, handle potential network errors more gracefully. - Reason this comment was not posted:
Comment was on unchanged code.
Workflow ID: wflow_wIA7W55QZ5HGXB8q
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 86adc9b in 16 seconds
More details
- Looked at
14lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. gptme/tools/tts.py:20
- Draft comment:
Good addition to handle OSError for missing PortAudio library. - Reason this comment was not posted:
Confidence changes required:0%
The PR adds an OSError exception to handle cases where the PortAudio library is not found. This is a good addition for robustness.
Workflow ID: wflow_sEtWqSUXOyrKX9BU
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
Codecov ReportAttention: Patch coverage is
✅ All tests successful. No failed tests found.
Additional details and impacted files@@ Coverage Diff @@
## master #401 +/- ##
==========================================
- Coverage 70.99% 69.11% -1.88%
==========================================
Files 68 69 +1
Lines 5540 5750 +210
==========================================
+ Hits 3933 3974 +41
- Misses 1607 1776 +169
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 7c8cc6b in 18 seconds
More details
- Looked at
25lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. pyproject.toml:142
- Draft comment:
Adding mypy overrides for ignoring missing imports for modules like numpy, scipy, sounddevice, and flask without explanation can hide potential issues. Ensure these are necessary and document the reason for each override. - Reason this comment was not posted:
Confidence changes required:50%
The PR adds mypy overrides for several modules, but the description does not mention why these overrides are necessary. This could lead to potential issues if the imports are actually missing or if there are type issues that need to be addressed.
Workflow ID: wflow_OWLftHMvOuFaH98A
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 5c0359b in 21 seconds
More details
- Looked at
16lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. gptme/chat.py:252
- Draft comment:
The PR description mentions integrating the Kokoro TTS model, but the code changes do not reflect this integration. Ensure that the Kokoro model is properly integrated and used for TTS. - Reason this comment was not posted:
Comment did not seem useful.
Workflow ID: wflow_e71i6UDlXbeAAzIp
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
|
Works great! |
New SOTA local TTS model dropped, figured I should try integrating it and get Bob to speak using it.
Closes #60
Important
Adds local TTS support using Kokoro, integrating a new TTS model and server into the system.
gptme/chat.py.step()ingptme/chat.pyto speak responses if TTS tool is available._clean_content_for_speech()ingptme/chat.pyto clean text for TTS.gptme/tools/tts.pyfile for TTS operations, includingspeak(),set_speed(), andset_volume()functions.scripts/tts_server.pyto run a TTS server using FastAPI and Kokoro model./ttsand/healthendpoints for TTS conversion and server health check.pyproject.tomlto includesounddevice,scipy, and other TTS-related packages.scripts/tts_server.pyfrom certain Makefile operations.This description was created by
for 5c0359b. It will automatically update as commits are pushed.