Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blinko doesn't honor ollama unloading of model #488

Open
elegast-me opened this issue Feb 8, 2025 · 1 comment
Open

blinko doesn't honor ollama unloading of model #488

elegast-me opened this issue Feb 8, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@elegast-me
Copy link

Describe the bug
even though OLLAMA_KEEP_ALIVE is set to 0. Blinko seems to keep the model in memory whenever any ai request is made.

To Reproduce
Steps to reproduce the behavior:
any ai request keeps model in memory even when ollama_keep_alive is set to 0 in system env.

Desktop (please complete the following information):

  • OS: win10
  • ollama
Copy link

dosubot bot commented Feb 8, 2025

Hey @elegast-me! I'm here to help you with any bugs, questions, or contributions you might have. Let's get this issue sorted out together!

Blinko has error handling and logging implemented in the AiStore class, which might help diagnose why the model is not unloading. Specifically, the completionsStream and writeStream methods log errors that occur during their execution. If an error occurs in completionsStream and it is not an "interrupted" error, it logs the error message using RootStore.Get(ToastPlugin).error(error.message). Similarly, in the writeStream method, errors are logged to the console with console.log('writeStream error', error). Checking these logs might provide insights into the issue [1].

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@dosubot dosubot bot added the bug Something isn't working label Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant