Error: connect ECONNREFUSED ::1:8000

[Joly0]

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

[ktfcaptain]

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

Ever figure anything out? I'm experiencing the same issue and was hoping someone would respond by now...

[Joly0]

Nope, was not able to solve this yet. I have no idea why this is happening

[edgar971]

Hey, i am trying to setup Open-Chat as a docker in Unraid and when starting, i get this error in the docker logs

though trying to open that http address through a browser (changing localhost to an actual ip of the host server) i am able to retrieve a correct response

My assumption is that the API crashed. Can you share more details on your setup? Are you using NVIDIA GPU and do you have the drivers? Any other logs?

[gellote]

same problem

[CodingAna]

Bumping this, I have the same error.

Platform: Windows 10 (VM in unRAID)
GPU: RTX 3070 (8GB VRAM) (Passthrough from unRAID), Driver version: 537.13
RAM: 20GB
Model: 7B, 10 Layers loaded to the GPU (barely 2GB VRAM iirc)

My full log: https://pastebin.com/9qqP5jdb
After some time of doing nothing I also get an out of memory error, even though my RAM and VRAM usage seems fine.
Here's the additional log including the memory error: https://pastebin.com/U4uEXiVU

I'm using, except for the gpu layer count = 10, the default config, whatsoever when following the README.

Edit: For whatever reason it works now. I didn't do anything and when I was about to close the browser, I saw the chat window and was able to select the model. The chat also works. I don't know how and why, but it works now.
Edit 2: Must have been the wind. When restarting the container, it throws the ECONNREFUSED error when loading the models.
Edit 3:Okay, I think I know how to reproduce it at least. Loading the models will fail with the error until I get the out of memory error (which doesn't seem to have an effect to (V)RAM) and then the model data loads and I can use the chat.

[gellote]

In my case I solved it by loading version 7B instead of 13B. Since I have 8GB (GTX 1080) of vram and apparently the 13B model exceeds the memory.

[gellote]

In model download url: https://huggingface.co/TheBloke/Nous-Hermes-Llama-2-7B-GGML/resolve/main/nous-hermes-llama-2-7b.ggmlv3.q4_0.bin

[mrwhistles]

Like most others my system was resolving localhost to IPv6 ::1. Setting environment variable "OPENAI_API_HOST=http://127.0.0.1:8000" solved the issue for me.