The returned results show garbled content?

[mxqin]

The running command used is：
python3 serve/gorilla_cli.py --model-path model/gorilla-7b-th-v0/

But the returned results show garbled content

How did this problem arise and how should it be resolved?

[Simplegram]

Hello, I got the same problem. How did you solve this?

[ShishirPatil]

Hey @Simplegram if you are getting garbled content, one possible reason is because you might have not applied the delta to the base weights. You need to get the base llama weights and apply the delta as described here: https://github.com/ShishirPatil/gorilla/tree/main/inference#applying-delta-weights

Or you can use the mpt and falcon based models, that can be directly downloaded from hf and don't need any deltas.