Merge pull request ggml-org#14 from hypnopump/update_macos

Update macOS, better instructions, streaming output
abetlen · Apr 4, 2023 · dc679bf · dc679bf
2 parents 77e1998 + d380134
commit dc679bf
Show file tree

Hide file tree

Showing 2 changed files with 7 additions and 3 deletions.
diff --git a/README.md b/README.md
@@ -57,6 +57,10 @@ cmake -DBUILD_SHARED_LIBS=ON .
 cmake --build . --config Release
 ```
 
+* **Anconda & M1 users**: Please verify that `CMAKE_SYSTEM_PROCESSOR: arm64` after running `cmake -DBUILD_SHARED_LIBS=ON .`
+  * If it detects `x86_64`, edit the `CMakeLists.txt` file under the `# Compile flags` to add `set(CMAKE_SYSTEM_PROCESSOR "arm64")`
+
+
 If everything went OK, `librwkv.so` (Linux) or `rwkv.o` (MacOS) file should appear in the base repo folder.
 
 
@@ -66,10 +70,10 @@ If everything went OK, `librwkv.so` (Linux) or `rwkv.o` (MacOS) file should appe
 
 ```commandline
 # Windows
-python rwkv\convert_rwkv_to_ggml.py C:\RWKV-4b-Pile-169M-20220807-8023.pth C:\rwkv.cpp-169M.bin float32
+python rwkv\convert_rwkv_to_ggml.py C:\RWKV-4-Pile-169M-20220807-8023.pth C:\rwkv.cpp-169M.bin float16
 
 # Linux / MacOS
-python rwkv/convert_pytorch_to_ggml.py ~/Downloads/RWKV-4b-Pile-169M-20220807-8023.pth ~/Downloads/rwkv.cpp-169M.bin float32
+python rwkv/convert_pytorch_to_ggml.py ~/Downloads/RWKV-4-Pile-169M-20220807-8023.pth ~/Downloads/rwkv.cpp-169M.bin float16
 ```
 
 #### 3.1. Optionally, quantize the model

diff --git a/rwkv/chat_with_bot.py b/rwkv/chat_with_bot.py
@@ -79,7 +79,7 @@
 
         decoded = tokenizer.decode([token])
 
-        print(decoded, end='')
+        print(decoded, end='', flush=True)
 
         if '\n' in decoded:
             break