File tree 3 files changed +8
-6
lines changed
3 files changed +8
-6
lines changed Original file line number Diff line number Diff line change @@ -70,9 +70,7 @@ conda activate moe_peft
70
70
# Install requirements
71
71
pip3 install -r requirements.txt --upgrade
72
72
# Install extra requirements
73
- pip3 install ninja
74
- pip3 install bitsandbytes==0.43.1
75
- pip3 install flash-attn==2.5.8 --no-build-isolation
73
+ bash misc/install-extra-linux.sh
76
74
```
77
75
78
76
## Verification
Original file line number Diff line number Diff line change @@ -34,15 +34,15 @@ You can use the `MOE_PEFT_EXECUTOR_TYPE` environment variable to force MoE-PEFT
34
34
| | Model | Model Size |
35
35
| ---------| --------------------------------------------------| -------------|
36
36
| &check ; | [ LLaMA 1/2] ( https://huggingface.co/meta-llama ) | 7B/13B/70B |
37
- | &check ; | [ LLaMA 3/3.1 ] ( https://huggingface.co/meta-llama ) | 8B/70B |
37
+ | &check ; | [ LLaMA 3.x ] ( https://huggingface.co/meta-llama ) | 3B/ 8B/70B |
38
38
| &check ; | [ Yi 1/1.5] ( https://huggingface.co/01-ai ) | 6B/9B/34B |
39
39
| &check ; | [ TinyLLaMA] ( https://huggingface.co/TinyLlama ) | 1.1B |
40
- | &check ; | [ Qwen 1.5/2] ( https://huggingface.co/Qwen ) | 0.5B ~ 72B |
40
+ | &check ; | [ Qwen 1.5/2.x ] ( https://huggingface.co/Qwen ) | 0.5B ~ 72B |
41
41
| &check ; | [ Gemma] ( https://huggingface.co/google ) | 2B/7B |
42
42
| &check ; | [ Gemma 2] ( https://huggingface.co/google ) | 9B/27B |
43
43
| &check ; | [ Mistral] ( https://huggingface.co/mistralai ) | 7B |
44
44
| &check ; | [ Phi 1.5/2] ( https://huggingface.co/microsoft ) | 2.7B |
45
- | &check ; | [ Phi 3/3.5 ] ( https://huggingface.co/microsoft ) | 3.8B/7B/14B |
45
+ | &check ; | [ Phi 3.x/4 ] ( https://huggingface.co/microsoft ) | 3.8B/7B/14B |
46
46
| &check ; | [ ChatGLM 1/2/3] ( https://huggingface.co/THUDM ) | 6B |
47
47
| &check ; | [ GLM 4] ( https://huggingface.co/THUDM ) | 6B |
48
48
Original file line number Diff line number Diff line change
1
+ # Install extra requirements
2
+ pip3 install ninja
3
+ pip3 install bitsandbytes==0.44.1
4
+ pip3 install flash-attn==2.6.3 --no-build-isolation
You can’t perform that action at this time.
0 commit comments