Skip to content

Commit

Permalink
Update LICENSE and TODOs in README (ggml-org#14)
Browse files Browse the repository at this point in the history
* Update LICENSE with our copyright notice

* Update README.md

* fix readme anchor

* Update README.md
  • Loading branch information
hodlen authored Dec 19, 2023
1 parent 7cebaba commit e3b4b85
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 10 deletions.
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
MIT License

Copyright (c) 2023 Georgi Gerganov
Copyright (c) 2023 SJTU-IPADS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
21 changes: 11 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,19 @@ We have tested PowerInfer on the following platforms:
- x86-64 CPU (with AVX2 instructions) on Linux
- x86-64 CPU and NVIDIA GPU on Linux
- Apple M Chips on macOS (As we do not optimize for Mac, the performance improvement is not significant now.)


And new features coming soon:

- Mistral-7B model
- Online fine-grained FFN offloading to GPU
- Metal backend for sparse inference on macOS

## Getting Started

- [Installation](##setup--installation)
- [Model Weights](##model-weights)
- [Installation](#setup-and-installation)
- [Model Weights](#model-weights)

## Setup & Installation
## Setup and Installation
### Get the Code

```bash
Expand All @@ -70,12 +75,7 @@ cd PowerInfer
### Build
In order to build PowerInfer you have two different options. These commands are supposed to be run from the root directory of the project.

Using `make` on Linux or macOS:
```bash
make
```

Using `CMake`:
Using `CMake` on Linux or macOS:
* If you have one GPU:
```bash
cmake -S . -B build -DLLAMA_CUBLAS=ON
Expand Down Expand Up @@ -130,6 +130,7 @@ PowerInfer achieves up to 11x and 8x speedup for FP16 and INT4 models!
We will release the code and data in the following order, please stay tuned!

- [x] Release core code of PowerInfer, supporting Llama-2, Falcon-40B.
- [ ] Support Mistral-7B
- [ ] Release perplexity evaluation code
- [ ] Support Metal for Mac
- [ ] Release code for OPT models
Expand Down

0 comments on commit e3b4b85

Please sign in to comment.