alihan/llama-cpp-python
1
0
Fork 0
mirror of https://github.com/abetlen/llama-cpp-python.git synced 2023-09-07 17:34:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update README to use cli options for server

Browse Source
This commit is contained in:
Andrei Betlen
2023-05-07 05:10:52 -04:00
parent 627811ea83
commit 3adc8fb3ae
1 changed files with 1 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@@ -68,18 +68,9 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
To install the server package and get started: To install the server package and get started:
Linux/MacOS
```bash ```bash
pip install llama-cpp-python[server] pip install llama-cpp-python[server]
export MODEL=./models/7B/ggml-model.bin python3 -m llama_cpp.server --model models/7B/ggml-model.bin
python3 -m llama_cpp.server
```
Windows
```cmd
pip install llama-cpp-python[server]
SET MODEL=..\models\7B\ggml-model.bin
python3 -m llama_cpp.server
``` ```
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation. Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.