alihan/llama-cpp-python
1
0
Fork 0
mirror of https://github.com/abetlen/llama-cpp-python.git synced 2023-09-07 17:34:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #140 from abetlen/Niek/main

Browse Source 
Add Dockerfile
This commit is contained in:
Andrei
2023-05-02 01:06:00 -04:00
committed by GitHub
parent e644e75915 36c81489e7
commit 25062cecd3
5 changed files with 223 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -72,6 +72,14 @@ python3 -m llama_cpp.server
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
## Docker image
A Docker image is available on [GHCR](https://ghcr.io/abetlen/llama-cpp-python). To run the server:
```bash
docker run --rm -it -p8000:8000 -v /path/to/models:/models -eMODEL=/models/ggml-model-name.bin ghcr.io/abetlen/llama-cpp-python:latest
```
## Low-level API
The low-level API is a direct `ctypes` binding to the C API provided by `llama.cpp`.