alihan/llama-cpp-python
1
0
Fork 0
mirror of https://github.com/abetlen/llama-cpp-python.git synced 2023-09-07 17:34:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update context size defaults Close #11

Browse Source
This commit is contained in:
Andrei Betlen
2023-04-03 20:11:13 -04:00
parent c0a5c0171f
commit f7ab8d55b2
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/fastapi_server.py
View File

@@ -34,6 +34,7 @@ llama = llama_cpp.Llama(
embedding=True, embedding=True,
n_threads=6, n_threads=6,
n_batch=2048, n_batch=2048,
n_ctx=2048,
) )