alihan/llama-cpp-python
1
0
Fork 0
mirror of https://github.com/abetlen/llama-cpp-python.git synced 2023-09-07 17:34:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

llama_cpp server: move logprobs to supported

Browse Source 
I think this is actually supported (its in the arguments of `LLama.__call__`, which is how the completion is invoked). decision: mark as supported
This commit is contained in:
Lucas Doyle
2023-04-29 11:48:49 -07:00
parent b47b9549d5
commit 1e42913599
1 changed files with 1 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
llama_cpp/server/app.py
View File

@@ -79,12 +79,11 @@ class CreateCompletionRequest(BaseModel):
echo: bool = False echo: bool = False
stop: Optional[List[str]] = [] stop: Optional[List[str]] = []
stream: bool = False stream: bool = False
logprobs: Optional[int] = Field(None)
# ignored, but marked as required for the sake of compatibility with openai's api # ignored, but marked as required for the sake of compatibility with openai's api
model: str = model_field model: str = model_field
logprobs: Optional[int] = Field(None)
# llama.cpp specific parameters # llama.cpp specific parameters
top_k: int = 40 top_k: int = 40
repeat_penalty: float = 1.1 repeat_penalty: float = 1.1