mirror of
https://github.com/docker/genai-stack.git
synced 2024-08-30 16:49:54 +03:00
Add linux-gpu profile to enable GPU (#64)
This commit is contained in:
committed by
GitHub
parent
4071ae1b69
commit
611bf84da2
@@ -1,11 +1,22 @@
|
||||
services:
|
||||
|
||||
llm:
|
||||
llm: &llm
|
||||
image: ollama/ollama:latest
|
||||
profiles: ["linux"]
|
||||
networks:
|
||||
- net
|
||||
|
||||
llm-gpu:
|
||||
<<: *llm
|
||||
profiles: ["linux-gpu"]
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
capabilities: [gpu]
|
||||
|
||||
pull-model:
|
||||
image: genai-stack/pull-model:latest
|
||||
build:
|
||||
|
||||
@@ -38,6 +38,8 @@ No need to install Ollama manually, it will run in a container as
|
||||
part of the stack when running with the Linux profile: run `docker compose --profile linux up`.
|
||||
Make sure to set the `OLLAMA_BASE_URL=http://llm:11434` in the `.env` file when using Ollama docker container.
|
||||
|
||||
To use the Linux-GPU profile: run `docker compose --profile linux-gpu up`. Also change `OLLAMA_BASE_URL=http://llm-gpu:11434` in the `.env` file.
|
||||
|
||||
**Windows**
|
||||
Not supported by Ollama, so Windows users need to generate a OpenAI API key and configure the stack to use `gpt-3.5` or `gpt-4` in the `.env` file.
|
||||
# Develop
|
||||
|
||||
Reference in New Issue
Block a user