alihan/text-generation-inference
1
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2023-08-23 10:47:54 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: aws sagemaker compatible image (#147)

Browse Source 
The only difference is that now it pushes to
registry.internal.huggingface.tech/api-inference/community/text-generation-inference/sagemaker:...
instead of
registry.internal.huggingface.tech/api-inference/community/text-generation-inference:sagemaker-...

---------

Co-authored-by: Philipp Schmid <32632186+philschmid@users.noreply.github.com>
This commit is contained in:
OlivierDehaene
2023-03-29 21:38:30 +02:00
committed by GitHub
parent c9bdaa8b73
commit d503e8f09d
4 changed files with 86 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
Dockerfile
View File

@@ -27,7 +27,7 @@ COPY router router
COPY launcher launcher
RUN cargo build --release
FROM nvidia/cuda:11.8.0-devel-ubuntu22.04
FROM nvidia/cuda:11.8.0-devel-ubuntu22.04 as base
ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
@@ -76,5 +76,16 @@ COPY --from=builder /usr/src/target/release/text-generation-router /usr/local/bi
# Install launcher
COPY --from=builder /usr/src/target/release/text-generation-launcher /usr/local/bin/text-generation-launcher
# AWS Sagemaker compatbile image
FROM base as sagemaker
COPY sagemaker-entrypoint.sh entrypoint.sh
RUN chmod +x entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
# Original image
FROM base
ENTRYPOINT ["text-generation-launcher"]
CMD ["--json-output"]