Kaushik Deka 5bb931e4d6 Add prompt caching (Sonnet, Haiku only) (#3411) ...

* Add prompt caching

* remove anthropic-version from extra_headers

* change supports_prompt_caching method to attribute

* change caching strat and log cache statistics

* add reminder as a new message to fix caching

* fix unit test

* append reminder to the end of the last message content

* move token logs to post completion function

* fix unit test failure

* fix reminder and prompt caching

* unit tests for prompt caching

* add test

* clean up tests

* separate reminder, use latest two messages

* fix tests

---------

Co-authored-by: tobitege <10787084+tobitege@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>

2024-08-26 20:46:44 -04:00

..

bedrock.py

Rename OpenDevin to OpenHands (#3472)

2024-08-20 00:44:54 +08:00

llm.py

Add prompt caching (Sonnet, Haiku only) (#3411)

2024-08-26 20:46:44 -04:00