alihan/tokencost
1
0
Fork 0
mirror of https://github.com/AgentOps-AI/tokencost.git synced 2024-06-22 04:30:40 +03:00
Code Issues Packages Projects Releases Wiki Activity

update to everything

Browse Source
This commit is contained in:
reibs
2023-12-24 23:58:17 -08:00
parent 1b1f0ba6ec
commit a140b1093e
4 changed files with 136 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
README.md
View File

@@ -2,117 +2,98 @@
## Overview
TokenCost is a specialized tool designed for calculating the token count and associated U.S. dollar cost of strings and messages used in Large Language Models (LLMs). This utility is particularly useful for developers and researchers working with language models, enabling them to estimate the computational resources required for processing various inputs and their returned outputs.
TokenCost is a specialized tool designed for calculating the USD cost of using major Large Language Models (LLMs) APIs.
### Features
* **LLM Price Tracking** Major LLM providers frequently add new models and update pricing. This repo helps track the latest price changes
* **Token counting** Accurately count prompt tokens before sending OpenAI requests
* **Easy integration** Get the cost of a prompt or completion with a single function
### Example usage:
```python
from tokencost import calculate_cost
from tokencost import calculate_prompt_cost, calculate_completion_cost
prompt = "Sample input"
response = "Sample response text"
model = "gpt-3.5-turbo"
prompt = "Hello world"
completion = "How may I assist you today?"
cost = calculate_cost(prompt, response, model)
print(f"{cost=}) # in units of TPU, which is 1/10,000,000th of a USD.
# cost=90
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 30 + 140 = 170
# In TPUs (token price units), which is 1/10,000,000th of a USD.
```
## Features
- **Token Counting**: Accurately counts the number of tokens in a given string or message.
- **Cost Calculation**: Computes the cost of processing based on the token count, considering the specific pricing model of the LLM in use.
- **Support for Multiple LLMs**: Compatible with various Large Language Models.
- **Easy Integration**: Simple API for integrating with existing projects or workflows.
## Installation
Tokencost can be installed either via PyPI or GitHub.
#### Recommended: with [PyPI](https://pypi.org/project/tokencost/) (Python package):
#### Recommended: [PyPI](https://pypi.org/project/tokencost/):
```bash
pip install tokencost
```
#### OR: with [GitHub](https://github.com/AgentOps-AI/tokencost):
```bash
git clone git@github.com:AgentOps-AI/tokencost.git
cd tokencost
pip install -e .
```
## Usage
To use TokenCost, follow these steps:
1. Import the module:
- Recommended: If you want to call the functions as `function_name` directly:
### Counting tokens
```python
from tokencost import count_message_tokens, count_string_tokens, calculate_cost
from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{ "role": "user", "content": "Hello world"}]
# Counting tokens in prompts formatted as message lists
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo"))
# TODO:
# Alternatively, counting tokens in string prompts
print(count_string_prompt(prompt="Hello world", model="gpt-3.5-turbo"))
# TODO:
```
- OR if you want to call the functions as `tokencost.function_name`:
### Cost estimates
Calculating the cost of prompts and completions from OpenAI requests
```python
import tokencost
from openai import OpenAI
client = OpenAI()
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Say this is a test"}]
chat_completion = client.chat.completions.create(
messages=prompt, model=model
)
completion = chat_completion.choices[0].message.content
# "This is a test."
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 180 + 100 = 280
from tokencost import USD_PER_TPU
print(f"Cost USD: ${(prompt_cost + completion_cost)/USD_PER_TPU}")
# $2.8e-05
```
2. Calculate tokens and cost (using `from tokencost import count_message_tokens, count_string_tokens, calculate_cost`):
**Calculating cost using string prompts instead of messages:**
```python
# First example using string input.
string_prompt = "Your sample text here"
response = "Sample response text"
model= "gpt-3.5-turbo"
cost = calculate_cost(string_prompt, response, model)
prompt_string_token_count = count_string_tokens(string_prompt, model)
print(f"{prompt_string_token_count=}, {completion_string_token_count=}")
print(f"Cost: ${string_cost/USD_PER_TPU} ({cost/CENTS_PER_TPU} cents)")
# Prints the below:
# prompt_cost=15
# completion_cost=20
# prompt_string_token_count=4, completion_string_token_count=3
# Cost: $1.2e-05 (0.0012 cents)
```
**Calculating cost using chat messages instead of string inputs:**
```python
messages =[
{
"role": "user",
"content": "Hey how is your day",
},
{
"role": 'assistant',
"content": "As an LLM model I do not have days"
},
{
"role": "user",
"content": "Err sure okay fine"
}
]
response = "Sample response text"
messages = "Hello world"
response = "How may I assist you today?"
model= "gpt-3.5-turbo"
cost = calculate_cost(messages, response, model)
print(f"Cost: ${message_cost/USD_PER_TPU} ({cost/CENTS_PER_TPU} cents)")
# Cost: $5.7e-05 (0.0057 cents)
print(f"Cost: ${message_cost/USD_PER_TPU}")
# TODO:
prompt_message_token_count = count_message_tokens(messages, model)
print(f"{prompt_message_token_count=}")
# prompt_message_token_count=34
# TODO:
completion_string_token_count = count_string_tokens(response, model)
print(f"{completion_string_token_count=}
# completion_string_token_count=3
# TODO:
```
## Cost table
@@ -139,6 +120,16 @@ print(f"{completion_string_token_count=}
Units denominated in TPUs (Token Price Units = 1/10,000,000 USD)
### Running locally
#### Installation via [GitHub](https://github.com/AgentOps-AI/tokencost):
```bash
git clone git@github.com:AgentOps-AI/tokencost.git
cd tokencost
pip install -e .
```
## Running tests
0. Install `pytest` if you don't have it already
@@ -153,6 +144,8 @@ pip install pytest
pytest tests
```
This repo also supports `tox`, simply run `python -m tox`.
## Contributing
Contributions to TokenCost are welcome! Feel free to create an [issue](https://github.com/AgentOps-AI/tokencost/issues) for any bug reports, complaints, or feature suggestions.

4
tokencost/__init__.py
View File

@@ -1,2 +1,2 @@
from .costs import count_message_tokens, count_string_tokens, calculate_cost
from .constants import TOKEN_COSTS, USD_PER_TPU, CENTS_PER_TPU
from .costs import count_message_tokens, count_string_tokens, calculate_completion_cost, calculate_prompt_cost
from .constants import TOKEN_COSTS, USD_PER_TPU

1
tokencost/constants.py
View File

@@ -19,7 +19,6 @@ is considered a prompt (for the purpose of context) and will thus cost prompt to
"""
USD_PER_TPU = float(10_000_000)
CENTS_PER_TPU = float(100_000)
# How to read TOKEN_COSTS:
# Each prompt token costs __ TPUs per token.

86
tokencost/costs.py
View File

@@ -2,7 +2,7 @@
Costs dictionary and utility tool for counting tokens
"""
import tiktoken
from typing import Union, List
from typing import Union, List, Dict
from .constants import TOKEN_COSTS
@@ -12,10 +12,17 @@ from .constants import TOKEN_COSTS
# https://github.com/anthropics/anthropic-tokenizer-typescript/blob/main/index.ts
def count_message_tokens(messages: List, model: str) -> int:
"""Return the total number of tokens in a list of (prompt or completion) messages."""
if not messages:
raise KeyError("Empty message list provided.")
def count_message_tokens(messages: List[Dict[str, str]], model: str) -> int:
"""
Return the total number of tokens in a prompt's messages.
Args:
messages (List[Dict[str, str]]): Message format for prompt requests. e.g.:
[{ "role": "user", "content": "Hello world"},
{ "role": "assistant", "content": "How may I assist you today?"}]
model (str): Name of LLM to choose encoding for.
Returns:
Total number of tokens in message.
"""
model = model.lower()
try:
encoding = tiktoken.encoding_for_model(model)
@@ -46,8 +53,6 @@ def count_message_tokens(messages: List, model: str) -> int:
else:
raise KeyError(
f"""num_tokens_from_messages() is not implemented for model {model}.
Double check your spelling, or open an issue/PR:
https://github.com/AgentOps-AI/tokencost/blob/main/tokencost/constants.py
See https://github.com/openai/openai-python/blob/main/chatml.md for how messages are converted to tokens."""
)
num_tokens = 0
@@ -66,49 +71,84 @@ def count_string_tokens(string: str, model: str) -> int:
Returns the number of tokens in a (prompt or completion) text string.
Args:
string (str): The text string.
string (str): The text string
model_name (str): The name of the encoding to use. (e.g., "gpt-3.5-turbo")
Returns:
int: The number of tokens in the text string.
"""
model = model.lower()
encoding = tiktoken.encoding_for_model(model)
try:
encoding = tiktoken.encoding_for_model(model)
except KeyError:
print("Warning: model not found. Using cl100k_base encoding.")
encoding = tiktoken.get_encoding("cl100k_base")
return len(encoding.encode(string))
def calculate_cost(prompt: Union[List[dict], str], completion: Union[List[dict], str], model: str) -> float:
def calculate_prompt_cost(prompt: Union[List[dict], str], model: str) -> int:
"""
Calculate the cost of tokens in TPUs. 1 TPU = 1/10,000,000 of $1 (USD), so 100,000 TPUs = $0.01.
Calculate the prompt's cost in token price units (TPU). 1 TPU = $1/10,000,000.
e.g. 100,000 TPUs = $0.01.
Args:
prompt (Union[List[dict], str]): List of message objects or single string prompt.
completion (Union[List[dict], str]): List of message objects or single string completion.
model (str): The model name.
Returns:
float: The calculated cost in TPUs.
int: The calculated cost in TPUs.
e.g.:
>>> prompt = [{ "role": "user", "content": "Hello world"},
{ "role": "assistant", "content": "How may I assist you today?"}]
# or
>>> prompt = "Hello world"
>>> calculate_prompt_cost(prompt, "gpt-3.5-turbo")
# TODO:
"""
model = model.lower()
if model not in TOKEN_COSTS:
raise KeyError(
f"""calculate_cost() is not implemented for model {model}.
Double-check your spelling, or submit an issue/PR:
https://github.com/AgentOps-AI/tokencost/blob/main/tokencost/constants.py
"""
f"""Model {model} is not implemented.
Double-check your spelling, or submit an issue/PR"""
)
if not isinstance(prompt, (list, str)) or not isinstance(completion, (list, str)):
if not isinstance(prompt, (list, str)) or not isinstance(prompt, (list, str)):
raise TypeError(
f"""Prompt and completion each must be either a string or list of message objects.
They are {type(prompt)} and {type(completion)}, respectively.
They are {type(prompt)} and {type(prompt)}, respectively.
"""
)
prompt_tokens = count_string_tokens(prompt, model) if isinstance(
prompt, str) else count_message_tokens(prompt, model)
prompt_cost = TOKEN_COSTS[model]["prompt"]
completion_tokens = count_string_tokens(completion, model) if isinstance(
completion, str) else count_message_tokens(completion, model)
return prompt_cost * prompt_tokens
def calculate_completion_cost(completion: str, model: str) -> int:
"""
Calculate the prompt's cost in token price units (TPU). 1 TPU = $1/10,000,000.
e.g. 100,000 TPUs = $0.01.
Args:
completion (str): Completion string.
model (str): The model name.
Returns:
int: The calculated cost in TPUs.
e.g.:
>>> completion = "How may I assist you today?"
>>> calculate_completion_cost(completion, "gpt-3.5-turbo")
# TODO:
"""
if model not in TOKEN_COSTS:
raise KeyError(
f"""Model {model} is not implemented.
Double-check your spelling, or submit an issue/PR"""
)
completion_tokens = count_string_tokens(completion, model)
completion_cost = TOKEN_COSTS[model]["completion"]
cost = (prompt_tokens * prompt_cost + completion_tokens * completion_cost)
return cost
return completion_cost * completion_tokens