47 lines
2.4 KiB
Markdown
47 lines
2.4 KiB
Markdown
# 🔑 QueryKey
|
|
|
|
QueryKey is a flexible playground for comparing and optimizing LLM prompts. It lets you quickly generate, test and compare candidate prompts with realistic sample data.
|
|
|
|
<img src="https://github.com/corbt/querykey/assets/176426/fc7624c6-5b65-4d4d-82b7-4a816f3e5678" alt="demo" height="400px">
|
|
|
|
Currently there's a public playground available at [https://querykey.dev/](https://querykey.dev/), but the recommended approach is to [run locally](#running-locally).
|
|
|
|
## High-Level Features
|
|
|
|
**Configure Multiple Prompts**
|
|
Set up multiple prompt configurations and compare their output side-by-side. Each configuration can be configured independently.
|
|
|
|
**Visualize Responses**
|
|
Inspect prompt completions side-by-side.
|
|
|
|
**Test Many Inputs**
|
|
QueryKey lets you *template* a prompt. Use the templating feature to run the prompts you're testing against many potential inputs for broader coverage of your problem space than you'd get with manual testing.
|
|
|
|
**🪄 Auto-generate Test Scenarios**
|
|
QueryKey includes a tool to generate new test scenarios based on your existing prompts and scenarios. Just click "Autogenerate Scenario" to try it out!
|
|
|
|
**Prompt Validation and Typeahead**
|
|
We use OpenAI's OpenAPI spec to automatically provide typeahead and validate prompts.
|
|
|
|
<img alt="typeahead" src="https://github.com/corbt/querykey/assets/176426/acc638f8-d851-4742-8d01-fe6f98890840" height="300px">
|
|
|
|
**Function Call Support**
|
|
Natively supports [OpenAI function calls](https://openai.com/blog/function-calling-and-other-api-updates) on supported models.
|
|
|
|
<img height="300px" alt="function calls" src="https://github.com/corbt/querykey/assets/176426/48ad13fe-af2f-4294-bf32-62015597fd9b">
|
|
|
|
## Supported Models
|
|
QueryKey currently supports GPT-3.5 and GPT-4. Wider model support is planned.
|
|
|
|
## Running Locally
|
|
|
|
1. Install [Postgresql](https://www.postgresql.org/download/).
|
|
2. Install [NodeJS 20](https://nodejs.org/en/download/current) (earlier versions will very likely work but aren't tested).
|
|
3. Install `pnpm`: `npm i -g pnpm`
|
|
4. Clone this repository: `git clone https://github.com/querykey/querykey`
|
|
5. Install the dependencies: `cd querykey && pnpm install`
|
|
6. Create a `.env` file (`cp .env.example .env`) and enter your `OPENAI_API_KEY`.
|
|
7. Update `DATABASE_URL` if necessary to point to your Postgres instance and run `pnpm prisma db push` to create the database.
|
|
8. Start the app: `pnpm dev`.
|
|
9. Navigate to [http://localhost:3000](http://localhost:3000)
|