alihan/OpenPipe-llm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

README changes

Browse Source
This commit is contained in:
Kyle Corbitt
2023-06-28 15:10:15 -07:00
parent 92b6ada0b5
commit dc8440c44b
2 changed files with 20 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.env.example
View File

@@ -11,7 +11,8 @@
# Prisma # Prisma
# https://www.prisma.io/docs/reference/database-reference/connection-urls#env # https://www.prisma.io/docs/reference/database-reference/connection-urls#env
DATABASE_URL="postgresql://username:password@localhost:5432/prompt-lab?schema=public" DATABASE_URL="postgresql://postgres:postgres@localhost:5432/prompt-lab?schema=public"
# OpenAI # OpenAI API key. Instructions on generating a key can be found here:
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key
OPENAI_API_KEY="" OPENAI_API_KEY=""

28
README.md
View File

@@ -2,27 +2,33 @@
Prompt Lab is a flexible playground for comparing and optimizing LLM prompts. It lets you quickly generate, test and compare candidate prompts with realistic sample data. Prompt Lab is a flexible playground for comparing and optimizing LLM prompts. It lets you quickly generate, test and compare candidate prompts with realistic sample data.
![main](https://github.com/corbt/prompt-lab/assets/176426/bc5ceb5d-b898-4710-8d62-af622c0d2d19) <img src="https://github.com/corbt/prompt-lab/assets/176426/fc7624c6-5b65-4d4d-82b7-4a816f3e5678" alt="demo" height="400px">
Currently there's a public playground available at [https://promptlab.corbt.com/](https://promptlab.corbt.com/), but the recommended approach is to [run locally](#running-locally). Currently there's a public playground available at [https://promptlab.corbt.com/](https://promptlab.corbt.com/), but the recommended approach is to [run locally](#running-locally).
## High-Level Features ## High-Level Features
- **Configure Multiple Prompts** - Set up multiple prompt configurations and compare their output side-by-side. Each configuration can be configured independently. **Configure Multiple Prompts**
Set up multiple prompt configurations and compare their output side-by-side. Each configuration can be configured independently.
- **Visualize Responses** - Inspect prompt completions side-by-side. **Visualize Responses**
Inspect prompt completions side-by-side.
- **Test Many Inputs** - Prompt Lab lets you *template* a prompt. Use the templating feature to run the prompts you're testing against many potential inputs for broader coverage of your problem space than you'd get with manual testing. **Test Many Inputs**
Prompt Lab lets you *template* a prompt. Use the templating feature to run the prompts you're testing against many potential inputs for broader coverage of your problem space than you'd get with manual testing.
- **🪄 Auto-generate Test Scenarios** - Prompt Lab includes a tool to generate new test scenarios based on your existing prompts and scenarios. Just click "Autogenerate Scenario" to try it out! **🪄 Auto-generate Test Scenarios**
Prompt Lab includes a tool to generate new test scenarios based on your existing prompts and scenarios. Just click "Autogenerate Scenario" to try it out!
- **Prompt Validation and Typeahead** - We use OpenAI's OpenAPI spec to automatically provide typeahead and validate prompts. **Prompt Validation and Typeahead**
We use OpenAI's OpenAPI spec to automatically provide typeahead and validate prompts.
![typeahead](https://github.com/corbt/prompt-lab/assets/176426/d475d88e-473a-4e7d-b20a-a1e7ed097c3e) <img alt="typeahead" src="https://github.com/corbt/prompt-lab/assets/176426/acc638f8-d851-4742-8d01-fe6f98890840" height="300px">
- **Function Call Support** - Natively supports [OpenAI function calls](https://openai.com/blog/function-calling-and-other-api-updates) on supported models. **Function Call Support**
Natively supports [OpenAI function calls](https://openai.com/blog/function-calling-and-other-api-updates) on supported models.
<img width="811" alt="function calls" src="https://github.com/corbt/prompt-lab/assets/176426/48ad13fe-af2f-4294-bf32-62015597fd9b"> <img height="300px" alt="function calls" src="https://github.com/corbt/prompt-lab/assets/176426/48ad13fe-af2f-4294-bf32-62015597fd9b">
## Supported Models ## Supported Models
Prompt Lab currently supports GPT-3.5 and GPT-4. Wider model support is planned. Prompt Lab currently supports GPT-3.5 and GPT-4. Wider model support is planned.
@@ -35,6 +41,6 @@ Prompt Lab currently supports GPT-3.5 and GPT-4. Wider model support is planned.
4. Clone this repository: `git clone https://github.com/prompt-lab/prompt-lab` 4. Clone this repository: `git clone https://github.com/prompt-lab/prompt-lab`
5. Install the dependencies: `cd prompt-lab && pnpm install` 5. Install the dependencies: `cd prompt-lab && pnpm install`
6. Create a `.env` file (`cp .env.example .env`) and enter your `OPENAI_API_KEY`. 6. Create a `.env` file (`cp .env.example .env`) and enter your `OPENAI_API_KEY`.
7. Update `DATABASE_URL` and run `pnpm prisma db push` to create the database. 7. Update `DATABASE_URL` if necessary to point to your Postgres instance and run `pnpm prisma db push` to create the database.
8. Start the app: `pnpm dev` 8. Start the app: `pnpm dev`.
9. Navigate to [http://localhost:3000](http://localhost:3000) 9. Navigate to [http://localhost:3000](http://localhost:3000)