alihan/OpenPipe-llm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
170 Commits 60 Branches 0 Tags
011b12abb9f18a2283578dbb68fc04876cf4a1a9
Commit Graph

170 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kyle Corbitt
011b12abb9 cache output evals 2023-07-17 17:52:30 -07:00
Kyle Corbitt
1ba18015bc Merge pull request #53 from OpenPipe/more-eval 
Fix seeds and update eval field names
 2023-07-17 14:26:29 -07:00
Kyle Corbitt
54369dba54 Fix seeds and update eval field names 2023-07-17 14:14:20 -07:00
arcticfly
6b84a59372 Properly catch completion errors (#51) 2023-07-17 10:50:25 -07:00
Kyle Corbitt
8db8aeacd3 Replace function chrome with comment 
Use a block comment to explain the expected prompt formatting instead of function chrome. The advantage here is that once a user builds a mental model of how OpenPipe works they can just delete the comment, instead of the function chrome sitting around and taking up space in the UI forever.
 2023-07-17 10:30:22 -07:00
Kyle Corbitt
64bd71e370 Merge pull request #50 from OpenPipe/remove-default 
remove the default value for PromptVariant.model
 2023-07-14 17:55:38 -07:00
Kyle Corbitt
ca21a7af06 Run checks on main 
This will (1) make sure that anything we push directly passes CI, and also (2) cache the pnpm store on the main branch, which will make it available to PR runs as well and hopefully speed up CI a bit (see https://stackoverflow.com/a/75250061).``
 2023-07-14 17:49:20 -07:00
Kyle Corbitt
3b99b7bd2b remove the default value for PromptVariant.model 
We should be explicit about setting the appropriate model so it always matches the constructFn.
 2023-07-14 17:43:52 -07:00
Kyle Corbitt
0c3bdbe4f2 Merge pull request #49 from OpenPipe/save-button 
Make save button disappear on save
 2023-07-14 17:39:06 -07:00
Kyle Corbitt
74c201d3a8 Make save button disappear on save 
Fixes a bug where often the "Save" button wouldn't disappear as expected the first time you clicked it.
 2023-07-14 17:35:57 -07:00
David Corbitt
ab9c721d09 Revert change to scenarios header 2023-07-14 17:51:12 -06:00
David Corbitt
0a2578a1d8 Update scenarios header negative margin 2023-07-14 17:41:51 -06:00
David Corbitt
1bebaff386 Merge branch 'main' of github.com:corbt/prompt-lab 2023-07-14 16:55:12 -06:00
David Corbitt
3bf5eaf4a2 Properly extract scenario id in new experiment creation 2023-07-14 16:55:09 -06:00
Kyle Corbitt
ded97f8bb9 fix lockfile 2023-07-14 15:55:01 -07:00
Kyle Corbitt
26ee8698be Make it so you can't delete the last prompt or scenario 
No reason for an experiment to have 0 prompts or 0 scenarios and it makes the UI look bad.
 2023-07-14 15:49:42 -07:00
arcticfly
b98eb9b729 Trigger llm output retrieval on server (#39) 
* Rename tables, add graphile workers, update types

* Add dev:worker command

* Update pnpm-lock.yaml

* Remove sentry config import from worker.ts

* Stop generating new cells in cell router get query

* Generate new cells for new scenarios, variants, and experiments

* Remove most error throwing from queryLLM.task.ts

* Remove promptVariantId and testScenarioId from ModelOutput

* Remove duplicate index from ModelOutput

* Move inputHash from cell to output

* Add TODO

* Add todo

* Show cost and time for each cell

* Always show output stats if there is output

* Trigger LLM outputs when scenario variables are updated

* Add newlines to ends of files

* Add another newline

* Cascade ModelOutput deletion

* Fix linting and prettier

* Return instead of throwing for non-pending cell

* Remove pnpm dev:worker from pnpm:dev

* Update pnpm-lock.yaml
 2023-07-14 16:38:46 -06:00
Kyle Corbitt
032c07ec65 Merge pull request #45 from OpenPipe/node-version 
warn folks if they use a lower node version
 2023-07-14 15:03:49 -07:00
Kyle Corbitt
80c0d13bb9 warn folks if they use a lower node version 2023-07-14 14:59:33 -07:00
Kyle Corbitt
f7c94be3f6 Merge pull request #44 from OpenPipe/strip-types 
Strip types from prompt variants
 2023-07-14 14:07:07 -07:00
Kyle Corbitt
c3e85607e0 Strip types from prompt variants 
We want Monaco to treat the prompt constructor as Typescript so we get type checks, but we actually want to save the prompt constructor as Javascript so we can run it directly without transpiling.
 2023-07-14 14:03:28 -07:00
Kyle Corbitt
cd5927b8f5 Merge pull request #43 from OpenPipe/function-ux 
Pseudo function signatures
 2023-07-14 14:01:10 -07:00
Kyle Corbitt
731406d1f4 Pseudo function signatures 
Show pseudo function signatures in the variant editor box as a UX hint that you're typing in javascript and have access to the scenario.
 2023-07-14 13:56:45 -07:00
Kyle Corbitt
3c59e4b774 Merge pull request #42 from OpenPipe/autoformat 
implement format on save
 2023-07-14 12:56:41 -07:00
Kyle Corbitt
a20f81939d implement format on save 2023-07-14 12:33:57 -07:00
Kyle Corbitt
972b1f2333 Merge pull request #41 from OpenPipe/github-actions 
CI checks
 2023-07-14 11:40:42 -07:00
Kyle Corbitt
7321f3deda CI checks 2023-07-14 11:36:47 -07:00
Kyle Corbitt
2bd41fdfbf Merge pull request #40 from OpenPipe:completion-costs 
store model and use to calculate completion costs
 2023-07-14 11:07:15 -07:00
Kyle Corbitt
a5378b106b store model and use to calculate completion costs 2023-07-14 11:06:07 -07:00
Kyle Corbitt
0371dacfca Merge pull request #38 from OpenPipe/prompt-templates 
Use javascript functions for prompt completions instead of templated json
 2023-07-14 07:19:28 -07:00
Kyle Corbitt
2cd82eedd0 rename variantEditor -> sharedVariantEditor 2023-07-13 23:27:39 -07:00
Kyle Corbitt
4770ea34a8 Use javascript functions for prompt completions instead of templated json 2023-07-13 18:01:07 -07:00
arcticfly
1776da937a Update README.md 2023-07-12 01:10:03 -06:00
David Corbitt
357e28c463 Properly pass color into IconLink 2023-07-11 16:46:47 -06:00
David Corbitt
ebe4d5bef6 Reduce left logo padding 2023-07-11 14:08:57 -06:00
David Corbitt
31c581d0fb Update logo 2023-07-11 13:53:55 -06:00
arcticfly
163f6ab23f Update README.md 2023-07-11 12:38:08 -06:00
arcticfly
82a042582c Update logo and favicon (#36) 2023-07-11 11:58:41 -06:00
arcticfly
b01ae677f5 Show more experiment content on screen, hide individual cost and time (#35) 
* Fit more text on screen

* Hide individual cost and time
 2023-07-11 01:17:01 -06:00
David Corbitt
e555d13dd7 Limit prompt tokens to outputs from visible scenarios 2023-07-10 16:33:16 -06:00
David Corbitt
d29e273faf Fix experiment header section 2023-07-10 16:30:53 -06:00
David Corbitt
3f4e607e7c Resize AppShell when orientation changes 2023-07-10 14:24:00 -06:00
arcticfly
187d6492f8 Reevaluate all prompt stats when scenario is hidden (#32) 
* Reevaluate when scenario is hidden

* Add newline
 2023-07-10 13:51:40 -06:00
David Corbitt
ab3445eaaa Add more padding to bottom of outputs table 2023-07-10 13:34:54 -06:00
arcticfly
b99b2e715d Dynamically resize viewport height on mobile (#31) 2023-07-10 13:15:09 -06:00
arcticfly
8ae344a286 Increase scenarios header negative margin (#30) 2023-07-10 12:55:21 -06:00
arcticfly
96dacb0378 Move experiment scrollbar to bottom of page, make scenarios header sticky (#29) 
* Remove newline from promptVariants router

* Move horizontal scroll bar to bottom of OutputsTable

* Make scenarios header sticky
 2023-07-10 12:40:02 -06:00
arcticfly
e64a94e06e Record experiment updated in more places (#24) 
* Record experiment updated in more places

* Update experiment updatedAt in same transaction
 2023-07-10 12:00:24 -06:00
Kyle Corbitt
d6a46b9e9d fix lockfile 2023-07-10 06:47:16 -07:00
arcticfly
32a80f8475 Limit evaluations to visible test scenarios (#28) 2023-07-10 02:10:23 -06:00