Commit Graph

75 Commits

Author SHA1 Message Date
Kyle Corbitt
8db8aeacd3 Replace function chrome with comment
Use a block comment to explain the expected prompt formatting instead of function chrome. The advantage here is that once a user builds a mental model of how OpenPipe works they can just delete the comment, instead of the function chrome sitting around and taking up space in the UI forever.
2023-07-17 10:30:22 -07:00
Kyle Corbitt
74c201d3a8 Make save button disappear on save
Fixes a bug where often the "Save" button wouldn't disappear as expected the first time you clicked it.
2023-07-14 17:35:57 -07:00
David Corbitt
ab9c721d09 Revert change to scenarios header 2023-07-14 17:51:12 -06:00
David Corbitt
0a2578a1d8 Update scenarios header negative margin 2023-07-14 17:41:51 -06:00
Kyle Corbitt
26ee8698be Make it so you can't delete the last prompt or scenario
No reason for an experiment to have 0 prompts or 0 scenarios and it makes the UI look bad.
2023-07-14 15:49:42 -07:00
arcticfly
b98eb9b729 Trigger llm output retrieval on server (#39)
* Rename tables, add graphile workers, update types

* Add dev:worker command

* Update pnpm-lock.yaml

* Remove sentry config import from worker.ts

* Stop generating new cells in cell router get query

* Generate new cells for new scenarios, variants, and experiments

* Remove most error throwing from queryLLM.task.ts

* Remove promptVariantId and testScenarioId from ModelOutput

* Remove duplicate index from ModelOutput

* Move inputHash from cell to output

* Add TODO

* Add todo

* Show cost and time for each cell

* Always show output stats if there is output

* Trigger LLM outputs when scenario variables are updated

* Add newlines to ends of files

* Add another newline

* Cascade ModelOutput deletion

* Fix linting and prettier

* Return instead of throwing for non-pending cell

* Remove pnpm dev:worker from pnpm:dev

* Update pnpm-lock.yaml
2023-07-14 16:38:46 -06:00
Kyle Corbitt
c3e85607e0 Strip types from prompt variants
We want Monaco to treat the prompt constructor as Typescript so we get type checks, but we actually want to save the prompt constructor as Javascript so we can run it directly without transpiling.
2023-07-14 14:03:28 -07:00
Kyle Corbitt
731406d1f4 Pseudo function signatures
Show pseudo function signatures in the variant editor box as a UX hint that you're typing in javascript and have access to the scenario.
2023-07-14 13:56:45 -07:00
Kyle Corbitt
3c59e4b774 Merge pull request #42 from OpenPipe/autoformat
implement format on save
2023-07-14 12:56:41 -07:00
Kyle Corbitt
a20f81939d implement format on save 2023-07-14 12:33:57 -07:00
Kyle Corbitt
a5378b106b store model and use to calculate completion costs 2023-07-14 11:06:07 -07:00
Kyle Corbitt
2cd82eedd0 rename variantEditor -> sharedVariantEditor 2023-07-13 23:27:39 -07:00
Kyle Corbitt
4770ea34a8 Use javascript functions for prompt completions instead of templated json 2023-07-13 18:01:07 -07:00
arcticfly
b01ae677f5 Show more experiment content on screen, hide individual cost and time (#35)
* Fit more text on screen

* Hide individual cost and time
2023-07-11 01:17:01 -06:00
David Corbitt
d29e273faf Fix experiment header section 2023-07-10 16:30:53 -06:00
arcticfly
187d6492f8 Reevaluate all prompt stats when scenario is hidden (#32)
* Reevaluate when scenario is hidden

* Add newline
2023-07-10 13:51:40 -06:00
David Corbitt
ab3445eaaa Add more padding to bottom of outputs table 2023-07-10 13:34:54 -06:00
arcticfly
8ae344a286 Increase scenarios header negative margin (#30) 2023-07-10 12:55:21 -06:00
arcticfly
96dacb0378 Move experiment scrollbar to bottom of page, make scenarios header sticky (#29)
* Remove newline from promptVariants router

* Move horizontal scroll bar to bottom of OutputsTable

* Make scenarios header sticky
2023-07-10 12:40:02 -06:00
Kyle Corbitt
a8db6cadfd format with prettier 3 2023-07-08 22:12:47 -07:00
jarenm
d7fb4a7236 add cost tooltip (#22)
* add cost tooltip

* add padding and style
2023-07-07 19:34:45 -07:00
Kyle Corbitt
46344d8fc4 small bugfixes 2023-07-07 12:22:27 -07:00
arcticfly
8e1384fed9 Break error words without breakLongWord (#16)
* List number of scenarios

* Remove breakLongWords
2023-07-06 21:43:56 -07:00
arcticfly
a2c7ef73ec Retry requests that receive 429 (#15)
* List number of scenarios

* Retry requests after 429

* Rename requestCallback

* Add sleep function

* Allow manual retry on frontend

* Remove unused utility functions

* Auto refetch

* Display wait time with Math.ceil

* Take one second modulo into account

* Add pluralize
2023-07-06 21:39:23 -07:00
Kyle Corbitt
cb15216d0b fix more issues with output cell mutation 2023-07-06 18:35:55 -07:00
Kyle Corbitt
97f44ea5c4 handle issues with OutputCell mutation 2023-07-06 17:56:08 -07:00
arcticfly
d2a50c3d1f List number of scenarios (#14) 2023-07-06 17:21:19 -07:00
arcticfly
fe501a80cb Add total token cost to variant stats (#13)
* Add total token cost to variant stats

* Copy over token counts for new variants

* Update invalidate call
2023-07-06 15:33:49 -07:00
Kyle Corbitt
1fa0d7bc62 bugfixes 2023-07-06 15:22:35 -07:00
arcticfly
92c240e7b8 Add request cost to OutputStats (#12) 2023-07-06 14:36:31 -07:00
Kyle Corbitt
f728027ef6 add evaluations 2023-07-06 13:44:03 -07:00
Kyle Corbitt
4275e6b19b settings drawer 2023-07-05 21:34:00 -07:00
Kyle Corbitt
6510b26b1e re-attach scenarios 2023-07-05 21:34:00 -07:00
Kyle Corbitt
e4c101a843 change outputs table to make room for evals 2023-07-05 21:34:00 -07:00
arcticfly
e1f05a7b59 Add seed file for reddit, restrict max height of variable value editor (#10)
* Add reddit seeds

* Restrict scenario variable value editor maxH to 60

* Add seed file for reddit experiments

* Further decrease max height

* Make score a string
2023-07-05 18:51:34 -07:00
David Corbitt
d434545fdf Rename channelId to channel 2023-07-03 20:22:38 -07:00
David Corbitt
5f11b258ca Streaming works for normal text 2023-07-03 19:51:34 -07:00
arcticfly
c2c512d751 Prevent scenario from dragging when input is hovered (#8)
* Prevent scenario from dragging when input is hovered

* Prevent VariantHeader from dragging when input is hovered
2023-07-01 10:23:04 -07:00
arcticfly
3f6ef5adcf Add AutoResizeTextArea (#7) 2023-07-01 08:52:20 -07:00
arcticfly
6389bd54de Save and display timeToComplete on model outputs (#6)
* Calculate and save timeToComplete on model outputs

* Add output stats to function call as well

* Record timeToComplete before parsing response json

* Add default value for timeToComplete
2023-06-30 21:29:28 -07:00
David Corbitt
ae9b37cf16 Add margin to autogenerating NewScenarioButton 2023-06-30 18:31:28 -07:00
David Corbitt
b45fece88f Center loading indicators in output cells 2023-06-30 18:26:20 -07:00
David Corbitt
210efdcd6b Move scenario variables to the top of their container 2023-06-30 18:12:06 -07:00
David Corbitt
0daa68fa7d Cease centering model output in cell 2023-06-30 17:52:55 -07:00
David Corbitt
f553c26c52 Maintain spaces and newlines in model output 2023-06-30 17:43:04 -07:00
David Corbitt
ac833a44d2 Mess around with error message 2023-06-28 16:05:02 -07:00
David Corbitt
99ca8bec25 Remove unnecessary scroll bar 2023-06-28 15:59:49 -07:00
David Corbitt
8f8e8121d1 Catch argument parsing error 2023-06-28 15:46:18 -07:00
Kyle Corbitt
15e4fe7e5a more robust sidebar layout 2023-06-28 10:37:37 -07:00
Kyle Corbitt
5d3d51d1a5 signal when new scenario is in progress 2023-06-28 06:37:40 -07:00