Commit Graph

11 Commits

Author SHA1 Message Date
arcticfly
d6b97b29f7 Allow experiment forking (#89)
* Move DeleteButton into a separate file

* Rename plural relations

* Add ability to fork

* Fork automatically after auth upon return

* Add experiment card skeleton

* Create HeaderButtons component

* return no header buttons while experiment loading

* Fix prettier

* Remove unused variable

* Remove newline

* Default json values to undefined

* Change header styles

* Fix prettier

* Give AddScenario icon less width

* Move useEffect

* Skip invalidating experiments list after forking

* Require user to be able to view experiment to fork it

* Move experiment creation into same transaction

* Only return the forked experiment id

* Put delete button in experiment settings drawer

* Move useEffect hook
2023-07-24 18:10:59 -07:00
Kyle Corbitt
7d41e94ca2 cache eval outputs and add gpt4 eval 2023-07-17 17:55:36 -07:00
Kyle Corbitt
011b12abb9 cache output evals 2023-07-17 17:52:30 -07:00
arcticfly
b98eb9b729 Trigger llm output retrieval on server (#39)
* Rename tables, add graphile workers, update types

* Add dev:worker command

* Update pnpm-lock.yaml

* Remove sentry config import from worker.ts

* Stop generating new cells in cell router get query

* Generate new cells for new scenarios, variants, and experiments

* Remove most error throwing from queryLLM.task.ts

* Remove promptVariantId and testScenarioId from ModelOutput

* Remove duplicate index from ModelOutput

* Move inputHash from cell to output

* Add TODO

* Add todo

* Show cost and time for each cell

* Always show output stats if there is output

* Trigger LLM outputs when scenario variables are updated

* Add newlines to ends of files

* Add another newline

* Cascade ModelOutput deletion

* Fix linting and prettier

* Return instead of throwing for non-pending cell

* Remove pnpm dev:worker from pnpm:dev

* Update pnpm-lock.yaml
2023-07-14 16:38:46 -06:00
Kyle Corbitt
4770ea34a8 Use javascript functions for prompt completions instead of templated json 2023-07-13 18:01:07 -07:00
arcticfly
187d6492f8 Reevaluate all prompt stats when scenario is hidden (#32)
* Reevaluate when scenario is hidden

* Add newline
2023-07-10 13:51:40 -06:00
arcticfly
32a80f8475 Limit evaluations to visible test scenarios (#28) 2023-07-10 02:10:23 -06:00
Kyle Corbitt
a8db6cadfd format with prettier 3 2023-07-08 22:12:47 -07:00
Kyle Corbitt
8e0722cd22 wrong denominator 2023-07-07 17:48:34 -07:00
Kyle Corbitt
46344d8fc4 small bugfixes 2023-07-07 12:22:27 -07:00
Kyle Corbitt
f728027ef6 add evaluations 2023-07-06 13:44:03 -07:00