Commit Graph

32 Commits

Author SHA1 Message Date
arcticfly
b98eb9b729 Trigger llm output retrieval on server (#39)
* Rename tables, add graphile workers, update types

* Add dev:worker command

* Update pnpm-lock.yaml

* Remove sentry config import from worker.ts

* Stop generating new cells in cell router get query

* Generate new cells for new scenarios, variants, and experiments

* Remove most error throwing from queryLLM.task.ts

* Remove promptVariantId and testScenarioId from ModelOutput

* Remove duplicate index from ModelOutput

* Move inputHash from cell to output

* Add TODO

* Add todo

* Show cost and time for each cell

* Always show output stats if there is output

* Trigger LLM outputs when scenario variables are updated

* Add newlines to ends of files

* Add another newline

* Cascade ModelOutput deletion

* Fix linting and prettier

* Return instead of throwing for non-pending cell

* Remove pnpm dev:worker from pnpm:dev

* Update pnpm-lock.yaml
2023-07-14 16:38:46 -06:00
Kyle Corbitt
80c0d13bb9 warn folks if they use a lower node version 2023-07-14 14:59:33 -07:00
Kyle Corbitt
c3e85607e0 Strip types from prompt variants
We want Monaco to treat the prompt constructor as Typescript so we get type checks, but we actually want to save the prompt constructor as Javascript so we can run it directly without transpiling.
2023-07-14 14:03:28 -07:00
Kyle Corbitt
a20f81939d implement format on save 2023-07-14 12:33:57 -07:00
Kyle Corbitt
4770ea34a8 Use javascript functions for prompt completions instead of templated json 2023-07-13 18:01:07 -07:00
Kyle Corbitt
a8db6cadfd format with prettier 3 2023-07-08 22:12:47 -07:00
Kyle Corbitt
918f209227 rename querykey -> openpipe 2023-07-07 12:27:21 -07:00
arcticfly
a2c7ef73ec Retry requests that receive 429 (#15)
* List number of scenarios

* Retry requests after 429

* Rename requestCallback

* Add sleep function

* Allow manual retry on frontend

* Remove unused utility functions

* Auto refetch

* Display wait time with Math.ceil

* Take one second modulo into account

* Add pluralize
2023-07-06 21:39:23 -07:00
Kyle Corbitt
f728027ef6 add evaluations 2023-07-06 13:44:03 -07:00
arcticfly
1ae5612d55 Add promptTokens and completionTokens to model output (#11)
* Default to streaming in config

* Add tokens to database

* Add NEXT_PUBLIC_SOCKET_URL to .env.example

* Disable streaming for functions

* Add newline to types
2023-07-06 13:12:59 -07:00
Kyle Corbitt
4275e6b19b settings drawer 2023-07-05 21:34:00 -07:00
Kyle Corbitt
552199da4a rename prompt-lab to querykey 2023-07-05 10:43:28 -07:00
David Corbitt
5f11b258ca Streaming works for normal text 2023-07-03 19:51:34 -07:00
Kyle Corbitt
7641ea0747 prettier line length 2023-06-28 18:18:57 -07:00
Kyle Corbitt
f63c23f9fb relicense 2023-06-28 06:49:44 -07:00
Kyle Corbitt
0a675cd7f7 autogen scenarios 2023-06-27 13:19:41 -07:00
Kyle Corbitt
f6f93a1161 trying out analytics 2023-06-27 00:11:07 -07:00
Kyle Corbitt
bca35c9eb2 tighter types and linting 2023-06-26 23:40:05 -07:00
Kyle Corbitt
0646fc0ba3 lots of changes on making new experiments work 2023-06-26 18:03:26 -07:00
Kyle Corbitt
1d78a78b1d validate json schema for openai api 2023-06-26 12:36:34 -07:00
Kyle Corbitt
e6fdd2d5c5 prettifying the ux 2023-06-26 06:26:19 -07:00
Kyle Corbitt
8534477236 can add scenarios and it mostly works 2023-06-23 20:00:46 -07:00
Kyle Corbitt
c497b74208 ripped out mantine, replaced with chakra 2023-06-23 17:23:30 -07:00
Kyle Corbitt
5fcefad461 rip out mantine table 2023-06-23 17:03:26 -07:00
Kyle Corbitt
2b0c2ad603 editing scenarios is kinda working 2023-06-23 16:18:28 -07:00
Kyle Corbitt
ca78406ad1 saving prompt configs works 2023-06-23 12:18:09 -07:00
Kyle Corbitt
a31c112745 we're actually calling openai 2023-06-22 17:57:21 -07:00
Kyle Corbitt
788911898b basic table with no info 2023-06-22 08:05:22 -07:00
Kyle Corbitt
0f3c381e1d add mantine 2023-06-21 17:29:19 -07:00
Kyle Corbitt
3be022786a basic db seeding 2023-06-21 15:20:33 -07:00
Kyle Corbitt
3d85984f98 more schema work 2023-06-21 13:26:51 -07:00
Kyle Corbitt
451e7b1d90 initial commit 2023-06-19 16:15:35 -07:00