Adds a `modelProvider` field to `promptVariants`, currently just set to "openai/ChatCompletion" for all variants for now.
Adds a `modelProviders/` directory where we can define and store pluggable model providers. Currently just OpenAI. Not everything is pluggable yet -- notably the code to actually generate completions hasn't been migrated to this setup yet.
Does a lot of work to get the types working. Prompts are now defined with a function `definePrompt(modelProvider, config)` instead of `prompt = config`. Added a script to migrate old prompt definitions.
This is still partial work, but the diff is large enough that I want to get it in. I don't think anything is broken but I haven't tested thoroughly.
* Add dropdown header for model switching
* Allow variant duplication
* Fix prettier
* Use env variable to restrict prisma logs
* Fix env.mjs
* Remove unnecessary scroll bar from function call output
* Properly record when 404 error occurs in queryLLM task
* Add SelectedModelInfo in SelectModelModal
* Add react-select
* Calculate new prompt after switching model
* Send newly selected model with creation request
* Get new prompt construction function back from GPT-4
* Fix prettier
* Fix prettier
* Rename tables, add graphile workers, update types
* Add dev:worker command
* Update pnpm-lock.yaml
* Remove sentry config import from worker.ts
* Stop generating new cells in cell router get query
* Generate new cells for new scenarios, variants, and experiments
* Remove most error throwing from queryLLM.task.ts
* Remove promptVariantId and testScenarioId from ModelOutput
* Remove duplicate index from ModelOutput
* Move inputHash from cell to output
* Add TODO
* Add todo
* Show cost and time for each cell
* Always show output stats if there is output
* Trigger LLM outputs when scenario variables are updated
* Add newlines to ends of files
* Add another newline
* Cascade ModelOutput deletion
* Fix linting and prettier
* Return instead of throwing for non-pending cell
* Remove pnpm dev:worker from pnpm:dev
* Update pnpm-lock.yaml
We want Monaco to treat the prompt constructor as Typescript so we get type checks, but we actually want to save the prompt constructor as Javascript so we can run it directly without transpiling.
* List number of scenarios
* Retry requests after 429
* Rename requestCallback
* Add sleep function
* Allow manual retry on frontend
* Remove unused utility functions
* Auto refetch
* Display wait time with Math.ceil
* Take one second modulo into account
* Add pluralize
* Default to streaming in config
* Add tokens to database
* Add NEXT_PUBLIC_SOCKET_URL to .env.example
* Disable streaming for functions
* Add newline to types
