Update TopNavbar component to include a GitHub button

Update README.md
2023-08-02 10:11:41 -07:00 · 2023-08-02 00:59:05 -07:00 · 2023-08-02 00:57:24 -07:00 · 2023-08-02 00:56:19 -07:00 · 2023-08-02 00:51:57 -07:00 · 2023-08-02 00:46:48 -07:00
96 changed files with 3172 additions and 965 deletions
--- a/.eslintrc.cjs
+++ b/.eslintrc.cjs
@@ -37,6 +37,7 @@ const config = {
      "warn",
      { vars: "all", varsIgnorePattern: "^_", args: "after-used", argsIgnorePattern: "^_" },
    ],
+    "react/no-unescaped-entities": "off",
  },
 };

--- a/.prettierignore
+++ b/.prettierignore
@@ -1,2 +1,2 @@
-src/codegen/openai.schema.json
+*.schema.json
 pnpm-lock.yaml
--- a/@types/nextjs-routes.d.ts
+++ b/@types/nextjs-routes.d.ts
@@ -13,10 +13,13 @@ declare module "nextjs-routes" {
  export type Route =
    | StaticRoute<"/account/signin">
    | DynamicRoute<"/api/auth/[...nextauth]", { "nextauth": string[] }>
+    | StaticRoute<"/api/experiments/og-image">
    | DynamicRoute<"/api/trpc/[trpc]", { "trpc": string }>
    | DynamicRoute<"/experiments/[id]", { "id": string }>
    | StaticRoute<"/experiments">
-    | StaticRoute<"/">;
+    | StaticRoute<"/">
+    | StaticRoute<"/world-champs">
+    | StaticRoute<"/world-champs/signup">;

  interface StaticRoute<Pathname> {
    pathname: Pathname;
--- a/README.md
+++ b/README.md
@@ -6,45 +6,53 @@ OpenPipe is a flexible playground for comparing and optimizing LLM prompts. It l

 ## Sample Experiments

-These are simple experiments users have created that show how OpenPipe works.
+These are simple experiments users have created that show how OpenPipe works. Feel free to fork them and start experimenting yourself.

- [Country Capitals](https://app.openpipe.ai/experiments/11111111-1111-1111-1111-111111111111)
+- [Twitter Sentiment Analysis](https://app.openpipe.ai/experiments/62c20a73-2012-4a64-973c-4b665ad46a57)
 - [Reddit User Needs](https://app.openpipe.ai/experiments/22222222-2222-2222-2222-222222222222)
 - [OpenAI Function Calls](https://app.openpipe.ai/experiments/2ebbdcb3-ed51-456e-87dc-91f72eaf3e2b)
 - [Activity Classification](https://app.openpipe.ai/experiments/3950940f-ab6b-4b74-841d-7e9dbc4e4ff8)

-<img src="https://github.com/openpipe/openpipe/assets/176426/fc7624c6-5b65-4d4d-82b7-4a816f3e5678" alt="demo" height="400px">
+<img src="https://github.com/openpipe/openpipe/assets/41524992/219a844e-3f4e-4f6b-8066-41348b42977b" alt="demo">

-You can use our hosted version of OpenPipe at [https://openpipe.ai]. You can also clone this repository and [run it locally](#running-locally).
+You can use our hosted version of OpenPipe at https://openpipe.ai. You can also clone this repository and [run it locally](#running-locally).

 ## High-Level Features

-**Configure Multiple Prompts**  
-Set up multiple prompt configurations and compare their output side-by-side. Each configuration can be configured independently.
-
 **Visualize Responses**  
 Inspect prompt completions side-by-side.

-**Test Many Inputs**  
-OpenPipe lets you _template_ a prompt. Use the templating feature to run the prompts you're testing against many potential inputs for broader coverage of your problem space than you'd get with manual testing.
+<br>

+**Test Many Inputs**  
+OpenPipe lets you _template_ a prompt. Use the templating feature to run the prompts you're testing against many potential inputs for broad coverage of your problem space.
+
+<br>
+
+**Translate between Model APIs**  
+Write your prompt in one format and automatically convert it to work with any other model.
+
+<img width="480" alt="Screenshot 2023-08-01 at 11 55 38 PM" src="https://github.com/OpenPipe/OpenPipe/assets/41524992/1e19ccf2-96b6-4e93-a3a5-1449710d1b5b" alt="translate between models">
+
+<br><br>
+**Refine your prompts automatically**  
+Use a growing database of best-practice refinements to improve your prompts automatically.
+
+<img width="480" alt="Screenshot 2023-08-01 at 11 55 38 PM" src="https://github.com/OpenPipe/OpenPipe/assets/41524992/87a27fe7-daef-445c-a5e2-1c82b23f9f99" alt="add function call">
+
+<br><br>
 **🪄 Auto-generate Test Scenarios**  
 OpenPipe includes a tool to generate new test scenarios based on your existing prompts and scenarios. Just click "Autogenerate Scenario" to try it out!

-**Prompt Validation and Typeahead**  
-We use OpenAI's OpenAPI spec to automatically provide typeahead and validate prompts.
+<img width="600" src="https://github.com/openpipe/openpipe/assets/41524992/219a844e-3f4e-4f6b-8066-41348b42977b" alt="auto-generate">

-<img alt="typeahead" src="https://github.com/openpipe/openpipe/assets/176426/acc638f8-d851-4742-8d01-fe6f98890840" height="300px">
-
-**Function Call Support**  
-Natively supports [OpenAI function calls](https://openai.com/blog/function-calling-and-other-api-updates) on supported models.
-
-<img height="300px" alt="function calls" src="https://github.com/openpipe/openpipe/assets/176426/48ad13fe-af2f-4294-bf32-62015597fd9b">
+<br><br>

 ## Supported Models

 - All models available through the OpenAI [chat completion API](https://platform.openai.com/docs/guides/gpt/chat-completions-api)
 - Llama2 [7b chat](https://replicate.com/a16z-infra/llama7b-v2-chat), [13b chat](https://replicate.com/a16z-infra/llama13b-v2-chat), [70b chat](https://replicate.com/replicate/llama70b-v2-chat).
+- Anthropic's [Claude 1 Instant](https://www.anthropic.com/index/introducing-claude) and [Claude 2](https://www.anthropic.com/index/claude-2)

 ## Running Locally

--- a/package.json
+++ b/package.json
@@ -12,7 +12,7 @@
    "dev:next": "next dev",
    "dev:wss": "pnpm tsx --watch src/wss-server.ts",
    "dev:worker": "NODE_ENV='development' pnpm tsx --watch src/server/tasks/worker.ts",
-    "dev": "concurrently --kill-others 'pnpm dev:next' 'pnpm dev:wss'",
+    "dev": "concurrently --kill-others 'pnpm dev:next' 'pnpm dev:wss' 'pnpm dev:worker'",
    "postinstall": "prisma generate",
    "lint": "next lint",
    "start": "next start",
@@ -21,11 +21,14 @@
    "check": "concurrently 'pnpm lint' 'pnpm tsc' 'pnpm prettier . --check'"
  },
  "dependencies": {
+    "@anthropic-ai/sdk": "^0.5.8",
    "@apidevtools/json-schema-ref-parser": "^10.1.0",
    "@babel/preset-typescript": "^7.22.5",
    "@babel/standalone": "^7.22.9",
+    "@chakra-ui/anatomy": "^2.2.0",
    "@chakra-ui/next-js": "^2.1.4",
    "@chakra-ui/react": "^2.7.1",
+    "@chakra-ui/styled-system": "^2.9.1",
    "@emotion/react": "^11.11.1",
    "@emotion/server": "^11.11.0",
    "@emotion/styled": "^11.11.0",
@@ -40,6 +43,7 @@
    "@trpc/next": "^10.26.0",
    "@trpc/react-query": "^10.26.0",
    "@trpc/server": "^10.26.0",
+    "@vercel/og": "^0.5.9",
    "ast-types": "^0.14.2",
    "chroma-js": "^2.4.2",
    "concurrently": "^8.2.0",
@@ -61,7 +65,7 @@
    "next-auth": "^4.22.1",
    "next-query-params": "^4.2.3",
    "nextjs-routes": "^2.0.1",
-    "openai": "4.0.0-beta.2",
+    "openai": "4.0.0-beta.7",
    "pluralize": "^8.0.0",
    "posthog-js": "^1.68.4",
    "prettier": "^3.0.0",
@@ -69,7 +73,9 @@
    "react": "18.2.0",
    "react-diff-viewer": "^3.1.1",
    "react-dom": "18.2.0",
+    "react-github-btn": "^1.4.0",
    "react-icons": "^4.10.1",
+    "react-json-tree": "^0.18.0",
    "react-select": "^5.7.4",
    "react-syntax-highlighter": "^15.5.0",
    "react-textarea-autosize": "^8.5.0",
@@ -81,6 +87,7 @@
    "tsx": "^3.12.7",
    "type-fest": "^4.0.0",
    "use-query-params": "^2.2.1",
+    "uuid": "^9.0.0",
    "vite-tsconfig-paths": "^4.2.0",
    "zod": "^3.21.4",
    "zustand": "^4.3.9"
@@ -101,8 +108,10 @@
    "@types/react": "^18.2.6",
    "@types/react-dom": "^18.2.4",
    "@types/react-syntax-highlighter": "^15.5.7",
+    "@types/uuid": "^9.0.2",
    "@typescript-eslint/eslint-plugin": "^5.59.6",
    "@typescript-eslint/parser": "^5.59.6",
+    "csv-parse": "^5.4.0",
    "eslint": "^8.40.0",
    "eslint-config-next": "^13.4.2",
    "eslint-plugin-unused-imports": "^2.0.0",
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -5,6 +5,9 @@ settings:
  excludeLinksFromLockfile: false

 dependencies:
+  '@anthropic-ai/sdk':
+    specifier: ^0.5.8
+    version: 0.5.8
  '@apidevtools/json-schema-ref-parser':
    specifier: ^10.1.0
    version: 10.1.0
@@ -14,12 +17,18 @@ dependencies:
  '@babel/standalone':
    specifier: ^7.22.9
    version: 7.22.9
+  '@chakra-ui/anatomy':
+    specifier: ^2.2.0
+    version: 2.2.0
  '@chakra-ui/next-js':
    specifier: ^2.1.4
    version: 2.1.4(@chakra-ui/react@2.7.1)(@emotion/react@11.11.1)(next@13.4.2)(react@18.2.0)
  '@chakra-ui/react':
    specifier: ^2.7.1
    version: 2.7.1(@emotion/react@11.11.1)(@emotion/styled@11.11.0)(@types/react@18.2.6)(framer-motion@10.12.17)(react-dom@18.2.0)(react@18.2.0)
+  '@chakra-ui/styled-system':
+    specifier: ^2.9.1
+    version: 2.9.1
  '@emotion/react':
    specifier: ^11.11.1
    version: 11.11.1(@types/react@18.2.6)(react@18.2.0)
@@ -62,6 +71,9 @@ dependencies:
  '@trpc/server':
    specifier: ^10.26.0
    version: 10.26.0
+  '@vercel/og':
+    specifier: ^0.5.9
+    version: 0.5.9
  ast-types:
    specifier: ^0.14.2
    version: 0.14.2
@@ -126,8 +138,8 @@ dependencies:
    specifier: ^2.0.1
    version: 2.0.1(next@13.4.2)
  openai:
-    specifier: 4.0.0-beta.2
-    version: 4.0.0-beta.2
+    specifier: 4.0.0-beta.7
+    version: 4.0.0-beta.7
  pluralize:
    specifier: ^8.0.0
    version: 8.0.0
@@ -149,9 +161,15 @@ dependencies:
  react-dom:
    specifier: 18.2.0
    version: 18.2.0(react@18.2.0)
+  react-github-btn:
+    specifier: ^1.4.0
+    version: 1.4.0(react@18.2.0)
  react-icons:
    specifier: ^4.10.1
    version: 4.10.1(react@18.2.0)
+  react-json-tree:
+    specifier: ^0.18.0
+    version: 0.18.0(@types/react@18.2.6)(react@18.2.0)
  react-select:
    specifier: ^5.7.4
    version: 5.7.4(@types/react@18.2.6)(react-dom@18.2.0)(react@18.2.0)
@@ -185,6 +203,9 @@ dependencies:
  use-query-params:
    specifier: ^2.2.1
    version: 2.2.1(react-dom@18.2.0)(react@18.2.0)
+  uuid:
+    specifier: ^9.0.0
+    version: 9.0.0
  vite-tsconfig-paths:
    specifier: ^4.2.0
    version: 4.2.0(typescript@5.0.4)
@@ -241,12 +262,18 @@ devDependencies:
  '@types/react-syntax-highlighter':
    specifier: ^15.5.7
    version: 15.5.7
+  '@types/uuid':
+    specifier: ^9.0.2
+    version: 9.0.2
  '@typescript-eslint/eslint-plugin':
    specifier: ^5.59.6
    version: 5.59.6(@typescript-eslint/parser@5.59.6)(eslint@8.40.0)(typescript@5.0.4)
  '@typescript-eslint/parser':
    specifier: ^5.59.6
    version: 5.59.6(eslint@8.40.0)(typescript@5.0.4)
+  csv-parse:
+    specifier: ^5.4.0
+    version: 5.4.0
  eslint:
    specifier: ^8.40.0
    version: 8.40.0
@@ -292,6 +319,22 @@ packages:
      '@jridgewell/gen-mapping': 0.3.3
      '@jridgewell/trace-mapping': 0.3.18

+  /@anthropic-ai/sdk@0.5.8:
+    resolution: {integrity: sha512-iHenjcE2Q/az6VZiP1DueOSvKNRmxsly6Rx2yjJBoy7OBYVFGVjEdgs2mPQHtTX0ibKAR7tPq6F6MQbKDPWcKg==}
+    dependencies:
+      '@types/node': 18.16.0
+      '@types/node-fetch': 2.6.4
+      abort-controller: 3.0.0
+      agentkeepalive: 4.3.0
+      digest-fetch: 1.3.0
+      form-data-encoder: 1.7.2
+      formdata-node: 4.4.1
+      node-fetch: 2.6.12
+    transitivePeerDependencies:
+      - encoding
+      - supports-color
+    dev: false
+
  /@apidevtools/json-schema-ref-parser@10.1.0:
    resolution: {integrity: sha512-3e+viyMuXdrcK8v5pvP+SDoAQ77FH6OyRmuK48SZKmdHJRFm87RsSs8qm6kP39a/pOPURByJw+OXzQIqcfmKtA==}
    engines: {node: '>= 16'}
@@ -655,6 +698,10 @@ packages:
    resolution: {integrity: sha512-pKfOS/mztc4sUXHNc8ypJ1gPWSolWT770jrgVRfolVbYlki8y5Y+As996zMF6k5lewTu6j9DQequ7Cc9a69IVQ==}
    dev: false

+  /@chakra-ui/anatomy@2.2.0:
+    resolution: {integrity: sha512-cD8Ms5C8+dFda0LrORMdxiFhAZwOIY1BSlCadz6/mHUIgNdQy13AHPrXiq6qWdMslqVHq10k5zH7xMPLt6kjFg==}
+    dev: false
+
  /@chakra-ui/avatar@2.2.11(@chakra-ui/system@2.5.8)(react@18.2.0):
    resolution: {integrity: sha512-CJFkoWvlCTDJTUBrKA/aVyG5Zz6TBEIVmmsJtqC6VcQuVDTxkWod8ruXnjb0LT2DUveL7xR5qZM9a5IXcsH3zg==}
    peerDependencies:
@@ -2625,10 +2672,24 @@ packages:
    resolution: {integrity: sha512-PDNlhP/1vyTgmNyiucGqGCdXIp7HIkkvKO50si3y3PcceeHvqtiKPaH1iJdz63jCWMVMbj2MElSxXPOeBvEVIQ==}
    requiresBuild: true

+  /@resvg/resvg-wasm@2.4.1:
+    resolution: {integrity: sha512-yi6R0HyHtsoWTRA06Col4WoDs7SvlXU3DLMNP2bdAgs7HK18dTEVl1weXgxRzi8gwLteGUbIg29zulxIB3GSdg==}
+    engines: {node: '>= 10'}
+    dev: false
+
  /@rushstack/eslint-patch@1.3.2:
    resolution: {integrity: sha512-V+MvGwaHH03hYhY+k6Ef/xKd6RYlc4q8WBx+2ANmipHJcKuktNcI/NgEsJgdSUF6Lw32njT6OnrRsKYCdgHjYw==}
    dev: true

+  /@shuding/opentype.js@1.4.0-beta.0:
+    resolution: {integrity: sha512-3NgmNyH3l/Hv6EvsWJbsvpcpUba6R8IREQ83nH83cyakCw7uM1arZKNfHwv1Wz6jgqrF/j4x5ELvR6PnK9nTcA==}
+    engines: {node: '>= 8.0.0'}
+    hasBin: true
+    dependencies:
+      fflate: 0.7.4
+      string.prototype.codepointat: 0.2.1
+    dev: false
+
  /@sinclair/typebox@0.27.8:
    resolution: {integrity: sha512-+Fj43pSMwJs4KRrH/938Uf+uAELIgVBmQzg/q1YG10djyfA3TnrU8N8XzqCh/okZdszqBQTZf96idMfE5lnwTA==}
    dev: true
@@ -2786,6 +2847,10 @@ packages:
      '@babel/types': 7.22.5
    dev: true

+  /@types/base16@1.0.2:
+    resolution: {integrity: sha512-oYO/U4VD1DavwrKuCSQWdLG+5K22SLPem2OQaHmFcQuwHoVeGC+JGVRji2MUqZUAIQZHEonOeVfAX09hYiLsdg==}
+    dev: false
+
  /@types/body-parser@1.19.2:
    resolution: {integrity: sha512-ALYone6pm6QmwZoAgeyNksccT9Q4AWZQ6PvfwR37GT6r6FWUPguq6sUmNGSMV2Wr761oQoBxwGGa6DR5o1DC9g==}
    dependencies:
@@ -2968,6 +3033,7 @@ packages:

  /@types/qs@6.9.7:
    resolution: {integrity: sha512-FGa1F62FT09qcrueBA6qYTrJPVDzah9a+493+o2PCXsesWHIn27G98TsSMs3WPNbZIEj4+VJf6saSFpvD+3Zsw==}
+    dev: true

  /@types/range-parser@1.2.4:
    resolution: {integrity: sha512-EEhsLsD6UsDM1yFhAvy0Cjr6VwmpMWqFBCb9w07wVugF7w9nfajxLuVmngTIpgS6svCnm6Vaw+MZhoDCKnOfsw==}
@@ -3024,6 +3090,10 @@ packages:
    resolution: {integrity: sha512-cputDpIbFgLUaGQn6Vqg3/YsJwxUwHLO13v3i5ouxT4lat0khip9AEWxtERujXV9wxIB1EyF97BSJFt6vpdI8g==}
    dev: false

+  /@types/uuid@9.0.2:
+    resolution: {integrity: sha512-kNnC1GFBLuhImSnV7w4njQkUiJi0ZXUycu1rUaouPqiKlXkh77JKgdRnTAp1x5eBwcIwbtI+3otwzuIDEuDoxQ==}
+    dev: true
+
  /@typescript-eslint/eslint-plugin@5.59.6(@typescript-eslint/parser@5.59.6)(eslint@8.40.0)(typescript@5.0.4):
    resolution: {integrity: sha512-sXtOgJNEuRU5RLwPUb1jxtToZbgvq3M6FPpY4QENxoOggK+UpTxUBpj6tD8+Qh2g46Pi9We87E+eHnUw8YcGsw==}
    engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0}
@@ -3154,6 +3224,15 @@ packages:
      eslint-visitor-keys: 3.4.1
    dev: true

+  /@vercel/og@0.5.9:
+    resolution: {integrity: sha512-CtjaV/BVHtNCjRtxGqn8Q6AKFLqcG34Byxr91+mY+4eqyp/09LVe9jEeY9WXjbaKvu8syWPMteTpY+YQUQYzSg==}
+    engines: {node: '>=16'}
+    dependencies:
+      '@resvg/resvg-wasm': 2.4.1
+      satori: 0.10.1
+      yoga-wasm-web: 0.3.3
+    dev: false
+
  /@vitest/expect@0.33.0:
    resolution: {integrity: sha512-sVNf+Gla3mhTCxNJx+wJLDPp/WcstOe0Ksqz4Vec51MmgMth/ia0MGFEkIZmVGeTL5HtjYR4Wl/ZxBxBXZJTzQ==}
    dependencies:
@@ -3600,6 +3679,15 @@ packages:
    resolution: {integrity: sha512-Y5gU45svrR5tI2Vt/X9GPd3L0HNIKzGu202EjxrXMpuc2V2CiKgemAbUUsqYmZJvPtCXoUKjNZwBJzsNScUbXA==}
    dev: false

+  /base16@1.0.0:
+    resolution: {integrity: sha512-pNdYkNPiJUnEhnfXV56+sQy8+AaPcG3POZAUnwr4EeqCUZFz4u2PePbo3e5Gj4ziYPCWGUZT9RHisvJKnwFuBQ==}
+    dev: false
+
+  /base64-js@0.0.8:
+    resolution: {integrity: sha512-3XSA2cR/h/73EzlXXdU6YNycmYI7+kicTxks4eJg2g39biHR84slg2+des+p7iHYhbRg/udIS4TD53WabcOUkw==}
+    engines: {node: '>= 0.4'}
+    dev: false
+
  /base64-js@1.5.1:
    resolution: {integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==}
    dev: false
@@ -3721,6 +3809,10 @@ packages:
    resolution: {integrity: sha512-P8BjAsXvZS+VIDUI11hHCQEv74YT67YUi5JJFNWIqL235sBmjX4+qx9Muvls5ivyNENctx46xQLQ3aTuE7ssaQ==}
    engines: {node: '>=6'}

+  /camelize@1.0.1:
+    resolution: {integrity: sha512-dU+Tx2fsypxTgtLoE36npi3UqcjSSMNYfkqgmoEhtZrraP5VWq0K7FkWVTYa8eMPtnU/G2txVsfdCJTn9uzpuQ==}
+    dev: false
+
  /caniuse-lite@1.0.30001517:
    resolution: {integrity: sha512-Vdhm5S11DaFVLlyiKu4hiUTkpZu+y1KA/rZZqVQfOD5YdDT/eQKlkt7NaE0WGOFgX32diqt9MiP9CAiFeRklaA==}

@@ -3849,10 +3941,24 @@ packages:
  /color-name@1.1.4:
    resolution: {integrity: sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==}

+  /color-string@1.9.1:
+    resolution: {integrity: sha512-shrVawQFojnZv6xM40anx4CkoDP+fZsw/ZerEMsW/pyzsRbElpsL/DBVW7q3ExxwusdNXI3lXpuhEZkzs8p5Eg==}
+    dependencies:
+      color-name: 1.1.4
+      simple-swizzle: 0.2.2
+    dev: false
+
  /color2k@2.0.2:
    resolution: {integrity: sha512-kJhwH5nAwb34tmyuqq/lgjEKzlFXn1U99NlnB6Ws4qVaERcRUYeYP1cBw6BJ4vxaWStAUEef4WMr7WjOCnBt8w==}
    dev: false

+  /color@3.2.1:
+    resolution: {integrity: sha512-aBl7dZI9ENN6fUGC7mWpMTPNHmWUSNan9tuWN6ahh5ZLNk9baLJOnSMlrQkHcrfFgz2/RigjUVAjdx36VcemKA==}
+    dependencies:
+      color-convert: 1.9.3
+      color-string: 1.9.1
+    dev: false
+
  /combined-stream@1.0.8:
    resolution: {integrity: sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==}
    engines: {node: '>= 0.8'}
@@ -3989,12 +4095,33 @@ packages:
    resolution: {integrity: sha512-mCxBlsHFYh9C+HVpiEacem8FEBnMXgU9gy4zmNC+SXAZNB/1idgp/aulFJ4FgCi7GPEVbfyng092GqL2k2rmow==}
    dev: false

+  /css-background-parser@0.1.0:
+    resolution: {integrity: sha512-2EZLisiZQ+7m4wwur/qiYJRniHX4K5Tc9w93MT3AS0WS1u5kaZ4FKXlOTBhOjc+CgEgPiGY+fX1yWD8UwpEqUA==}
+    dev: false
+
  /css-box-model@1.2.1:
    resolution: {integrity: sha512-a7Vr4Q/kd/aw96bnJG332W9V9LkJO69JRcaCYDUqjp6/z0w6VcZjgAcTbgFxEPfBgdnAwlh3iwu+hLopa+flJw==}
    dependencies:
      tiny-invariant: 1.3.1
    dev: false

+  /css-box-shadow@1.0.0-3:
+    resolution: {integrity: sha512-9jaqR6e7Ohds+aWwmhe6wILJ99xYQbfmK9QQB9CcMjDbTxPZjwEmUQpU91OG05Xgm8BahT5fW+svbsQGjS/zPg==}
+    dev: false
+
+  /css-color-keywords@1.0.0:
+    resolution: {integrity: sha512-FyyrDHZKEjXDpNJYvVsV960FiqQyXc/LlYmsxl2BcdMb2WPx0OGRVgTg55rPSyLSNMqP52R9r8geSp7apN3Ofg==}
+    engines: {node: '>=4'}
+    dev: false
+
+  /css-to-react-native@3.2.0:
+    resolution: {integrity: sha512-e8RKaLXMOFii+02mOlqwjbD00KSEKqblnpO9e++1aXS1fPQOpS1YoqdVHBqPjHNoxeF2mimzVqawm2KCbEdtHQ==}
+    dependencies:
+      camelize: 1.0.1
+      css-color-keywords: 1.0.0
+      postcss-value-parser: 4.2.0
+    dev: false
+
  /csstype@2.6.21:
    resolution: {integrity: sha512-Z1PhmomIfypOpoMjRQB70jfvy/wxT50qW08YXO5lMIJkrdq4yOTR+AW7FqutScmB9NkLwxo+jU+kZLbofZZq/w==}
    dev: false
@@ -4002,6 +4129,10 @@ packages:
  /csstype@3.1.2:
    resolution: {integrity: sha512-I7K1Uu0MBPzaFKg4nI5Q7Vs2t+3gWWW648spaF+Rg7pI9ds18Ugn+lvg4SHczUdKlHI5LWBXyqfS8+DufyBsgQ==}

+  /csv-parse@5.4.0:
+    resolution: {integrity: sha512-JiQosUWiOFgp4hQn0an+SBoV9IKdqzhROM0iiN4LB7UpfJBlsSJlWl9nq4zGgxgMAzHJ6V4t29VAVD+3+2NJAg==}
+    dev: true
+
  /d@1.0.1:
    resolution: {integrity: sha512-m62ShEObQ39CfralilEQRjH6oAMtNCV1xJyEx5LpRYUVN+EviphDgUc/F3hnYbADmkiNs67Y+3ylmlG7Lnu+FA==}
    dependencies:
@@ -4195,6 +4326,10 @@ packages:
  /electron-to-chromium@1.4.465:
    resolution: {integrity: sha512-XQcuHvEJRMU97UJ75e170mgcITZoz0lIyiaVjk6R+NMTJ8KBIvUHYd1779swgOppUlzxR+JsLpq59PumaXS1jQ==}

+  /emoji-regex@10.2.1:
+    resolution: {integrity: sha512-97g6QgOk8zlDRdgq1WxwgTMgEWGVAQvB5Fdpgc1MkNy56la5SKP9GsMXKDOdqwn90/41a8yPwIGk1Y6WVbeMQA==}
+    dev: false
+
  /emoji-regex@8.0.0:
    resolution: {integrity: sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==}
    dev: false
@@ -4910,6 +5045,10 @@ packages:
    resolution: {integrity: sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA==}
    dev: false

+  /fflate@0.7.4:
+    resolution: {integrity: sha512-5u2V/CDW15QM1XbbgS+0DfPxVB+jUKhWEKuuFuHncbk3tEEqzmoXL+2KyOFuKGqOnmdIy0/davWF1CkuwtibCw==}
+    dev: false
+
  /file-entry-cache@6.0.1:
    resolution: {integrity: sha512-7Gps/XWymbLk2QLYK4NzpMOrYjMhdIxXuIvy2QBsLE6ljuodKvdkWs/cpyJJ3CVIVpH0Oi1Hvg1ovbMzLdFBBg==}
    engines: {node: ^10.12.0 || >=12.0.0}
@@ -5110,6 +5249,10 @@ packages:
    dependencies:
      resolve-pkg-maps: 1.0.0

+  /github-buttons@2.27.0:
+    resolution: {integrity: sha512-PmfRMI2Rttg/2jDfKBeSl621sEznrsKF019SuoLdoNlO7qRUZaOyEI5Li4uW+79pVqnDtKfIEVuHTIJ5lgy64w==}
+    dev: false
+
  /glob-parent@5.1.2:
    resolution: {integrity: sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==}
    engines: {node: '>= 6'}
@@ -5295,6 +5438,11 @@ packages:
      space-separated-tokens: 1.1.5
    dev: false

+  /hex-rgb@4.3.0:
+    resolution: {integrity: sha512-Ox1pJVrDCyGHMG9CFg1tmrRUMRPRsAWYc/PinY0XzJU4K7y7vjNoLKIQ7BR5UJMCxNN8EM1MNDmHWA/B3aZUuw==}
+    engines: {node: '>=6'}
+    dev: false
+
  /highlight.js@10.7.3:
    resolution: {integrity: sha512-tzcUFauisWKNHaRkN4Wjl/ZA07gENAjFl3J/c480dprkGTg5EQstgaNFqBfUqCq54kZRIEcreTsAgF/m2quD7A==}
    dev: false
@@ -5431,6 +5579,10 @@ packages:
    resolution: {integrity: sha512-zz06S8t0ozoDXMG+ube26zeCTNXcKIPJZJi8hBrF4idCLms4CG9QtK7qBl1boi5ODzFpjswb5JPmHCbMpjaYzg==}
    dev: false

+  /is-arrayish@0.3.2:
+    resolution: {integrity: sha512-eVRqCvVlZbuw3GrM63ovNSNAeA1K16kaR/LRY/92w0zxQ5/1YzwblUX652i4Xs9RwAGjW9d9y6X88t8OaAJfWQ==}
+    dev: false
+
  /is-bigint@1.0.4:
    resolution: {integrity: sha512-zB9CruMamjym81i2JZ3UMn54PKGsQzsJeo6xvN3HJJ4CAsQNB6iRutp2To77OfCNuoxspsIhzaPoO1zyCEhFOg==}
    dependencies:
@@ -5760,6 +5912,13 @@ packages:
      type-check: 0.4.0
    dev: true

+  /linebreak@1.1.0:
+    resolution: {integrity: sha512-MHp03UImeVhB7XZtjd0E4n6+3xr5Dq/9xI/5FptGk5FrbDR3zagPa2DS6U8ks/3HjbKWG9Q1M2ufOzxV2qLYSQ==}
+    dependencies:
+      base64-js: 0.0.8
+      unicode-trie: 2.0.0
+    dev: false
+
  /lines-and-columns@1.2.4:
    resolution: {integrity: sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg==}
    dev: false
@@ -5798,6 +5957,10 @@ packages:
    resolution: {integrity: sha512-H5ZhCF25riFd9uB5UCkVKo61m3S/xZk1x4wA6yp/L3RFP6Z/eHH1ymQcGLo7J3GMPfm0V/7m1tryHuGVxpqEBQ==}
    dev: false

+  /lodash.curry@4.1.1:
+    resolution: {integrity: sha512-/u14pXGviLaweY5JI0IUzgzF2J6Ne8INyzAZjImcryjgkZ+ebruBxy2/JaOOkTqScddcYtakjhSaeemV8lR0tA==}
+    dev: false
+
  /lodash.merge@4.6.2:
    resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==}
    dev: true
@@ -6275,19 +6438,17 @@ packages:
      is-wsl: 2.2.0
    dev: true

-  /openai@4.0.0-beta.2:
-    resolution: {integrity: sha512-zTuAxBFe5nSO7LngbV+/O0udtgHWfXb2lFei8/sDY4GB5cOdnrRoSOtiyUfV65ANdvlI4F75oYZX7w067cxj3w==}
+  /openai@4.0.0-beta.7:
+    resolution: {integrity: sha512-jHjwvpMuGkNxiQ3erwLZsOvPEhcVrMtwtfNeYmGCjhbdB+oStVw/7pIhIPkualu8rlhLwgMR7awknIaN3IQcOA==}
    dependencies:
      '@types/node': 18.16.0
      '@types/node-fetch': 2.6.4
-      '@types/qs': 6.9.7
      abort-controller: 3.0.0
      agentkeepalive: 4.3.0
      digest-fetch: 1.3.0
      form-data-encoder: 1.7.2
      formdata-node: 4.4.1
      node-fetch: 2.6.12
-      qs: 6.11.2
    transitivePeerDependencies:
      - encoding
      - supports-color
@@ -6364,12 +6525,23 @@ packages:
    resolution: {integrity: sha512-HAKu/fG3HpHFO0AA8WE8q2g+gBJaZ9MG7fcKk+IJPLTGAD6Psw4443l+9DGRbOIh3/aXr7Phy0TjilYivJo5XQ==}
    dev: false

+  /pako@0.2.9:
+    resolution: {integrity: sha512-NUcwaKxUxWrZLpDG+z/xZaCgQITkA/Dv4V/T6bw7VON6l1Xz/VnrBqrYjZQ12TamKHzITTfOEIYUj48y2KXImA==}
+    dev: false
+
  /parent-module@1.0.1:
    resolution: {integrity: sha512-GQ2EWRpQV8/o+Aw8YqtfZZPfNRWZYkbidE9k5rpl/hC3vtHHBfGm2Ifi6qWV+coDGkrUKZAxE3Lot5kcsRlh+g==}
    engines: {node: '>=6'}
    dependencies:
      callsites: 3.1.0

+  /parse-css-color@0.2.1:
+    resolution: {integrity: sha512-bwS/GGIFV3b6KS4uwpzCFj4w297Yl3uqnSgIPsoQkx7GMLROXfMnWvxfNkL0oh8HVhZA4hvJoEoEIqonfJ3BWg==}
+    dependencies:
+      color-name: 1.1.4
+      hex-rgb: 4.3.0
+    dev: false
+
  /parse-entities@2.0.0:
    resolution: {integrity: sha512-kkywGpCcRYhqQIchaWqZ875wzpS/bMKhz5HnN3p7wveJTkTtyAB/AlnS0f8DFSqYW1T82t6yEAkEcB+A1I3MbQ==}
    dependencies:
@@ -6536,6 +6708,10 @@ packages:
    engines: {node: '>=4'}
    dev: false

+  /postcss-value-parser@4.2.0:
+    resolution: {integrity: sha512-1NNCs6uurfkVbeXG4S8JFT9t19m45ICnif8zWLd5oPSZ50QnwMfK+H3jv408d4jw/7Bttv5axS5IiHoLaVNHeQ==}
+    dev: false
+
  /postcss@8.4.14:
    resolution: {integrity: sha512-E398TUmfAYFPBSdzgeieK2Y1+1cpdxJx8yXbK/m57nRhKSmk1GB2tO4lbLBtlkfPQTDKfe4Xqv1ASWPpayPEig==}
    engines: {node: ^10 || ^12 || >=14}
@@ -6705,13 +6881,6 @@ packages:
      side-channel: 1.0.4
    dev: false

-  /qs@6.11.2:
-    resolution: {integrity: sha512-tDNIz22aBzCDxLtVH++VnTfzxlfeK5CbqohpSqpJgj1Wg/cQbStNAz3NuqCs5vV+pjBsK4x4pN9HlVh7rcYRiA==}
-    engines: {node: '>=0.6'}
-    dependencies:
-      side-channel: 1.0.4
-    dev: false
-
  /queue-microtask@1.2.3:
    resolution: {integrity: sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==}
    dev: true
@@ -6748,6 +6917,18 @@ packages:
      webpack: 5.88.2
    dev: true

+  /react-base16-styling@0.9.1:
+    resolution: {integrity: sha512-1s0CY1zRBOQ5M3T61wetEpvQmsYSNtWEcdYzyZNxKa8t7oDvaOn9d21xrGezGAHFWLM7SHcktPuPTrvoqxSfKw==}
+    dependencies:
+      '@babel/runtime': 7.22.6
+      '@types/base16': 1.0.2
+      '@types/lodash': 4.14.195
+      base16: 1.0.0
+      color: 3.2.1
+      csstype: 3.1.2
+      lodash.curry: 4.1.1
+    dev: false
+
  /react-clientside-effect@1.2.6(react@18.2.0):
    resolution: {integrity: sha512-XGGGRQAKY+q25Lz9a/4EPqom7WRjz3z9R2k4jhVKA/puQFH/5Nt27vFZYql4m4NVNdUvX8PS3O7r/Zzm7cjUlg==}
    peerDependencies:
@@ -6807,6 +6988,15 @@ packages:
      use-sidecar: 1.1.2(@types/react@18.2.6)(react@18.2.0)
    dev: false

+  /react-github-btn@1.4.0(react@18.2.0):
+    resolution: {integrity: sha512-lV4FYClAfjWnBfv0iNlJUGhamDgIq6TayD0kPZED6VzHWdpcHmPfsYOZ/CFwLfPv4Zp+F4m8QKTj0oy2HjiGXg==}
+    peerDependencies:
+      react: '>=16.3.0'
+    dependencies:
+      github-buttons: 2.27.0
+      react: 18.2.0
+    dev: false
+
  /react-icons@4.10.1(react@18.2.0):
    resolution: {integrity: sha512-/ngzDP/77tlCfqthiiGNZeYFACw85fUjZtLbedmJ5DTlNDIwETxhwBzdOJ21zj4iJdvc0J3y7yOsX3PpxAJzrw==}
    peerDependencies:
@@ -6822,6 +7012,19 @@ packages:
    resolution: {integrity: sha512-xWGDIW6x921xtzPkhiULtthJHoJvBbF3q26fzloPCK0hsvxtPVelvftw3zjbHWSkR2km9Z+4uxbDDK/6Zw9B8w==}
    dev: true

+  /react-json-tree@0.18.0(@types/react@18.2.6)(react@18.2.0):
+    resolution: {integrity: sha512-Qe6HKSXrr++n9Y31nkRJ3XvQMATISpqigH1vEKhLwB56+nk5thTP0ITThpjxY6ZG/ubpVq/aEHIcyLP/OPHxeA==}
+    peerDependencies:
+      '@types/react': ^16.8.0 || ^17.0.0 || ^18.0.0
+      react: ^16.8.0 || ^17.0.0 || ^18.0.0
+    dependencies:
+      '@babel/runtime': 7.22.6
+      '@types/lodash': 4.14.195
+      '@types/react': 18.2.6
+      react: 18.2.0
+      react-base16-styling: 0.9.1
+    dev: false
+
  /react-remove-scroll-bar@2.3.4(@types/react@18.2.6)(react@18.2.0):
    resolution: {integrity: sha512-63C4YQBUt0m6ALadE9XV56hV8BgJWDmmTPY758iIJjfQKt2nYwoUrPk0LXRXcB/yIj82T1/Ixfdpdk68LwIB0A==}
    engines: {node: '>=10'}
@@ -7112,6 +7315,22 @@ packages:
    resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==}
    dev: false

+  /satori@0.10.1:
+    resolution: {integrity: sha512-F4bTCkDp931tLb7+UCNPBuSQwXhikrUkI4fBQo6fA8lF0Evqqgg3nDyUpRktQpR5Ry1DIiIVqLyEwkAms87ykg==}
+    engines: {node: '>=16'}
+    dependencies:
+      '@shuding/opentype.js': 1.4.0-beta.0
+      css-background-parser: 0.1.0
+      css-box-shadow: 1.0.0-3
+      css-to-react-native: 3.2.0
+      emoji-regex: 10.2.1
+      escape-html: 1.0.3
+      linebreak: 1.1.0
+      parse-css-color: 0.2.1
+      postcss-value-parser: 4.2.0
+      yoga-wasm-web: 0.3.3
+    dev: false
+
  /scheduler@0.23.0:
    resolution: {integrity: sha512-CtuThmgHNg7zIZWAXi3AsyIzA3n4xx7aNyjwC2VJldO2LMVDhFK+63xGqq6CsJH4rTAt6/M+N4GhZiDYPx9eUw==}
    dependencies:
@@ -7217,6 +7436,12 @@ packages:
    resolution: {integrity: sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ==}
    dev: true

+  /simple-swizzle@0.2.2:
+    resolution: {integrity: sha512-JA//kQgZtbuY83m+xT+tXJkmJncGMTFT+C+g2h2R9uxkYIrE2yy9sgmcLhCnw57/WSD+Eh3J97FPEDFnbXnDUg==}
+    dependencies:
+      is-arrayish: 0.3.2
+    dev: false
+
  /slash@3.0.0:
    resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==}
    engines: {node: '>=8'}
@@ -7339,6 +7564,10 @@ packages:
      strip-ansi: 6.0.1
    dev: false

+  /string.prototype.codepointat@0.2.1:
+    resolution: {integrity: sha512-2cBVCj6I4IOvEnjgO/hWqXjqBGsY+zwPmHl12Srk9IXSZ56Jwwmy+66XO5Iut/oQVR7t5ihYdLB0GMa4alEUcg==}
+    dev: false
+
  /string.prototype.matchall@4.0.8:
    resolution: {integrity: sha512-6zOCOcJ+RJAQshcTvXPHoxoQGONa3e/Lqx90wUA+wEzX78sg5Bo+1tQo4N0pohS0erG9qtCqJDjNCQBjeWVxyg==}
    dependencies:
@@ -7565,6 +7794,10 @@ packages:
      globrex: 0.1.2
    dev: true

+  /tiny-inflate@1.0.3:
+    resolution: {integrity: sha512-pkY1fj1cKHb2seWDy0B16HeWyczlJA9/WW3u3c4z/NiWDsO3DOU5D7nhTLE9CF0yXv/QZFY7sEJmj24dK+Rrqw==}
+    dev: false
+
  /tiny-invariant@1.3.1:
    resolution: {integrity: sha512-AD5ih2NlSssTCwsMznbvwMZpJ1cbhkGd2uueNxzv2jDlEeZdU04JQfRnggJQ8DrcVBGjAsCKwFBbDlVNtEMlzw==}
    dev: false
@@ -7771,6 +8004,13 @@ packages:
      busboy: 1.6.0
    dev: true

+  /unicode-trie@2.0.0:
+    resolution: {integrity: sha512-x7bc76x0bm4prf1VLg79uhAzKw8DVboClSN5VxJuQ+LKDOVEW9CdH+VY7SP+vX7xCYQqzzgQpFqz15zeLvAtZQ==}
+    dependencies:
+      pako: 0.2.9
+      tiny-inflate: 1.0.3
+    dev: false
+
  /unpipe@1.0.0:
    resolution: {integrity: sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==}
    engines: {node: '>= 0.8'}
@@ -7913,6 +8153,11 @@ packages:
    hasBin: true
    dev: false

+  /uuid@9.0.0:
+    resolution: {integrity: sha512-MXcSTerfPa4uqyzStbRoTgt5XIe3x5+42+q1sDuy3R5MDk66URdLMOZe5aPX/SQd+kuYAh0FdP/pO28IkQyTeg==}
+    hasBin: true
+    dev: false
+
  /vary@1.1.2:
    resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
    engines: {node: '>= 0.8'}
@@ -8271,6 +8516,10 @@ packages:
    engines: {node: '>=12.20'}
    dev: true

+  /yoga-wasm-web@0.3.3:
+    resolution: {integrity: sha512-N+d4UJSJbt/R3wqY7Coqs5pcV0aUj2j9IaQ3rNj9bVCLld8tTGKRa2USARjnvZJWVx1NDmQev8EknoczaOQDOA==}
+    dev: false
+
  /zod@3.21.4:
    resolution: {integrity: sha512-m46AKbrzKVzOzs/DZgVnG5H55N1sv1M8qZU3A8RIKbs3mrACDNeIOeilDymVb2HdmP8uwshOCF4uJ8uM9rCqJw==}
    dev: false
--- a/prisma/datasets/validated_tweets.csv
+++ b/prisma/datasets/validated_tweets.csv
@@ -0,0 +1,84 @@
+Text,sentiment,emotion
+@dell your customer service is horrible especially agent syedfaisal who has made this experience of purchasing a new computer downright awful and I’ll reconsider ever buying a Dell in the future @DellTech,negative,anger
+@zacokalo @Dell @DellCares @Dell give the man what he paid for!,neutral,anger
+"COOKING STREAM DAY!!! Ty to @Alienware for sponsoring this stream! I’ll be making a bunch of Japanese Alien themed foods hehe 
+
+Come check it out! https://t.co/m06tJQ06zk 
+
+#alienwarepartner #intelgaming @Dell @IntelGaming https://t.co/qOdQX2E8VD",positive,joy
+@emijuju_ @Alienware @Dell @intel Beautiful 😍❤️😻,positive,joy
+"What's your biggest data management challenge?     • Cloud complexity?   • Lengthy tech refresh cycles?   • Capital budget constraints?      Solve your challenges with as-a-Storage. Get simplicity, agility &amp; control with @Dell #APEX. https://t.co/mCblMtH931 https://t.co/eepKNZ4Ai3",neutral,optimism
+"This week we were at the ""Top Gun"" themed @Dell Product Expo. Eddie Muñoz met Maverick look-alike, California Tom Cruise (Jerome LeBlanc)!
+
+""I feel the need, the need for speed."" - Maverick
+#topgun #topgunmaverick #dell #delltechnologies #lockncharge https://t.co/QHYH2EbMjq",positive,joy
+"Itsss been more than a week...i m following up with dell for troubleshootings...my https://t.co/lWhg2YKhQa suffering so as my hard earned money...hightly disappointed...contd..
+@DellCares @Dell",negative,sadness
+"@ashu_k7 @Dell Pathetic!!!!! I Dont mind taking legal action, this is deficency of service for which the customer is nt getting help..",negative,anger
+@ashu_k7 @Dell Making life unhappy is the new tag line of #Dell,negative,sadness
+"@Dell If you are buying a Dell, make sure you are making your life hell.
+Better buy other laptops. If you wanted to opt for Dell better opt for garbage on the streets.",negative,anger
+"MY DESK'S FINAL FORM? Seriously, I'm finally happy with my monitor setup here... and I'll keep this setup whenever I move... FOREVER. What do you think?
+https://t.co/WJZ2JXtOnX
+@Alienware @Dell cheers. https://t.co/6Whhldfpv0",positive,joy
+"@Dell Dell Alienware computer has had software problems with SupportAssist since purchase.  Dell, despite paying for Premium Support, has never fixed issues.  Latest solution was to erase everything and reload....SupportAssist still doesn't work.",negative,anger
+"HUGE congratulations to Startup Battle 3.0 winner ➡️ @Ox_Fulfillment x @cyborgcharu for being featured in @BusinessInsider &amp; @Dell showcasing the journey at Ox! 🚀🚀🚀
+
+We love to see our portfolio companies continuing to BUILD SOMETHING FROM NOTHING! 🔥 https://t.co/awBkn5ippB",positive,joy
+@Dell happy Friday!,positive,joy
+"@intel Core i5 1135G7 - 4732 points
+@intel Core i5 1235 - 6619 points 
+@Dell Latitude 5420 x 5430.
+Cinebench R23. Good job Intel!",positive,joy
+@Dell india we purchased 52 docking station and we have around 100 users using dell laptop as well as dell monitor now they are refusing to replace my faulty product and disconnecting my every call....,negative,anger
+"It's another year ans another day But cant fill it in yet the child hood dreams.
+It's my birthdy today. Can anyone of you guys bless me with  a simplest gaming oc that can run 
+@DOTA2 ?
+@Dell @HP @VastGG @Acer @Alienware @Lenovo @toshiba @IBM @Fujitsu_Global @NEC https://t.co/69G8tL9sN8",neutral,joy
+"@idoccor @Dell That's always the decision—wait, or, look elsewhere. In this case, I think I unfortunately need to wait since there are only two monitors with these specs and I don't like the other one 😂",negative,sadness
+"@MichaelDell @Dell @DellCares For how long this will continue. It is high time you either fix the problem for good or replace the complete laptop. Spent over 60+ hours with Customer Care teams, which is not helping. Cannot keep going on like this.",negative,anger
+"@Dell @DellCares but no, not really",neutral,sadness
+"Business innovation requires insight, agility and efficiency. How do you get there? RP PRO, LLC recommends starting by proactively managing IT infrastructure with #OpenManage Systems from @Dell. https://t.co/fBcK1lfFMu https://t.co/xWHLkkHCjn",neutral,optimism
+@Dell Yessirrrrr #NationalCoffeeDay,positive,joy
+"New blog post from @Dell shared on https://t.co/EgfPChB8AT 
+
+Re-routing Our Connected and Autonomous Future https://t.co/AW8EHQrbd6
+
+#future #futuretech #techinnovation https://t.co/koX8stKPsr",neutral,joy
+"In a free-market economy, the folks @IronMountain can set prices as they see fit. Their customers are also free to find better prices at competitors like @Dell 
+@H3CGlobal @HPE
+https://t.co/reZ56DNTBI",neutral,optimism
+"Delighted to chat with many of our partners here in person at @Intel Innovation! @Dell, @Lenovo, @Supermicro_SMCI, @QuantaQCT #IntelON https://t.co/BxIeGW8deN",positive,joy
+"A special gracias to our Startup Chica San Antonio 2022 sponsors @eBay, @jcpenney, @Barbie, @HEB, @Dell, @Honda, @SouthsideSATX💜✨ https://t.co/lZ6WWkziHl",positive,joy
+"When your team decides to start supporting developers, your #ops must change too. More from @cote and @Dell Developer Community Manager @barton808: https://t.co/W6f1oMiTgV",neutral,optimism
+@EmDStowers @LASERGIANT1 @ohwormongod @Ludovician_Vega @Dell our boy snitchin,neutral,anger
+A 1st place dmi:Design Value Award goes to @Dell for a packaging modernization initiative that helped them get closer to their corporate Moonshot Sustainability Goal of 100% recycled or renewable packaging by 2030. More at https://t.co/dnhZWWLCQC #designvalue #DVA22,positive,optimism
+Reducing deployment and maintenance complexity is the goal behind @dell and @WindRiver's new collaboration. https://t.co/2PxQgPuHUU,positive,optimism
+@jaserhunter @Dell Love the sales pitch lol,positive,joy
+@Dell india we purchased 52 docking station and we have around 100 users using dell laptop as well as dell monitor now they are refusing to replace my faulty product and disconnecting my every call....,negative,anger
+@ashu_k7 @Dell One more example.. their technical support is also worse. https://t.co/20atSgI4fg,negative,anger
+*angry screeches about @Dell proprietary MBR windows 8.1 partitions not being able to save as an img in clonezilla *,negative,anger
+@socialitebooks @BBYC_Gamers @Dell @Alienware @BestBuyCanada @intelcanada Congratulations!!!,positive,joy
+"Thank you to the @dell team for coming out to volunteer today! We truly appreciate your hard work and look forward to seeing you again soon!
+
+If you and your team are interested in helping out at the UMLAUF, visit our website for more information: https://t.co/lVfsZT2ogS https://t.co/eLz0FY0y4M",positive,joy
+"@TheCaramelGamer @intel @bravadogaming @Intel_Africa @Dell @DellTech @DellTechMEA @Alienware @IntelUK we love to see it. 
+
+Also also actually actually whoever did that artwork? 🔥🔥🔥 am a fan.",positive,joy
+"LOVING MY DELL 2 IN 1 LAPTOP
+YAYY 🥳🥳
+@Dell #DellInspiron #DellLaptop https://t.co/vib96jf3tC",positive,joy
+@Azure @OracleItalia @AWS_Italy @lenovoitalia @Dell discussing the future of #HPC during the #hpcroundtable22 in Turin today #highperformancecomputing https://t.co/jJ1WqBulPF,neutral,joy
+Attracting talent @AmericanChamber. @marg_cola @Dell speaks of quality of life connectivity and the Opportunity for development being so crucial. Housing  availability is now impacting on decision making for potential candidates. #WhyCork,positive,optimism
+.@Dell partners with @WindRiver on modular cloud-native telecommunications infrastructure https://t.co/4SWATspwCP @SiliconANGLE @Mike_Wheatley @holgermu @constellationr,neutral,joy
+@Dell Not buy Dell Inspiron laptop,neutral,sadness
+"@dell #delltechforum reminding us IDC have predicted that by 2024, 50% of everything we consume in technology will be as a service https://t.co/3UBiZJX0LE",neutral,optimism
+@RachMurph @HETTShow @Dell Thank you for coming! Great evening,positive,joy
+Congratulations to Jason M of Moncton NB on winning a @Dell  @Alienware m15 R7 15.6″ gaming laptop from @BestBuyCanada and @intelcanada's gaming days #contest on the blog. Visit https://t.co/VryaY5Rvv9 to learn about tech and for chances to win new tech. https://t.co/T6n0dzF6oL,positive,joy
+@MattVisiwig @Dell Sour taste for sure 😶 But don't let ego distract you from what you really want to buy 😁,neutral,optimism
+"Massive thank you goes to sponsors @HendersonLoggie @lindsaysnews @Dell @unity, all of our fantastic judges and mentors and the team at @EGX and @ExCeLLondon. 
+
+Big congratulations also to all of our other @AbertayDare teams - an amazing year! #Dare2022 https://t.co/jYe4agO7lW",positive,joy
+"@timetcetera @rahaug Nah, I just need @Dell to start paying me comissions 😂",neutral,joy
+"""Whether you’re an engineer, a designer, or work in supply chain management or sales, there are always opportunities to think about sustainability and how you can do things more efficiently."" 👏 — Oliver Campbell, Director of Packaging Engineering, @Dell https://t.co/vUJLTWNFwP https://t.co/GJWAzGfAxJ",positive,optimism
+"Hi, my name is @listerepvp and I support @Dell, always.",positive,joy
--- a/prisma/migrations/20230725005817_use_id_as_streaming_channel/migration.sql
+++ b/prisma/migrations/20230725005817_use_id_as_streaming_channel/migration.sql
@@ -0,0 +1,8 @@
+/*
+  Warnings:
+
+  - You are about to drop the column `streamingChannel` on the `ScenarioVariantCell` table. All the data in the column will be lost.
+
+*/
+-- AlterTable
+ALTER TABLE "ScenarioVariantCell" DROP COLUMN "streamingChannel";
--- a/prisma/migrations/20230725191512_migrate_model_response/migration.sql
+++ b/prisma/migrations/20230725191512_migrate_model_response/migration.sql
@@ -0,0 +1,52 @@
+-- DropForeignKey
+ALTER TABLE "ModelOutput" DROP CONSTRAINT "ModelOutput_scenarioVariantCellId_fkey";
+
+-- DropForeignKey
+ALTER TABLE "OutputEvaluation" DROP CONSTRAINT "OutputEvaluation_modelOutputId_fkey";
+
+-- DropIndex
+DROP INDEX "OutputEvaluation_modelOutputId_evaluationId_key";
+
+-- AlterTable
+ALTER TABLE "OutputEvaluation" RENAME COLUMN "modelOutputId" TO "modelResponseId";
+
+-- AlterTable
+ALTER TABLE "ScenarioVariantCell" DROP COLUMN "retryTime",
+DROP COLUMN "statusCode",
+ADD COLUMN     "jobQueuedAt" TIMESTAMP(3),
+ADD COLUMN     "jobStartedAt" TIMESTAMP(3);
+
+ALTER TABLE "ModelOutput" RENAME TO "ModelResponse";
+
+ALTER TABLE "ModelResponse" 
+ADD COLUMN "requestedAt" TIMESTAMP(3),
+ADD COLUMN "receivedAt" TIMESTAMP(3),
+ADD COLUMN "statusCode" INTEGER,
+ADD COLUMN "errorMessage" TEXT,
+ADD COLUMN "retryTime" TIMESTAMP(3),
+ADD COLUMN "outdated" BOOLEAN NOT NULL DEFAULT false;
+
+-- 3. Remove the unnecessary column
+ALTER TABLE "ModelResponse"
+DROP COLUMN "timeToComplete";
+
+-- AlterTable
+ALTER TABLE "ModelResponse" RENAME CONSTRAINT "ModelOutput_pkey" TO "ModelResponse_pkey";
+ALTER TABLE "ModelResponse" ALTER COLUMN "output" DROP NOT NULL;
+
+-- DropIndex
+DROP INDEX "ModelOutput_scenarioVariantCellId_key";
+
+-- AddForeignKey
+ALTER TABLE "ModelResponse" ADD CONSTRAINT "ModelResponse_scenarioVariantCellId_fkey" FOREIGN KEY ("scenarioVariantCellId") REFERENCES "ScenarioVariantCell"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+
+-- RenameIndex
+ALTER INDEX "ModelOutput_inputHash_idx" RENAME TO "ModelResponse_inputHash_idx";
+
+-- CreateIndex
+CREATE UNIQUE INDEX "OutputEvaluation_modelResponseId_evaluationId_key" ON "OutputEvaluation"("modelResponseId", "evaluationId");
+
+-- AddForeignKey
+ALTER TABLE "OutputEvaluation" ADD CONSTRAINT "OutputEvaluation_modelResponseId_fkey" FOREIGN KEY ("modelResponseId") REFERENCES "ModelResponse"("id") ON DELETE CASCADE ON UPDATE CASCADE;
+
+
--- a/prisma/migrations/20230801195916_add_world_champs/migration.sql
+++ b/prisma/migrations/20230801195916_add_world_champs/migration.sql
@@ -0,0 +1,16 @@
+-- CreateTable
+CREATE TABLE "WorldChampEntrant" (
+    "id" UUID NOT NULL,
+    "userId" UUID NOT NULL,
+    "approved" BOOLEAN NOT NULL DEFAULT false,
+    "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    "updatedAt" TIMESTAMP(3) NOT NULL,
+
+    CONSTRAINT "WorldChampEntrant_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateIndex
+CREATE UNIQUE INDEX "WorldChampEntrant_userId_key" ON "WorldChampEntrant"("userId");
+
+-- AddForeignKey
+ALTER TABLE "WorldChampEntrant" ADD CONSTRAINT "WorldChampEntrant_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
--- a/prisma/migrations/20230801234208_add_created_at_and_updated_at_to_users_table/migration.sql
+++ b/prisma/migrations/20230801234208_add_created_at_and_updated_at_to_users_table/migration.sql
@@ -0,0 +1,3 @@
+-- AlterTable
+ALTER TABLE "User" ADD COLUMN     "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ADD COLUMN     "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP;
--- a/prisma/schema.prisma
+++ b/prisma/schema.prisma
@@ -22,10 +22,10 @@ model Experiment {
    createdAt DateTime @default(now())
    updatedAt DateTime @updatedAt

-    TemplateVariable TemplateVariable[]
-    PromptVariant    PromptVariant[]
-    TestScenario     TestScenario[]
-    Evaluation       Evaluation[]
+    templateVariables TemplateVariable[]
+    promptVariants    PromptVariant[]
+    testScenarios     TestScenario[]
+    evaluations       Evaluation[]
 }

 model PromptVariant {
@@ -90,13 +90,11 @@ enum CellRetrievalStatus {
 model ScenarioVariantCell {
    id String @id @default(uuid()) @db.Uuid

-    statusCode       Int?
-    errorMessage     String?
-    retryTime        DateTime?
-    streamingChannel String?
    retrievalStatus CellRetrievalStatus @default(COMPLETE)
-
-    modelOutput ModelOutput?
+    jobQueuedAt     DateTime?
+    jobStartedAt    DateTime?
+    modelResponses  ModelResponse[]
+    errorMessage    String? // Contains errors that occurred independently of model responses

    promptVariantId String        @db.Uuid
    promptVariant   PromptVariant @relation(fields: [promptVariantId], references: [id], onDelete: Cascade)
@@ -111,24 +109,28 @@ model ScenarioVariantCell {
    @@unique([promptVariantId, testScenarioId])
 }

-model ModelOutput {
+model ModelResponse {
    id String @id @default(uuid()) @db.Uuid

    inputHash        String
-    output           Json
-    timeToComplete   Int    @default(0)
+    requestedAt      DateTime?
+    receivedAt       DateTime?
+    output           Json?
    cost             Float?
    promptTokens     Int?
    completionTokens Int?
+    statusCode       Int?
+    errorMessage     String?
+    retryTime        DateTime?
+    outdated         Boolean   @default(false)

    createdAt DateTime @default(now())
    updatedAt DateTime @updatedAt

    scenarioVariantCellId String              @db.Uuid
    scenarioVariantCell   ScenarioVariantCell @relation(fields: [scenarioVariantCellId], references: [id], onDelete: Cascade)
-    outputEvaluation      OutputEvaluation[]
+    outputEvaluations     OutputEvaluation[]

-    @@unique([scenarioVariantCellId])
    @@index([inputHash])
 }

@@ -150,7 +152,7 @@ model Evaluation {

    createdAt         DateTime           @default(now())
    updatedAt         DateTime           @updatedAt
-    OutputEvaluation OutputEvaluation[]
+    outputEvaluations OutputEvaluation[]
 }

 model OutputEvaluation {
@@ -160,8 +162,8 @@ model OutputEvaluation {
    result  Float
    details String?

-    modelOutputId String      @db.Uuid
-    modelOutput   ModelOutput @relation(fields: [modelOutputId], references: [id], onDelete: Cascade)
+    modelResponseId String        @db.Uuid
+    modelResponse   ModelResponse @relation(fields: [modelResponseId], references: [id], onDelete: Cascade)

    evaluationId String     @db.Uuid
    evaluation   Evaluation @relation(fields: [evaluationId], references: [id], onDelete: Cascade)
@@ -169,7 +171,7 @@ model OutputEvaluation {
    createdAt DateTime @default(now())
    updatedAt DateTime @updatedAt

-    @@unique([modelOutputId, evaluationId])
+    @@unique([modelResponseId, evaluationId])
 }

 model Organization {
@@ -179,8 +181,8 @@ model Organization {

    createdAt         DateTime           @default(now())
    updatedAt         DateTime           @updatedAt
-    OrganizationUser OrganizationUser[]
-    Experiment       Experiment[]
+    organizationUsers OrganizationUser[]
+    experiments       Experiment[]
 }

 enum OrganizationUserRole {
@@ -206,6 +208,20 @@ model OrganizationUser {
    @@unique([organizationId, userId])
 }

+model WorldChampEntrant {
+    id String @id @default(uuid()) @db.Uuid
+
+    userId String @db.Uuid
+    user   User   @relation(fields: [userId], references: [id], onDelete: Cascade)
+
+    approved Boolean @default(false)
+
+    createdAt DateTime @default(now())
+    updatedAt DateTime @updatedAt
+
+    @@unique([userId])
+}
+
 model Account {
    id                       String  @id @default(uuid()) @db.Uuid
    userId                   String  @db.Uuid
@@ -241,8 +257,12 @@ model User {
    image             String?
    accounts          Account[]
    sessions          Session[]
-    OrganizationUser OrganizationUser[]
-    Organization     Organization[]
+    organizationUsers OrganizationUser[]
+    organizations     Organization[]
+    worldChampEntrant WorldChampEntrant?
+
+    createdAt DateTime @default(now())
+    updatedAt DateTime @default(now()) @updatedAt
 }

 model VerificationToken {
--- a/prisma/seed.ts
+++ b/prisma/seed.ts
@@ -164,5 +164,5 @@ await Promise.all(
        testScenarioId: scenario.id,
      })),
    )
-    .map((cell) => generateNewCell(cell.promptVariantId, cell.testScenarioId)),
+    .map((cell) => generateNewCell(cell.promptVariantId, cell.testScenarioId, { stream: false })),
 );
--- a/prisma/seedAgiEval.ts
+++ b/prisma/seedAgiEval.ts
@@ -0,0 +1,127 @@
+import { prisma } from "~/server/db";
+import { generateNewCell } from "~/server/utils/generateNewCell";
+import dedent from "dedent";
+import { execSync } from "child_process";
+import fs from "fs";
+
+const defaultId = "11111111-1111-1111-1111-111111111112";
+
+await prisma.organization.deleteMany({
+  where: { id: defaultId },
+});
+
+// If there's an existing org, just seed into it
+const org =
+  (await prisma.organization.findFirst({})) ??
+  (await prisma.organization.create({
+    data: { id: defaultId },
+  }));
+
+// Clone the repo from git@github.com:microsoft/AGIEval.git into a tmp dir if it doesn't exist
+const tmpDir = "/tmp/agi-eval";
+if (!fs.existsSync(tmpDir)) {
+  execSync(`git clone git@github.com:microsoft/AGIEval.git ${tmpDir}`);
+}
+
+const datasets = [
+  "sat-en",
+  "sat-math",
+  "lsat-rc",
+  "lsat-ar",
+  "aqua-rat",
+  "logiqa-en",
+  "lsat-lr",
+  "math",
+];
+
+type Scenario = {
+  passage: string | null;
+  question: string;
+  options: string[] | null;
+  label: string;
+};
+
+for (const dataset of datasets) {
+  const experimentName = `AGI-Eval: ${dataset}`;
+  const oldExperiment = await prisma.experiment.findFirst({
+    where: {
+      label: experimentName,
+      organizationId: org.id,
+    },
+  });
+  if (oldExperiment) {
+    await prisma.experiment.deleteMany({
+      where: { id: oldExperiment.id },
+    });
+  }
+
+  const experiment = await prisma.experiment.create({
+    data: {
+      id: oldExperiment?.id ?? undefined,
+      label: experimentName,
+      organizationId: org.id,
+    },
+  });
+
+  const scenarios: Scenario[] = fs
+    .readFileSync(`${tmpDir}/data/v1/${dataset}.jsonl`, "utf8")
+    .split("\n")
+    .filter((line) => line.length > 0)
+    .map((line) => JSON.parse(line) as Scenario);
+  console.log("scenarios", scenarios.length);
+
+  await prisma.testScenario.createMany({
+    data: scenarios.slice(0, 30).map((scenario, i) => ({
+      experimentId: experiment.id,
+      sortIndex: i,
+      variableValues: {
+        passage: scenario.passage,
+        question: scenario.question,
+        options: scenario.options?.join("\n"),
+        label: scenario.label,
+      },
+    })),
+  });
+
+  await prisma.templateVariable.createMany({
+    data: ["passage", "question", "options", "label"].map((label) => ({
+      experimentId: experiment.id,
+      label,
+    })),
+  });
+
+  await prisma.promptVariant.createMany({
+    data: [
+      {
+        experimentId: experiment.id,
+        label: "Prompt Variant 1",
+        sortIndex: 0,
+        model: "gpt-3.5-turbo-0613",
+        modelProvider: "openai/ChatCompletion",
+        constructFnVersion: 1,
+        constructFn: dedent`
+          definePrompt("openai/ChatCompletion", {
+            model: "gpt-3.5-turbo-0613",
+            messages: [
+              {
+                role: "user",
+                content: \`Passage: ${"$"}{scenario.passage}\n\nQuestion: ${"$"}{scenario.question}\n\nOptions: ${"$"}{scenario.options}\n\n Respond with just the letter of the best option in the format Answer: (A).\`
+              }
+            ],
+            temperature: 0,
+          })`,
+      },
+    ],
+  });
+
+  await prisma.evaluation.createMany({
+    data: [
+      {
+        experimentId: experiment.id,
+        label: "Eval",
+        evalType: "CONTAINS",
+        value: "Answer: ({{label}})",
+      },
+    ],
+  });
+}
--- a/prisma/seedTwitterSentiment.ts
+++ b/prisma/seedTwitterSentiment.ts
@@ -0,0 +1,113 @@
+import { prisma } from "~/server/db";
+import dedent from "dedent";
+import fs from "fs";
+import { parse } from "csv-parse/sync";
+
+const defaultId = "11111111-1111-1111-1111-111111111112";
+
+await prisma.organization.deleteMany({
+  where: { id: defaultId },
+});
+
+// If there's an existing org, just seed into it
+const org =
+  (await prisma.organization.findFirst({})) ??
+  (await prisma.organization.create({
+    data: { id: defaultId },
+  }));
+
+type Scenario = {
+  text: string;
+  sentiment: string;
+  emotion: string;
+};
+
+const experimentName = `Twitter Sentiment Analysis`;
+const oldExperiment = await prisma.experiment.findFirst({
+  where: {
+    label: experimentName,
+    organizationId: org.id,
+  },
+});
+if (oldExperiment) {
+  await prisma.experiment.deleteMany({
+    where: { id: oldExperiment.id },
+  });
+}
+
+const experiment = await prisma.experiment.create({
+  data: {
+    id: oldExperiment?.id ?? undefined,
+    label: experimentName,
+    organizationId: org.id,
+  },
+});
+
+const content = fs.readFileSync("./prisma/datasets/validated_tweets.csv", "utf8");
+
+// eslint-disable-next-line @typescript-eslint/no-explicit-any
+const records: any[] = parse(content, { delimiter: ",", from_line: 2 });
+
+console.log("records", records);
+
+const scenarios: Scenario[] = records.map((row) => ({
+  text: row[0],
+  sentiment: row[1],
+  emotion: row[2],
+}));
+
+console.log("scenarios", scenarios.length);
+
+await prisma.testScenario.createMany({
+  data: scenarios.slice(0, 30).map((scenario, i) => ({
+    experimentId: experiment.id,
+    sortIndex: i,
+    variableValues: {
+      text: scenario.text,
+      sentiment: scenario.sentiment,
+      emotion: scenario.emotion,
+    },
+  })),
+});
+
+await prisma.templateVariable.createMany({
+  data: ["text", "sentiment", "emotion"].map((label) => ({
+    experimentId: experiment.id,
+    label,
+  })),
+});
+
+await prisma.promptVariant.createMany({
+  data: [
+    {
+      experimentId: experiment.id,
+      label: "Prompt Variant 1",
+      sortIndex: 0,
+      model: "gpt-3.5-turbo-0613",
+      modelProvider: "openai/ChatCompletion",
+      constructFnVersion: 1,
+      constructFn: dedent`
+          definePrompt("openai/ChatCompletion", {
+            model: "gpt-3.5-turbo-0613",
+            messages: [
+              {
+                role: "user",
+                content: \`Text: ${"$"}{scenario.text}\n\nRespond with the sentiment (negative|neutral|positive) and emotion (optimism|joy|anger|sadness) of the tweet in this format: "answer: <sentiment>-<emotion>".\`
+              }
+            ],
+            temperature: 0,
+          })`,
+    },
+  ],
+});
+
+await prisma.evaluation.createMany({
+  data: [
+    {
+      experimentId: experiment.id,
+      label: "Eval",
+      evalType: "CONTAINS",
+      value: "answer: {{sentiment}}-{{emotion}}",
+    },
+  ],
+});
--- a/public/fonts/Inconsolata_SemiExpanded-Medium.ttf
+++ b/public/fonts/Inconsolata_SemiExpanded-Medium.ttf
--- a/public/og.png
+++ b/public/og.png
--- a/run-prod.sh
+++ b/run-prod.sh
@@ -6,4 +6,7 @@ echo "Migrating the database"
 pnpm prisma migrate deploy

 echo "Starting the server"
-pnpm start
+
+pnpm concurrently --kill-others \
+  "pnpm start" \
+  "pnpm tsx src/server/tasks/worker.ts"
--- a/src/components/ChangeModelModal/ChangeModelModal.tsx
+++ b/src/components/ChangeModelModal/ChangeModelModal.tsx
@@ -1,5 +1,7 @@
 import {
  Button,
+  HStack,
+  Icon,
  Modal,
  ModalBody,
  ModalCloseButton,
@@ -7,24 +9,21 @@ import {
  ModalFooter,
  ModalHeader,
  ModalOverlay,
-  VStack,
-  Text,
  Spinner,
-  HStack,
-  Icon,
+  Text,
+  VStack,
 } from "@chakra-ui/react";
-import { RiExchangeFundsFill } from "react-icons/ri";
-import { useState } from "react";
-import { ModelStatsCard } from "./ModelStatsCard";
-import { ModelSearch } from "./ModelSearch";
-import { api } from "~/utils/api";
-import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
-import CompareFunctions from "../RefinePromptModal/CompareFunctions";
 import { type PromptVariant } from "@prisma/client";
 import { isObject, isString } from "lodash-es";
-import { type Model, type SupportedProvider } from "~/modelProviders/types";
-import frontendModelProviders from "~/modelProviders/frontendModelProviders";
-import { keyForModel } from "~/utils/utils";
+import { useState } from "react";
+import { RiExchangeFundsFill } from "react-icons/ri";
+import { type ProviderModel } from "~/modelProviders/types";
+import { api } from "~/utils/api";
+import { useExperiment, useHandledAsyncCallback, useVisibleScenarioIds } from "~/utils/hooks";
+import { lookupModel, modelLabel } from "~/utils/utils";
+import CompareFunctions from "../RefinePromptModal/CompareFunctions";
+import { ModelSearch } from "./ModelSearch";
+import { ModelStatsCard } from "./ModelStatsCard";

 export const ChangeModelModal = ({
  variant,
@@ -33,11 +32,14 @@ export const ChangeModelModal = ({
  variant: PromptVariant;
  onClose: () => void;
 }) => {
-  const originalModelProviderName = variant.modelProvider as SupportedProvider;
-  const originalModelProvider = frontendModelProviders[originalModelProviderName];
-  const originalModel = originalModelProvider.models[variant.model] as Model;
-  const [selectedModel, setSelectedModel] = useState<Model>(originalModel);
-  const [convertedModel, setConvertedModel] = useState<Model | undefined>(undefined);
+  const originalModel = lookupModel(variant.modelProvider, variant.model);
+  const [selectedModel, setSelectedModel] = useState({
+    provider: variant.modelProvider,
+    model: variant.model,
+  } as ProviderModel);
+  const [convertedModel, setConvertedModel] = useState<ProviderModel | undefined>();
+  const visibleScenarios = useVisibleScenarioIds();
+
  const utils = api.useContext();

  const experiment = useExperiment();
@@ -67,14 +69,16 @@ export const ChangeModelModal = ({
    await replaceVariantMutation.mutateAsync({
      id: variant.id,
      constructFn: modifiedPromptFn,
+      streamScenarios: visibleScenarios,
    });
    await utils.promptVariants.list.invalidate();
    onClose();
  }, [replaceVariantMutation, variant, onClose, modifiedPromptFn]);

-  const originalModelLabel = keyForModel(originalModel);
-  const selectedModelLabel = keyForModel(selectedModel);
-  const convertedModelLabel = convertedModel ? keyForModel(convertedModel) : undefined;
+  const originalLabel = modelLabel(variant.modelProvider, variant.model);
+  const selectedLabel = modelLabel(selectedModel.provider, selectedModel.model);
+  const convertedLabel =
+    convertedModel && modelLabel(convertedModel.provider, convertedModel.model);

  return (
    <Modal
@@ -94,16 +98,19 @@ export const ChangeModelModal = ({
        <ModalBody maxW="unset">
          <VStack spacing={8}>
            <ModelStatsCard label="Original Model" model={originalModel} />
-            {originalModelLabel !== selectedModelLabel && (
-              <ModelStatsCard label="New Model" model={selectedModel} />
+            {originalLabel !== selectedLabel && (
+              <ModelStatsCard
+                label="New Model"
+                model={lookupModel(selectedModel.provider, selectedModel.model)}
+              />
            )}
            <ModelSearch selectedModel={selectedModel} setSelectedModel={setSelectedModel} />
            {isString(modifiedPromptFn) && (
              <CompareFunctions
                originalFunction={variant.constructFn}
                newFunction={modifiedPromptFn}
-                leftTitle={originalModelLabel}
-                rightTitle={convertedModelLabel}
+                leftTitle={originalLabel}
+                rightTitle={convertedLabel}
              />
            )}
          </VStack>
@@ -115,7 +122,7 @@ export const ChangeModelModal = ({
              colorScheme="gray"
              onClick={getModifiedPromptFn}
              minW={24}
-              isDisabled={originalModel === selectedModel || modificationInProgress}
+              isDisabled={originalLabel === selectedLabel || modificationInProgress}
            >
              {modificationInProgress ? <Spinner boxSize={4} /> : <Text>Convert</Text>}
            </Button>
--- a/src/components/ChangeModelModal/ModelSearch.tsx
+++ b/src/components/ChangeModelModal/ModelSearch.tsx
@@ -1,49 +1,35 @@
-import { VStack, Text } from "@chakra-ui/react";
-import { type LegacyRef, useCallback } from "react";
-import Select, { type SingleValue } from "react-select";
+import { Text, VStack } from "@chakra-ui/react";
+import { type LegacyRef } from "react";
+import Select from "react-select";
 import { useElementDimensions } from "~/utils/hooks";

+import { flatMap } from "lodash-es";
 import frontendModelProviders from "~/modelProviders/frontendModelProviders";
-import { type Model } from "~/modelProviders/types";
-import { keyForModel } from "~/utils/utils";
+import { type ProviderModel } from "~/modelProviders/types";
+import { modelLabel } from "~/utils/utils";

-const modelOptions: { label: string; value: Model }[] = [];
+const modelOptions = flatMap(Object.entries(frontendModelProviders), ([providerId, provider]) =>
+  Object.entries(provider.models).map(([modelId]) => ({
+    provider: providerId,
+    model: modelId,
+  })),
+) as ProviderModel[];

-for (const [_, providerValue] of Object.entries(frontendModelProviders)) {
-  for (const [_, modelValue] of Object.entries(providerValue.models)) {
-    modelOptions.push({
-      label: keyForModel(modelValue),
-      value: modelValue,
-    });
-  }
-}
-
-export const ModelSearch = ({
-  selectedModel,
-  setSelectedModel,
-}: {
-  selectedModel: Model;
-  setSelectedModel: (model: Model) => void;
+export const ModelSearch = (props: {
+  selectedModel: ProviderModel;
+  setSelectedModel: (model: ProviderModel) => void;
 }) => {
-  const handleSelection = useCallback(
-    (option: SingleValue<{ label: string; value: Model }>) => {
-      if (!option) return;
-      setSelectedModel(option.value);
-    },
-    [setSelectedModel],
-  );
-  const selectedOption = modelOptions.find((option) => option.label === keyForModel(selectedModel));
-
  const [containerRef, containerDimensions] = useElementDimensions();

  return (
-    <VStack ref={containerRef as LegacyRef<HTMLDivElement>} w="full">
-      <Text>Browse Models</Text>
-      <Select
+    <VStack ref={containerRef as LegacyRef<HTMLDivElement>} w="full" fontFamily="inconsolata">
+      <Text fontWeight="bold">Browse Models</Text>
+      <Select<ProviderModel>
        styles={{ control: (provided) => ({ ...provided, width: containerDimensions?.width }) }}
-        value={selectedOption}
+        getOptionLabel={(data) => modelLabel(data.provider, data.model)}
+        getOptionValue={(data) => modelLabel(data.provider, data.model)}
        options={modelOptions}
-        onChange={handleSelection}
+        onChange={(option) => option && props.setSelectedModel(option)}
      />
    </VStack>
  );
--- a/src/components/ChangeModelModal/ModelStatsCard.tsx
+++ b/src/components/ChangeModelModal/ModelStatsCard.tsx
@@ -1,31 +1,46 @@
 import {
-  VStack,
-  Text,
-  HStack,
-  type StackProps,
  GridItem,
-  SimpleGrid,
+  HStack,
  Link,
+  SimpleGrid,
+  Text,
+  VStack,
+  type StackProps,
 } from "@chakra-ui/react";
-import { type Model } from "~/modelProviders/types";
+import { type lookupModel } from "~/utils/utils";

-export const ModelStatsCard = ({ label, model }: { label: string; model: Model }) => {
+export const ModelStatsCard = ({
+  label,
+  model,
+}: {
+  label: string;
+  model: ReturnType<typeof lookupModel>;
+}) => {
+  if (!model) return null;
  return (
    <VStack w="full" align="start">
      <Text fontWeight="bold" fontSize="sm" textTransform="uppercase">
        {label}
      </Text>

-      <VStack w="full" spacing={6} bgColor="gray.100" p={4} borderRadius={4}>
+      <VStack
+        w="full"
+        spacing={6}
+        borderWidth={1}
+        borderColor="gray.300"
+        p={4}
+        borderRadius={8}
+        fontFamily="inconsolata"
+      >
        <HStack w="full" align="flex-start">
-          <Text flex={1} fontSize="lg">
-            <Text as="span" color="gray.600">
-              {model.provider} /{" "}
-            </Text>
+          <VStack flex={1} fontSize="lg" alignItems="flex-start">
            <Text as="span" fontWeight="bold" color="gray.900">
              {model.name}
            </Text>
+            <Text as="span" color="gray.600" fontSize="sm">
+              Provider: {model.provider}
            </Text>
+          </VStack>
          <Link
            href={model.learnMoreUrl}
            isExternal
--- a/src/components/ExperimentSettingsDrawer/DeleteButton.tsx
+++ b/src/components/ExperimentSettingsDrawer/DeleteButton.tsx
@@ -0,0 +1,69 @@
+import {
+  Button,
+  Icon,
+  AlertDialog,
+  AlertDialogBody,
+  AlertDialogFooter,
+  AlertDialogHeader,
+  AlertDialogContent,
+  AlertDialogOverlay,
+  useDisclosure,
+  Text,
+} from "@chakra-ui/react";
+
+import { useRouter } from "next/router";
+import { useRef } from "react";
+import { BsTrash } from "react-icons/bs";
+import { api } from "~/utils/api";
+import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
+
+export const DeleteButton = () => {
+  const experiment = useExperiment();
+  const mutation = api.experiments.delete.useMutation();
+  const utils = api.useContext();
+  const router = useRouter();
+
+  const { isOpen, onOpen, onClose } = useDisclosure();
+  const cancelRef = useRef<HTMLButtonElement>(null);
+
+  const [onDeleteConfirm] = useHandledAsyncCallback(async () => {
+    if (!experiment.data?.id) return;
+    await mutation.mutateAsync({ id: experiment.data.id });
+    await utils.experiments.list.invalidate();
+    await router.push({ pathname: "/experiments" });
+    onClose();
+  }, [mutation, experiment.data?.id, router]);
+
+  return (
+    <>
+      <Button size="sm" variant="ghost" colorScheme="red" fontWeight="normal" onClick={onOpen}>
+        <Icon as={BsTrash} boxSize={4} />
+        <Text ml={2}>Delete Experiment</Text>
+      </Button>
+
+      <AlertDialog isOpen={isOpen} leastDestructiveRef={cancelRef} onClose={onClose}>
+        <AlertDialogOverlay>
+          <AlertDialogContent>
+            <AlertDialogHeader fontSize="lg" fontWeight="bold">
+              Delete Experiment
+            </AlertDialogHeader>
+
+            <AlertDialogBody>
+              If you delete this experiment all the associated prompts and scenarios will be deleted
+              as well. Are you sure?
+            </AlertDialogBody>
+
+            <AlertDialogFooter>
+              <Button ref={cancelRef} onClick={onClose}>
+                Cancel
+              </Button>
+              <Button colorScheme="red" onClick={onDeleteConfirm} ml={3}>
+                Delete
+              </Button>
+            </AlertDialogFooter>
+          </AlertDialogContent>
+        </AlertDialogOverlay>
+      </AlertDialog>
+    </>
+  );
+};
--- a/src/components/ExperimentSettingsDrawer/ExperimentSettingsDrawer.tsx
+++ b/src/components/ExperimentSettingsDrawer/ExperimentSettingsDrawer.tsx
@@ -6,13 +6,14 @@ import {
  DrawerHeader,
  DrawerOverlay,
  Heading,
-  Stack,
+  VStack,
 } from "@chakra-ui/react";
-import EditScenarioVars from "./EditScenarioVars";
-import EditEvaluations from "./EditEvaluations";
+import EditScenarioVars from "../OutputsTable/EditScenarioVars";
+import EditEvaluations from "../OutputsTable/EditEvaluations";
 import { useAppStore } from "~/state/store";
+import { DeleteButton } from "./DeleteButton";

-export default function SettingsDrawer() {
+export default function ExperimentSettingsDrawer() {
  const isOpen = useAppStore((state) => state.drawerOpen);
  const closeDrawer = useAppStore((state) => state.closeDrawer);

@@ -22,13 +23,16 @@ export default function SettingsDrawer() {
      <DrawerContent>
        <DrawerCloseButton />
        <DrawerHeader>
-          <Heading size="md">Settings</Heading>
+          <Heading size="md">Experiment Settings</Heading>
        </DrawerHeader>
-        <DrawerBody>
-          <Stack spacing={6}>
+        <DrawerBody h="full" pb={4}>
+          <VStack h="full" justifyContent="space-between">
+            <VStack spacing={6}>
              <EditScenarioVars />
              <EditEvaluations />
-          </Stack>
+            </VStack>
+            <DeleteButton />
+          </VStack>
        </DrawerBody>
      </DrawerContent>
    </Drawer>
--- a/src/components/OutputsTable/AddVariantButton.tsx
+++ b/src/components/OutputsTable/AddVariantButton.tsx
@@ -2,7 +2,12 @@ import { Box, Flex, Icon, Spinner } from "@chakra-ui/react";
 import { BsPlus } from "react-icons/bs";
 import { Text } from "@chakra-ui/react";
 import { api } from "~/utils/api";
-import { useExperiment, useExperimentAccess, useHandledAsyncCallback } from "~/utils/hooks";
+import {
+  useExperiment,
+  useExperimentAccess,
+  useHandledAsyncCallback,
+  useVisibleScenarioIds,
+} from "~/utils/hooks";
 import { cellPadding } from "../constants";
 import { ActionButton } from "./ScenariosHeader";

@@ -10,11 +15,13 @@ export default function AddVariantButton() {
  const experiment = useExperiment();
  const mutation = api.promptVariants.create.useMutation();
  const utils = api.useContext();
+  const visibleScenarios = useVisibleScenarioIds();

  const [onClick, loading] = useHandledAsyncCallback(async () => {
    if (!experiment.data) return;
    await mutation.mutateAsync({
      experimentId: experiment.data.id,
+      streamScenarios: visibleScenarios,
    });
    await utils.promptVariants.list.invalidate();
  }, [mutation]);
--- a/src/components/OutputsTable/FloatingLabelInput.tsx
+++ b/src/components/OutputsTable/FloatingLabelInput.tsx
@@ -37,7 +37,6 @@ export const FloatingLabelInput = ({
        borderColor={isFocused ? "blue.500" : "gray.400"}
        autoComplete="off"
        value={value}
-        maxHeight={32}
        overflowY="auto"
        overflowX="hidden"
        {...props}
--- a/src/components/OutputsTable/OutputCell/CellOptions.tsx
+++ b/src/components/OutputsTable/OutputCell/CellOptions.tsx
@@ -1,37 +0,0 @@
-import { Button, HStack, Icon, Tooltip } from "@chakra-ui/react";
-import { BsArrowClockwise } from "react-icons/bs";
-import { useExperimentAccess } from "~/utils/hooks";
-
-export const CellOptions = ({
-  refetchingOutput,
-  refetchOutput,
-}: {
-  refetchingOutput: boolean;
-  refetchOutput: () => void;
-}) => {
-  const { canModify } = useExperimentAccess();
-  return (
-    <HStack justifyContent="flex-end" w="full">
-      {!refetchingOutput && canModify && (
-        <Tooltip label="Refetch output" aria-label="refetch output">
-          <Button
-            size="xs"
-            w={4}
-            h={4}
-            py={4}
-            px={4}
-            minW={0}
-            borderRadius={8}
-            color="gray.500"
-            variant="ghost"
-            cursor="pointer"
-            onClick={refetchOutput}
-            aria-label="refetch output"
-          >
-            <Icon as={BsArrowClockwise} boxSize={4} />
-          </Button>
-        </Tooltip>
-      )}
-    </HStack>
-  );
-};
--- a/src/components/OutputsTable/OutputCell/OutputCell.tsx
+++ b/src/components/OutputsTable/OutputCell/OutputCell.tsx
@@ -1,16 +1,19 @@
 import { api } from "~/utils/api";
 import { type PromptVariant, type Scenario } from "../types";
-import { Spinner, Text, Center, VStack } from "@chakra-ui/react";
+import { type StackProps, Text, VStack } from "@chakra-ui/react";
 import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
 import SyntaxHighlighter from "react-syntax-highlighter";
 import { docco } from "react-syntax-highlighter/dist/cjs/styles/hljs";
 import stringify from "json-stringify-pretty-compact";
-import { type ReactElement, useState, useEffect } from "react";
+import { type ReactElement, useState, useEffect, Fragment, useCallback } from "react";
 import useSocket from "~/utils/useSocket";
 import { OutputStats } from "./OutputStats";
-import { ErrorHandler } from "./ErrorHandler";
-import { CellOptions } from "./CellOptions";
+import { RetryCountdown } from "./RetryCountdown";
 import frontendModelProviders from "~/modelProviders/frontendModelProviders";
+import { ResponseLog } from "./ResponseLog";
+import { CellOptions } from "./TopActions";
+
+const WAITING_MESSAGE_INTERVAL = 20000;

 export default function OutputCell({
  scenario,
@@ -60,62 +63,115 @@ export default function OutputCell({

  const awaitingOutput =
    !cell ||
+    !cell.evalsComplete ||
    cell.retrievalStatus === "PENDING" ||
    cell.retrievalStatus === "IN_PROGRESS" ||
    hardRefetching;
  useEffect(() => setRefetchInterval(awaitingOutput ? 1000 : 0), [awaitingOutput]);

-  const modelOutput = cell?.modelOutput;
+  // TODO: disconnect from socket if we're not streaming anymore
+  const streamedMessage = useSocket<OutputSchema>(cell?.id);

-  // Disconnect from socket if we're not streaming anymore
-  const streamedMessage = useSocket<OutputSchema>(cell?.streamingChannel);
+  const mostRecentResponse = cell?.modelResponses[cell.modelResponses.length - 1];
+
+  const CellWrapper = useCallback(
+    ({ children, ...props }: StackProps) => (
+      <VStack w="full" alignItems="flex-start" {...props} px={2} py={2} h="100%">
+        {cell && (
+          <CellOptions refetchingOutput={hardRefetching} refetchOutput={hardRefetch} cell={cell} />
+        )}
+        <VStack w="full" alignItems="flex-start" maxH={500} overflowY="auto" flex={1}>
+          {children}
+        </VStack>
+        {mostRecentResponse && (
+          <OutputStats modelResponse={mostRecentResponse} scenario={scenario} />
+        )}
+      </VStack>
+    ),
+    [hardRefetching, hardRefetch, mostRecentResponse, scenario],
+  );

  if (!vars) return null;

-  if (disabledReason) return <Text color="gray.500">{disabledReason}</Text>;
-
-  if (awaitingOutput && !streamedMessage)
-    return (
-      <Center h="100%" w="100%">
-        <Spinner />
-      </Center>
-    );
-
  if (!cell && !fetchingOutput)
    return (
-      <VStack>
-        <CellOptions refetchingOutput={hardRefetching} refetchOutput={hardRefetch} />
+      <CellWrapper>
        <Text color="gray.500">Error retrieving output</Text>
-      </VStack>
+      </CellWrapper>
    );

  if (cell && cell.errorMessage) {
    return (
-      <VStack>
-        <CellOptions refetchingOutput={hardRefetching} refetchOutput={hardRefetch} />
-        <ErrorHandler cell={cell} refetchOutput={hardRefetch} />
-      </VStack>
+      <CellWrapper>
+        <Text color="red.500">{cell.errorMessage}</Text>
+      </CellWrapper>
    );
  }

-  const normalizedOutput = modelOutput
-    ? provider.normalizeOutput(modelOutput.output)
+  if (disabledReason) return <Text color="gray.500">{disabledReason}</Text>;
+
+  const showLogs = !streamedMessage && !mostRecentResponse?.output;
+
+  if (showLogs)
+    return (
+      <CellWrapper alignItems="flex-start" fontFamily="inconsolata, monospace" spacing={0}>
+        {cell?.jobQueuedAt && <ResponseLog time={cell.jobQueuedAt} title="Job queued" />}
+        {cell?.jobStartedAt && <ResponseLog time={cell.jobStartedAt} title="Job started" />}
+        {cell?.modelResponses?.map((response) => {
+          let numWaitingMessages = 0;
+          const relativeWaitingTime = response.receivedAt
+            ? response.receivedAt.getTime()
+            : Date.now();
+          if (response.requestedAt) {
+            numWaitingMessages = Math.floor(
+              (relativeWaitingTime - response.requestedAt.getTime()) / WAITING_MESSAGE_INTERVAL,
+            );
+          }
+          return (
+            <Fragment key={response.id}>
+              {response.requestedAt && (
+                <ResponseLog time={response.requestedAt} title="Request sent to API" />
+              )}
+              {response.requestedAt &&
+                Array.from({ length: numWaitingMessages }, (_, i) => (
+                  <ResponseLog
+                    key={`waiting-${i}`}
+                    time={
+                      new Date(
+                        (response.requestedAt?.getTime?.() ?? 0) +
+                          (i + 1) * WAITING_MESSAGE_INTERVAL,
+                      )
+                    }
+                    title="Waiting for response..."
+                  />
+                ))}
+              {response.receivedAt && (
+                <ResponseLog
+                  time={response.receivedAt}
+                  title="Response received from API"
+                  message={`statusCode: ${response.statusCode ?? ""}\n ${
+                    response.errorMessage ?? ""
+                  }`}
+                />
+              )}
+            </Fragment>
+          );
+        }) ?? null}
+        {mostRecentResponse?.retryTime && (
+          <RetryCountdown retryTime={mostRecentResponse.retryTime} />
+        )}
+      </CellWrapper>
+    );
+
+  const normalizedOutput = mostRecentResponse?.output
+    ? provider.normalizeOutput(mostRecentResponse?.output)
    : streamedMessage
    ? provider.normalizeOutput(streamedMessage)
    : null;

-  if (modelOutput && normalizedOutput?.type === "json") {
+  if (mostRecentResponse?.output && normalizedOutput?.type === "json") {
    return (
-      <VStack
-        w="100%"
-        h="100%"
-        fontSize="xs"
-        flexWrap="wrap"
-        overflowX="hidden"
-        justifyContent="space-between"
-      >
-        <VStack w="full" flex={1} spacing={0}>
-          <CellOptions refetchingOutput={hardRefetching} refetchOutput={hardRefetch} />
+      <CellWrapper>
        <SyntaxHighlighter
          customStyle={{ overflowX: "unset", width: "100%", flex: 1 }}
          language="json"
@@ -127,21 +183,15 @@ export default function OutputCell({
        >
          {stringify(normalizedOutput.value, { maxLength: 40 })}
        </SyntaxHighlighter>
-        </VStack>
-        <OutputStats modelOutput={modelOutput} scenario={scenario} />
-      </VStack>
+      </CellWrapper>
    );
  }

  const contentToDisplay = (normalizedOutput?.type === "text" && normalizedOutput.value) || "";

  return (
-    <VStack w="100%" h="100%" justifyContent="space-between" whiteSpace="pre-wrap">
-      <VStack w="full" alignItems="flex-start" spacing={0}>
-        <CellOptions refetchingOutput={hardRefetching} refetchOutput={hardRefetch} />
+    <CellWrapper>
      <Text>{contentToDisplay}</Text>
-      </VStack>
-      {modelOutput && <OutputStats modelOutput={modelOutput} scenario={scenario} />}
-    </VStack>
+    </CellWrapper>
  );
 }
--- a/src/components/OutputsTable/OutputCell/OutputStats.tsx
+++ b/src/components/OutputsTable/OutputCell/OutputStats.tsx
@@ -7,28 +7,32 @@ import { CostTooltip } from "~/components/tooltip/CostTooltip";
 const SHOW_TIME = true;

 export const OutputStats = ({
-  modelOutput,
+  modelResponse,
 }: {
-  modelOutput: NonNullable<
-    NonNullable<RouterOutputs["scenarioVariantCells"]["get"]>["modelOutput"]
+  modelResponse: NonNullable<
+    NonNullable<RouterOutputs["scenarioVariantCells"]["get"]>["modelResponses"][0]
  >;
  scenario: Scenario;
 }) => {
-  const timeToComplete = modelOutput.timeToComplete;
+  const timeToComplete =
+    modelResponse.receivedAt && modelResponse.requestedAt
+      ? modelResponse.receivedAt.getTime() - modelResponse.requestedAt.getTime()
+      : 0;

-  const promptTokens = modelOutput.promptTokens;
-  const completionTokens = modelOutput.completionTokens;
+  const promptTokens = modelResponse.promptTokens;
+  const completionTokens = modelResponse.completionTokens;

  return (
    <HStack w="full" align="center" color="gray.500" fontSize="2xs" mt={{ base: 0, md: 1 }}>
      <HStack flex={1}>
-        {modelOutput.outputEvaluation.map((evaluation) => {
+        {modelResponse.outputEvaluations.map((evaluation) => {
          const passed = evaluation.result > 0.5;
          return (
            <Tooltip
              isDisabled={!evaluation.details}
              label={evaluation.details}
              key={evaluation.id}
+              shouldWrapChildren
            >
              <HStack spacing={0}>
                <Text>{evaluation.evaluation.label}</Text>
@@ -42,15 +46,15 @@ export const OutputStats = ({
          );
        })}
      </HStack>
-      {modelOutput.cost && (
+      {modelResponse.cost && (
        <CostTooltip
          promptTokens={promptTokens}
          completionTokens={completionTokens}
-          cost={modelOutput.cost}
+          cost={modelResponse.cost}
        >
          <HStack spacing={0}>
            <Icon as={BsCurrencyDollar} />
-            <Text mr={1}>{modelOutput.cost.toFixed(3)}</Text>
+            <Text mr={1}>{modelResponse.cost.toFixed(3)}</Text>
          </HStack>
        </CostTooltip>
      )}
--- a/src/components/OutputsTable/OutputCell/PromptModal.tsx
+++ b/src/components/OutputsTable/OutputCell/PromptModal.tsx
@@ -0,0 +1,36 @@
+import {
+  Modal,
+  ModalBody,
+  ModalCloseButton,
+  ModalContent,
+  ModalHeader,
+  ModalOverlay,
+  type UseDisclosureReturn,
+} from "@chakra-ui/react";
+import { type RouterOutputs } from "~/utils/api";
+import { JSONTree } from "react-json-tree";
+
+export default function ExpandedModal(props: {
+  cell: NonNullable<RouterOutputs["scenarioVariantCells"]["get"]>;
+  disclosure: UseDisclosureReturn;
+}) {
+  return (
+    <Modal isOpen={props.disclosure.isOpen} onClose={props.disclosure.onClose} size="2xl">
+      <ModalOverlay />
+      <ModalContent>
+        <ModalHeader>Prompt</ModalHeader>
+        <ModalCloseButton />
+        <ModalBody>
+          <JSONTree
+            data={props.cell.prompt}
+            invertTheme={true}
+            theme="chalk"
+            shouldExpandNodeInitially={() => true}
+            getItemString={() => ""}
+            hideRoot
+          />
+        </ModalBody>
+      </ModalContent>
+    </Modal>
+  );
+}
--- a/src/components/OutputsTable/OutputCell/ResponseLog.tsx
+++ b/src/components/OutputsTable/OutputCell/ResponseLog.tsx
@@ -0,0 +1,22 @@
+import { HStack, VStack, Text } from "@chakra-ui/react";
+import dayjs from "dayjs";
+
+export const ResponseLog = ({
+  time,
+  title,
+  message,
+}: {
+  time: Date;
+  title: string;
+  message?: string;
+}) => {
+  return (
+    <VStack spacing={0} alignItems="flex-start">
+      <HStack>
+        <Text>{dayjs(time).format("HH:mm:ss")}</Text>
+        <Text>{title}</Text>
+      </HStack>
+      {message && <Text pl={4}>{message}</Text>}
+    </VStack>
+  );
+};
--- a/src/components/OutputsTable/OutputCell/RetryCountdown.tsx
+++ b/src/components/OutputsTable/OutputCell/RetryCountdown.tsx
@@ -1,21 +1,12 @@
-import { type ScenarioVariantCell } from "@prisma/client";
-import { VStack, Text } from "@chakra-ui/react";
+import { Text } from "@chakra-ui/react";
 import { useEffect, useState } from "react";
 import pluralize from "pluralize";

-export const ErrorHandler = ({
-  cell,
-  refetchOutput,
-}: {
-  cell: ScenarioVariantCell;
-  refetchOutput: () => void;
-}) => {
+export const RetryCountdown = ({ retryTime }: { retryTime: Date }) => {
  const [msToWait, setMsToWait] = useState(0);

  useEffect(() => {
-    if (!cell.retryTime) return;
-
-    const initialWaitTime = cell.retryTime.getTime() - Date.now();
+    const initialWaitTime = retryTime.getTime() - Date.now();
    const msModuloOneSecond = initialWaitTime % 1000;
    let remainingTime = initialWaitTime - msModuloOneSecond;
    setMsToWait(remainingTime);
@@ -36,18 +27,13 @@ export const ErrorHandler = ({
      clearInterval(interval);
      clearTimeout(timeout);
    };
-  }, [cell.retryTime, cell.statusCode, setMsToWait, refetchOutput]);
+  }, [retryTime]);
+
+  if (msToWait <= 0) return null;

  return (
-    <VStack w="full">
-      <Text color="red.600" wordBreak="break-word">
-        {cell.errorMessage}
-      </Text>
-      {msToWait > 0 && (
    <Text color="red.600" fontSize="sm">
      Retrying in {pluralize("second", Math.ceil(msToWait / 1000), true)}...
    </Text>
-      )}
-    </VStack>
  );
 };
--- a/src/components/OutputsTable/OutputCell/TopActions.tsx
+++ b/src/components/OutputsTable/OutputCell/TopActions.tsx
@@ -0,0 +1,53 @@
+import { HStack, Icon, IconButton, Spinner, Tooltip, useDisclosure } from "@chakra-ui/react";
+import { BsArrowClockwise, BsInfoCircle } from "react-icons/bs";
+import { useExperimentAccess } from "~/utils/hooks";
+import ExpandedModal from "./PromptModal";
+import { type RouterOutputs } from "~/utils/api";
+
+export const CellOptions = ({
+  cell,
+  refetchingOutput,
+  refetchOutput,
+}: {
+  cell: RouterOutputs["scenarioVariantCells"]["get"];
+  refetchingOutput: boolean;
+  refetchOutput: () => void;
+}) => {
+  const { canModify } = useExperimentAccess();
+
+  const modalDisclosure = useDisclosure();
+
+  return (
+    <HStack justifyContent="flex-end" w="full">
+      {cell && (
+        <>
+          <Tooltip label="See Prompt">
+            <IconButton
+              aria-label="See Prompt"
+              icon={<Icon as={BsInfoCircle} boxSize={4} />}
+              onClick={modalDisclosure.onOpen}
+              size="xs"
+              colorScheme="gray"
+              color="gray.500"
+              variant="ghost"
+            />
+          </Tooltip>
+          <ExpandedModal cell={cell} disclosure={modalDisclosure} />
+        </>
+      )}
+      {canModify && (
+        <Tooltip label="Refetch output">
+          <IconButton
+            size="xs"
+            color="gray.500"
+            variant="ghost"
+            cursor="pointer"
+            onClick={refetchOutput}
+            aria-label="refetch output"
+            icon={<Icon as={refetchingOutput ? Spinner : BsArrowClockwise} boxSize={4} />}
+          />
+        </Tooltip>
+      )}
+    </HStack>
+  );
+};
--- a/src/components/OutputsTable/ScenarioEditor.tsx
+++ b/src/components/OutputsTable/ScenarioEditor.tsx
@@ -1,15 +1,24 @@
-import { type DragEvent } from "react";
-import { api } from "~/utils/api";
 import { isEqual } from "lodash-es";
-import { type Scenario } from "./types";
+import { useEffect, useState, type DragEvent } from "react";
+import { api } from "~/utils/api";
 import { useExperiment, useExperimentAccess, useHandledAsyncCallback } from "~/utils/hooks";
-import { useState } from "react";
+import { type Scenario } from "./types";

-import { Box, Button, Flex, HStack, Icon, Spinner, Stack, Tooltip, VStack } from "@chakra-ui/react";
+import {
+  Box,
+  Button,
+  HStack,
+  Icon,
+  IconButton,
+  Spinner,
+  Text,
+  Tooltip,
+  VStack,
+} from "@chakra-ui/react";
+import { BsArrowsAngleExpand, BsX } from "react-icons/bs";
 import { cellPadding } from "../constants";
-import { BsX } from "react-icons/bs";
-import { RiDraggable } from "react-icons/ri";
 import { FloatingLabelInput } from "./FloatingLabelInput";
+import { ScenarioEditorModal } from "./ScenarioEditorModal";

 export default function ScenarioEditor({
  scenario,
@@ -28,6 +37,10 @@ export default function ScenarioEditor({

  const [values, setValues] = useState<Record<string, string>>(savedValues);

+  useEffect(() => {
+    if (savedValues) setValues(savedValues);
+  }, [savedValues]);
+
  const experiment = useExperiment();
  const vars = api.templateVars.list.useQuery({ experimentId: experiment.data?.id ?? "" });

@@ -71,7 +84,10 @@ export default function ScenarioEditor({
    [reorderMutation, scenario.id],
  );

+  const [scenarioEditorModalOpen, setScenarioEditorModalOpen] = useState(false);
+
  return (
+    <>
      <HStack
        alignItems="flex-start"
        px={cellPadding.x}
@@ -96,58 +112,54 @@ export default function ScenarioEditor({
        onDrop={onReorder}
        backgroundColor={isDragTarget ? "gray.100" : "transparent"}
      >
-      {canModify && props.canHide && (
-        <Stack
-          alignSelf="flex-start"
-          opacity={props.hovered ? 1 : 0}
-          spacing={0}
-          ml={-cellPadding.x}
-        >
-          <Tooltip label="Hide scenario" hasArrow>
-            {/* for some reason the tooltip can't position itself properly relative to the icon without the wrapping box */}
-            <Button
-              variant="unstyled"
-              color="gray.400"
-              height="unset"
-              width="unset"
-              minW="unset"
-              onClick={onHide}
-              _hover={{
-                color: "gray.800",
-                cursor: "pointer",
-              }}
-            >
-              <Icon as={hidingInProgress ? Spinner : BsX} boxSize={hidingInProgress ? 4 : 6} />
-            </Button>
-          </Tooltip>
-          <Icon
-            as={RiDraggable}
-            boxSize={6}
-            color="gray.400"
-            _hover={{ color: "gray.800", cursor: "pointer" }}
-          />
-        </Stack>
-      )}
-
        {variableLabels.length === 0 ? (
-        <Box color="gray.500">{vars.data ? "No scenario variables configured" : "Loading..."}</Box>
+          <Box color="gray.500">
+            {vars.data ? "No scenario variables configured" : "Loading..."}
+          </Box>
        ) : (
          <VStack spacing={4} flex={1} py={2}>
+            <HStack justifyContent="space-between" w="100%" align="center" spacing={0}>
+              <Text flex={1}>Scenario</Text>
+              <Tooltip label="Expand" hasArrow>
+                <IconButton
+                  aria-label="Expand"
+                  icon={<Icon as={BsArrowsAngleExpand} boxSize={3} />}
+                  onClick={() => setScenarioEditorModalOpen(true)}
+                  size="xs"
+                  colorScheme="gray"
+                  color="gray.500"
+                  variant="ghost"
+                />
+              </Tooltip>
+              {canModify && props.canHide && (
+                <Tooltip label="Delete" hasArrow>
+                  <IconButton
+                    aria-label="Delete"
+                    icon={
+                      <Icon
+                        as={hidingInProgress ? Spinner : BsX}
+                        boxSize={hidingInProgress ? 4 : 6}
+                      />
+                    }
+                    onClick={onHide}
+                    size="xs"
+                    display="flex"
+                    colorScheme="gray"
+                    color="gray.500"
+                    variant="ghost"
+                  />
+                </Tooltip>
+              )}
+            </HStack>
            {variableLabels.map((key) => {
              const value = values[key] ?? "";
-            const layoutDirection = value.length > 20 ? "column" : "row";
              return (
-              <Flex
-                key={key}
-                direction={layoutDirection}
-                alignItems={layoutDirection === "column" ? "flex-start" : "center"}
-                flexWrap="wrap"
-                width="full"
-              >
                <FloatingLabelInput
+                  key={key}
                  label={key}
                  isDisabled={!canModify}
                  style={{ width: "100%" }}
+                  maxHeight={32}
                  value={value}
                  onChange={(e) => {
                    setValues((prev) => ({ ...prev, [key]: e.target.value }));
@@ -162,7 +174,6 @@ export default function ScenarioEditor({
                  onMouseEnter={() => setVariableInputHovered(true)}
                  onMouseLeave={() => setVariableInputHovered(false)}
                />
-              </Flex>
              );
            })}
            {hasChanged && (
@@ -184,5 +195,13 @@ export default function ScenarioEditor({
          </VStack>
        )}
      </HStack>
+      {scenarioEditorModalOpen && (
+        <ScenarioEditorModal
+          scenarioId={scenario.id}
+          initialValues={savedValues}
+          onClose={() => setScenarioEditorModalOpen(false)}
+        />
+      )}
+    </>
  );
 }
--- a/src/components/OutputsTable/ScenarioEditorModal.tsx
+++ b/src/components/OutputsTable/ScenarioEditorModal.tsx
@@ -0,0 +1,123 @@
+import {
+  Button,
+  HStack,
+  Modal,
+  ModalBody,
+  ModalCloseButton,
+  ModalContent,
+  ModalFooter,
+  ModalHeader,
+  ModalOverlay,
+  Spinner,
+  Text,
+  VStack,
+} from "@chakra-ui/react";
+import { useEffect, useState } from "react";
+import { isEqual } from "lodash-es";
+
+import { api } from "~/utils/api";
+import {
+  useScenario,
+  useHandledAsyncCallback,
+  useExperiment,
+  useExperimentAccess,
+} from "~/utils/hooks";
+import { FloatingLabelInput } from "./FloatingLabelInput";
+
+export const ScenarioEditorModal = ({
+  scenarioId,
+  initialValues,
+  onClose,
+}: {
+  scenarioId: string;
+  initialValues: Record<string, string>;
+  onClose: () => void;
+}) => {
+  const utils = api.useContext();
+  const experiment = useExperiment();
+  const { canModify } = useExperimentAccess();
+  const scenario = useScenario(scenarioId);
+
+  const savedValues = scenario.data?.variableValues as Record<string, string>;
+
+  const [values, setValues] = useState<Record<string, string>>(initialValues);
+
+  useEffect(() => {
+    if (savedValues) setValues(savedValues);
+  }, [savedValues]);
+
+  const hasChanged = !isEqual(savedValues, values);
+
+  const mutation = api.scenarios.replaceWithValues.useMutation();
+
+  const [onSave, saving] = useHandledAsyncCallback(async () => {
+    await mutation.mutateAsync({
+      id: scenarioId,
+      values,
+    });
+    await utils.scenarios.list.invalidate();
+  }, [mutation, values]);
+
+  const vars = api.templateVars.list.useQuery({ experimentId: experiment.data?.id ?? "" });
+  const variableLabels = vars.data?.map((v) => v.label) ?? [];
+
+  return (
+    <Modal
+      isOpen
+      onClose={onClose}
+      size={{ base: "xl", sm: "2xl", md: "3xl", lg: "5xl", xl: "7xl" }}
+    >
+      <ModalOverlay />
+      <ModalContent w={1200}>
+        <ModalHeader />
+        <ModalCloseButton />
+        <ModalBody maxW="unset">
+          <VStack spacing={8}>
+            {values &&
+              variableLabels.map((key) => {
+                const value = values[key] ?? "";
+                return (
+                  <FloatingLabelInput
+                    key={key}
+                    label={key}
+                    isDisabled={!canModify}
+                    _disabled={{ opacity: 1 }}
+                    style={{ width: "100%" }}
+                    value={value}
+                    onChange={(e) => {
+                      setValues((prev) => ({ ...prev, [key]: e.target.value }));
+                    }}
+                    onKeyDown={(e) => {
+                      if (e.key === "Enter" && (e.metaKey || e.ctrlKey)) {
+                        e.preventDefault();
+                        e.currentTarget.blur();
+                        onSave();
+                      }
+                    }}
+                  />
+                );
+              })}
+          </VStack>
+        </ModalBody>
+
+        <ModalFooter>
+          {canModify && (
+            <HStack>
+              <Button
+                colorScheme="gray"
+                onClick={() => setValues(savedValues)}
+                minW={24}
+                isDisabled={!hasChanged}
+              >
+                <Text>Reset</Text>
+              </Button>
+              <Button colorScheme="blue" onClick={onSave} minW={24} isDisabled={!hasChanged}>
+                {saving ? <Spinner boxSize={4} /> : <Text>Save</Text>}
+              </Button>
+            </HStack>
+          )}
+        </ModalFooter>
+      </ModalContent>
+    </Modal>
+  );
+};
--- a/src/components/OutputsTable/ScenarioRow.tsx
+++ b/src/components/OutputsTable/ScenarioRow.tsx
@@ -1,6 +1,5 @@
-import { Box, GridItem } from "@chakra-ui/react";
+import { GridItem } from "@chakra-ui/react";
 import React, { useState } from "react";
-import { cellPadding } from "../constants";
 import OutputCell from "./OutputCell/OutputCell";
 import ScenarioEditor from "./ScenarioEditor";
 import type { PromptVariant, Scenario } from "./types";
@@ -39,9 +38,7 @@ const ScenarioRow = (props: {
          colStart={i + 2}
          {...borders}
        >
-          <Box h="100%" w="100%" px={cellPadding.x} py={cellPadding.y}>
          <OutputCell key={variant.id} scenario={props.scenario} variant={variant} />
-          </Box>
        </GridItem>
      ))}
    </>
--- a/src/components/OutputsTable/ScenariosHeader.tsx
+++ b/src/components/OutputsTable/ScenariosHeader.tsx
@@ -54,16 +54,16 @@ export const ScenariosHeader = () => {
      </Text>
      {canModify && (
        <Menu>
-          <MenuButton mt={1}>
-            <IconButton
+          <MenuButton
+            as={IconButton}
+            mt={1}
            variant="ghost"
            aria-label="Edit Scenarios"
            icon={<Icon as={loading ? Spinner : BsGear} />}
          />
-          </MenuButton>
          <MenuList fontSize="md" zIndex="dropdown" mt={-3}>
            <MenuItem
-              icon={<Icon as={BsPlus} boxSize={6} mx={-1} />}
+              icon={<Icon as={BsPlus} boxSize={6} mx="-5px" />}
              onClick={() => onAddScenario(false)}
            >
              Add Scenario
--- a/src/components/OutputsTable/VariantEditor.tsx
+++ b/src/components/OutputsTable/VariantEditor.tsx
@@ -2,19 +2,24 @@ import {
  Box,
  Button,
  HStack,
+  IconButton,
  Spinner,
+  Text,
  Tooltip,
  useToast,
-  Text,
-  IconButton,
 } from "@chakra-ui/react";
-import { useRef, useEffect, useState, useCallback } from "react";
-import { useExperimentAccess, useHandledAsyncCallback, useModifierKeyLabel } from "~/utils/hooks";
-import { type PromptVariant } from "./types";
-import { api } from "~/utils/api";
-import { useAppStore } from "~/state/store";
+import { useCallback, useEffect, useRef, useState } from "react";
 import { FiMaximize, FiMinimize } from "react-icons/fi";
 import { editorBackground } from "~/state/sharedVariantEditor.slice";
+import { useAppStore } from "~/state/store";
+import { api } from "~/utils/api";
+import {
+  useExperimentAccess,
+  useHandledAsyncCallback,
+  useModifierKeyLabel,
+  useVisibleScenarioIds,
+} from "~/utils/hooks";
+import { type PromptVariant } from "./types";

 export default function VariantEditor(props: { variant: PromptVariant }) {
  const { canModify } = useExperimentAccess();
@@ -63,6 +68,7 @@ export default function VariantEditor(props: { variant: PromptVariant }) {
  const replaceVariant = api.promptVariants.replaceVariant.useMutation();
  const utils = api.useContext();
  const toast = useToast();
+  const visibleScenarios = useVisibleScenarioIds();

  const [onSave, saveInProgress] = useHandledAsyncCallback(async () => {
    if (!editorRef.current) return;
@@ -91,6 +97,7 @@ export default function VariantEditor(props: { variant: PromptVariant }) {
    const resp = await replaceVariant.mutateAsync({
      id: props.variant.id,
      constructFn: currentFn,
+      streamScenarios: visibleScenarios,
    });
    if (resp.status === "error") {
      return toast({
--- a/src/components/OutputsTable/VariantStats.tsx
+++ b/src/components/OutputsTable/VariantStats.tsx
@@ -21,17 +21,14 @@ export default function VariantStats(props: { variant: PromptVariant }) {
        completionTokens: 0,
        scenarioCount: 0,
        outputCount: 0,
-        awaitingRetrievals: false,
+        awaitingEvals: false,
      },
      refetchInterval,
    },
  );

  // Poll every two seconds while we are waiting for LLM retrievals to finish
-  useEffect(
-    () => setRefetchInterval(data.awaitingRetrievals ? 2000 : 0),
-    [data.awaitingRetrievals],
-  );
+  useEffect(() => setRefetchInterval(data.awaitingEvals ? 5000 : 0), [data.awaitingEvals]);

  const [passColor, neutralColor, failColor] = useToken("colors", [
    "green.500",
@@ -51,12 +48,12 @@ export default function VariantStats(props: { variant: PromptVariant }) {
      fontSize="xs"
      py={cellPadding.y}
    >
+      <HStack px={cellPadding.x}>
        {showNumFinished && (
          <Text>
            {data.outputCount} / {data.scenarioCount}
          </Text>
        )}
-      <HStack px={cellPadding.x}>
        {data.evalResults.map((result) => {
          const passedFrac = result.passCount / result.totalCount;
          return (
@@ -69,7 +66,7 @@ export default function VariantStats(props: { variant: PromptVariant }) {
          );
        })}
      </HStack>
-      {data.overallCost && !data.awaitingRetrievals && (
+      {data.overallCost && (
        <CostTooltip
          promptTokens={data.promptTokens}
          completionTokens={data.completionTokens}
--- a/src/components/OutputsTable/index.tsx
+++ b/src/components/OutputsTable/index.tsx
@@ -33,9 +33,9 @@ export default function OutputsTable({ experimentId }: { experimentId: string |
    <Grid
      pt={4}
      pb={24}
-      pl={4}
+      pl={8}
      display="grid"
-      gridTemplateColumns={`250px repeat(${variants.data.length}, minmax(300px, 1fr)) auto`}
+      gridTemplateColumns={`250px repeat(${variants.data.length}, minmax(360px, 1fr)) auto`}
      sx={{
        "> *": {
          borderColor: "gray.300",
@@ -53,6 +53,7 @@ export default function OutputsTable({ experimentId }: { experimentId: string |
          colStart: i + 2,
          borderLeftWidth: i === 0 ? 1 : 0,
          marginLeft: i === 0 ? "-1px" : 0,
+          backgroundColor: "gray.100",
        };
        return (
          <Fragment key={variant.uiId}>
--- a/src/components/OutputsTable/styles.ts
+++ b/src/components/OutputsTable/styles.ts
@@ -1,11 +1,4 @@
-import { type GridItemProps, type SystemStyleObject } from "@chakra-ui/react";
-
-export const stickyHeaderStyle: SystemStyleObject = {
-  position: "sticky",
-  top: "0",
-  backgroundColor: "#fff",
-  zIndex: 10,
-};
+import { type GridItemProps } from "@chakra-ui/react";

 export const borders: GridItemProps = {
  borderRightWidth: 1,
--- a/src/components/RefinePromptModal/RefinePromptModal.tsx
+++ b/src/components/RefinePromptModal/RefinePromptModal.tsx
@@ -16,7 +16,7 @@ import {
 } from "@chakra-ui/react";
 import { BsStars } from "react-icons/bs";
 import { api } from "~/utils/api";
-import { useHandledAsyncCallback } from "~/utils/hooks";
+import { useHandledAsyncCallback, useVisibleScenarioIds } from "~/utils/hooks";
 import { type PromptVariant } from "@prisma/client";
 import { useState } from "react";
 import CompareFunctions from "./CompareFunctions";
@@ -34,6 +34,7 @@ export const RefinePromptModal = ({
  onClose: () => void;
 }) => {
  const utils = api.useContext();
+  const visibleScenarios = useVisibleScenarioIds();

  const refinementActions =
    frontendModelProviders[variant.modelProvider as SupportedProvider].refinementActions || {};
@@ -73,6 +74,7 @@ export const RefinePromptModal = ({
    await replaceVariantMutation.mutateAsync({
      id: variant.id,
      constructFn: refinedPromptFn,
+      streamScenarios: visibleScenarios,
    });
    await utils.promptVariants.list.invalidate();
    onClose();
@@ -95,7 +97,7 @@ export const RefinePromptModal = ({
        <ModalCloseButton />
        <ModalBody maxW="unset">
          <VStack spacing={8}>
-            <VStack spacing={4}>
+            <VStack spacing={4} w="full">
              {Object.keys(refinementActions).length && (
                <>
                  <SimpleGrid columns={{ base: 1, md: 2 }} spacing={8}>
--- a/src/components/VariantHeader/VariantHeader.tsx
+++ b/src/components/VariantHeader/VariantHeader.tsx
@@ -6,7 +6,6 @@ import { useExperimentAccess, useHandledAsyncCallback } from "~/utils/hooks";
 import { HStack, Icon, Text, GridItem, type GridItemProps } from "@chakra-ui/react"; // Changed here
 import { cellPadding, headerMinHeight } from "../constants";
 import AutoResizeTextArea from "../AutoResizeTextArea";
-import { stickyHeaderStyle } from "../OutputsTable/styles";
 import VariantHeaderMenuButton from "./VariantHeaderMenuButton";

 export default function VariantHeader(
@@ -53,7 +52,17 @@ export default function VariantHeader(

  if (!canModify) {
    return (
-      <GridItem padding={0} sx={stickyHeaderStyle} borderTopWidth={1} {...gridItemProps}>
+      <GridItem
+        padding={0}
+        sx={{
+          position: "sticky",
+          top: "0",
+          // Ensure that the menu always appears above the sticky header of other variants
+          zIndex: menuOpen ? "dropdown" : 10,
+        }}
+        borderTopWidth={1}
+        {...gridItemProps}
+      >
        <Text fontSize={16} fontWeight="bold" px={cellPadding.x} py={cellPadding.y}>
          {variant.label}
        </Text>
@@ -65,15 +74,16 @@ export default function VariantHeader(
    <GridItem
      padding={0}
      sx={{
-        ...stickyHeaderStyle,
+        position: "sticky",
+        top: "0",
        // Ensure that the menu always appears above the sticky header of other variants
-        zIndex: menuOpen ? "dropdown" : stickyHeaderStyle.zIndex,
+        zIndex: menuOpen ? "dropdown" : 10,
      }}
      borderTopWidth={1}
      {...gridItemProps}
    >
      <HStack
-        spacing={4}
+        spacing={2}
        alignItems="flex-start"
        minH={headerMinHeight}
        draggable={!isInputHovered}
@@ -92,7 +102,8 @@ export default function VariantHeader(
          setIsDragTarget(false);
        }}
        onDrop={onReorder}
-        backgroundColor={isDragTarget ? "gray.100" : "transparent"}
+        backgroundColor={isDragTarget ? "gray.200" : "gray.100"}
+        h="full"
      >
        <Icon
          as={RiDraggable}
--- a/src/components/VariantHeader/VariantHeaderMenuButton.tsx
+++ b/src/components/VariantHeader/VariantHeaderMenuButton.tsx
@@ -1,8 +1,7 @@
 import { type PromptVariant } from "../OutputsTable/types";
 import { api } from "~/utils/api";
-import { useHandledAsyncCallback } from "~/utils/hooks";
+import { useHandledAsyncCallback, useVisibleScenarioIds } from "~/utils/hooks";
 import {
-  Button,
  Icon,
  Menu,
  MenuButton,
@@ -11,6 +10,7 @@ import {
  MenuDivider,
  Text,
  Spinner,
+  IconButton,
 } from "@chakra-ui/react";
 import { BsFillTrashFill, BsGear, BsStars } from "react-icons/bs";
 import { FaRegClone } from "react-icons/fa";
@@ -33,11 +33,13 @@ export default function VariantHeaderMenuButton({
  const utils = api.useContext();

  const duplicateMutation = api.promptVariants.create.useMutation();
+  const visibleScenarios = useVisibleScenarioIds();

  const [duplicateVariant, duplicationInProgress] = useHandledAsyncCallback(async () => {
    await duplicateMutation.mutateAsync({
      experimentId: variant.experimentId,
      variantId: variant.id,
+      streamScenarios: visibleScenarios,
    });
    await utils.promptVariants.list.invalidate();
  }, [duplicateMutation, variant.experimentId, variant.id]);
@@ -56,15 +58,12 @@ export default function VariantHeaderMenuButton({
  return (
    <>
      <Menu isOpen={menuOpen} onOpen={() => setMenuOpen(true)} onClose={() => setMenuOpen(false)}>
-        {duplicationInProgress ? (
-          <Spinner boxSize={4} mx={3} my={3} />
-        ) : (
-          <MenuButton>
-            <Button variant="ghost">
-              <Icon as={BsGear} />
-            </Button>
-          </MenuButton>
-        )}
+        <MenuButton
+          as={IconButton}
+          variant="ghost"
+          aria-label="Edit Scenarios"
+          icon={<Icon as={duplicationInProgress ? Spinner : BsGear} />}
+        />

        <MenuList mt={-3} fontSize="md">
          <MenuItem icon={<Icon as={FaRegClone} boxSize={4} w={5} />} onClick={duplicateVariant}>
--- a/src/components/experiments/ExperimentCard.tsx
+++ b/src/components/experiments/ExperimentCard.tsx
@@ -1,4 +1,13 @@
-import { HStack, Icon, VStack, Text, Divider, Spinner, AspectRatio } from "@chakra-ui/react";
+import {
+  HStack,
+  Icon,
+  VStack,
+  Text,
+  Divider,
+  Spinner,
+  AspectRatio,
+  SkeletonText,
+} from "@chakra-ui/react";
 import { RiFlaskLine } from "react-icons/ri";
 import { formatTimePast } from "~/utils/dayjs";
 import Link from "next/link";
@@ -93,3 +102,13 @@ export const NewExperimentCard = () => {
    </AspectRatio>
  );
 };
+
+export const ExperimentCardSkeleton = () => (
+  <AspectRatio ratio={1.2} w="full">
+    <VStack align="center" borderColor="gray.200" borderWidth={1} p={4} bg="gray.50">
+      <SkeletonText noOfLines={1} w="80%" />
+      <SkeletonText noOfLines={2} w="60%" />
+      <SkeletonText noOfLines={1} w="80%" />
+    </VStack>
+  </AspectRatio>
+);
--- a/src/components/experiments/HeaderButtons/DeleteDialog.tsx
+++ b/src/components/experiments/HeaderButtons/DeleteDialog.tsx
@@ -0,0 +1,57 @@
+import {
+  Button,
+  AlertDialog,
+  AlertDialogBody,
+  AlertDialogFooter,
+  AlertDialogHeader,
+  AlertDialogContent,
+  AlertDialogOverlay,
+} from "@chakra-ui/react";
+
+import { useRouter } from "next/router";
+import { useRef } from "react";
+import { api } from "~/utils/api";
+import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
+
+export const DeleteDialog = ({ onClose }: { onClose: () => void }) => {
+  const experiment = useExperiment();
+  const deleteMutation = api.experiments.delete.useMutation();
+  const utils = api.useContext();
+  const router = useRouter();
+
+  const cancelRef = useRef<HTMLButtonElement>(null);
+
+  const [onDeleteConfirm] = useHandledAsyncCallback(async () => {
+    if (!experiment.data?.id) return;
+    await deleteMutation.mutateAsync({ id: experiment.data.id });
+    await utils.experiments.list.invalidate();
+    await router.push({ pathname: "/experiments" });
+    onClose();
+  }, [deleteMutation, experiment.data?.id, router]);
+
+  return (
+    <AlertDialog isOpen leastDestructiveRef={cancelRef} onClose={onClose}>
+      <AlertDialogOverlay>
+        <AlertDialogContent>
+          <AlertDialogHeader fontSize="lg" fontWeight="bold">
+            Delete Experiment
+          </AlertDialogHeader>
+
+          <AlertDialogBody>
+            If you delete this experiment all the associated prompts and scenarios will be deleted
+            as well. Are you sure?
+          </AlertDialogBody>
+
+          <AlertDialogFooter>
+            <Button ref={cancelRef} onClick={onClose}>
+              Cancel
+            </Button>
+            <Button colorScheme="red" onClick={onDeleteConfirm} ml={3}>
+              Delete
+            </Button>
+          </AlertDialogFooter>
+        </AlertDialogContent>
+      </AlertDialogOverlay>
+    </AlertDialog>
+  );
+};
--- a/src/components/experiments/HeaderButtons/HeaderButtons.tsx
+++ b/src/components/experiments/HeaderButtons/HeaderButtons.tsx
@@ -0,0 +1,42 @@
+import { Button, HStack, Icon, Spinner, Text } from "@chakra-ui/react";
+import { useOnForkButtonPressed } from "./useOnForkButtonPressed";
+import { useExperiment } from "~/utils/hooks";
+import { BsGearFill } from "react-icons/bs";
+import { TbGitFork } from "react-icons/tb";
+import { useAppStore } from "~/state/store";
+
+export const HeaderButtons = () => {
+  const experiment = useExperiment();
+
+  const canModify = experiment.data?.access.canModify ?? false;
+
+  const { onForkButtonPressed, isForking } = useOnForkButtonPressed();
+
+  const openDrawer = useAppStore((s) => s.openDrawer);
+
+  if (experiment.isLoading) return null;
+
+  return (
+    <HStack spacing={0} mt={{ base: 2, md: 0 }}>
+      <Button
+        onClick={onForkButtonPressed}
+        mr={4}
+        colorScheme={canModify ? undefined : "orange"}
+        bgColor={canModify ? undefined : "orange.400"}
+        minW={0}
+        variant={{ base: "solid", md: canModify ? "ghost" : "solid" }}
+      >
+        {isForking ? <Spinner boxSize={5} /> : <Icon as={TbGitFork} boxSize={5} />}
+        <Text ml={2}>Fork</Text>
+      </Button>
+      {canModify && (
+        <Button variant={{ base: "solid", md: "ghost" }} onClick={openDrawer}>
+          <HStack>
+            <Icon as={BsGearFill} />
+            <Text>Settings</Text>
+          </HStack>
+        </Button>
+      )}
+    </HStack>
+  );
+};
--- a/src/components/experiments/HeaderButtons/useOnForkButtonPressed.tsx
+++ b/src/components/experiments/HeaderButtons/useOnForkButtonPressed.tsx
@@ -0,0 +1,30 @@
+import { useCallback } from "react";
+import { api } from "~/utils/api";
+import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
+import { signIn, useSession } from "next-auth/react";
+import { useRouter } from "next/router";
+
+export const useOnForkButtonPressed = () => {
+  const router = useRouter();
+
+  const user = useSession().data;
+  const experiment = useExperiment();
+
+  const forkMutation = api.experiments.fork.useMutation();
+
+  const [onFork, isForking] = useHandledAsyncCallback(async () => {
+    if (!experiment.data?.id) return;
+    const forkedExperimentId = await forkMutation.mutateAsync({ id: experiment.data.id });
+    await router.push({ pathname: "/experiments/[id]", query: { id: forkedExperimentId } });
+  }, [forkMutation, experiment.data?.id, router]);
+
+  const onForkButtonPressed = useCallback(() => {
+    if (user === null) {
+      signIn("github").catch(console.error);
+    } else {
+      onFork();
+    }
+  }, [onFork, user]);
+
+  return { onForkButtonPressed, isForking };
+};
--- a/src/components/nav/AppShell.tsx
+++ b/src/components/nav/AppShell.tsx
@@ -84,7 +84,11 @@ const NavSidebar = () => {
          />
        )}
      </VStack>
-      {user ? <UserMenu user={user} /> : <Divider />}
+      {user ? (
+        <UserMenu user={user} borderColor={"gray.200"} borderTopWidth={1} borderBottomWidth={1} />
+      ) : (
+        <Divider />
+      )}
      <VStack spacing={0} align="center">
        <Link
          href="https://github.com/openpipe/openpipe"
--- a/src/components/nav/UserMenu.tsx
+++ b/src/components/nav/UserMenu.tsx
@@ -8,12 +8,16 @@ import {
  PopoverTrigger,
  PopoverContent,
  Link,
+  useColorMode,
+  type StackProps,
 } from "@chakra-ui/react";
 import { type Session } from "next-auth";
 import { signOut } from "next-auth/react";
 import { BsBoxArrowRight, BsChevronRight, BsPersonCircle } from "react-icons/bs";

-export default function UserMenu({ user }: { user: Session }) {
+export default function UserMenu({ user, ...rest }: { user: Session } & StackProps) {
+  const { colorMode } = useColorMode();
+
  const profileImage = user.user.image ? (
    <Image src={user.user.image} alt="profile picture" boxSize={8} borderRadius="50%" />
  ) : (
@@ -29,12 +33,10 @@ export default function UserMenu({ user }: { user: Session }) {
            px={3}
            spacing={3}
            py={2}
-            borderColor={"gray.200"}
-            borderTopWidth={1}
-            borderBottomWidth={1}
+            {...rest}
            cursor="pointer"
            _hover={{
-              bgColor: "gray.200",
+              bgColor: colorMode === "light" ? "gray.200" : "gray.700",
            }}
          >
            {profileImage}
--- a/src/env.mjs
+++ b/src/env.mjs
@@ -18,6 +18,7 @@ export const env = createEnv({
    GITHUB_CLIENT_SECRET: z.string().min(1),
    OPENAI_API_KEY: z.string().min(1),
    REPLICATE_API_TOKEN: z.string().default("placeholder"),
+    ANTHROPIC_API_KEY: z.string().default("placeholder"),
  },

  /**
@@ -44,6 +45,7 @@ export const env = createEnv({
    GITHUB_CLIENT_ID: process.env.GITHUB_CLIENT_ID,
    GITHUB_CLIENT_SECRET: process.env.GITHUB_CLIENT_SECRET,
    REPLICATE_API_TOKEN: process.env.REPLICATE_API_TOKEN,
+    ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY,
  },
  /**
   * Run `build` or `dev` with `SKIP_ENV_VALIDATION` to skip env validation.
--- a/src/modelProviders/anthropic/codegen/codegen.ts
+++ b/src/modelProviders/anthropic/codegen/codegen.ts
@@ -0,0 +1,69 @@
+/* eslint-disable @typescript-eslint/no-var-requires */
+
+import YAML from "yaml";
+import fs from "fs";
+import path from "path";
+import { openapiSchemaToJsonSchema } from "@openapi-contrib/openapi-schema-to-json-schema";
+import $RefParser from "@apidevtools/json-schema-ref-parser";
+import { type JSONObject } from "superjson/dist/types";
+import assert from "assert";
+import { type JSONSchema4Object } from "json-schema";
+import { isObject } from "lodash-es";
+
+// @ts-expect-error for some reason missing from types
+import parserEstree from "prettier/plugins/estree";
+import parserBabel from "prettier/plugins/babel";
+import prettier from "prettier/standalone";
+
+const OPENAPI_URL =
+  "https://raw.githubusercontent.com/tryAGI/Anthropic/1c0871e861de60a4c3a843cb90e17d63e86c234a/docs/openapi.yaml";
+
+// Fetch the openapi document
+const response = await fetch(OPENAPI_URL);
+const openApiYaml = await response.text();
+
+// Parse the yaml document
+let schema = YAML.parse(openApiYaml) as JSONObject;
+schema = openapiSchemaToJsonSchema(schema);
+
+const jsonSchema = await $RefParser.dereference(schema);
+
+assert("components" in jsonSchema);
+const completionRequestSchema = jsonSchema.components.schemas
+  .CreateCompletionRequest as JSONSchema4Object;
+
+// We need to do a bit of surgery here since the Monaco editor doesn't like
+// the fact that the schema says `model` can be either a string or an enum,
+// and displays a warning in the editor. Let's stick with just an enum for
+// now and drop the string option.
+assert(
+  "properties" in completionRequestSchema &&
+    isObject(completionRequestSchema.properties) &&
+    "model" in completionRequestSchema.properties &&
+    isObject(completionRequestSchema.properties.model),
+);
+
+const modelProperty = completionRequestSchema.properties.model;
+assert(
+  "oneOf" in modelProperty &&
+    Array.isArray(modelProperty.oneOf) &&
+    modelProperty.oneOf.length === 2 &&
+    isObject(modelProperty.oneOf[1]) &&
+    "enum" in modelProperty.oneOf[1],
+  "Expected model to have oneOf length of 2",
+);
+modelProperty.type = "string";
+modelProperty.enum = modelProperty.oneOf[1].enum;
+delete modelProperty["oneOf"];
+
+// Get the directory of the current script
+const currentDirectory = path.dirname(import.meta.url).replace("file://", "");
+
+// Write the JSON schema to a file in the current directory
+fs.writeFileSync(
+  path.join(currentDirectory, "input.schema.json"),
+  await prettier.format(JSON.stringify(completionRequestSchema, null, 2), {
+    parser: "json",
+    plugins: [parserBabel, parserEstree],
+  }),
+);
--- a/src/modelProviders/anthropic/codegen/input.schema.json
+++ b/src/modelProviders/anthropic/codegen/input.schema.json
@@ -0,0 +1,63 @@
+{
+  "type": "object",
+  "properties": {
+    "model": {
+      "description": "The model that will complete your prompt.",
+      "x-oaiTypeLabel": "string",
+      "type": "string",
+      "enum": [
+        "claude-2",
+        "claude-2.0",
+        "claude-instant-1",
+        "claude-instant-1.1"
+      ]
+    },
+    "prompt": {
+      "description": "The prompt that you want Claude to complete.\n\nFor proper response generation you will need to format your prompt as follows:\n\"\\n\\nHuman: all instructions for the assistant\\n\\nAssistant:\". The prompt string should begin with the characters \"Human:\" and end with \"Assistant:\".",
+      "default": "<|endoftext|>",
+      "example": "\\n\\nHuman: What is the correct translation of ${scenario.input}? I would like a long analysis followed by a short answer.\\n\\nAssistant:",
+      "type": "string"
+    },
+    "max_tokens_to_sample": {
+      "type": "integer",
+      "minimum": 1,
+      "default": 256,
+      "nullable": true,
+      "description": "The maximum number of tokens to generate before stopping."
+    },
+    "temperature": {
+      "type": "number",
+      "minimum": 0,
+      "maximum": 1,
+      "nullable": true,
+      "description": "Amount of randomness injected into the response.\n\nDefaults to 1."
+    },
+    "top_p": {
+      "type": "number",
+      "minimum": 0,
+      "maximum": 1,
+      "nullable": true,
+      "description": "Use nucleus sampling.\n\nYou should either alter temperature or top_p, but not both.\n"
+    },
+    "top_k": {
+      "type": "number",
+      "minimum": 0,
+      "default": 5,
+      "nullable": true,
+      "description": "Only sample from the top K options for each subsequent token."
+    },
+    "stream": {
+      "description": "Whether to incrementally stream the response using server-sent events.",
+      "type": "boolean",
+      "nullable": true,
+      "default": false
+    },
+    "stop_sequences": {
+      "description": "Sequences that will cause the model to stop generating completion text.\nBy default, our models stop on \"\\n\\nHuman:\".",
+      "default": null,
+      "nullable": true,
+      "type": "array"
+    }
+  },
+  "required": ["model", "prompt", "max_tokens_to_sample"]
+}
--- a/src/modelProviders/anthropic/frontend.ts
+++ b/src/modelProviders/anthropic/frontend.ts
@@ -0,0 +1,42 @@
+import { type Completion } from "@anthropic-ai/sdk/resources";
+import { type SupportedModel } from ".";
+import { type FrontendModelProvider } from "../types";
+import { refinementActions } from "./refinementActions";
+
+const frontendModelProvider: FrontendModelProvider<SupportedModel, Completion> = {
+  name: "Replicate Llama2",
+
+  models: {
+    "claude-2.0": {
+      name: "Claude 2.0",
+      contextWindow: 100000,
+      promptTokenPrice: 11.02 / 1000000,
+      completionTokenPrice: 32.68 / 1000000,
+      speed: "medium",
+      provider: "anthropic",
+      learnMoreUrl: "https://www.anthropic.com/product",
+      apiDocsUrl: "https://docs.anthropic.com/claude/reference/complete_post",
+    },
+    "claude-instant-1.1": {
+      name: "Claude Instant 1.1",
+      contextWindow: 100000,
+      promptTokenPrice: 1.63 / 1000000,
+      completionTokenPrice: 5.51 / 1000000,
+      speed: "fast",
+      provider: "anthropic",
+      learnMoreUrl: "https://www.anthropic.com/product",
+      apiDocsUrl: "https://docs.anthropic.com/claude/reference/complete_post",
+    },
+  },
+
+  refinementActions,
+
+  normalizeOutput: (output) => {
+    return {
+      type: "text",
+      value: output.completion,
+    };
+  },
+};
+
+export default frontendModelProvider;
--- a/src/modelProviders/anthropic/getCompletion.ts
+++ b/src/modelProviders/anthropic/getCompletion.ts
@@ -0,0 +1,86 @@
+import { env } from "~/env.mjs";
+import { type CompletionResponse } from "../types";
+
+import Anthropic, { APIError } from "@anthropic-ai/sdk";
+import { type Completion, type CompletionCreateParams } from "@anthropic-ai/sdk/resources";
+import { isObject, isString } from "lodash-es";
+
+const anthropic = new Anthropic({
+  apiKey: env.ANTHROPIC_API_KEY,
+});
+
+export async function getCompletion(
+  input: CompletionCreateParams,
+  onStream: ((partialOutput: Completion) => void) | null,
+): Promise<CompletionResponse<Completion>> {
+  const start = Date.now();
+  let finalCompletion: Completion | null = null;
+
+  try {
+    if (onStream) {
+      const resp = await anthropic.completions.create(
+        { ...input, stream: true },
+        {
+          maxRetries: 0,
+        },
+      );
+
+      for await (const part of resp) {
+        if (finalCompletion === null) {
+          finalCompletion = part;
+        } else {
+          finalCompletion = { ...part, completion: finalCompletion.completion + part.completion };
+        }
+        onStream(finalCompletion);
+      }
+      if (!finalCompletion) {
+        return {
+          type: "error",
+          message: "Streaming failed to return a completion",
+          autoRetry: false,
+        };
+      }
+    } else {
+      const resp = await anthropic.completions.create(
+        { ...input, stream: false },
+        {
+          maxRetries: 0,
+        },
+      );
+      finalCompletion = resp;
+    }
+    const timeToComplete = Date.now() - start;
+
+    return {
+      type: "success",
+      statusCode: 200,
+      value: finalCompletion,
+      timeToComplete,
+    };
+  } catch (error: unknown) {
+    console.log("CAUGHT ERROR", error);
+    if (error instanceof APIError) {
+      const message =
+        isObject(error.error) &&
+        "error" in error.error &&
+        isObject(error.error.error) &&
+        "message" in error.error.error &&
+        isString(error.error.error.message)
+          ? error.error.error.message
+          : error.message;
+
+      return {
+        type: "error",
+        message,
+        autoRetry: error.status === 429 || error.status === 503,
+        statusCode: error.status,
+      };
+    } else {
+      return {
+        type: "error",
+        message: (error as Error).message,
+        autoRetry: true,
+      };
+    }
+  }
+}
--- a/src/modelProviders/anthropic/index.ts
+++ b/src/modelProviders/anthropic/index.ts
@@ -0,0 +1,34 @@
+import { type JSONSchema4 } from "json-schema";
+import { type ModelProvider } from "../types";
+import inputSchema from "./codegen/input.schema.json";
+import { getCompletion } from "./getCompletion";
+import frontendModelProvider from "./frontend";
+import type { Completion, CompletionCreateParams } from "@anthropic-ai/sdk/resources";
+
+const supportedModels = ["claude-2.0", "claude-instant-1.1"] as const;
+
+export type SupportedModel = (typeof supportedModels)[number];
+
+export type AnthropicProvider = ModelProvider<SupportedModel, CompletionCreateParams, Completion>;
+
+const modelProvider: AnthropicProvider = {
+  getModel: (input) => {
+    if (supportedModels.includes(input.model as SupportedModel))
+      return input.model as SupportedModel;
+
+    const modelMaps: Record<string, SupportedModel> = {
+      "claude-2": "claude-2.0",
+      "claude-instant-1": "claude-instant-1.1",
+    };
+
+    if (input.model in modelMaps) return modelMaps[input.model] as SupportedModel;
+
+    return null;
+  },
+  inputSchema: inputSchema as JSONSchema4,
+  canStream: true,
+  getCompletion,
+  ...frontendModelProvider,
+};
+
+export default modelProvider;
--- a/src/modelProviders/anthropic/refinementActions.ts
+++ b/src/modelProviders/anthropic/refinementActions.ts
@@ -0,0 +1,3 @@
+import { type RefinementAction } from "../types";
+
+export const refinementActions: Record<string, RefinementAction> = {};
--- a/src/modelProviders/frontendModelProviders.ts
+++ b/src/modelProviders/frontendModelProviders.ts
@@ -1,15 +1,15 @@
 import openaiChatCompletionFrontend from "./openai-ChatCompletion/frontend";
 import replicateLlama2Frontend from "./replicate-llama2/frontend";
+import anthropicFrontend from "./anthropic/frontend";
 import { type SupportedProvider, type FrontendModelProvider } from "./types";

-// TODO: make sure we get a typescript error if you forget to add a provider here
-
 // Keep attributes here that need to be accessible from the frontend. We can't
 // just include them in the default `modelProviders` object because it has some
 // transient dependencies that can only be imported on the server.
 const frontendModelProviders: Record<SupportedProvider, FrontendModelProvider<any, any>> = {
  "openai/ChatCompletion": openaiChatCompletionFrontend,
  "replicate/llama2": replicateLlama2Frontend,
+  anthropic: anthropicFrontend,
 };

 export default frontendModelProviders;
--- a/src/modelProviders/modelProviders.ts
+++ b/src/modelProviders/modelProviders.ts
@@ -1,10 +1,12 @@
 import openaiChatCompletion from "./openai-ChatCompletion";
 import replicateLlama2 from "./replicate-llama2";
+import anthropic from "./anthropic";
 import { type SupportedProvider, type ModelProvider } from "./types";

 const modelProviders: Record<SupportedProvider, ModelProvider<any, any, any>> = {
  "openai/ChatCompletion": openaiChatCompletion,
  "replicate/llama2": replicateLlama2,
+  anthropic,
 };

 export default modelProviders;
--- a/src/modelProviders/openai-ChatCompletion/getCompletion.ts
+++ b/src/modelProviders/openai-ChatCompletion/getCompletion.ts
@@ -6,7 +6,7 @@ import {
 } from "openai/resources/chat";
 import { countOpenAIChatTokens } from "~/utils/countTokens";
 import { type CompletionResponse } from "../types";
-import { omit } from "lodash-es";
+import { isArray, isString, omit } from "lodash-es";
 import { openai } from "~/server/utils/openai";
 import { truthyFilter } from "~/utils/utils";
 import { APIError } from "openai";
@@ -40,6 +40,8 @@ const mergeStreamedChunks = (
          ((choice.delta.function_call.arguments as string) ?? "");
      }
    } else {
+      // @ts-expect-error the types are correctly telling us that finish_reason
+      // could be null, but don't want to fix it right now.
      choices.push({ ...omit(choice, "delta"), message: { role: "assistant", ...choice.delta } });
    }
  }
@@ -64,6 +66,7 @@ export async function getCompletion(

  try {
    if (onStream) {
+      console.log("got started");
      const resp = await openai.chat.completions.create(
        { ...input, stream: true },
        {
@@ -71,9 +74,11 @@ export async function getCompletion(
        },
      );
      for await (const part of resp) {
+        console.log("got part", part);
        finalCompletion = mergeStreamedChunks(finalCompletion, part);
        onStream(finalCompletion);
      }
+      console.log("got final", finalCompletion);
      if (!finalCompletion) {
        return {
          type: "error",
@@ -120,11 +125,18 @@ export async function getCompletion(
      cost,
    };
  } catch (error: unknown) {
-    console.error("ERROR IS", error);
    if (error instanceof APIError) {
+      // The types from the sdk are wrong
+      const rawMessage = error.message as string | string[];
+      // If the message is not a string, stringify it
+      const message = isString(rawMessage)
+        ? rawMessage
+        : isArray(rawMessage)
+        ? rawMessage.map((m) => m.toString()).join("\n")
+        : (rawMessage as any).toString();
      return {
        type: "error",
-        message: error.message,
+        message,
        autoRetry: error.status === 429 || error.status === 503,
        statusCode: error.status,
      };
--- a/src/modelProviders/openai-ChatCompletion/index.ts
+++ b/src/modelProviders/openai-ChatCompletion/index.ts
@@ -37,7 +37,7 @@ const modelProvider: OpenaiChatModelProvider = {
    return null;
  },
  inputSchema: inputSchema as JSONSchema4,
-  shouldStream: (input) => input.stream ?? false,
+  canStream: true,
  getCompletion,
  ...frontendModelProvider,
 };
--- a/src/modelProviders/replicate-llama2/getCompletion.ts
+++ b/src/modelProviders/replicate-llama2/getCompletion.ts
@@ -8,9 +8,9 @@ const replicate = new Replicate({
 });

 const modelIds: Record<ReplicateLlama2Input["model"], string> = {
-  "7b-chat": "3725a659b5afff1a0ba9bead5fac3899d998feaad00e07032ca2b0e35eb14f8a",
-  "13b-chat": "5c785d117c5bcdd1928d5a9acb1ffa6272d6cf13fcb722e90886a0196633f9d3",
-  "70b-chat": "e951f18578850b652510200860fc4ea62b3b16fac280f83ff32282f87bbd2e48",
+  "7b-chat": "058333670f2a6e88cf1b29b8183405b17bb997767282f790b82137df8c090c1f",
+  "13b-chat": "d5da4236b006f967ceb7da037be9cfc3924b20d21fed88e1e94f19d56e2d3111",
+  "70b-chat": "2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1",
 };

 export async function getCompletion(
@@ -19,7 +19,7 @@ export async function getCompletion(
 ): Promise<CompletionResponse<ReplicateLlama2Output>> {
  const start = Date.now();

-  const { model, stream, ...rest } = input;
+  const { model, ...rest } = input;

  try {
    const prediction = await replicate.predictions.create({
--- a/src/modelProviders/replicate-llama2/index.ts
+++ b/src/modelProviders/replicate-llama2/index.ts
@@ -9,7 +9,6 @@ export type SupportedModel = (typeof supportedModels)[number];
 export type ReplicateLlama2Input = {
  model: SupportedModel;
  prompt: string;
-  stream?: boolean;
  max_length?: number;
  temperature?: number;
  top_p?: number;
@@ -47,10 +46,6 @@ const modelProvider: ReplicateLlama2Provider = {
        type: "string",
        description: "Prompt to send to Llama v2.",
      },
-      stream: {
-        type: "boolean",
-        description: "Whether to stream output from Llama v2.",
-      },
      max_new_tokens: {
        type: "number",
        description:
@@ -59,7 +54,7 @@ const modelProvider: ReplicateLlama2Provider = {
      temperature: {
        type: "number",
        description:
-          "Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value. (minimum: 0.01; maximum: 5)",
+          "Adjusts randomness of outputs, 0.1 is a good starting value. (minimum: 0.01; maximum: 5)",
      },
      top_p: {
        type: "number",
@@ -78,7 +73,7 @@ const modelProvider: ReplicateLlama2Provider = {
    },
    required: ["model", "prompt"],
  },
-  shouldStream: (input) => input.stream ?? false,
+  canStream: true,
  getCompletion,
  ...frontendModelProvider,
 };
--- a/src/modelProviders/types.ts
+++ b/src/modelProviders/types.ts
@@ -3,26 +3,28 @@ import { type IconType } from "react-icons";
 import { type JsonValue } from "type-fest";
 import { z } from "zod";

-const ZodSupportedProvider = z.union([
+export const ZodSupportedProvider = z.union([
  z.literal("openai/ChatCompletion"),
  z.literal("replicate/llama2"),
+  z.literal("anthropic"),
 ]);

 export type SupportedProvider = z.infer<typeof ZodSupportedProvider>;

-export const ZodModel = z.object({
-  name: z.string(),
-  contextWindow: z.number(),
-  promptTokenPrice: z.number().optional(),
-  completionTokenPrice: z.number().optional(),
-  pricePerSecond: z.number().optional(),
-  speed: z.union([z.literal("fast"), z.literal("medium"), z.literal("slow")]),
-  provider: ZodSupportedProvider,
-  description: z.string().optional(),
-  learnMoreUrl: z.string().optional(),
-});
+export type Model = {
+  name: string;
+  contextWindow: number;
+  promptTokenPrice?: number;
+  completionTokenPrice?: number;
+  pricePerSecond?: number;
+  speed: "fast" | "medium" | "slow";
+  provider: SupportedProvider;
+  description?: string;
+  learnMoreUrl?: string;
+  apiDocsUrl?: string;
+};

-export type Model = z.infer<typeof ZodModel>;
+export type ProviderModel = { provider: z.infer<typeof ZodSupportedProvider>; model: string };

 export type RefinementAction = { icon?: IconType; description: string; instructions: string };

@@ -48,7 +50,7 @@ export type CompletionResponse<T> =

 export type ModelProvider<SupportedModels extends string, InputSchema, OutputSchema> = {
  getModel: (input: InputSchema) => SupportedModels | null;
-  shouldStream: (input: InputSchema) => boolean;
+  canStream: boolean;
  inputSchema: JSONSchema4;
  getCompletion: (
    input: InputSchema,
--- a/src/pages/_app.tsx
+++ b/src/pages/_app.tsx
@@ -21,6 +21,17 @@ const MyApp: AppType<{ session: Session | null }> = ({
          name="viewport"
          content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=0"
        />
+        <meta name="og:title" content="OpenPipe: Open-Source Lab for LLMs" key="title" />
+        <meta
+          name="og:description"
+          content="OpenPipe is a powerful playground for quickly optimizing performance, cost, and speed across models."
+          key="description"
+        />
+        <meta name="og:image" content="/og.png" key="og-image" />
+        <meta property="og:image:height" content="630" />
+        <meta property="og:image:width" content="1200" />
+        <meta name="twitter:card" content="summary_large_image" />
+        <meta name="twitter:image" content="/og.png" />
      </Head>
      <SessionProvider session={session}>
        <SyncAppStore />
--- a/src/pages/api/experiments/og-image.tsx
+++ b/src/pages/api/experiments/og-image.tsx
@@ -0,0 +1,81 @@
+import { ImageResponse } from "@vercel/og";
+import { type NextApiRequest, type NextApiResponse } from "next";
+
+export const config = {
+  runtime: "experimental-edge",
+};
+
+const inconsolataRegularFontP = fetch(
+  new URL("../../../../public/fonts/Inconsolata_SemiExpanded-Medium.ttf", import.meta.url),
+).then((res) => res.arrayBuffer());
+
+const OgImage = async (req: NextApiRequest, _res: NextApiResponse) => {
+  // @ts-expect-error - nextUrl is not defined on NextApiRequest for some reason
+  const searchParams = req.nextUrl?.searchParams as URLSearchParams;
+  const experimentLabel = searchParams.get("experimentLabel");
+  const variantsCount = searchParams.get("variantsCount");
+  const scenariosCount = searchParams.get("scenariosCount");
+
+  const inconsolataRegularFont = await inconsolataRegularFontP;
+
+  return new ImageResponse(
+    (
+      <div
+        style={{
+          width: "100%",
+          height: "100%",
+          display: "flex",
+          flexDirection: "column",
+          alignItems: "center",
+          justifyContent: "center",
+          fontSize: 48,
+          padding: "48px",
+          background: "white",
+          position: "relative",
+        }}
+      >
+        <div
+          style={{
+            position: "absolute",
+            top: 0,
+            left: 0,
+            display: "flex",
+            alignItems: "center",
+            padding: 48,
+          }}
+        >
+          {/* eslint-disable-next-line @next/next/no-img-element */}
+          <img
+            src="https://app.openpipe.ai/logo.svg"
+            alt="OpenPipe Logo"
+            height={100}
+            width={120}
+          />
+          <div style={{ marginLeft: 24, fontSize: 64, fontFamily: "Inconsolata" }}>OpenPipe</div>
+        </div>
+
+        <div style={{ display: "flex", fontSize: 72, marginTop: 108 }}>{experimentLabel}</div>
+        <div style={{ display: "flex", flexDirection: "column", marginTop: 36 }}>
+          <div style={{ display: "flex" }}>
+            <span style={{ width: 320 }}>Variants:</span> {variantsCount}
+          </div>
+          <div style={{ display: "flex", marginTop: 24 }}>
+            <span style={{ width: 320 }}>Scenarios:</span> {scenariosCount}
+          </div>
+        </div>
+      </div>
+    ),
+    {
+      fonts: [
+        {
+          name: "inconsolata",
+          data: inconsolataRegularFont,
+          style: "normal",
+          weight: 400,
+        },
+      ],
+    },
+  );
+};
+
+export default OgImage;
--- a/src/pages/experiments/[id].tsx
+++ b/src/pages/experiments/[id].tsx
@@ -2,102 +2,62 @@ import {
  Box,
  Breadcrumb,
  BreadcrumbItem,
-  Button,
  Center,
  Flex,
  Icon,
  Input,
-  AlertDialog,
-  AlertDialogBody,
-  AlertDialogFooter,
-  AlertDialogHeader,
-  AlertDialogContent,
-  AlertDialogOverlay,
-  useDisclosure,
  Text,
-  HStack,
  VStack,
 } from "@chakra-ui/react";
 import Link from "next/link";

 import { useRouter } from "next/router";
-import { useState, useEffect, useRef } from "react";
-import { BsGearFill, BsTrash } from "react-icons/bs";
+import { useState, useEffect } from "react";
 import { RiFlaskLine } from "react-icons/ri";
 import OutputsTable from "~/components/OutputsTable";
-import SettingsDrawer from "~/components/OutputsTable/SettingsDrawer";
+import ExperimentSettingsDrawer from "~/components/ExperimentSettingsDrawer/ExperimentSettingsDrawer";
 import AppShell from "~/components/nav/AppShell";
 import { api } from "~/utils/api";
 import { useExperiment, useHandledAsyncCallback } from "~/utils/hooks";
 import { useAppStore } from "~/state/store";
 import { useSyncVariantEditor } from "~/state/sync";
+import { HeaderButtons } from "~/components/experiments/HeaderButtons/HeaderButtons";
+import Head from "next/head";

-const DeleteButton = () => {
-  const experiment = useExperiment();
-  const mutation = api.experiments.delete.useMutation();
-  const utils = api.useContext();
-  const router = useRouter();
+// TODO: import less to fix deployment with server side props
+// export const getServerSideProps = async (context: GetServerSidePropsContext<{ id: string }>) => {
+//   const experimentId = context.params?.id as string;

-  const { isOpen, onOpen, onClose } = useDisclosure();
-  const cancelRef = useRef<HTMLButtonElement>(null);
+//   const helpers = createServerSideHelpers({
+//     router: appRouter,
+//     ctx: createInnerTRPCContext({ session: null }),
+//     transformer: superjson, // optional - adds superjson serialization
+//   });

-  const [onDeleteConfirm] = useHandledAsyncCallback(async () => {
-    if (!experiment.data?.id) return;
-    await mutation.mutateAsync({ id: experiment.data.id });
-    await utils.experiments.list.invalidate();
-    await router.push({ pathname: "/experiments" });
-    onClose();
-  }, [mutation, experiment.data?.id, router]);
+//   // prefetch query
+//   await helpers.experiments.stats.prefetch({ id: experimentId });

-  return (
-    <>
-      <Button
-        size="sm"
-        variant={{ base: "outline", lg: "ghost" }}
-        colorScheme="gray"
-        fontWeight="normal"
-        onClick={onOpen}
-      >
-        <Icon as={BsTrash} boxSize={4} color="gray.600" />
-        <Text display={{ base: "none", lg: "block" }} ml={2}>
-          Delete Experiment
-        </Text>
-      </Button>
-
-      <AlertDialog isOpen={isOpen} leastDestructiveRef={cancelRef} onClose={onClose}>
-        <AlertDialogOverlay>
-          <AlertDialogContent>
-            <AlertDialogHeader fontSize="lg" fontWeight="bold">
-              Delete Experiment
-            </AlertDialogHeader>
-
-            <AlertDialogBody>
-              If you delete this experiment all the associated prompts and scenarios will be deleted
-              as well. Are you sure?
-            </AlertDialogBody>
-
-            <AlertDialogFooter>
-              <Button ref={cancelRef} onClick={onClose}>
-                Cancel
-              </Button>
-              <Button colorScheme="red" onClick={onDeleteConfirm} ml={3}>
-                Delete
-              </Button>
-            </AlertDialogFooter>
-          </AlertDialogContent>
-        </AlertDialogOverlay>
-      </AlertDialog>
-    </>
-  );
-};
+//   return {
+//     props: {
+//       trpcState: helpers.dehydrate(),
+//     },
+//   };
+// };

 export default function Experiment() {
  const router = useRouter();
-  const experiment = useExperiment();
  const utils = api.useContext();
-  const openDrawer = useAppStore((s) => s.openDrawer);
  useSyncVariantEditor();

+  const experiment = useExperiment();
+  const experimentStats = api.experiments.stats.useQuery(
+    { id: router.query.id as string },
+    {
+      enabled: !!router.query.id,
+    },
+  );
+  const stats = experimentStats.data;
+
  useEffect(() => {
    useAppStore.getState().sharedVariantEditor.loadMonaco().catch(console.error);
  });
@@ -131,6 +91,17 @@ export default function Experiment() {
  const canModify = experiment.data?.access.canModify ?? false;

  return (
+    <>
+      {stats && (
+        <Head>
+          <meta property="og:title" content={stats.experimentLabel} key="title" />
+          <meta
+            property="og:image"
+            content={`/api/experiments/og-image?experimentLabel=${stats.experimentLabel}&variantsCount=${stats.promptVariantCount}&scenariosCount=${stats.testScenarioCount}`}
+            key="og-image"
+          />
+        </Head>
+      )}
      <AppShell title={experiment.data?.label}>
        <VStack h="full">
          <Flex
@@ -138,7 +109,7 @@ export default function Experiment() {
            py={2}
            w="full"
            direction={{ base: "column", sm: "row" }}
-          alignItems="flex-start"
+            alignItems={{ base: "flex-start", sm: "center" }}
          >
            <Breadcrumb flex={1}>
              <BreadcrumbItem>
@@ -171,29 +142,14 @@ export default function Experiment() {
                )}
              </BreadcrumbItem>
            </Breadcrumb>
-          {canModify && (
-            <HStack>
-              <Button
-                size="sm"
-                variant={{ base: "outline", lg: "ghost" }}
-                colorScheme="gray"
-                fontWeight="normal"
-                onClick={openDrawer}
-              >
-                <Icon as={BsGearFill} boxSize={4} color="gray.600" />
-                <Text display={{ base: "none", lg: "block" }} ml={2}>
-                  Edit Vars & Evals
-                </Text>
-              </Button>
-              <DeleteButton />
-            </HStack>
-          )}
+            <HeaderButtons />
          </Flex>
-        <SettingsDrawer />
+          <ExperimentSettingsDrawer />
          <Box w="100%" overflowX="auto" flex={1}>
            <OutputsTable experimentId={router.query.id as string | undefined} />
          </Box>
        </VStack>
      </AppShell>
+    </>
  );
 }
--- a/src/pages/experiments/index.tsx
+++ b/src/pages/experiments/index.tsx
@@ -13,7 +13,11 @@ import {
 import { RiFlaskLine } from "react-icons/ri";
 import AppShell from "~/components/nav/AppShell";
 import { api } from "~/utils/api";
-import { ExperimentCard, NewExperimentCard } from "~/components/experiments/ExperimentCard";
+import {
+  ExperimentCard,
+  ExperimentCardSkeleton,
+  NewExperimentCard,
+} from "~/components/experiments/ExperimentCard";
 import { signIn, useSession } from "next-auth/react";

 export default function ExperimentsPage() {
@@ -47,7 +51,7 @@ export default function ExperimentsPage() {
  return (
    <AppShell title="Experiments">
      <VStack alignItems={"flex-start"} px={4} py={2}>
-        <HStack minH={8} align="center">
+        <HStack minH={8} align="center" pt={2}>
          <Breadcrumb flex={1}>
            <BreadcrumbItem>
              <Flex alignItems="center">
@@ -58,7 +62,15 @@ export default function ExperimentsPage() {
        </HStack>
        <SimpleGrid w="full" columns={{ base: 1, md: 2, lg: 3, xl: 4 }} spacing={8} p="4">
          <NewExperimentCard />
-          {experiments?.data?.map((exp) => <ExperimentCard key={exp.id} exp={exp} />)}
+          {experiments.data && !experiments.isLoading ? (
+            experiments?.data?.map((exp) => <ExperimentCard key={exp.id} exp={exp} />)
+          ) : (
+            <>
+              <ExperimentCardSkeleton />
+              <ExperimentCardSkeleton />
+              <ExperimentCardSkeleton />
+            </>
+          )}
        </SimpleGrid>
      </VStack>
    </AppShell>
--- a/src/pages/world-champs/index.tsx
+++ b/src/pages/world-champs/index.tsx
@@ -0,0 +1,15 @@
+import { type GetServerSideProps } from "next";
+
+// eslint-disable-next-line @typescript-eslint/require-await
+export const getServerSideProps: GetServerSideProps = async () => {
+  return {
+    redirect: {
+      destination: "/world-champs/signup",
+      permanent: false,
+    },
+  };
+};
+
+export default function WorldChamps() {
+  return null;
+}
--- a/src/pages/world-champs/signup.tsx
+++ b/src/pages/world-champs/signup.tsx
@@ -0,0 +1,244 @@
+import {
+  Box,
+  type BoxProps,
+  Button,
+  DarkMode,
+  GlobalStyle,
+  HStack,
+  Heading,
+  Icon,
+  Link,
+  Table,
+  Tbody,
+  Td,
+  Text,
+  type TextProps,
+  Th,
+  Tr,
+  VStack,
+  useInterval,
+  Image,
+} from "@chakra-ui/react";
+import { signIn, useSession } from "next-auth/react";
+import Head from "next/head";
+import { useCallback, useState } from "react";
+import { BsGithub } from "react-icons/bs";
+import UserMenu from "~/components/nav/UserMenu";
+import { api } from "~/utils/api";
+import dayjs from "~/utils/dayjs";
+import { useHandledAsyncCallback } from "~/utils/hooks";
+import GitHubButton from "react-github-btn";
+
+const TopNavbar = () => (
+  <HStack px={4} py={2} align="center" justify="center">
+    <HStack
+      as={Link}
+      href="/"
+      _hover={{ textDecoration: "none" }}
+      spacing={0}
+      py={2}
+      pr={16}
+      flex={1}
+      sx={{
+        ".widget": {
+          display: "block",
+        },
+      }}
+    >
+      <Image src="/logo.svg" alt="" boxSize={6} mr={4} />
+      <Heading size="md" fontFamily="inconsolata, monospace">
+        OpenPipe
+      </Heading>
+    </HStack>
+    <Box pt="6px">
+      <GitHubButton
+        href="https://github.com/openpipe/openpipe"
+        data-color-scheme="no-preference: dark; light: dark; dark: dark;"
+        data-size="large"
+        aria-label="Follow @openpipe on GitHub"
+      >
+        Github
+      </GitHubButton>
+    </Box>
+  </HStack>
+);
+
+// Shows how long until the competition starts. Refreshes every second
+function CountdownTimer(props: { date: Date } & TextProps) {
+  const [now, setNow] = useState(dayjs());
+
+  useInterval(() => {
+    setNow(dayjs());
+  }, 1000);
+
+  const { date, ...rest } = props;
+
+  const kickoff = dayjs(date);
+  const diff = kickoff.diff(now, "second");
+  const days = Math.floor(diff / 86400);
+  const hours = Math.floor((diff % 86400) / 3600);
+  const minutes = Math.floor((diff % 3600) / 60);
+  const seconds = Math.floor(diff % 60);
+
+  return (
+    <Text {...rest} suppressHydrationWarning>
+      <Text as="span" fontWeight="bold">
+        Kickoff in
+      </Text>{" "}
+      {days}d {hours}h {minutes}m {seconds}s
+    </Text>
+  );
+}
+
+function ApplicationStatus(props: BoxProps) {
+  const user = useSession().data;
+  const entrant = api.worldChamps.userStatus.useQuery().data;
+  const applyMutation = api.worldChamps.apply.useMutation();
+
+  const utils = api.useContext();
+
+  const [onSignIn] = useHandledAsyncCallback(async () => {
+    await signIn("github");
+  }, []);
+
+  const [onApply] = useHandledAsyncCallback(async () => {
+    await applyMutation.mutateAsync();
+    await utils.worldChamps.userStatus.invalidate();
+  }, []);
+
+  const Wrapper = useCallback(
+    (wrapperProps: BoxProps) => (
+      <Box {...props} {...wrapperProps} minH="120px" alignItems="center" justifyItems="center" />
+    ),
+    [props],
+  );
+
+  if (user === null) {
+    return (
+      <Wrapper>
+        <Button onClick={onSignIn} colorScheme="orange" leftIcon={<Icon as={BsGithub} />}>
+          Connect GitHub to apply
+        </Button>
+      </Wrapper>
+    );
+  } else if (user) {
+    return (
+      <Wrapper>
+        <HStack spacing={8}>
+          <UserMenu user={user} borderRadius={2} borderColor={"gray.700"} borderWidth={1} pr={6} />
+          <Box flex={1}>
+            {entrant?.approved ? (
+              <Text fontSize="sm">
+                You're accepted! We'll send you more details before August 14th.
+              </Text>
+            ) : entrant ? (
+              <Text fontSize="sm">
+                ✅ Application submitted successfully. We'll notify you by email before August 14th.{" "}
+                <Link href="https://github.com/openpipe/openpipe" isExternal textDecor="underline">
+                  Star our Github
+                </Link>{" "}
+                for updates while you wait!
+              </Text>
+            ) : (
+              <Button onClick={onApply} colorScheme="orange">
+                Apply to compete
+              </Button>
+            )}
+          </Box>
+        </HStack>
+      </Wrapper>
+    );
+  }
+
+  return <Wrapper />;
+}
+
+export default function Signup() {
+  return (
+    <DarkMode>
+      <GlobalStyle />
+
+      <Head>
+        <title>🏆 Prompt Engineering World Championships</title>
+        <meta property="og:title" content="🏆 Prompt Engineering World Championships" key="title" />
+        <meta
+          property="og:description"
+          content="Think you have what it takes to be the best? Compete with the world's top prompt engineers and see where you rank!"
+          key="description"
+        />
+      </Head>
+
+      <Box color="gray.200" minH="100vh" w="full">
+        <TopNavbar />
+        <VStack mx="auto" py={24} maxW="2xl" px={4} align="center" fontSize="lg">
+          <Heading size="lg" textAlign="center">
+            🏆 Prompt Engineering World Championships
+          </Heading>
+          <CountdownTimer
+            date={new Date("2023-08-14T00:00:00Z")}
+            fontSize="2xl"
+            alignSelf="center"
+            color="gray.500"
+          />
+
+          <ApplicationStatus py={8} alignSelf="center" />
+
+          <Text fontSize="lg" textAlign="left">
+            Think you have what it takes to be the best? Compete with the world's top prompt
+            engineers and see where you rank!
+          </Text>
+
+          <Heading size="lg" pt={12} alignSelf="left">
+            Event Details
+          </Heading>
+          <Table variant="simple">
+            <Tbody>
+              <Tr>
+                <Th>Kickoff</Th>
+                <Td>August 14</Td>
+              </Tr>
+              <Tr>
+                <Th>Prize</Th>
+                <Td>$15,000 grand prize + smaller category prizes.</Td>
+              </Tr>
+              <Tr>
+                <Th>Events</Th>
+                <Td>
+                  Optimize prompts for multiple tasks selected from academic benchmarks and
+                  real-world applications.
+                </Td>
+              </Tr>
+              <Tr>
+                <Th>Models</Th>
+                <Td>Separate "weight classes" for GPT 3.5, Claude Instant, and Llama 2.</Td>
+              </Tr>
+              <Tr>
+                <Th>Qualifications</Th>
+                <Td>Open to entrants with any level of experience.</Td>
+              </Tr>
+              <Tr>
+                <Th>Certificates</Th>
+                <Td>Certificate of mastery for all qualifying participants.</Td>
+              </Tr>
+              <Tr>
+                <Th>Cost</Th>
+                <Td>
+                  <strong>Free</strong>. We'll cover your inference budget.
+                </Td>
+              </Tr>
+              <Tr>
+                <Th>Questions?</Th>
+                <Td>
+                  <Link href="mailto:world-champs@openpipe.ai" textDecor="underline">
+                    Email us
+                  </Link>{" "}
+                  with any follow-up questions!
+                </Td>
+              </Tr>
+            </Tbody>
+          </Table>
+        </VStack>
+      </Box>
+    </DarkMode>
+  );
+}
--- a/src/server/api/root.router.ts
+++ b/src/server/api/root.router.ts
@@ -5,6 +5,7 @@ import { scenariosRouter } from "./routers/scenarios.router";
 import { scenarioVariantCellsRouter } from "./routers/scenarioVariantCells.router";
 import { templateVarsRouter } from "./routers/templateVariables.router";
 import { evaluationsRouter } from "./routers/evaluations.router";
+import { worldChampsRouter } from "./routers/worldChamps.router";

 /**
 * This is the primary router for your server.
@@ -18,6 +19,7 @@ export const appRouter = createTRPCRouter({
  scenarioVariantCells: scenarioVariantCellsRouter,
  templateVars: templateVarsRouter,
  evaluations: evaluationsRouter,
+  worldChamps: worldChampsRouter,
 });

 // export type definition of API
--- a/src/server/api/routers/evaluations.router.ts
+++ b/src/server/api/routers/evaluations.router.ts
@@ -2,7 +2,7 @@ import { EvalType } from "@prisma/client";
 import { z } from "zod";
 import { createTRPCRouter, protectedProcedure, publicProcedure } from "~/server/api/trpc";
 import { prisma } from "~/server/db";
-import { runAllEvals } from "~/server/utils/evaluations";
+import { queueRunNewEval } from "~/server/tasks/runNewEval.task";
 import { requireCanModifyExperiment, requireCanViewExperiment } from "~/utils/accessControl";

 export const evaluationsRouter = createTRPCRouter({
@@ -40,9 +40,7 @@ export const evaluationsRouter = createTRPCRouter({
        },
      });

-      // TODO: this may be a bad UX for slow evals (eg. GPT-4 evals) Maybe need
-      // to kick off a background job or something instead
-      await runAllEvals(input.experimentId);
+      await queueRunNewEval(input.experimentId);
    }),

  update: protectedProcedure
@@ -78,7 +76,7 @@ export const evaluationsRouter = createTRPCRouter({
      });
      // Re-run all evals. Other eval results will already be cached, so this
      // should only re-run the updated one.
-      await runAllEvals(evaluation.experimentId);
+      await queueRunNewEval(experimentId);
    }),

  delete: protectedProcedure
--- a/src/server/api/routers/experiments.router.ts
+++ b/src/server/api/routers/experiments.router.ts
@@ -1,5 +1,7 @@
 import { z } from "zod";
+import { v4 as uuidv4 } from "uuid";
 import { createTRPCRouter, protectedProcedure, publicProcedure } from "~/server/api/trpc";
+import { type Prisma } from "@prisma/client";
 import { prisma } from "~/server/db";
 import dedent from "dedent";
 import { generateNewCell } from "~/server/utils/generateNewCell";
@@ -13,6 +15,33 @@ import userOrg from "~/server/utils/userOrg";
 import generateTypes from "~/modelProviders/generateTypes";

 export const experimentsRouter = createTRPCRouter({
+  stats: publicProcedure.input(z.object({ id: z.string() })).query(async ({ input, ctx }) => {
+    await requireCanViewExperiment(input.id, ctx);
+
+    const [experiment, promptVariantCount, testScenarioCount] = await prisma.$transaction([
+      prisma.experiment.findFirstOrThrow({
+        where: { id: input.id },
+      }),
+      prisma.promptVariant.count({
+        where: {
+          experimentId: input.id,
+          visible: true,
+        },
+      }),
+      prisma.testScenario.count({
+        where: {
+          experimentId: input.id,
+          visible: true,
+        },
+      }),
+    ]);
+
+    return {
+      experimentLabel: experiment.label,
+      promptVariantCount,
+      testScenarioCount,
+    };
+  }),
  list: protectedProcedure.query(async ({ ctx }) => {
    // Anyone can list experiments
    requireNothing(ctx);
@@ -20,7 +49,7 @@ export const experimentsRouter = createTRPCRouter({
    const experiments = await prisma.experiment.findMany({
      where: {
        organization: {
-          OrganizationUser: {
+          organizationUsers: {
            some: { userId: ctx.session.user.id },
          },
        },
@@ -77,6 +106,189 @@ export const experimentsRouter = createTRPCRouter({
    };
  }),

+  fork: protectedProcedure.input(z.object({ id: z.string() })).mutation(async ({ input, ctx }) => {
+    await requireCanViewExperiment(input.id, ctx);
+
+    const [
+      existingExp,
+      existingVariants,
+      existingScenarios,
+      existingCells,
+      evaluations,
+      templateVariables,
+    ] = await prisma.$transaction([
+      prisma.experiment.findUniqueOrThrow({
+        where: {
+          id: input.id,
+        },
+      }),
+      prisma.promptVariant.findMany({
+        where: {
+          experimentId: input.id,
+          visible: true,
+        },
+      }),
+      prisma.testScenario.findMany({
+        where: {
+          experimentId: input.id,
+          visible: true,
+        },
+      }),
+      prisma.scenarioVariantCell.findMany({
+        where: {
+          testScenario: {
+            visible: true,
+          },
+          promptVariant: {
+            experimentId: input.id,
+            visible: true,
+          },
+        },
+        include: {
+          modelResponses: {
+            include: {
+              outputEvaluations: true,
+            },
+          },
+        },
+      }),
+      prisma.evaluation.findMany({
+        where: {
+          experimentId: input.id,
+        },
+      }),
+      prisma.templateVariable.findMany({
+        where: {
+          experimentId: input.id,
+        },
+      }),
+    ]);
+
+    const newExperimentId = uuidv4();
+
+    const existingToNewVariantIds = new Map<string, string>();
+    const variantsToCreate: Prisma.PromptVariantCreateManyInput[] = [];
+    for (const variant of existingVariants) {
+      const newVariantId = uuidv4();
+      existingToNewVariantIds.set(variant.id, newVariantId);
+      variantsToCreate.push({
+        ...variant,
+        id: newVariantId,
+        experimentId: newExperimentId,
+      });
+    }
+
+    const existingToNewScenarioIds = new Map<string, string>();
+    const scenariosToCreate: Prisma.TestScenarioCreateManyInput[] = [];
+    for (const scenario of existingScenarios) {
+      const newScenarioId = uuidv4();
+      existingToNewScenarioIds.set(scenario.id, newScenarioId);
+      scenariosToCreate.push({
+        ...scenario,
+        id: newScenarioId,
+        experimentId: newExperimentId,
+        variableValues: scenario.variableValues as Prisma.InputJsonValue,
+      });
+    }
+
+    const existingToNewEvaluationIds = new Map<string, string>();
+    const evaluationsToCreate: Prisma.EvaluationCreateManyInput[] = [];
+    for (const evaluation of evaluations) {
+      const newEvaluationId = uuidv4();
+      existingToNewEvaluationIds.set(evaluation.id, newEvaluationId);
+      evaluationsToCreate.push({
+        ...evaluation,
+        id: newEvaluationId,
+        experimentId: newExperimentId,
+      });
+    }
+
+    const cellsToCreate: Prisma.ScenarioVariantCellCreateManyInput[] = [];
+    const modelResponsesToCreate: Prisma.ModelResponseCreateManyInput[] = [];
+    const outputEvaluationsToCreate: Prisma.OutputEvaluationCreateManyInput[] = [];
+    for (const cell of existingCells) {
+      const newCellId = uuidv4();
+      const { modelResponses, ...cellData } = cell;
+      cellsToCreate.push({
+        ...cellData,
+        id: newCellId,
+        promptVariantId: existingToNewVariantIds.get(cell.promptVariantId) ?? "",
+        testScenarioId: existingToNewScenarioIds.get(cell.testScenarioId) ?? "",
+        prompt: (cell.prompt as Prisma.InputJsonValue) ?? undefined,
+      });
+      for (const modelResponse of modelResponses) {
+        const newModelResponseId = uuidv4();
+        const { outputEvaluations, ...modelResponseData } = modelResponse;
+        modelResponsesToCreate.push({
+          ...modelResponseData,
+          id: newModelResponseId,
+          scenarioVariantCellId: newCellId,
+          output: (modelResponse.output as Prisma.InputJsonValue) ?? undefined,
+        });
+        for (const evaluation of outputEvaluations) {
+          outputEvaluationsToCreate.push({
+            ...evaluation,
+            id: uuidv4(),
+            modelResponseId: newModelResponseId,
+            evaluationId: existingToNewEvaluationIds.get(evaluation.evaluationId) ?? "",
+          });
+        }
+      }
+    }
+
+    const templateVariablesToCreate: Prisma.TemplateVariableCreateManyInput[] = [];
+    for (const templateVariable of templateVariables) {
+      templateVariablesToCreate.push({
+        ...templateVariable,
+        id: uuidv4(),
+        experimentId: newExperimentId,
+      });
+    }
+
+    const maxSortIndex =
+      (
+        await prisma.experiment.aggregate({
+          _max: {
+            sortIndex: true,
+          },
+        })
+      )._max?.sortIndex ?? 0;
+
+    await prisma.$transaction([
+      prisma.experiment.create({
+        data: {
+          id: newExperimentId,
+          sortIndex: maxSortIndex + 1,
+          label: `${existingExp.label} (forked)`,
+          organizationId: (await userOrg(ctx.session.user.id)).id,
+        },
+      }),
+      prisma.promptVariant.createMany({
+        data: variantsToCreate,
+      }),
+      prisma.testScenario.createMany({
+        data: scenariosToCreate,
+      }),
+      prisma.scenarioVariantCell.createMany({
+        data: cellsToCreate,
+      }),
+      prisma.modelResponse.createMany({
+        data: modelResponsesToCreate,
+      }),
+      prisma.evaluation.createMany({
+        data: evaluationsToCreate,
+      }),
+      prisma.outputEvaluation.createMany({
+        data: outputEvaluationsToCreate,
+      }),
+      prisma.templateVariable.createMany({
+        data: templateVariablesToCreate,
+      }),
+    ]);
+
+    return newExperimentId;
+  }),
+
  create: protectedProcedure.input(z.object({})).mutation(async ({ ctx }) => {
    // Anyone can create an experiment
    requireNothing(ctx);
--- a/src/server/api/routers/promptVariants.router.ts
+++ b/src/server/api/routers/promptVariants.router.ts
@@ -1,6 +1,7 @@
 import { z } from "zod";
 import { createTRPCRouter, protectedProcedure, publicProcedure } from "~/server/api/trpc";
 import { prisma } from "~/server/db";
+import { Prisma } from "@prisma/client";
 import { generateNewCell } from "~/server/utils/generateNewCell";
 import userError from "~/server/utils/error";
 import { recordExperimentUpdated } from "~/server/utils/recordExperimentUpdated";
@@ -9,7 +10,8 @@ import { type PromptVariant } from "@prisma/client";
 import { deriveNewConstructFn } from "~/server/utils/deriveNewContructFn";
 import { requireCanModifyExperiment, requireCanViewExperiment } from "~/utils/accessControl";
 import parseConstructFn from "~/server/utils/parseConstructFn";
-import { ZodModel } from "~/modelProviders/types";
+import modelProviders from "~/modelProviders/modelProviders";
+import { ZodSupportedProvider } from "~/modelProviders/types";

 export const promptVariantsRouter = createTRPCRouter({
  list: publicProcedure
@@ -50,7 +52,9 @@ export const promptVariantsRouter = createTRPCRouter({
          id: true,
        },
        where: {
-          modelOutput: {
+          modelResponse: {
+            outdated: false,
+            output: { not: Prisma.AnyNull },
            scenarioVariantCell: {
              promptVariant: {
                id: input.variantId,
@@ -92,14 +96,23 @@ export const promptVariantsRouter = createTRPCRouter({
        where: {
          promptVariantId: input.variantId,
          testScenario: { visible: true },
-          modelOutput: {
-            is: {},
+          modelResponses: {
+            some: {
+              outdated: false,
+              output: {
+                not: Prisma.AnyNull,
+              },
+            },
          },
        },
      });

-      const overallTokens = await prisma.modelOutput.aggregate({
+      const overallTokens = await prisma.modelResponse.aggregate({
        where: {
+          outdated: false,
+          output: {
+            not: Prisma.AnyNull,
+          },
          scenarioVariantCell: {
            promptVariantId: input.variantId,
            testScenario: {
@@ -117,16 +130,9 @@ export const promptVariantsRouter = createTRPCRouter({
      const promptTokens = overallTokens._sum?.promptTokens ?? 0;
      const completionTokens = overallTokens._sum?.completionTokens ?? 0;

-      const awaitingRetrievals = !!(await prisma.scenarioVariantCell.findFirst({
-        where: {
-          promptVariantId: input.variantId,
-          testScenario: { visible: true },
-          // Check if is PENDING or IN_PROGRESS
-          retrievalStatus: {
-            in: ["PENDING", "IN_PROGRESS"],
-          },
-        },
-      }));
+      const awaitingEvals = !!evalResults.find(
+        (result) => result.totalCount < scenarioCount * evals.length,
+      );

      return {
        evalResults,
@@ -135,7 +141,7 @@ export const promptVariantsRouter = createTRPCRouter({
        overallCost: overallTokens._sum?.cost ?? 0,
        scenarioCount,
        outputCount,
-        awaitingRetrievals,
+        awaitingEvals,
      };
    }),

@@ -144,7 +150,7 @@ export const promptVariantsRouter = createTRPCRouter({
      z.object({
        experimentId: z.string(),
        variantId: z.string().optional(),
-        newModel: ZodModel.optional(),
+        streamScenarios: z.array(z.string()),
      }),
    )
    .mutation(async ({ input, ctx }) => {
@@ -186,7 +192,7 @@ export const promptVariantsRouter = createTRPCRouter({
          ? `${originalVariant?.label} Copy`
          : `Prompt Variant ${largestSortIndex + 2}`;

-      const newConstructFn = await deriveNewConstructFn(originalVariant, input.newModel);
+      const newConstructFn = await deriveNewConstructFn(originalVariant);

      const createNewVariantAction = prisma.promptVariant.create({
        data: {
@@ -218,7 +224,9 @@ export const promptVariantsRouter = createTRPCRouter({
      });

      for (const scenario of scenarios) {
-        await generateNewCell(newVariant.id, scenario.id);
+        await generateNewCell(newVariant.id, scenario.id, {
+          stream: input.streamScenarios.includes(scenario.id),
+        });
      }

      return newVariant;
@@ -286,7 +294,12 @@ export const promptVariantsRouter = createTRPCRouter({
      z.object({
        id: z.string(),
        instructions: z.string().optional(),
-        newModel: ZodModel.optional(),
+        newModel: z
+          .object({
+            provider: ZodSupportedProvider,
+            model: z.string(),
+          })
+          .optional(),
      }),
    )
    .mutation(async ({ input, ctx }) => {
@@ -303,11 +316,11 @@ export const promptVariantsRouter = createTRPCRouter({
        return userError(constructedPrompt.error);
      }

-      const promptConstructionFn = await deriveNewConstructFn(
-        existing,
-        input.newModel,
-        input.instructions,
-      );
+      const model = input.newModel
+        ? modelProviders[input.newModel.provider].models[input.newModel.model]
+        : undefined;
+
+      const promptConstructionFn = await deriveNewConstructFn(existing, model, input.instructions);

      // TODO: Validate promptConstructionFn
      // TODO: Record in some sort of history
@@ -320,6 +333,7 @@ export const promptVariantsRouter = createTRPCRouter({
      z.object({
        id: z.string(),
        constructFn: z.string(),
+        streamScenarios: z.array(z.string()),
      }),
    )
    .mutation(async ({ input, ctx }) => {
@@ -377,7 +391,9 @@ export const promptVariantsRouter = createTRPCRouter({
      });

      for (const scenario of scenarios) {
-        await generateNewCell(newVariant.id, scenario.id);
+        await generateNewCell(newVariant.id, scenario.id, {
+          stream: input.streamScenarios.includes(scenario.id),
+        });
      }

      return { status: "ok" } as const;
--- a/src/server/api/routers/scenarioVariantCells.router.ts
+++ b/src/server/api/routers/scenarioVariantCells.router.ts
@@ -1,8 +1,8 @@
 import { z } from "zod";
 import { createTRPCRouter, protectedProcedure, publicProcedure } from "~/server/api/trpc";
 import { prisma } from "~/server/db";
+import { queueQueryModel } from "~/server/tasks/queryModel.task";
 import { generateNewCell } from "~/server/utils/generateNewCell";
-import { queueLLMRetrievalTask } from "~/server/utils/queueLLMRetrievalTask";
 import { requireCanModifyExperiment, requireCanViewExperiment } from "~/utils/accessControl";

 export const scenarioVariantCellsRouter = createTRPCRouter({
@@ -19,7 +19,8 @@ export const scenarioVariantCellsRouter = createTRPCRouter({
      });
      await requireCanViewExperiment(experimentId, ctx);

-      return await prisma.scenarioVariantCell.findUnique({
+      const [cell, numTotalEvals] = await prisma.$transaction([
+        prisma.scenarioVariantCell.findUnique({
          where: {
            promptVariantId_testScenarioId: {
              promptVariantId: input.variantId,
@@ -27,9 +28,12 @@ export const scenarioVariantCellsRouter = createTRPCRouter({
            },
          },
          include: {
-          modelOutput: {
+            modelResponses: {
+              where: {
+                outdated: false,
+              },
              include: {
-              outputEvaluation: {
+                outputEvaluations: {
                  include: {
                    evaluation: {
                      select: { label: true },
@@ -39,7 +43,21 @@ export const scenarioVariantCellsRouter = createTRPCRouter({
              },
            },
          },
-      });
+        }),
+        prisma.evaluation.count({
+          where: { experimentId },
+        }),
+      ]);
+
+      if (!cell) return null;
+
+      const lastResponse = cell.modelResponses?.[cell.modelResponses?.length - 1];
+      const evalsComplete = lastResponse?.outputEvaluations?.length === numTotalEvals;
+
+      return {
+        ...cell,
+        evalsComplete,
+      };
    }),
  forceRefetch: protectedProcedure
    .input(
@@ -62,29 +80,20 @@ export const scenarioVariantCellsRouter = createTRPCRouter({
            testScenarioId: input.scenarioId,
          },
        },
-        include: {
-          modelOutput: true,
-        },
      });

      if (!cell) {
-        await generateNewCell(input.variantId, input.scenarioId);
-        return true;
+        await generateNewCell(input.variantId, input.scenarioId, { stream: true });
+        return;
      }

-      if (cell.modelOutput) {
-        // TODO: Maybe keep these around to show previous generations?
-        await prisma.modelOutput.delete({
-          where: { id: cell.modelOutput.id },
-        });
-      }
-
-      await prisma.scenarioVariantCell.update({
-        where: { id: cell.id },
-        data: { retrievalStatus: "PENDING" },
+      await prisma.modelResponse.updateMany({
+        where: { scenarioVariantCellId: cell.id },
+        data: {
+          outdated: true,
+        },
      });

-      await queueLLMRetrievalTask(cell.id);
-      return true;
+      await queueQueryModel(cell.id, true);
    }),
 });
--- a/src/server/api/routers/scenarios.router.ts
+++ b/src/server/api/routers/scenarios.router.ts
@@ -41,7 +41,21 @@ export const scenariosRouter = createTRPCRouter({
        count,
      };
    }),
+  get: protectedProcedure.input(z.object({ id: z.string() })).query(async ({ input, ctx }) => {
+    const scenario = await prisma.testScenario.findUnique({
+      where: {
+        id: input.id,
+      },
+    });

+    if (!scenario) {
+      throw new Error(`Scenario with id ${input.id} does not exist`);
+    }
+
+    await requireCanViewExperiment(scenario.experimentId, ctx);
+
+    return scenario;
+  }),
  create: protectedProcedure
    .input(
      z.object({
@@ -86,7 +100,7 @@ export const scenariosRouter = createTRPCRouter({
      });

      for (const variant of promptVariants) {
-        await generateNewCell(variant.id, scenario.id);
+        await generateNewCell(variant.id, scenario.id, { stream: true });
      }
    }),

@@ -230,7 +244,7 @@ export const scenariosRouter = createTRPCRouter({
      });

      for (const variant of promptVariants) {
-        await generateNewCell(variant.id, newScenario.id);
+        await generateNewCell(variant.id, newScenario.id, { stream: true });
      }

      return newScenario;
--- a/src/server/api/routers/worldChamps.router.ts
+++ b/src/server/api/routers/worldChamps.router.ts
@@ -0,0 +1,36 @@
+import { createTRPCRouter, protectedProcedure, publicProcedure } from "~/server/api/trpc";
+import { prisma } from "~/server/db";
+import { requireNothing } from "~/utils/accessControl";
+
+export const worldChampsRouter = createTRPCRouter({
+  userStatus: publicProcedure.query(async ({ ctx }) => {
+    const userId = ctx.session?.user.id;
+
+    if (!userId) {
+      return null;
+    }
+
+    return await prisma.worldChampEntrant.findUnique({
+      where: { userId },
+    });
+  }),
+
+  apply: protectedProcedure.mutation(async ({ ctx }) => {
+    const userId = ctx.session.user.id;
+    requireNothing(ctx);
+
+    const existingEntrant = await prisma.worldChampEntrant.findUnique({
+      where: { userId },
+    });
+
+    if (existingEntrant) {
+      return existingEntrant;
+    }
+
+    return await prisma.worldChampEntrant.create({
+      data: {
+        userId,
+      },
+    });
+  }),
+});
--- a/src/server/api/trpc.ts
+++ b/src/server/api/trpc.ts
@@ -40,7 +40,7 @@ const noOp = () => {};
 *
 * @see https://create.t3.gg/en/usage/trpc#-serverapitrpcts
 */
-const createInnerTRPCContext = (opts: CreateContextOptions) => {
+export const createInnerTRPCContext = (opts: CreateContextOptions) => {
  return {
    session: opts.session,
    prisma,
--- a/src/server/scripts/studio-prod.sh
+++ b/src/server/scripts/studio-prod.sh
@@ -0,0 +1,12 @@
+#! /bin/bash
+
+set -e
+cd "$(dirname "$0")/../../.."
+
+
+set -o allexport
+source .env
+set +o allexport
+
+echo "Connecting to prod db"
+DATABASE_URL=$PROD_DATABASE_URL pnpm prisma studio
--- a/src/server/tasks/defineTask.ts
+++ b/src/server/tasks/defineTask.ts
@@ -7,9 +7,9 @@ function defineTask<TPayload>(
  taskIdentifier: string,
  taskHandler: (payload: TPayload, helpers: Helpers) => Promise<void>,
 ) {
-  const enqueue = async (payload: TPayload) => {
+  const enqueue = async (payload: TPayload, runAt?: Date) => {
    console.log("Enqueuing task", taskIdentifier, payload);
-    await quickAddJob({ connectionString: env.DATABASE_URL }, taskIdentifier, payload);
+    await quickAddJob({ connectionString: env.DATABASE_URL }, taskIdentifier, payload, { runAt });
  };

  const handler = (payload: TPayload, helpers: Helpers) => {
--- a/src/server/tasks/queryLLM.task.ts
+++ b/src/server/tasks/queryLLM.task.ts
@@ -1,165 +0,0 @@
-import { prisma } from "~/server/db";
-import defineTask from "./defineTask";
-import { sleep } from "../utils/sleep";
-import { generateChannel } from "~/utils/generateChannel";
-import { runEvalsForOutput } from "../utils/evaluations";
-import { type Prisma } from "@prisma/client";
-import parseConstructFn from "../utils/parseConstructFn";
-import hashPrompt from "../utils/hashPrompt";
-import { type JsonObject } from "type-fest";
-import modelProviders from "~/modelProviders/modelProviders";
-import { wsConnection } from "~/utils/wsConnection";
-
-export type queryLLMJob = {
-  scenarioVariantCellId: string;
-};
-
-const MAX_AUTO_RETRIES = 10;
-const MIN_DELAY = 500; // milliseconds
-const MAX_DELAY = 15000; // milliseconds
-
-function calculateDelay(numPreviousTries: number): number {
-  const baseDelay = Math.min(MAX_DELAY, MIN_DELAY * Math.pow(2, numPreviousTries));
-  const jitter = Math.random() * baseDelay;
-  return baseDelay + jitter;
-}
-
-export const queryLLM = defineTask<queryLLMJob>("queryLLM", async (task) => {
-  const { scenarioVariantCellId } = task;
-  const cell = await prisma.scenarioVariantCell.findUnique({
-    where: { id: scenarioVariantCellId },
-    include: { modelOutput: true },
-  });
-  if (!cell) {
-    await prisma.scenarioVariantCell.update({
-      where: { id: scenarioVariantCellId },
-      data: {
-        statusCode: 404,
-        errorMessage: "Cell not found",
-        retrievalStatus: "ERROR",
-      },
-    });
-    return;
-  }
-
-  // If cell is not pending, then some other job is already processing it
-  if (cell.retrievalStatus !== "PENDING") {
-    return;
-  }
-  await prisma.scenarioVariantCell.update({
-    where: { id: scenarioVariantCellId },
-    data: {
-      retrievalStatus: "IN_PROGRESS",
-    },
-  });
-
-  const variant = await prisma.promptVariant.findUnique({
-    where: { id: cell.promptVariantId },
-  });
-  if (!variant) {
-    await prisma.scenarioVariantCell.update({
-      where: { id: scenarioVariantCellId },
-      data: {
-        statusCode: 404,
-        errorMessage: "Prompt Variant not found",
-        retrievalStatus: "ERROR",
-      },
-    });
-    return;
-  }
-
-  const scenario = await prisma.testScenario.findUnique({
-    where: { id: cell.testScenarioId },
-  });
-  if (!scenario) {
-    await prisma.scenarioVariantCell.update({
-      where: { id: scenarioVariantCellId },
-      data: {
-        statusCode: 404,
-        errorMessage: "Scenario not found",
-        retrievalStatus: "ERROR",
-      },
-    });
-    return;
-  }
-
-  const prompt = await parseConstructFn(variant.constructFn, scenario.variableValues as JsonObject);
-
-  if ("error" in prompt) {
-    await prisma.scenarioVariantCell.update({
-      where: { id: scenarioVariantCellId },
-      data: {
-        statusCode: 400,
-        errorMessage: prompt.error,
-        retrievalStatus: "ERROR",
-      },
-    });
-    return;
-  }
-
-  const provider = modelProviders[prompt.modelProvider];
-
-  const streamingChannel = provider.shouldStream(prompt.modelInput) ? generateChannel() : null;
-
-  if (streamingChannel) {
-    // Save streaming channel so that UI can connect to it
-    await prisma.scenarioVariantCell.update({
-      where: { id: scenarioVariantCellId },
-      data: { streamingChannel },
-    });
-  }
-  const onStream = streamingChannel
-    ? (partialOutput: (typeof provider)["_outputSchema"]) => {
-        wsConnection.emit("message", { channel: streamingChannel, payload: partialOutput });
-      }
-    : null;
-
-  for (let i = 0; true; i++) {
-    const response = await provider.getCompletion(prompt.modelInput, onStream);
-    if (response.type === "success") {
-      const inputHash = hashPrompt(prompt);
-
-      const modelOutput = await prisma.modelOutput.create({
-        data: {
-          scenarioVariantCellId,
-          inputHash,
-          output: response.value as Prisma.InputJsonObject,
-          timeToComplete: response.timeToComplete,
-          promptTokens: response.promptTokens,
-          completionTokens: response.completionTokens,
-          cost: response.cost,
-        },
-      });
-
-      await prisma.scenarioVariantCell.update({
-        where: { id: scenarioVariantCellId },
-        data: {
-          statusCode: response.statusCode,
-          retrievalStatus: "COMPLETE",
-        },
-      });
-
-      await runEvalsForOutput(variant.experimentId, scenario, modelOutput);
-      break;
-    } else {
-      const shouldRetry = response.autoRetry && i < MAX_AUTO_RETRIES;
-      const delay = calculateDelay(i);
-
-      await prisma.scenarioVariantCell.update({
-        where: { id: scenarioVariantCellId },
-        data: {
-          errorMessage: response.message,
-          statusCode: response.statusCode,
-          retryTime: shouldRetry ? new Date(Date.now() + delay) : null,
-          retrievalStatus: "ERROR",
-        },
-      });
-
-      if (shouldRetry) {
-        await sleep(delay);
-      } else {
-        break;
-      }
-    }
-  }
-});
--- a/src/server/tasks/queryModel.task.ts
+++ b/src/server/tasks/queryModel.task.ts
@@ -0,0 +1,185 @@
+import { type Prisma } from "@prisma/client";
+import { type JsonObject } from "type-fest";
+import modelProviders from "~/modelProviders/modelProviders";
+import { prisma } from "~/server/db";
+import { wsConnection } from "~/utils/wsConnection";
+import { runEvalsForOutput } from "../utils/evaluations";
+import hashPrompt from "../utils/hashPrompt";
+import parseConstructFn from "../utils/parseConstructFn";
+import defineTask from "./defineTask";
+
+export type QueryModelJob = {
+  cellId: string;
+  stream: boolean;
+  numPreviousTries: number;
+};
+
+const MAX_AUTO_RETRIES = 50;
+const MIN_DELAY = 500; // milliseconds
+const MAX_DELAY = 15000; // milliseconds
+
+function calculateDelay(numPreviousTries: number): number {
+  const baseDelay = Math.min(MAX_DELAY, MIN_DELAY * Math.pow(2, numPreviousTries));
+  const jitter = Math.random() * baseDelay;
+  return baseDelay + jitter;
+}
+
+export const queryModel = defineTask<QueryModelJob>("queryModel", async (task) => {
+  console.log("RUNNING TASK", task);
+  const { cellId, stream, numPreviousTries } = task;
+  const cell = await prisma.scenarioVariantCell.findUnique({
+    where: { id: cellId },
+    include: { modelResponses: true },
+  });
+  if (!cell) {
+    return;
+  }
+
+  // If cell is not pending, then some other job is already processing it
+  if (cell.retrievalStatus !== "PENDING") {
+    return;
+  }
+  await prisma.scenarioVariantCell.update({
+    where: { id: cellId },
+    data: {
+      retrievalStatus: "IN_PROGRESS",
+      jobStartedAt: new Date(),
+    },
+  });
+
+  const variant = await prisma.promptVariant.findUnique({
+    where: { id: cell.promptVariantId },
+  });
+  if (!variant) {
+    await prisma.scenarioVariantCell.update({
+      where: { id: cellId },
+      data: {
+        errorMessage: "Prompt Variant not found",
+        retrievalStatus: "ERROR",
+      },
+    });
+    return;
+  }
+
+  const scenario = await prisma.testScenario.findUnique({
+    where: { id: cell.testScenarioId },
+  });
+  if (!scenario) {
+    await prisma.scenarioVariantCell.update({
+      where: { id: cellId },
+      data: {
+        errorMessage: "Scenario not found",
+        retrievalStatus: "ERROR",
+      },
+    });
+    return;
+  }
+
+  const prompt = await parseConstructFn(variant.constructFn, scenario.variableValues as JsonObject);
+
+  if ("error" in prompt) {
+    await prisma.scenarioVariantCell.update({
+      where: { id: cellId },
+      data: {
+        errorMessage: prompt.error,
+        retrievalStatus: "ERROR",
+      },
+    });
+    return;
+  }
+
+  const provider = modelProviders[prompt.modelProvider];
+
+  const onStream = stream
+    ? (partialOutput: (typeof provider)["_outputSchema"]) => {
+        wsConnection.emit("message", { channel: cell.id, payload: partialOutput });
+      }
+    : null;
+
+  const inputHash = hashPrompt(prompt);
+
+  let modelResponse = await prisma.modelResponse.create({
+    data: {
+      inputHash,
+      scenarioVariantCellId: cellId,
+      requestedAt: new Date(),
+    },
+  });
+  const response = await provider.getCompletion(prompt.modelInput, onStream);
+  if (response.type === "success") {
+    modelResponse = await prisma.modelResponse.update({
+      where: { id: modelResponse.id },
+      data: {
+        output: response.value as Prisma.InputJsonObject,
+        statusCode: response.statusCode,
+        receivedAt: new Date(),
+        promptTokens: response.promptTokens,
+        completionTokens: response.completionTokens,
+        cost: response.cost,
+      },
+    });
+
+    await prisma.scenarioVariantCell.update({
+      where: { id: cellId },
+      data: {
+        retrievalStatus: "COMPLETE",
+      },
+    });
+
+    await runEvalsForOutput(variant.experimentId, scenario, modelResponse, prompt.modelProvider);
+  } else {
+    const shouldRetry = response.autoRetry && numPreviousTries < MAX_AUTO_RETRIES;
+    const delay = calculateDelay(numPreviousTries);
+    const retryTime = new Date(Date.now() + delay);
+
+    await prisma.modelResponse.update({
+      where: { id: modelResponse.id },
+      data: {
+        statusCode: response.statusCode,
+        errorMessage: response.message,
+        receivedAt: new Date(),
+        retryTime: shouldRetry ? retryTime : null,
+      },
+    });
+
+    if (shouldRetry) {
+      await queryModel.enqueue(
+        {
+          cellId,
+          stream,
+          numPreviousTries: numPreviousTries + 1,
+        },
+        retryTime,
+      );
+      await prisma.scenarioVariantCell.update({
+        where: { id: cellId },
+        data: {
+          retrievalStatus: "PENDING",
+        },
+      });
+    } else {
+      await prisma.scenarioVariantCell.update({
+        where: { id: cellId },
+        data: {
+          retrievalStatus: "ERROR",
+        },
+      });
+    }
+  }
+});
+
+export const queueQueryModel = async (cellId: string, stream: boolean) => {
+  await Promise.all([
+    prisma.scenarioVariantCell.update({
+      where: {
+        id: cellId,
+      },
+      data: {
+        retrievalStatus: "PENDING",
+        errorMessage: null,
+        jobQueuedAt: new Date(),
+      },
+    }),
+    queryModel.enqueue({ cellId, stream, numPreviousTries: 0 }),
+  ]);
+};
--- a/src/server/tasks/runNewEval.task.ts
+++ b/src/server/tasks/runNewEval.task.ts
@@ -0,0 +1,17 @@
+import { runAllEvals } from "../utils/evaluations";
+import defineTask from "./defineTask";
+
+export type RunNewEvalJob = {
+  experimentId: string;
+};
+
+// When a new eval is created, we want to run it on all existing outputs, but return the new eval first
+export const runNewEval = defineTask<RunNewEvalJob>("runNewEval", async (task) => {
+  console.log("RUNNING TASK", task);
+  const { experimentId } = task;
+  await runAllEvals(experimentId);
+});
+
+export const queueRunNewEval = async (experimentId: string) => {
+  await runNewEval.enqueue({ experimentId });
+};
--- a/src/server/tasks/worker.ts
+++ b/src/server/tasks/worker.ts
@@ -2,39 +2,28 @@ import { type TaskList, run } from "graphile-worker";
 import "dotenv/config";

 import { env } from "~/env.mjs";
-import { queryLLM } from "./queryLLM.task";
+import { queryModel } from "./queryModel.task";
+import { runNewEval } from "./runNewEval.task";

-const registeredTasks = [queryLLM];
+console.log("Starting worker");
+
+const registeredTasks = [queryModel, runNewEval];

 const taskList = registeredTasks.reduce((acc, task) => {
  acc[task.task.identifier] = task.task.handler;
  return acc;
 }, {} as TaskList);

-async function main() {
-  // Run a worker to execute jobs:
-  const runner = await run({
+// Run a worker to execute jobs:
+const runner = await run({
  connectionString: env.DATABASE_URL,
-    concurrency: 20,
+  concurrency: 50,
  // Install signal handlers for graceful shutdown on SIGINT, SIGTERM, etc
  noHandleSignals: false,
  pollInterval: 1000,
-    // you can set the taskList or taskDirectory but not both
  taskList,
-    // or:
-    //   taskDirectory: `${__dirname}/tasks`,
-  });
-
-  // Immediately await (or otherwise handled) the resulting promise, to avoid
-  // "unhandled rejection" errors causing a process crash in the event of
-  // something going wrong.
-  await runner.promise;
-
-  // If the worker exits (whether through fatal error or otherwise), the above
-  // promise will resolve/reject.
-}
-
-main().catch((err) => {
-  console.error("Unhandled error occurred running worker: ", err);
-  process.exit(1);
 });
+
+console.log("Worker successfully started");
+
+await runner.promise;
--- a/src/server/utils/deriveNewContructFn.ts
+++ b/src/server/utils/deriveNewContructFn.ts
@@ -3,7 +3,7 @@ import ivm from "isolated-vm";
 import dedent from "dedent";
 import { openai } from "./openai";
 import { isObject } from "lodash-es";
-import { type CompletionCreateParams } from "openai/resources/chat/completions";
+import type { CreateChatCompletionRequestMessage } from "openai/resources/chat/completions";
 import formatPromptConstructor from "~/utils/formatPromptConstructor";
 import { type SupportedProvider, type Model } from "~/modelProviders/types";
 import modelProviders from "~/modelProviders/modelProviders";
@@ -44,7 +44,7 @@ const requestUpdatedPromptFunction = async (
  let newContructionFn = "";
  for (let i = 0; i < NUM_RETRIES; i++) {
    try {
-      const messages: CompletionCreateParams.CreateChatCompletionRequestNonStreaming.Message[] = [
+      const messages: CreateChatCompletionRequestMessage[] = [
        {
          role: "system",
          content: `Your job is to update prompt constructor functions. Here is the api shape for the current model:\n---\n${JSON.stringify(
@@ -66,9 +66,11 @@ const requestUpdatedPromptFunction = async (
        if (newModel.provider !== originalModel.provider) {
          messages.push({
            role: "user",
-            content: `The old provider was ${originalModel.provider}. The new provider is ${
+            content: `As seen in the first argument to definePrompt, the old provider endpoint was "${
+              originalModel.provider
+            }". The new provider endpoint is "${
              newModel.provider
-            }. Here is the schema for the new model:\n---\n${JSON.stringify(
+            }". Here is the schema for the new model:\n---\n${JSON.stringify(
              modelProviders[newModel.provider].inputSchema,
              null,
              2,
--- a/src/server/utils/evaluations.ts
+++ b/src/server/utils/evaluations.ts
@@ -1,19 +1,25 @@
-import { type ModelOutput, type Evaluation } from "@prisma/client";
+import { type ModelResponse, type Evaluation, Prisma } from "@prisma/client";
 import { prisma } from "../db";
 import { runOneEval } from "./runOneEval";
 import { type Scenario } from "~/components/OutputsTable/types";
+import { type SupportedProvider } from "~/modelProviders/types";

-const saveResult = async (evaluation: Evaluation, scenario: Scenario, modelOutput: ModelOutput) => {
-  const result = await runOneEval(evaluation, scenario, modelOutput);
+const runAndSaveEval = async (
+  evaluation: Evaluation,
+  scenario: Scenario,
+  modelResponse: ModelResponse,
+  provider: SupportedProvider,
+) => {
+  const result = await runOneEval(evaluation, scenario, modelResponse, provider);
  return await prisma.outputEvaluation.upsert({
    where: {
-      modelOutputId_evaluationId: {
-        modelOutputId: modelOutput.id,
+      modelResponseId_evaluationId: {
+        modelResponseId: modelResponse.id,
        evaluationId: evaluation.id,
      },
    },
    create: {
-      modelOutputId: modelOutput.id,
+      modelResponseId: modelResponse.id,
      evaluationId: evaluation.id,
      ...result,
    },
@@ -26,20 +32,28 @@ const saveResult = async (evaluation: Evaluation, scenario: Scenario, modelOutpu
 export const runEvalsForOutput = async (
  experimentId: string,
  scenario: Scenario,
-  modelOutput: ModelOutput,
+  modelResponse: ModelResponse,
+  provider: SupportedProvider,
 ) => {
  const evaluations = await prisma.evaluation.findMany({
    where: { experimentId },
  });

  await Promise.all(
-    evaluations.map(async (evaluation) => await saveResult(evaluation, scenario, modelOutput)),
+    evaluations.map(
+      async (evaluation) => await runAndSaveEval(evaluation, scenario, modelResponse, provider),
+    ),
  );
 };

+// Will not run eval-output pairs that already exist in the database
 export const runAllEvals = async (experimentId: string) => {
-  const outputs = await prisma.modelOutput.findMany({
+  const outputs = await prisma.modelResponse.findMany({
    where: {
+      outdated: false,
+      output: {
+        not: Prisma.AnyNull,
+      },
      scenarioVariantCell: {
        promptVariant: {
          experimentId,
@@ -54,9 +68,10 @@ export const runAllEvals = async (experimentId: string) => {
      scenarioVariantCell: {
        include: {
          testScenario: true,
+          promptVariant: true,
        },
      },
-      outputEvaluation: true,
+      outputEvaluations: true,
    },
  });
  const evals = await prisma.evaluation.findMany({
@@ -65,13 +80,18 @@ export const runAllEvals = async (experimentId: string) => {

  await Promise.all(
    outputs.map(async (output) => {
-      const unrunEvals = evals.filter(
-        (evaluation) => !output.outputEvaluation.find((e) => e.evaluationId === evaluation.id),
+      const evalsToBeRun = evals.filter(
+        (evaluation) => !output.outputEvaluations.find((e) => e.evaluationId === evaluation.id),
      );

      await Promise.all(
-        unrunEvals.map(async (evaluation) => {
-          await saveResult(evaluation, output.scenarioVariantCell.testScenario, output);
+        evalsToBeRun.map(async (evaluation) => {
+          await runAndSaveEval(
+            evaluation,
+            output.scenarioVariantCell.testScenario,
+            output,
+            output.scenarioVariantCell.promptVariant.modelProvider as SupportedProvider,
+          );
        }),
      );
    }),
--- a/src/server/utils/generateNewCell.ts
+++ b/src/server/utils/generateNewCell.ts
@@ -1,12 +1,18 @@
-import { type Prisma } from "@prisma/client";
+import { Prisma } from "@prisma/client";
 import { prisma } from "../db";
-import { queueLLMRetrievalTask } from "./queueLLMRetrievalTask";
 import parseConstructFn from "./parseConstructFn";
 import { type JsonObject } from "type-fest";
 import hashPrompt from "./hashPrompt";
 import { omit } from "lodash-es";
+import { queueQueryModel } from "../tasks/queryModel.task";
+
+export const generateNewCell = async (
+  variantId: string,
+  scenarioId: string,
+  options?: { stream?: boolean },
+): Promise<void> => {
+  const stream = options?.stream ?? false;

-export const generateNewCell = async (variantId: string, scenarioId: string): Promise<void> => {
  const variant = await prisma.promptVariant.findUnique({
    where: {
      id: variantId,
@@ -29,7 +35,7 @@ export const generateNewCell = async (variantId: string, scenarioId: string): Pr
      },
    },
    include: {
-      modelOutput: true,
+      modelResponses: true,
    },
  });

@@ -45,8 +51,6 @@ export const generateNewCell = async (variantId: string, scenarioId: string): Pr
      data: {
        promptVariantId: variantId,
        testScenarioId: scenarioId,
-        statusCode: 400,
-        errorMessage: parsedConstructFn.error,
        retrievalStatus: "ERROR",
      },
    });
@@ -63,41 +67,60 @@ export const generateNewCell = async (variantId: string, scenarioId: string): Pr
      retrievalStatus: "PENDING",
    },
    include: {
-      modelOutput: true,
+      modelResponses: true,
    },
  });

-  const matchingModelOutput = await prisma.modelOutput.findFirst({
-    where: { inputHash },
+  const matchingModelResponse = await prisma.modelResponse.findFirst({
+    where: {
+      inputHash,
+      output: {
+        not: Prisma.AnyNull,
+      },
+    },
+    orderBy: {
+      receivedAt: "desc",
+    },
+    include: {
+      scenarioVariantCell: true,
+    },
+    take: 1,
  });

-  if (matchingModelOutput) {
-    const newModelOutput = await prisma.modelOutput.create({
+  if (matchingModelResponse) {
+    const newModelResponse = await prisma.modelResponse.create({
      data: {
-        ...omit(matchingModelOutput, ["id"]),
+        ...omit(matchingModelResponse, ["id", "scenarioVariantCell"]),
        scenarioVariantCellId: cell.id,
-        output: matchingModelOutput.output as Prisma.InputJsonValue,
+        output: matchingModelResponse.output as Prisma.InputJsonValue,
      },
    });
+
    await prisma.scenarioVariantCell.update({
      where: { id: cell.id },
-      data: { retrievalStatus: "COMPLETE" },
+      data: {
+        retrievalStatus: "COMPLETE",
+        jobStartedAt: matchingModelResponse.scenarioVariantCell.jobStartedAt,
+        jobQueuedAt: matchingModelResponse.scenarioVariantCell.jobQueuedAt,
+      },
    });

    // Copy over all eval results as well
    await Promise.all(
      (
-        await prisma.outputEvaluation.findMany({ where: { modelOutputId: matchingModelOutput.id } })
+        await prisma.outputEvaluation.findMany({
+          where: { modelResponseId: matchingModelResponse.id },
+        })
      ).map(async (evaluation) => {
        await prisma.outputEvaluation.create({
          data: {
            ...omit(evaluation, ["id"]),
-            modelOutputId: newModelOutput.id,
+            modelResponseId: newModelResponse.id,
          },
        });
      }),
    );
  } else {
-    cell = await queueLLMRetrievalTask(cell.id);
+    await queueQueryModel(cell.id, stream);
  }
 };
--- a/src/server/utils/openai.ts
+++ b/src/server/utils/openai.ts
@@ -2,4 +2,5 @@ import { env } from "~/env.mjs";

 import OpenAI from "openai";

-export const openai = new OpenAI({ apiKey: env.OPENAI_API_KEY });
+// Set a dummy key so it doesn't fail at build time
+export const openai = new OpenAI({ apiKey: env.OPENAI_API_KEY ?? "dummy-key" });
--- a/src/server/utils/queueLLMRetrievalTask.ts
+++ b/src/server/utils/queueLLMRetrievalTask.ts
@@ -1,22 +0,0 @@
-import { prisma } from "../db";
-import { queryLLM } from "../tasks/queryLLM.task";
-
-export const queueLLMRetrievalTask = async (cellId: string) => {
-  const updatedCell = await prisma.scenarioVariantCell.update({
-    where: {
-      id: cellId,
-    },
-    data: {
-      retrievalStatus: "PENDING",
-      errorMessage: null,
-    },
-    include: {
-      modelOutput: true,
-    },
-  });
-
-  // @ts-expect-error we aren't passing the helpers but that's ok
-  void queryLLM.task.handler({ scenarioVariantCellId: cellId }, { logger: console });
-
-  return updatedCell;
-};
--- a/src/server/utils/runOneEval.ts
+++ b/src/server/utils/runOneEval.ts
@@ -1,13 +1,14 @@
-import { type Evaluation, type ModelOutput, type TestScenario } from "@prisma/client";
-import { type ChatCompletion } from "openai/resources/chat";
+import { type Evaluation, type ModelResponse, type TestScenario } from "@prisma/client";
 import { type VariableMap, fillTemplate, escapeRegExp, escapeQuotes } from "./fillTemplate";
 import { openai } from "./openai";
 import dedent from "dedent";
+import modelProviders from "~/modelProviders/modelProviders";
+import { type SupportedProvider } from "~/modelProviders/types";

 export const runGpt4Eval = async (
  evaluation: Evaluation,
  scenario: TestScenario,
-  message: ChatCompletion.Choice.Message,
+  stringifiedOutput: string,
 ): Promise<{ result: number; details: string }> => {
  const output = await openai.chat.completions.create({
    model: "gpt-4-0613",
@@ -26,11 +27,7 @@ export const runGpt4Eval = async (
      },
      {
        role: "user",
-        content: `The full output of the simpler message:\n---\n${JSON.stringify(
-          message.content ?? message.function_call,
-          null,
-          2,
-        )}`,
+        content: `The full output of the simpler message:\n---\n${stringifiedOutput}`,
      },
    ],
    function_call: {
@@ -70,15 +67,16 @@ export const runGpt4Eval = async (
 export const runOneEval = async (
  evaluation: Evaluation,
  scenario: TestScenario,
-  modelOutput: ModelOutput,
+  modelResponse: ModelResponse,
+  provider: SupportedProvider,
 ): Promise<{ result: number; details?: string }> => {
-  const output = modelOutput.output as unknown as ChatCompletion;
-
-  const message = output?.choices?.[0]?.message;
+  const modelProvider = modelProviders[provider];
+  const message = modelProvider.normalizeOutput(modelResponse.output);

  if (!message) return { result: 0 };

-  const stringifiedMessage = message.content ?? JSON.stringify(message.function_call);
+  const stringifiedOutput =
+    message.type === "json" ? JSON.stringify(message.value, null, 2) : message.value;

  const matchRegex = escapeRegExp(
    fillTemplate(escapeQuotes(evaluation.value), scenario.variableValues as VariableMap),
@@ -86,10 +84,10 @@ export const runOneEval = async (

  switch (evaluation.evalType) {
    case "CONTAINS":
-      return { result: stringifiedMessage.match(matchRegex) !== null ? 1 : 0 };
+      return { result: stringifiedOutput.match(matchRegex) !== null ? 1 : 0 };
    case "DOES_NOT_CONTAIN":
-      return { result: stringifiedMessage.match(matchRegex) === null ? 1 : 0 };
+      return { result: stringifiedOutput.match(matchRegex) === null ? 1 : 0 };
    case "GPT4_EVAL":
-      return await runGpt4Eval(evaluation, scenario, message);
+      return await runGpt4Eval(evaluation, scenario, stringifiedOutput);
  }
 };
--- a/src/server/utils/userOrg.ts
+++ b/src/server/utils/userOrg.ts
@@ -8,7 +8,7 @@ export default async function userOrg(userId: string) {
    update: {},
    create: {
      personalOrgUserId: userId,
-      OrganizationUser: {
+      organizationUsers: {
        create: {
          userId: userId,
          role: "ADMIN",
--- a/src/theme/ChakraThemeProvider.tsx
+++ b/src/theme/ChakraThemeProvider.tsx
@@ -1,11 +1,31 @@
 import { extendTheme } from "@chakra-ui/react";
 import "@fontsource/inconsolata";
 import { ChakraProvider } from "@chakra-ui/react";
+import { modalAnatomy } from "@chakra-ui/anatomy";
+import { createMultiStyleConfigHelpers } from "@chakra-ui/styled-system";

 const systemFont =
  'ui-sans-serif, -apple-system, "system-ui", "Segoe UI", Helvetica, "Apple Color Emoji", Arial, sans-serif, "Segoe UI Emoji", "Segoe UI Symbol"';

+/* eslint-disable @typescript-eslint/unbound-method */
+const { definePartsStyle, defineMultiStyleConfig } = createMultiStyleConfigHelpers(
+  modalAnatomy.keys,
+);
+
+const modalTheme = defineMultiStyleConfig({
+  baseStyle: definePartsStyle({
+    dialog: { borderRadius: "sm" },
+  }),
+});
+
 const theme = extendTheme({
+  styles: {
+    global: (props: { colorMode: "dark" | "light" }) => ({
+      "html, body": {
+        backgroundColor: props.colorMode === "dark" ? "gray.900" : "white",
+      },
+    }),
+  },
  fonts: {
    heading: systemFont,
    body: systemFont,
@@ -32,6 +52,7 @@ const theme = extendTheme({
        },
      },
    },
+    Modal: modalTheme,
  },
 });

--- a/src/utils/accessControl.ts
+++ b/src/utils/accessControl.ts
@@ -22,7 +22,7 @@ export const canModifyExperiment = async (experimentId: string, userId: string)
    where: {
      id: experimentId,
      organization: {
-        OrganizationUser: {
+        organizationUsers: {
          some: {
            role: { in: [OrganizationUserRole.ADMIN, OrganizationUserRole.MEMBER] },
            userId,
--- a/src/utils/dayjs.ts
+++ b/src/utils/dayjs.ts
@@ -7,3 +7,5 @@ dayjs.extend(relativeTime);

 export const formatTimePast = (date: Date) =>
  dayjs.duration(dayjs(date).diff(dayjs())).humanize(true);
+
+export default dayjs;
--- a/src/utils/generateChannel.ts
+++ b/src/utils/generateChannel.ts
@@ -1,5 +0,0 @@
-// generate random channel id
-
-export const generateChannel = () => {
-  return Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15);
-};
--- a/src/utils/hooks.ts
+++ b/src/utils/hooks.ts
@@ -106,3 +106,9 @@ export const useScenarios = () => {
    { enabled: experiment.data?.id != null },
  );
 };
+
+export const useScenario = (scenarioId: string) => {
+  return api.scenarios.get.useQuery({ id: scenarioId });
+};
+
+export const useVisibleScenarioIds = () => useScenarios().data?.scenarios.map((s) => s.id) ?? [];
--- a/src/utils/utils.ts
+++ b/src/utils/utils.ts
@@ -1,5 +1,12 @@
-import { type Model } from "~/modelProviders/types";
+import frontendModelProviders from "~/modelProviders/frontendModelProviders";
+import { type ProviderModel } from "~/modelProviders/types";

 export const truthyFilter = <T>(x: T | null | undefined): x is T => Boolean(x);

-export const keyForModel = (model: Model) => `${model.provider}/${model.name}`;
+export const lookupModel = (provider: string, model: string) => {
+  const modelObj = frontendModelProviders[provider as ProviderModel["provider"]]?.models[model];
+  return modelObj ? { ...modelObj, provider } : null;
+};
+
+export const modelLabel = (provider: string, model: string) =>
+  `${provider}/${lookupModel(provider, model)?.name ?? model}`;
Author	SHA1	Message	Date
Kyle Corbitt	5945f0ed6b	Update TopNavbar component to include a GitHub button	2023-08-02 10:11:41 -07:00
arcticfly	6bc7d76d15	Update README.md	2023-08-02 00:59:05 -07:00
arcticfly	e9ed173e34	Update README.md	2023-08-02 00:57:24 -07:00
arcticfly	75d58d7021	Update README.md	2023-08-02 00:56:19 -07:00
arcticfly	896c8c5c57	Update README.md	2023-08-02 00:51:57 -07:00
arcticfly	ec5547d0b0	Update README.md with new features and gifs (#110 )	2023-08-02 00:46:48 -07:00
Kyle Corbitt	77e4e3b8c3	mobile styles	2023-08-01 23:08:35 -07:00
Kyle Corbitt	a1b03ddad1	Merge pull request #109 from OpenPipe/debug-prompts Add debug modal for output cells	2023-08-01 22:51:39 -07:00
Kyle Corbitt	6be32bea4c	Add debug modal for output cells See the actual input that a model got for a specific cell. The formatting isn't great right now; should probably iterate on that.	2023-08-01 22:49:38 -07:00
arcticfly	72c70e2a55	Improve conversion to/from Claude (#108 ) * Increase min width of prompt variant * Increase width of custom instructions input * Start recording API docs * Provide better instructions for converting to/from Claude * Fix prettier	2023-08-01 21:03:23 -07:00
arcticfly	026532f2c2	Model selection styling changes (#107 ) * Model selection styling changes * Fix prettier	2023-08-01 18:45:15 -07:00
Kyle Corbitt	f88538336f	fix types	2023-08-01 18:31:34 -07:00
Kyle Corbitt	3c7178115e	Merge pull request #105 from OpenPipe/bump-models Bump Replicate models	2023-08-01 18:26:16 -07:00
Kyle Corbitt	292aaf090a	Merge pull request #106 from OpenPipe/dark-mode Update global background color in ChakraThemeProvider	2023-08-01 18:25:57 -07:00
Kyle Corbitt	d9915dc41b	Update global background color in ChakraThemeProvider	2023-08-01 18:25:29 -07:00
David Corbitt	3560bcff14	Correct time stamps on waiting message	2023-08-01 18:09:23 -07:00
Kyle Corbitt	6982339a1a	Bump Replicate models	2023-08-01 18:08:02 -07:00
arcticfly	d348b130d5	Add navbar to world-champs (#104 ) * Add navbar to world-champs * Move TopNavbar to signup.tsx	2023-08-01 16:59:46 -07:00
Kyle Corbitt	bf67580991	Merge pull request #103 from OpenPipe/world-champs add created_at and updated_at to users	2023-08-01 16:49:11 -07:00
Kyle Corbitt	156f248c3a	add created_at and updated_at to users	2023-08-01 16:43:35 -07:00
Kyle Corbitt	6184498810	Merge pull request #102 from OpenPipe/world-champs world champs signup	2023-08-01 13:09:06 -07:00
Kyle Corbitt	65a76cddc5	world champs signup Basic landing page to sign up for the "world champs"	2023-08-01 13:07:14 -07:00
David Corbitt	c88266bcd4	Add twitter card	2023-08-01 12:47:05 -07:00
David Corbitt	1bf9554eca	Comment out getServerSideProps in [id].tsx	2023-08-01 01:29:28 -07:00
arcticfly	1fb428ef4a	Add scenario editing modal, twitter sentiment seeding (#101 ) * testing agi-eval benchmark * Add scenario modal editor * Add initial values to ScenarioEditorModal * Add seedTwitterSentiment.ts --------- Co-authored-by: Kyle Corbitt <kyle@corbt.com>	2023-08-01 01:26:43 -07:00
Kyle Corbitt	6316eaae6d	dummy key at build time	2023-07-31 18:03:51 -07:00
Kyle Corbitt	8513924ea5	give the openai lib a dummy default value to try to fix the build	2023-07-31 17:39:45 -07:00
arcticfly	51d64baae9	Add og image for experiments (#100 ) * Add og image for experiments * Update meta tags * Fix prettier * Add key to meta tags * Remove ngrok from og:image	2023-07-31 16:50:15 -07:00
arcticfly	26b6fa4f0c	Requeue rate-limited query model tasks (#99 ) * Continue polling stats until all evals complete * Return evaluation changes early, before it has run * Add task for running new eval * requeue rate-limited tasks * Fix prettier	2023-07-26 16:30:50 -07:00
Kyle Corbitt	807665fdc1	Merge pull request #98 from OpenPipe/anthropic Add Anthropic provider	2023-07-26 15:44:39 -07:00
Kyle Corbitt	d6597d2c8a	Add Anthropic provider Adds support for Claude 1 and 2 in experiments! 🎉	2023-07-26 15:37:27 -07:00
arcticfly	566d67bf48	Change prompt variant section background color (#97 ) * Change backgroundColor of variant editor section * Move refresh control out of scrollable area	2023-07-26 10:51:23 -07:00
arcticfly	d4fb8b689a	Ensure evals run properly (#96 ) * Run evals against llama output * Continue polling in OutputCell until evals complete * Remove unnecessary check	2023-07-25 20:01:58 -07:00
arcticfly	98b231c8bd	Store multiple ModelResponses (#95 ) * Store multiple ModelResponses * Fix prettier * Add CellContent container	2023-07-25 18:54:38 -07:00
Kyle Corbitt	45afb1f1f4	Merge pull request #94 from OpenPipe/scenario-shape updated replicate llama2 models	2023-07-25 17:40:05 -07:00
Kyle Corbitt	2bffb03766	updated replicate llama2 models	2023-07-25 16:40:12 -07:00
arcticfly	223b990005	Change experiment button styling (#93 ) * Change button styling * Fix prettier	2023-07-25 11:41:02 -07:00
Kyle Corbitt	fa61c9c472	prettier	2023-07-25 09:24:28 -07:00
Kyle Corbitt	1309a6ec5d	Merge pull request #91 from OpenPipe/better-streaming Better streaming	2023-07-25 06:46:01 -07:00
arcticfly	17a6fd31a5	Update HeaderButtons.tsx marginTop (#92 )	2023-07-24 21:09:08 -07:00
Kyle Corbitt	e1cbeccb90	Better streaming - Always stream the visible scenarios, if the modelProvider supports it - Never stream the invisible scenarios Also actually runs our query tasks in a background worker, which we weren't quite doing before.	2023-07-24 18:34:30 -07:00
arcticfly	d6b97b29f7	Allow experiment forking (#89 ) * Move DeleteButton into a separate file * Rename plural relations * Add ability to fork * Fork automatically after auth upon return * Add experiment card skeleton * Create HeaderButtons component * return no header buttons while experiment loading * Fix prettier * Remove unused variable * Remove newline * Default json values to undefined * Change header styles * Fix prettier * Give AddScenario icon less width * Move useEffect * Skip invalidating experiments list after forking * Require user to be able to view experiment to fork it * Move experiment creation into same transaction * Only return the forked experiment id * Put delete button in experiment settings drawer * Move useEffect hook	2023-07-24 18:10:59 -07:00
Kyle Corbitt	09140f8b5f	Merge pull request #90 from OpenPipe/simplify-model Only pass in model and provider	2023-07-24 17:43:24 -07:00
Kyle Corbitt	9952dd93d8	Only pass in model and provider I got somewhat confused by the extra fields, sorry. Also makes some frontend changes to track that state more directly although in retrospect not sure the frontend changes make things any better.	2023-07-24 17:21:25 -07:00
Kyle Corbitt	e0b457c6c5	Merge pull request #88 from OpenPipe/load-delete Bugfix: show prompts for view-only experiments	2023-07-24 14:58:58 -07:00