move app to app/ subdir
This commit is contained in:
Kyle Corbitt
84
app/prisma/datasets/validated_tweets.csv
Normal file
84
app/prisma/datasets/validated_tweets.csv
Normal file
@@ -0,0 +1,84 @@
|
||||
Text,sentiment,emotion
|
||||
@dell your customer service is horrible especially agent syedfaisal who has made this experience of purchasing a new computer downright awful and I’ll reconsider ever buying a Dell in the future @DellTech,negative,anger
|
||||
@zacokalo @Dell @DellCares @Dell give the man what he paid for!,neutral,anger
|
||||
"COOKING STREAM DAY!!! Ty to @Alienware for sponsoring this stream! I’ll be making a bunch of Japanese Alien themed foods hehe
|
||||
|
||||
Come check it out! https://t.co/m06tJQ06zk
|
||||
|
||||
#alienwarepartner #intelgaming @Dell @IntelGaming https://t.co/qOdQX2E8VD",positive,joy
|
||||
@emijuju_ @Alienware @Dell @intel Beautiful 😍❤️😻,positive,joy
|
||||
"What's your biggest data management challenge? • Cloud complexity? • Lengthy tech refresh cycles? • Capital budget constraints? Solve your challenges with as-a-Storage. Get simplicity, agility & control with @Dell #APEX. https://t.co/mCblMtH931 https://t.co/eepKNZ4Ai3",neutral,optimism
|
||||
"This week we were at the ""Top Gun"" themed @Dell Product Expo. Eddie Muñoz met Maverick look-alike, California Tom Cruise (Jerome LeBlanc)!
|
||||
|
||||
""I feel the need, the need for speed."" - Maverick
|
||||
#topgun #topgunmaverick #dell #delltechnologies #lockncharge https://t.co/QHYH2EbMjq",positive,joy
|
||||
"Itsss been more than a week...i m following up with dell for troubleshootings...my https://t.co/lWhg2YKhQa suffering so as my hard earned money...hightly disappointed...contd..
|
||||
@DellCares @Dell",negative,sadness
|
||||
"@ashu_k7 @Dell Pathetic!!!!! I Dont mind taking legal action, this is deficency of service for which the customer is nt getting help..",negative,anger
|
||||
@ashu_k7 @Dell Making life unhappy is the new tag line of #Dell,negative,sadness
|
||||
"@Dell If you are buying a Dell, make sure you are making your life hell.
|
||||
Better buy other laptops. If you wanted to opt for Dell better opt for garbage on the streets.",negative,anger
|
||||
"MY DESK'S FINAL FORM? Seriously, I'm finally happy with my monitor setup here... and I'll keep this setup whenever I move... FOREVER. What do you think?
|
||||
https://t.co/WJZ2JXtOnX
|
||||
@Alienware @Dell cheers. https://t.co/6Whhldfpv0",positive,joy
|
||||
"@Dell Dell Alienware computer has had software problems with SupportAssist since purchase. Dell, despite paying for Premium Support, has never fixed issues. Latest solution was to erase everything and reload....SupportAssist still doesn't work.",negative,anger
|
||||
"HUGE congratulations to Startup Battle 3.0 winner ➡️ @Ox_Fulfillment x @cyborgcharu for being featured in @BusinessInsider & @Dell showcasing the journey at Ox! 🚀🚀🚀
|
||||
|
||||
We love to see our portfolio companies continuing to BUILD SOMETHING FROM NOTHING! 🔥 https://t.co/awBkn5ippB",positive,joy
|
||||
@Dell happy Friday!,positive,joy
|
||||
"@intel Core i5 1135G7 - 4732 points
|
||||
@intel Core i5 1235 - 6619 points
|
||||
@Dell Latitude 5420 x 5430.
|
||||
Cinebench R23. Good job Intel!",positive,joy
|
||||
@Dell india we purchased 52 docking station and we have around 100 users using dell laptop as well as dell monitor now they are refusing to replace my faulty product and disconnecting my every call....,negative,anger
|
||||
"It's another year ans another day But cant fill it in yet the child hood dreams.
|
||||
It's my birthdy today. Can anyone of you guys bless me with a simplest gaming oc that can run
|
||||
@DOTA2 ?
|
||||
@Dell @HP @VastGG @Acer @Alienware @Lenovo @toshiba @IBM @Fujitsu_Global @NEC https://t.co/69G8tL9sN8",neutral,joy
|
||||
"@idoccor @Dell That's always the decision—wait, or, look elsewhere. In this case, I think I unfortunately need to wait since there are only two monitors with these specs and I don't like the other one 😂",negative,sadness
|
||||
"@MichaelDell @Dell @DellCares For how long this will continue. It is high time you either fix the problem for good or replace the complete laptop. Spent over 60+ hours with Customer Care teams, which is not helping. Cannot keep going on like this.",negative,anger
|
||||
"@Dell @DellCares but no, not really",neutral,sadness
|
||||
"Business innovation requires insight, agility and efficiency. How do you get there? RP PRO, LLC recommends starting by proactively managing IT infrastructure with #OpenManage Systems from @Dell. https://t.co/fBcK1lfFMu https://t.co/xWHLkkHCjn",neutral,optimism
|
||||
@Dell Yessirrrrr #NationalCoffeeDay,positive,joy
|
||||
"New blog post from @Dell shared on https://t.co/EgfPChB8AT
|
||||
|
||||
Re-routing Our Connected and Autonomous Future https://t.co/AW8EHQrbd6
|
||||
|
||||
#future #futuretech #techinnovation https://t.co/koX8stKPsr",neutral,joy
|
||||
"In a free-market economy, the folks @IronMountain can set prices as they see fit. Their customers are also free to find better prices at competitors like @Dell
|
||||
@H3CGlobal @HPE
|
||||
https://t.co/reZ56DNTBI",neutral,optimism
|
||||
"Delighted to chat with many of our partners here in person at @Intel Innovation! @Dell, @Lenovo, @Supermicro_SMCI, @QuantaQCT #IntelON https://t.co/BxIeGW8deN",positive,joy
|
||||
"A special gracias to our Startup Chica San Antonio 2022 sponsors @eBay, @jcpenney, @Barbie, @HEB, @Dell, @Honda, @SouthsideSATX💜✨ https://t.co/lZ6WWkziHl",positive,joy
|
||||
"When your team decides to start supporting developers, your #ops must change too. More from @cote and @Dell Developer Community Manager @barton808: https://t.co/W6f1oMiTgV",neutral,optimism
|
||||
@EmDStowers @LASERGIANT1 @ohwormongod @Ludovician_Vega @Dell our boy snitchin,neutral,anger
|
||||
A 1st place dmi:Design Value Award goes to @Dell for a packaging modernization initiative that helped them get closer to their corporate Moonshot Sustainability Goal of 100% recycled or renewable packaging by 2030. More at https://t.co/dnhZWWLCQC #designvalue #DVA22,positive,optimism
|
||||
Reducing deployment and maintenance complexity is the goal behind @dell and @WindRiver's new collaboration. https://t.co/2PxQgPuHUU,positive,optimism
|
||||
@jaserhunter @Dell Love the sales pitch lol,positive,joy
|
||||
@Dell india we purchased 52 docking station and we have around 100 users using dell laptop as well as dell monitor now they are refusing to replace my faulty product and disconnecting my every call....,negative,anger
|
||||
@ashu_k7 @Dell One more example.. their technical support is also worse. https://t.co/20atSgI4fg,negative,anger
|
||||
*angry screeches about @Dell proprietary MBR windows 8.1 partitions not being able to save as an img in clonezilla *,negative,anger
|
||||
@socialitebooks @BBYC_Gamers @Dell @Alienware @BestBuyCanada @intelcanada Congratulations!!!,positive,joy
|
||||
"Thank you to the @dell team for coming out to volunteer today! We truly appreciate your hard work and look forward to seeing you again soon!
|
||||
|
||||
If you and your team are interested in helping out at the UMLAUF, visit our website for more information: https://t.co/lVfsZT2ogS https://t.co/eLz0FY0y4M",positive,joy
|
||||
"@TheCaramelGamer @intel @bravadogaming @Intel_Africa @Dell @DellTech @DellTechMEA @Alienware @IntelUK we love to see it.
|
||||
|
||||
Also also actually actually whoever did that artwork? 🔥🔥🔥 am a fan.",positive,joy
|
||||
"LOVING MY DELL 2 IN 1 LAPTOP
|
||||
YAYY 🥳🥳
|
||||
@Dell #DellInspiron #DellLaptop https://t.co/vib96jf3tC",positive,joy
|
||||
@Azure @OracleItalia @AWS_Italy @lenovoitalia @Dell discussing the future of #HPC during the #hpcroundtable22 in Turin today #highperformancecomputing https://t.co/jJ1WqBulPF,neutral,joy
|
||||
Attracting talent @AmericanChamber. @marg_cola @Dell speaks of quality of life connectivity and the Opportunity for development being so crucial. Housing availability is now impacting on decision making for potential candidates. #WhyCork,positive,optimism
|
||||
.@Dell partners with @WindRiver on modular cloud-native telecommunications infrastructure https://t.co/4SWATspwCP @SiliconANGLE @Mike_Wheatley @holgermu @constellationr,neutral,joy
|
||||
@Dell Not buy Dell Inspiron laptop,neutral,sadness
|
||||
"@dell #delltechforum reminding us IDC have predicted that by 2024, 50% of everything we consume in technology will be as a service https://t.co/3UBiZJX0LE",neutral,optimism
|
||||
@RachMurph @HETTShow @Dell Thank you for coming! Great evening,positive,joy
|
||||
Congratulations to Jason M of Moncton NB on winning a @Dell @Alienware m15 R7 15.6″ gaming laptop from @BestBuyCanada and @intelcanada's gaming days #contest on the blog. Visit https://t.co/VryaY5Rvv9 to learn about tech and for chances to win new tech. https://t.co/T6n0dzF6oL,positive,joy
|
||||
@MattVisiwig @Dell Sour taste for sure 😶 But don't let ego distract you from what you really want to buy 😁,neutral,optimism
|
||||
"Massive thank you goes to sponsors @HendersonLoggie @lindsaysnews @Dell @unity, all of our fantastic judges and mentors and the team at @EGX and @ExCeLLondon.
|
||||
|
||||
Big congratulations also to all of our other @AbertayDare teams - an amazing year! #Dare2022 https://t.co/jYe4agO7lW",positive,joy
|
||||
"@timetcetera @rahaug Nah, I just need @Dell to start paying me comissions 😂",neutral,joy
|
||||
"""Whether you’re an engineer, a designer, or work in supply chain management or sales, there are always opportunities to think about sustainability and how you can do things more efficiently."" 👏 — Oliver Campbell, Director of Packaging Engineering, @Dell https://t.co/vUJLTWNFwP https://t.co/GJWAzGfAxJ",positive,optimism
|
||||
"Hi, my name is @listerepvp and I support @Dell, always.",positive,joy
|
||||
|
156
app/prisma/migrations/20230701015916_init/migration.sql
Normal file
156
app/prisma/migrations/20230701015916_init/migration.sql
Normal file
@@ -0,0 +1,156 @@
|
||||
-- CreateTable
|
||||
CREATE TABLE "Experiment" (
|
||||
"id" UUID NOT NULL,
|
||||
"label" TEXT NOT NULL,
|
||||
"sortIndex" INTEGER NOT NULL DEFAULT 0,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "Experiment_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "PromptVariant" (
|
||||
"id" UUID NOT NULL,
|
||||
"label" TEXT NOT NULL,
|
||||
"config" JSONB NOT NULL,
|
||||
"uiId" UUID NOT NULL,
|
||||
"visible" BOOLEAN NOT NULL DEFAULT true,
|
||||
"sortIndex" INTEGER NOT NULL DEFAULT 0,
|
||||
"experimentId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "PromptVariant_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "TestScenario" (
|
||||
"id" UUID NOT NULL,
|
||||
"variableValues" JSONB NOT NULL,
|
||||
"uiId" UUID NOT NULL,
|
||||
"visible" BOOLEAN NOT NULL DEFAULT true,
|
||||
"sortIndex" INTEGER NOT NULL DEFAULT 0,
|
||||
"experimentId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "TestScenario_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "TemplateVariable" (
|
||||
"id" UUID NOT NULL,
|
||||
"label" TEXT NOT NULL,
|
||||
"experimentId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "TemplateVariable_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "ModelOutput" (
|
||||
"id" UUID NOT NULL,
|
||||
"inputHash" TEXT NOT NULL,
|
||||
"output" JSONB NOT NULL,
|
||||
"statusCode" INTEGER NOT NULL,
|
||||
"errorMessage" TEXT,
|
||||
"promptVariantId" UUID NOT NULL,
|
||||
"testScenarioId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "ModelOutput_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Account" (
|
||||
"id" TEXT NOT NULL,
|
||||
"userId" TEXT NOT NULL,
|
||||
"type" TEXT NOT NULL,
|
||||
"provider" TEXT NOT NULL,
|
||||
"providerAccountId" TEXT NOT NULL,
|
||||
"refresh_token" TEXT,
|
||||
"access_token" TEXT,
|
||||
"expires_at" INTEGER,
|
||||
"token_type" TEXT,
|
||||
"scope" TEXT,
|
||||
"id_token" TEXT,
|
||||
"session_state" TEXT,
|
||||
|
||||
CONSTRAINT "Account_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Session" (
|
||||
"id" TEXT NOT NULL,
|
||||
"sessionToken" TEXT NOT NULL,
|
||||
"userId" TEXT NOT NULL,
|
||||
"expires" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "Session_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "User" (
|
||||
"id" TEXT NOT NULL,
|
||||
"name" TEXT,
|
||||
"email" TEXT,
|
||||
"emailVerified" TIMESTAMP(3),
|
||||
"image" TEXT,
|
||||
|
||||
CONSTRAINT "User_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "VerificationToken" (
|
||||
"identifier" TEXT NOT NULL,
|
||||
"token" TEXT NOT NULL,
|
||||
"expires" TIMESTAMP(3) NOT NULL
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX "PromptVariant_uiId_idx" ON "PromptVariant"("uiId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX "ModelOutput_inputHash_idx" ON "ModelOutput"("inputHash");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "ModelOutput_promptVariantId_testScenarioId_key" ON "ModelOutput"("promptVariantId", "testScenarioId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "Account_provider_providerAccountId_key" ON "Account"("provider", "providerAccountId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "Session_sessionToken_key" ON "Session"("sessionToken");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "User_email_key" ON "User"("email");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "VerificationToken_token_key" ON "VerificationToken"("token");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "VerificationToken_identifier_token_key" ON "VerificationToken"("identifier", "token");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "PromptVariant" ADD CONSTRAINT "PromptVariant_experimentId_fkey" FOREIGN KEY ("experimentId") REFERENCES "Experiment"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "TestScenario" ADD CONSTRAINT "TestScenario_experimentId_fkey" FOREIGN KEY ("experimentId") REFERENCES "Experiment"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "TemplateVariable" ADD CONSTRAINT "TemplateVariable_experimentId_fkey" FOREIGN KEY ("experimentId") REFERENCES "Experiment"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "ModelOutput" ADD CONSTRAINT "ModelOutput_promptVariantId_fkey" FOREIGN KEY ("promptVariantId") REFERENCES "PromptVariant"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "ModelOutput" ADD CONSTRAINT "ModelOutput_testScenarioId_fkey" FOREIGN KEY ("testScenarioId") REFERENCES "TestScenario"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Account" ADD CONSTRAINT "Account_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Session" ADD CONSTRAINT "Session_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,2 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "ModelOutput" ADD COLUMN "timeToComplete" INTEGER NOT NULL DEFAULT 0;
|
||||
@@ -0,0 +1,3 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "ModelOutput" ADD COLUMN "completionTokens" INTEGER,
|
||||
ADD COLUMN "promptTokens" INTEGER;
|
||||
@@ -0,0 +1,40 @@
|
||||
-- CreateEnum
|
||||
CREATE TYPE "EvaluationMatchType" AS ENUM ('CONTAINS', 'DOES_NOT_CONTAIN');
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Evaluation" (
|
||||
"id" UUID NOT NULL,
|
||||
"name" TEXT NOT NULL,
|
||||
"matchString" TEXT NOT NULL,
|
||||
"matchType" "EvaluationMatchType" NOT NULL,
|
||||
"experimentId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "Evaluation_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "EvaluationResult" (
|
||||
"id" UUID NOT NULL,
|
||||
"passCount" INTEGER NOT NULL,
|
||||
"failCount" INTEGER NOT NULL,
|
||||
"evaluationId" UUID NOT NULL,
|
||||
"promptVariantId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "EvaluationResult_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "EvaluationResult_evaluationId_promptVariantId_key" ON "EvaluationResult"("evaluationId", "promptVariantId");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Evaluation" ADD CONSTRAINT "Evaluation_experimentId_fkey" FOREIGN KEY ("experimentId") REFERENCES "Experiment"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "EvaluationResult" ADD CONSTRAINT "EvaluationResult_evaluationId_fkey" FOREIGN KEY ("evaluationId") REFERENCES "Evaluation"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "EvaluationResult" ADD CONSTRAINT "EvaluationResult_promptVariantId_fkey" FOREIGN KEY ("promptVariantId") REFERENCES "PromptVariant"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,15 @@
|
||||
-- 1. Add a nullable constructFn column
|
||||
ALTER TABLE "PromptVariant"
|
||||
ADD COLUMN "constructFn" TEXT;
|
||||
|
||||
-- 2. Populate constructFn based on the config column
|
||||
UPDATE "PromptVariant"
|
||||
SET "constructFn" = 'prompt = ' || "config"::text;
|
||||
|
||||
-- 3. Remove the config column
|
||||
ALTER TABLE "PromptVariant"
|
||||
DROP COLUMN "config";
|
||||
|
||||
-- 4. Make constructFn not null
|
||||
ALTER TABLE "PromptVariant"
|
||||
ALTER COLUMN "constructFn" SET NOT NULL;
|
||||
@@ -0,0 +1,49 @@
|
||||
-- Drop the foreign key constraints on the original ModelOutput
|
||||
ALTER TABLE "ModelOutput" DROP CONSTRAINT "ModelOutput_promptVariantId_fkey";
|
||||
ALTER TABLE "ModelOutput" DROP CONSTRAINT "ModelOutput_testScenarioId_fkey";
|
||||
|
||||
-- Rename the old table
|
||||
ALTER TABLE "ModelOutput" RENAME TO "ScenarioVariantCell";
|
||||
ALTER TABLE "ScenarioVariantCell" RENAME CONSTRAINT "ModelOutput_pkey" TO "ScenarioVariantCell_pkey";
|
||||
ALTER INDEX "ModelOutput_inputHash_idx" RENAME TO "ScenarioVariantCell_inputHash_idx";
|
||||
ALTER INDEX "ModelOutput_promptVariantId_testScenarioId_key" RENAME TO "ScenarioVariantCell_promptVariantId_testScenarioId_key";
|
||||
|
||||
-- Add the new fields to the renamed table
|
||||
ALTER TABLE "ScenarioVariantCell" ADD COLUMN "retryTime" TIMESTAMP(3);
|
||||
ALTER TABLE "ScenarioVariantCell" ADD COLUMN "streamingChannel" TEXT;
|
||||
ALTER TABLE "ScenarioVariantCell" ALTER COLUMN "inputHash" DROP NOT NULL;
|
||||
ALTER TABLE "ScenarioVariantCell" ALTER COLUMN "output" DROP NOT NULL,
|
||||
ALTER COLUMN "statusCode" DROP NOT NULL,
|
||||
ALTER COLUMN "timeToComplete" DROP NOT NULL;
|
||||
|
||||
-- Create the new table
|
||||
CREATE TABLE "ModelOutput" (
|
||||
"id" UUID NOT NULL,
|
||||
"inputHash" TEXT NOT NULL,
|
||||
"output" JSONB NOT NULL,
|
||||
"timeToComplete" INTEGER NOT NULL DEFAULT 0,
|
||||
"promptTokens" INTEGER,
|
||||
"completionTokens" INTEGER,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
"scenarioVariantCellId" UUID
|
||||
);
|
||||
|
||||
-- Move inputHash index
|
||||
DROP INDEX "ScenarioVariantCell_inputHash_idx";
|
||||
CREATE INDEX "ModelOutput_inputHash_idx" ON "ModelOutput"("inputHash");
|
||||
|
||||
CREATE UNIQUE INDEX "ModelOutput_scenarioVariantCellId_key" ON "ModelOutput"("scenarioVariantCellId");
|
||||
ALTER TABLE "ModelOutput" ADD CONSTRAINT "ModelOutput_scenarioVariantCellId_fkey" FOREIGN KEY ("scenarioVariantCellId") REFERENCES "ScenarioVariantCell"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
ALTER TABLE "ModelOutput" ALTER COLUMN "scenarioVariantCellId" SET NOT NULL,
|
||||
ADD CONSTRAINT "ModelOutput_pkey" PRIMARY KEY ("id");
|
||||
|
||||
ALTER TABLE "ScenarioVariantCell" ADD CONSTRAINT "ScenarioVariantCell_promptVariantId_fkey" FOREIGN KEY ("promptVariantId") REFERENCES "PromptVariant"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
ALTER TABLE "ScenarioVariantCell" ADD CONSTRAINT "ScenarioVariantCell_testScenarioId_fkey" FOREIGN KEY ("testScenarioId") REFERENCES "TestScenario"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- CreateEnum
|
||||
CREATE TYPE "CellRetrievalStatus" AS ENUM ('PENDING', 'IN_PROGRESS', 'COMPLETE', 'ERROR');
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "ScenarioVariantCell" ADD COLUMN "retrievalStatus" "CellRetrievalStatus" NOT NULL DEFAULT 'COMPLETE';
|
||||
@@ -0,0 +1,2 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "PromptVariant" ADD COLUMN "model" TEXT NOT NULL DEFAULT 'gpt-3.5-turbo';
|
||||
@@ -0,0 +1,2 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "PromptVariant" ALTER COLUMN "model" DROP DEFAULT;
|
||||
@@ -0,0 +1,24 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to rename the column `matchString` on the `Evaluation` table. If there is any code or views referring to the old name, they will break.
|
||||
- You are about to rename the column `matchType` on the `Evaluation` table. If there is any code or views referring to the old name, they will break.
|
||||
- You are about to rename the column `name` on the `Evaluation` table. If there is any code or views referring to the old name, they will break.
|
||||
- You are about to rename the enum `EvaluationMatchType` to `EvalType`. If there is any code or views referring to the old name, they will break.
|
||||
*/
|
||||
|
||||
-- RenameEnum
|
||||
ALTER TYPE "EvaluationMatchType" RENAME TO "EvalType";
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "Evaluation" RENAME COLUMN "matchString" TO "value";
|
||||
ALTER TABLE "Evaluation" RENAME COLUMN "matchType" TO "evalType";
|
||||
ALTER TABLE "Evaluation" RENAME COLUMN "name" TO "label";
|
||||
|
||||
-- AlterColumnType
|
||||
ALTER TABLE "Evaluation" ALTER COLUMN "evalType" TYPE "EvalType" USING "evalType"::text::"EvalType";
|
||||
|
||||
-- SetNotNullConstraint
|
||||
ALTER TABLE "Evaluation" ALTER COLUMN "evalType" SET NOT NULL;
|
||||
ALTER TABLE "Evaluation" ALTER COLUMN "label" SET NOT NULL;
|
||||
ALTER TABLE "Evaluation" ALTER COLUMN "value" SET NOT NULL;
|
||||
@@ -0,0 +1,39 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the `EvaluationResult` table. If the table is not empty, all the data it contains will be lost.
|
||||
|
||||
*/
|
||||
-- AlterEnum
|
||||
ALTER TYPE "EvalType" ADD VALUE 'GPT4_EVAL';
|
||||
|
||||
-- DropForeignKey
|
||||
ALTER TABLE "EvaluationResult" DROP CONSTRAINT "EvaluationResult_evaluationId_fkey";
|
||||
|
||||
-- DropForeignKey
|
||||
ALTER TABLE "EvaluationResult" DROP CONSTRAINT "EvaluationResult_promptVariantId_fkey";
|
||||
|
||||
-- DropTable
|
||||
DROP TABLE "EvaluationResult";
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "OutputEvaluation" (
|
||||
"id" UUID NOT NULL,
|
||||
"result" DOUBLE PRECISION NOT NULL,
|
||||
"details" TEXT,
|
||||
"modelOutputId" UUID NOT NULL,
|
||||
"evaluationId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "OutputEvaluation_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "OutputEvaluation_modelOutputId_evaluationId_key" ON "OutputEvaluation"("modelOutputId", "evaluationId");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "OutputEvaluation" ADD CONSTRAINT "OutputEvaluation_modelOutputId_fkey" FOREIGN KEY ("modelOutputId") REFERENCES "ModelOutput"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "OutputEvaluation" ADD CONSTRAINT "OutputEvaluation_evaluationId_fkey" FOREIGN KEY ("evaluationId") REFERENCES "Evaluation"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,124 @@
|
||||
DROP TABLE "Account";
|
||||
DROP TABLE "Session";
|
||||
DROP TABLE "User";
|
||||
DROP TABLE "VerificationToken";
|
||||
|
||||
CREATE TYPE "OrganizationUserRole" AS ENUM ('ADMIN', 'MEMBER', 'VIEWER');
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Organization" (
|
||||
"id" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
"personalOrgUserId" UUID,
|
||||
|
||||
CONSTRAINT "Organization_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "OrganizationUser" (
|
||||
"id" UUID NOT NULL,
|
||||
"role" "OrganizationUserRole" NOT NULL,
|
||||
"organizationId" UUID NOT NULL,
|
||||
"userId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "OrganizationUser_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Account" (
|
||||
"id" UUID NOT NULL,
|
||||
"userId" UUID NOT NULL,
|
||||
"type" TEXT NOT NULL,
|
||||
"provider" TEXT NOT NULL,
|
||||
"providerAccountId" TEXT NOT NULL,
|
||||
"refresh_token" TEXT,
|
||||
"refresh_token_expires_in" INTEGER,
|
||||
"access_token" TEXT,
|
||||
"expires_at" INTEGER,
|
||||
"token_type" TEXT,
|
||||
"scope" TEXT,
|
||||
"id_token" TEXT,
|
||||
"session_state" TEXT,
|
||||
|
||||
CONSTRAINT "Account_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "Session" (
|
||||
"id" UUID NOT NULL,
|
||||
"sessionToken" TEXT NOT NULL,
|
||||
"userId" UUID NOT NULL,
|
||||
"expires" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "Session_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "User" (
|
||||
"id" UUID NOT NULL,
|
||||
"name" TEXT,
|
||||
"email" TEXT,
|
||||
"emailVerified" TIMESTAMP(3),
|
||||
"image" TEXT,
|
||||
|
||||
CONSTRAINT "User_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "VerificationToken" (
|
||||
"identifier" TEXT NOT NULL,
|
||||
"token" TEXT NOT NULL,
|
||||
"expires" TIMESTAMP(3) NOT NULL
|
||||
);
|
||||
|
||||
INSERT INTO "Organization" ("id", "updatedAt") VALUES ('11111111-1111-1111-1111-111111111111', CURRENT_TIMESTAMP);
|
||||
|
||||
-- AlterTable add organizationId as NULLABLE
|
||||
ALTER TABLE "Experiment" ADD COLUMN "organizationId" UUID;
|
||||
|
||||
-- Set default organization for existing experiments
|
||||
UPDATE "Experiment" SET "organizationId" = '11111111-1111-1111-1111-111111111111';
|
||||
|
||||
-- AlterTable set organizationId as NOT NULL
|
||||
ALTER TABLE "Experiment" ALTER COLUMN "organizationId" SET NOT NULL;
|
||||
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "OrganizationUser_organizationId_userId_key" ON "OrganizationUser"("organizationId", "userId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "Account_provider_providerAccountId_key" ON "Account"("provider", "providerAccountId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "Session_sessionToken_key" ON "Session"("sessionToken");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "User_email_key" ON "User"("email");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "VerificationToken_token_key" ON "VerificationToken"("token");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "VerificationToken_identifier_token_key" ON "VerificationToken"("identifier", "token");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Experiment" ADD CONSTRAINT "Experiment_organizationId_fkey" FOREIGN KEY ("organizationId") REFERENCES "Organization"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "OrganizationUser" ADD CONSTRAINT "OrganizationUser_organizationId_fkey" FOREIGN KEY ("organizationId") REFERENCES "Organization"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "OrganizationUser" ADD CONSTRAINT "OrganizationUser_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Account" ADD CONSTRAINT "Account_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Session" ADD CONSTRAINT "Session_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
CREATE UNIQUE INDEX "Organization_personalOrgUserId_key" ON "Organization"("personalOrgUserId");
|
||||
|
||||
ALTER TABLE "Organization" ADD CONSTRAINT "Organization_personalOrgUserId_fkey" FOREIGN KEY ("personalOrgUserId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,16 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the column `completionTokens` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
- You are about to drop the column `inputHash` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
- You are about to drop the column `output` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
- You are about to drop the column `promptTokens` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
- You are about to drop the column `timeToComplete` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
|
||||
*/
|
||||
-- AlterTable
|
||||
ALTER TABLE "ScenarioVariantCell" DROP COLUMN "completionTokens",
|
||||
DROP COLUMN "inputHash",
|
||||
DROP COLUMN "output",
|
||||
DROP COLUMN "promptTokens",
|
||||
DROP COLUMN "timeToComplete";
|
||||
@@ -0,0 +1,8 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the column `model` on the `PromptVariant` table. All the data in the column will be lost.
|
||||
|
||||
*/
|
||||
-- AlterTable
|
||||
ALTER TABLE "ModelOutput" ADD COLUMN "cost" DOUBLE PRECISION;
|
||||
@@ -0,0 +1,17 @@
|
||||
-- Add new columns allowing NULL values
|
||||
ALTER TABLE "PromptVariant"
|
||||
ADD COLUMN "constructFnVersion" INTEGER,
|
||||
ADD COLUMN "modelProvider" TEXT;
|
||||
|
||||
-- Update existing records to have the default values
|
||||
UPDATE "PromptVariant"
|
||||
SET "constructFnVersion" = 1,
|
||||
"modelProvider" = 'openai/ChatCompletion'
|
||||
WHERE "constructFnVersion" IS NULL OR "modelProvider" IS NULL;
|
||||
|
||||
-- Alter table to set NOT NULL constraint
|
||||
ALTER TABLE "PromptVariant"
|
||||
ALTER COLUMN "constructFnVersion" SET NOT NULL,
|
||||
ALTER COLUMN "modelProvider" SET NOT NULL;
|
||||
|
||||
ALTER TABLE "ScenarioVariantCell" ADD COLUMN "prompt" JSONB;
|
||||
@@ -0,0 +1,8 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the column `streamingChannel` on the `ScenarioVariantCell` table. All the data in the column will be lost.
|
||||
|
||||
*/
|
||||
-- AlterTable
|
||||
ALTER TABLE "ScenarioVariantCell" DROP COLUMN "streamingChannel";
|
||||
@@ -0,0 +1,52 @@
|
||||
-- DropForeignKey
|
||||
ALTER TABLE "ModelOutput" DROP CONSTRAINT "ModelOutput_scenarioVariantCellId_fkey";
|
||||
|
||||
-- DropForeignKey
|
||||
ALTER TABLE "OutputEvaluation" DROP CONSTRAINT "OutputEvaluation_modelOutputId_fkey";
|
||||
|
||||
-- DropIndex
|
||||
DROP INDEX "OutputEvaluation_modelOutputId_evaluationId_key";
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "OutputEvaluation" RENAME COLUMN "modelOutputId" TO "modelResponseId";
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "ScenarioVariantCell" DROP COLUMN "retryTime",
|
||||
DROP COLUMN "statusCode",
|
||||
ADD COLUMN "jobQueuedAt" TIMESTAMP(3),
|
||||
ADD COLUMN "jobStartedAt" TIMESTAMP(3);
|
||||
|
||||
ALTER TABLE "ModelOutput" RENAME TO "ModelResponse";
|
||||
|
||||
ALTER TABLE "ModelResponse"
|
||||
ADD COLUMN "requestedAt" TIMESTAMP(3),
|
||||
ADD COLUMN "receivedAt" TIMESTAMP(3),
|
||||
ADD COLUMN "statusCode" INTEGER,
|
||||
ADD COLUMN "errorMessage" TEXT,
|
||||
ADD COLUMN "retryTime" TIMESTAMP(3),
|
||||
ADD COLUMN "outdated" BOOLEAN NOT NULL DEFAULT false;
|
||||
|
||||
-- 3. Remove the unnecessary column
|
||||
ALTER TABLE "ModelResponse"
|
||||
DROP COLUMN "timeToComplete";
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "ModelResponse" RENAME CONSTRAINT "ModelOutput_pkey" TO "ModelResponse_pkey";
|
||||
ALTER TABLE "ModelResponse" ALTER COLUMN "output" DROP NOT NULL;
|
||||
|
||||
-- DropIndex
|
||||
DROP INDEX "ModelOutput_scenarioVariantCellId_key";
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "ModelResponse" ADD CONSTRAINT "ModelResponse_scenarioVariantCellId_fkey" FOREIGN KEY ("scenarioVariantCellId") REFERENCES "ScenarioVariantCell"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- RenameIndex
|
||||
ALTER INDEX "ModelOutput_inputHash_idx" RENAME TO "ModelResponse_inputHash_idx";
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "OutputEvaluation_modelResponseId_evaluationId_key" ON "OutputEvaluation"("modelResponseId", "evaluationId");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "OutputEvaluation" ADD CONSTRAINT "OutputEvaluation_modelResponseId_fkey" FOREIGN KEY ("modelResponseId") REFERENCES "ModelResponse"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
-- CreateTable
|
||||
CREATE TABLE "WorldChampEntrant" (
|
||||
"id" UUID NOT NULL,
|
||||
"userId" UUID NOT NULL,
|
||||
"approved" BOOLEAN NOT NULL DEFAULT false,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "WorldChampEntrant_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "WorldChampEntrant_userId_key" ON "WorldChampEntrant"("userId");
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "WorldChampEntrant" ADD CONSTRAINT "WorldChampEntrant_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,3 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "User" ADD COLUMN "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
ADD COLUMN "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP;
|
||||
@@ -0,0 +1,5 @@
|
||||
-- CreateEnum
|
||||
CREATE TYPE "UserRole" AS ENUM ('ADMIN', 'USER');
|
||||
|
||||
-- AlterTable
|
||||
ALTER TABLE "User" ADD COLUMN "role" "UserRole" NOT NULL DEFAULT 'USER';
|
||||
@@ -0,0 +1,28 @@
|
||||
-- CreateTable
|
||||
CREATE TABLE "Dataset" (
|
||||
"id" UUID NOT NULL,
|
||||
"name" TEXT NOT NULL,
|
||||
"organizationId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "Dataset_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "DatasetEntry" (
|
||||
"id" UUID NOT NULL,
|
||||
"input" TEXT NOT NULL,
|
||||
"output" TEXT,
|
||||
"datasetId" UUID NOT NULL,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "DatasetEntry_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "Dataset" ADD CONSTRAINT "Dataset_organizationId_fkey" FOREIGN KEY ("organizationId") REFERENCES "Organization"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "DatasetEntry" ADD CONSTRAINT "DatasetEntry_datasetId_fkey" FOREIGN KEY ("datasetId") REFERENCES "Dataset"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
@@ -0,0 +1,13 @@
|
||||
/*
|
||||
Warnings:
|
||||
|
||||
- You are about to drop the column `constructFn` on the `PromptVariant` table. All the data in the column will be lost.
|
||||
- You are about to drop the column `constructFnVersion` on the `PromptVariant` table. All the data in the column will be lost.
|
||||
- Added the required column `promptConstructor` to the `PromptVariant` table without a default value. This is not possible if the table is not empty.
|
||||
- Added the required column `promptConstructorVersion` to the `PromptVariant` table without a default value. This is not possible if the table is not empty.
|
||||
|
||||
*/
|
||||
-- AlterTable
|
||||
|
||||
ALTER TABLE "PromptVariant" RENAME COLUMN "constructFn" TO "promptConstructor";
|
||||
ALTER TABLE "PromptVariant" RENAME COLUMN "constructFnVersion" TO "promptConstructorVersion";
|
||||
3
app/prisma/migrations/migration_lock.toml
Normal file
3
app/prisma/migrations/migration_lock.toml
Normal file
@@ -0,0 +1,3 @@
|
||||
# Please do not edit this file manually
|
||||
# It should be added in your version-control system (i.e. Git)
|
||||
provider = "postgresql"
|
||||
309
app/prisma/schema.prisma
Normal file
309
app/prisma/schema.prisma
Normal file
@@ -0,0 +1,309 @@
|
||||
// This is your Prisma schema file,
|
||||
// learn more about it in the docs: https://pris.ly/d/prisma-schema
|
||||
|
||||
generator client {
|
||||
provider = "prisma-client-js"
|
||||
}
|
||||
|
||||
datasource db {
|
||||
provider = "postgresql"
|
||||
url = env("DATABASE_URL")
|
||||
}
|
||||
|
||||
model Experiment {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
label String
|
||||
|
||||
sortIndex Int @default(0)
|
||||
|
||||
organizationId String @db.Uuid
|
||||
organization Organization? @relation(fields: [organizationId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
templateVariables TemplateVariable[]
|
||||
promptVariants PromptVariant[]
|
||||
testScenarios TestScenario[]
|
||||
evaluations Evaluation[]
|
||||
}
|
||||
|
||||
model PromptVariant {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
label String
|
||||
promptConstructor String
|
||||
promptConstructorVersion Int
|
||||
model String
|
||||
modelProvider String
|
||||
|
||||
uiId String @default(uuid()) @db.Uuid
|
||||
visible Boolean @default(true)
|
||||
sortIndex Int @default(0)
|
||||
|
||||
experimentId String @db.Uuid
|
||||
experiment Experiment @relation(fields: [experimentId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
scenarioVariantCells ScenarioVariantCell[]
|
||||
|
||||
@@index([uiId])
|
||||
}
|
||||
|
||||
model TestScenario {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
variableValues Json
|
||||
|
||||
uiId String @default(uuid()) @db.Uuid
|
||||
visible Boolean @default(true)
|
||||
sortIndex Int @default(0)
|
||||
|
||||
experimentId String @db.Uuid
|
||||
experiment Experiment @relation(fields: [experimentId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
scenarioVariantCells ScenarioVariantCell[]
|
||||
}
|
||||
|
||||
model TemplateVariable {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
label String
|
||||
|
||||
experimentId String @db.Uuid
|
||||
experiment Experiment @relation(fields: [experimentId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
|
||||
enum CellRetrievalStatus {
|
||||
PENDING
|
||||
IN_PROGRESS
|
||||
COMPLETE
|
||||
ERROR
|
||||
}
|
||||
|
||||
model ScenarioVariantCell {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
retrievalStatus CellRetrievalStatus @default(COMPLETE)
|
||||
jobQueuedAt DateTime?
|
||||
jobStartedAt DateTime?
|
||||
modelResponses ModelResponse[]
|
||||
errorMessage String? // Contains errors that occurred independently of model responses
|
||||
|
||||
promptVariantId String @db.Uuid
|
||||
promptVariant PromptVariant @relation(fields: [promptVariantId], references: [id], onDelete: Cascade)
|
||||
prompt Json?
|
||||
|
||||
testScenarioId String @db.Uuid
|
||||
testScenario TestScenario @relation(fields: [testScenarioId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@unique([promptVariantId, testScenarioId])
|
||||
}
|
||||
|
||||
model ModelResponse {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
inputHash String
|
||||
requestedAt DateTime?
|
||||
receivedAt DateTime?
|
||||
output Json?
|
||||
cost Float?
|
||||
promptTokens Int?
|
||||
completionTokens Int?
|
||||
statusCode Int?
|
||||
errorMessage String?
|
||||
retryTime DateTime?
|
||||
outdated Boolean @default(false)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
scenarioVariantCellId String @db.Uuid
|
||||
scenarioVariantCell ScenarioVariantCell @relation(fields: [scenarioVariantCellId], references: [id], onDelete: Cascade)
|
||||
outputEvaluations OutputEvaluation[]
|
||||
|
||||
@@index([inputHash])
|
||||
}
|
||||
|
||||
enum EvalType {
|
||||
CONTAINS
|
||||
DOES_NOT_CONTAIN
|
||||
GPT4_EVAL
|
||||
}
|
||||
|
||||
model Evaluation {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
label String
|
||||
evalType EvalType
|
||||
value String
|
||||
|
||||
experimentId String @db.Uuid
|
||||
experiment Experiment @relation(fields: [experimentId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
outputEvaluations OutputEvaluation[]
|
||||
}
|
||||
|
||||
model OutputEvaluation {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
// Number between 0 (fail) and 1 (pass)
|
||||
result Float
|
||||
details String?
|
||||
|
||||
modelResponseId String @db.Uuid
|
||||
modelResponse ModelResponse @relation(fields: [modelResponseId], references: [id], onDelete: Cascade)
|
||||
|
||||
evaluationId String @db.Uuid
|
||||
evaluation Evaluation @relation(fields: [evaluationId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@unique([modelResponseId, evaluationId])
|
||||
}
|
||||
|
||||
model Dataset {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
name String
|
||||
datasetEntries DatasetEntry[]
|
||||
|
||||
organizationId String @db.Uuid
|
||||
organization Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
|
||||
model DatasetEntry {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
input String
|
||||
output String?
|
||||
|
||||
datasetId String @db.Uuid
|
||||
dataset Dataset? @relation(fields: [datasetId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
|
||||
model Organization {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
personalOrgUserId String? @unique @db.Uuid
|
||||
PersonalOrgUser User? @relation(fields: [personalOrgUserId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
organizationUsers OrganizationUser[]
|
||||
experiments Experiment[]
|
||||
datasets Dataset[]
|
||||
}
|
||||
|
||||
enum OrganizationUserRole {
|
||||
ADMIN
|
||||
MEMBER
|
||||
VIEWER
|
||||
}
|
||||
|
||||
model OrganizationUser {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
role OrganizationUserRole
|
||||
|
||||
organizationId String @db.Uuid
|
||||
organization Organization? @relation(fields: [organizationId], references: [id], onDelete: Cascade)
|
||||
|
||||
userId String @db.Uuid
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@unique([organizationId, userId])
|
||||
}
|
||||
|
||||
model WorldChampEntrant {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
|
||||
userId String @db.Uuid
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
|
||||
approved Boolean @default(false)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@unique([userId])
|
||||
}
|
||||
|
||||
model Account {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
userId String @db.Uuid
|
||||
type String
|
||||
provider String
|
||||
providerAccountId String
|
||||
refresh_token String? @db.Text
|
||||
refresh_token_expires_in Int?
|
||||
access_token String? @db.Text
|
||||
expires_at Int?
|
||||
token_type String?
|
||||
scope String?
|
||||
id_token String? @db.Text
|
||||
session_state String?
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@unique([provider, providerAccountId])
|
||||
}
|
||||
|
||||
model Session {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
sessionToken String @unique
|
||||
userId String @db.Uuid
|
||||
expires DateTime
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
}
|
||||
|
||||
enum UserRole {
|
||||
ADMIN
|
||||
USER
|
||||
}
|
||||
|
||||
model User {
|
||||
id String @id @default(uuid()) @db.Uuid
|
||||
name String?
|
||||
email String? @unique
|
||||
emailVerified DateTime?
|
||||
image String?
|
||||
|
||||
role UserRole @default(USER)
|
||||
|
||||
accounts Account[]
|
||||
sessions Session[]
|
||||
organizationUsers OrganizationUser[]
|
||||
organizations Organization[]
|
||||
worldChampEntrant WorldChampEntrant?
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @default(now()) @updatedAt
|
||||
}
|
||||
|
||||
model VerificationToken {
|
||||
identifier String
|
||||
token String @unique
|
||||
expires DateTime
|
||||
|
||||
@@unique([identifier, token])
|
||||
}
|
||||
169
app/prisma/seed.ts
Normal file
169
app/prisma/seed.ts
Normal file
@@ -0,0 +1,169 @@
|
||||
import { prisma } from "~/server/db";
|
||||
import dedent from "dedent";
|
||||
import { generateNewCell } from "~/server/utils/generateNewCell";
|
||||
import { promptConstructorVersion } from "~/promptConstructor/version";
|
||||
|
||||
const defaultId = "11111111-1111-1111-1111-111111111111";
|
||||
|
||||
await prisma.organization.deleteMany({
|
||||
where: { id: defaultId },
|
||||
});
|
||||
|
||||
// If there's an existing org, just seed into it
|
||||
const org =
|
||||
(await prisma.organization.findFirst({})) ??
|
||||
(await prisma.organization.create({
|
||||
data: { id: defaultId },
|
||||
}));
|
||||
|
||||
await prisma.experiment.deleteMany({
|
||||
where: {
|
||||
id: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.experiment.create({
|
||||
data: {
|
||||
id: defaultId,
|
||||
label: "Country Capitals Example",
|
||||
organizationId: org.id,
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.scenarioVariantCell.deleteMany({
|
||||
where: {
|
||||
promptVariant: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.promptVariant.deleteMany({
|
||||
where: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.promptVariant.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: defaultId,
|
||||
label: "Prompt Variant 1",
|
||||
sortIndex: 0,
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
modelProvider: "openai/ChatCompletion",
|
||||
promptConstructorVersion,
|
||||
promptConstructor: dedent`
|
||||
definePrompt("openai/ChatCompletion", {
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: \`What is the capital of ${"$"}{scenario.country}?\`
|
||||
}
|
||||
],
|
||||
temperature: 0,
|
||||
})`,
|
||||
},
|
||||
{
|
||||
experimentId: defaultId,
|
||||
label: "Prompt Variant 2",
|
||||
sortIndex: 1,
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
modelProvider: "openai/ChatCompletion",
|
||||
promptConstructorVersion,
|
||||
promptConstructor: dedent`
|
||||
definePrompt("openai/ChatCompletion", {
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: \`What is the capital of ${"$"}{scenario.country}? Return just the city name and nothing else.\`
|
||||
}
|
||||
],
|
||||
temperature: 0,
|
||||
})`,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await prisma.templateVariable.deleteMany({
|
||||
where: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
await prisma.templateVariable.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: defaultId,
|
||||
label: "country",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await prisma.testScenario.deleteMany({
|
||||
where: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
const countries = [
|
||||
"Afghanistan",
|
||||
"Albania",
|
||||
"Algeria",
|
||||
"Andorra",
|
||||
"Angola",
|
||||
"Antigua and Barbuda",
|
||||
"Argentina",
|
||||
"Armenia",
|
||||
"Australia",
|
||||
"Austria",
|
||||
"Austrian Empire",
|
||||
"Azerbaijan",
|
||||
"Baden",
|
||||
"Bahamas, The",
|
||||
"Bahrain",
|
||||
"Bangladesh",
|
||||
"Barbados",
|
||||
"Bavaria",
|
||||
"Belarus",
|
||||
"Belgium",
|
||||
"Belize",
|
||||
"Benin (Dahomey)",
|
||||
"Bolivia",
|
||||
"Bosnia and Herzegovina",
|
||||
"Botswana",
|
||||
];
|
||||
await prisma.testScenario.createMany({
|
||||
data: countries.map((country, i) => ({
|
||||
experimentId: defaultId,
|
||||
sortIndex: i,
|
||||
variableValues: {
|
||||
country: country,
|
||||
},
|
||||
})),
|
||||
});
|
||||
|
||||
const variants = await prisma.promptVariant.findMany({
|
||||
where: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
const scenarios = await prisma.testScenario.findMany({
|
||||
where: {
|
||||
experimentId: defaultId,
|
||||
},
|
||||
});
|
||||
|
||||
await Promise.all(
|
||||
variants
|
||||
.flatMap((variant) =>
|
||||
scenarios.map((scenario) => ({
|
||||
promptVariantId: variant.id,
|
||||
testScenarioId: scenario.id,
|
||||
})),
|
||||
)
|
||||
.map((cell) => generateNewCell(cell.promptVariantId, cell.testScenarioId, { stream: false })),
|
||||
);
|
||||
128
app/prisma/seedAgiEval.ts
Normal file
128
app/prisma/seedAgiEval.ts
Normal file
@@ -0,0 +1,128 @@
|
||||
import { prisma } from "~/server/db";
|
||||
import { generateNewCell } from "~/server/utils/generateNewCell";
|
||||
import dedent from "dedent";
|
||||
import { execSync } from "child_process";
|
||||
import fs from "fs";
|
||||
import { promptConstructorVersion } from "~/promptConstructor/version";
|
||||
|
||||
const defaultId = "11111111-1111-1111-1111-111111111112";
|
||||
|
||||
await prisma.organization.deleteMany({
|
||||
where: { id: defaultId },
|
||||
});
|
||||
|
||||
// If there's an existing org, just seed into it
|
||||
const org =
|
||||
(await prisma.organization.findFirst({})) ??
|
||||
(await prisma.organization.create({
|
||||
data: { id: defaultId },
|
||||
}));
|
||||
|
||||
// Clone the repo from git@github.com:microsoft/AGIEval.git into a tmp dir if it doesn't exist
|
||||
const tmpDir = "/tmp/agi-eval";
|
||||
if (!fs.existsSync(tmpDir)) {
|
||||
execSync(`git clone git@github.com:microsoft/AGIEval.git ${tmpDir}`);
|
||||
}
|
||||
|
||||
const datasets = [
|
||||
"sat-en",
|
||||
"sat-math",
|
||||
"lsat-rc",
|
||||
"lsat-ar",
|
||||
"aqua-rat",
|
||||
"logiqa-en",
|
||||
"lsat-lr",
|
||||
"math",
|
||||
];
|
||||
|
||||
type Scenario = {
|
||||
passage: string | null;
|
||||
question: string;
|
||||
options: string[] | null;
|
||||
label: string;
|
||||
};
|
||||
|
||||
for (const dataset of datasets) {
|
||||
const experimentName = `AGI-Eval: ${dataset}`;
|
||||
const oldExperiment = await prisma.experiment.findFirst({
|
||||
where: {
|
||||
label: experimentName,
|
||||
organizationId: org.id,
|
||||
},
|
||||
});
|
||||
if (oldExperiment) {
|
||||
await prisma.experiment.deleteMany({
|
||||
where: { id: oldExperiment.id },
|
||||
});
|
||||
}
|
||||
|
||||
const experiment = await prisma.experiment.create({
|
||||
data: {
|
||||
id: oldExperiment?.id ?? undefined,
|
||||
label: experimentName,
|
||||
organizationId: org.id,
|
||||
},
|
||||
});
|
||||
|
||||
const scenarios: Scenario[] = fs
|
||||
.readFileSync(`${tmpDir}/data/v1/${dataset}.jsonl`, "utf8")
|
||||
.split("\n")
|
||||
.filter((line) => line.length > 0)
|
||||
.map((line) => JSON.parse(line) as Scenario);
|
||||
console.log("scenarios", scenarios.length);
|
||||
|
||||
await prisma.testScenario.createMany({
|
||||
data: scenarios.slice(0, 30).map((scenario, i) => ({
|
||||
experimentId: experiment.id,
|
||||
sortIndex: i,
|
||||
variableValues: {
|
||||
passage: scenario.passage,
|
||||
question: scenario.question,
|
||||
options: scenario.options?.join("\n"),
|
||||
label: scenario.label,
|
||||
},
|
||||
})),
|
||||
});
|
||||
|
||||
await prisma.templateVariable.createMany({
|
||||
data: ["passage", "question", "options", "label"].map((label) => ({
|
||||
experimentId: experiment.id,
|
||||
label,
|
||||
})),
|
||||
});
|
||||
|
||||
await prisma.promptVariant.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: experiment.id,
|
||||
label: "Prompt Variant 1",
|
||||
sortIndex: 0,
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
modelProvider: "openai/ChatCompletion",
|
||||
promptConstructorVersion,
|
||||
promptConstructor: dedent`
|
||||
definePrompt("openai/ChatCompletion", {
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: \`Passage: ${"$"}{scenario.passage}\n\nQuestion: ${"$"}{scenario.question}\n\nOptions: ${"$"}{scenario.options}\n\n Respond with just the letter of the best option in the format Answer: (A).\`
|
||||
}
|
||||
],
|
||||
temperature: 0,
|
||||
})`,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await prisma.evaluation.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: experiment.id,
|
||||
label: "Eval",
|
||||
evalType: "CONTAINS",
|
||||
value: "Answer: ({{label}})",
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
114
app/prisma/seedTwitterSentiment.ts
Normal file
114
app/prisma/seedTwitterSentiment.ts
Normal file
@@ -0,0 +1,114 @@
|
||||
import { prisma } from "~/server/db";
|
||||
import dedent from "dedent";
|
||||
import fs from "fs";
|
||||
import { parse } from "csv-parse/sync";
|
||||
import { promptConstructorVersion } from "~/promptConstructor/version";
|
||||
|
||||
const defaultId = "11111111-1111-1111-1111-111111111112";
|
||||
|
||||
await prisma.organization.deleteMany({
|
||||
where: { id: defaultId },
|
||||
});
|
||||
|
||||
// If there's an existing org, just seed into it
|
||||
const org =
|
||||
(await prisma.organization.findFirst({})) ??
|
||||
(await prisma.organization.create({
|
||||
data: { id: defaultId },
|
||||
}));
|
||||
|
||||
type Scenario = {
|
||||
text: string;
|
||||
sentiment: string;
|
||||
emotion: string;
|
||||
};
|
||||
|
||||
const experimentName = `Twitter Sentiment Analysis`;
|
||||
const oldExperiment = await prisma.experiment.findFirst({
|
||||
where: {
|
||||
label: experimentName,
|
||||
organizationId: org.id,
|
||||
},
|
||||
});
|
||||
if (oldExperiment) {
|
||||
await prisma.experiment.deleteMany({
|
||||
where: { id: oldExperiment.id },
|
||||
});
|
||||
}
|
||||
|
||||
const experiment = await prisma.experiment.create({
|
||||
data: {
|
||||
id: oldExperiment?.id ?? undefined,
|
||||
label: experimentName,
|
||||
organizationId: org.id,
|
||||
},
|
||||
});
|
||||
|
||||
const content = fs.readFileSync("./prisma/datasets/validated_tweets.csv", "utf8");
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const records: any[] = parse(content, { delimiter: ",", from_line: 2 });
|
||||
|
||||
console.log("records", records);
|
||||
|
||||
const scenarios: Scenario[] = records.map((row) => ({
|
||||
text: row[0],
|
||||
sentiment: row[1],
|
||||
emotion: row[2],
|
||||
}));
|
||||
|
||||
console.log("scenarios", scenarios.length);
|
||||
|
||||
await prisma.testScenario.createMany({
|
||||
data: scenarios.slice(0, 30).map((scenario, i) => ({
|
||||
experimentId: experiment.id,
|
||||
sortIndex: i,
|
||||
variableValues: {
|
||||
text: scenario.text,
|
||||
sentiment: scenario.sentiment,
|
||||
emotion: scenario.emotion,
|
||||
},
|
||||
})),
|
||||
});
|
||||
|
||||
await prisma.templateVariable.createMany({
|
||||
data: ["text", "sentiment", "emotion"].map((label) => ({
|
||||
experimentId: experiment.id,
|
||||
label,
|
||||
})),
|
||||
});
|
||||
|
||||
await prisma.promptVariant.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: experiment.id,
|
||||
label: "Prompt Variant 1",
|
||||
sortIndex: 0,
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
modelProvider: "openai/ChatCompletion",
|
||||
promptConstructorVersion,
|
||||
promptConstructor: dedent`
|
||||
definePrompt("openai/ChatCompletion", {
|
||||
model: "gpt-3.5-turbo-0613",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: \`Text: ${"$"}{scenario.text}\n\nRespond with the sentiment (negative|neutral|positive) and emotion (optimism|joy|anger|sadness) of the tweet in this format: "answer: <sentiment>-<emotion>".\`
|
||||
}
|
||||
],
|
||||
temperature: 0,
|
||||
})`,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
await prisma.evaluation.createMany({
|
||||
data: [
|
||||
{
|
||||
experimentId: experiment.id,
|
||||
label: "Eval",
|
||||
evalType: "CONTAINS",
|
||||
value: "answer: {{sentiment}}-{{emotion}}",
|
||||
},
|
||||
],
|
||||
});
|
||||
Reference in New Issue
Block a user