69 Commits

Author	SHA1	Message	Date
Ian Webster	4e6b580f7b	Add ollama provider (#102)	2023-08-08 16:30:55 -07:00
Ian Webster	508ab41159	Disable nunjucks autoescaping by default (#101)	2023-08-08 11:43:42 -07:00
Ian Webster	747d6f3434	Add assertions in webview (#100)	2023-08-07 16:14:55 -07:00
Ian Webster	690729c3c8	Add support for llama.cpp server (#94)	2023-07-30 20:13:10 -07:00
Ian Webster	aca0821924	Theories -> Scenarios and load scenarios from default config	2023-07-27 22:49:22 -07:00
Skylertodd	2187908b44	Promptfoo Theories (#89)	2023-07-27 22:46:33 -07:00
Ian Webster	5aff43afcf	Add support for native function ApiProviders and Assertions (#93)	2023-07-27 07:34:34 -07:00
Ian Webster	c8a8bb25cf	Add support for loading test cases from file/directory path (#88)	2023-07-23 21:46:38 -07:00
Ian Webster	67ed3d6f4d	Record latency in results (#85)	2023-07-22 10:20:42 -07:00
Ian Webster	6a9b5ac284	Add support for postprocessing in test cases (#84)	2023-07-21 20:38:49 -07:00
Ian Webster	9b7e40d947	formatting	2023-07-21 09:58:23 -07:00
Ian Webster	59b1df1b72	Add support for provider-specific prompts in the test suite configuration. (#76)	2023-07-20 08:31:32 -07:00
Ian Webster	f3de0e48e2	Add confirmation for promptfoo share	2023-07-14 12:56:20 -07:00
Paul	d15dbc9464	Add Azure OpenAI Provider (#66) ... * update local command azure providers other naming updates Add azure providers * unit tests	2023-07-13 17:47:17 -07:00
Ian Webster	af994ffbc7	formatting	2023-07-13 17:27:38 -07:00
Ian Webster	ed2589e2ae	Add starts-with assertion type (#64)	2023-07-13 10:25:21 -07:00
Ian Webster	ddef314dd6	Add telemetry notice (#39)	2023-07-09 23:16:00 -07:00
Kavan McEachern	88989a21df	Add Anthropic provider (#58)	2023-07-09 11:16:15 -07:00
Ian Webster	cc01c1f64b	Add ability to view raw prompts in web viewer (#54)	2023-07-05 22:04:03 -07:00
Ian Webster	463d79439a	fix tests	2023-06-29 17:18:15 -07:00
Ian Webster	73524bd65d	Add support for var objects that can be manipulated in nunjucks templates (#50)	2023-06-28 08:39:31 -07:00
Ian Webster	3d0998d622	Add retries for failed HTTP requests (#47)	2023-06-26 08:11:18 -07:00
Ian Webster	96843f3ed8	Add support for continuous scoring (#44)	2023-06-24 11:57:34 -07:00
Ian Webster	04daaa780b	fix tests	2023-06-18 16:07:35 -07:00
Ian Webster	dffb091d4c	Make files in config yaml resolve from config location, not working directory	2023-06-17 12:18:46 -07:00
Ian Webster	ecf79a42a5	add --no-table cli option	2023-06-17 10:02:54 -07:00
Ian Webster	a5c045bf32	Add JSONL prompt file support (#40)	2023-06-15 18:27:15 -07:00
Ian Webster	47976bb6a4	Add support for OpenAI functions and custom provider args (#34)	2023-06-15 16:33:55 -07:00
Ian Webster	5b4350e918	Add check for latest package version (#31)	2023-06-11 16:53:08 -07:00
Ian Webster	7f8be158a7	add csv support for rouge-n, webhook assert types	2023-06-11 15:14:58 -07:00
Ian Webster	c50f8ab9f5	formatting	2023-06-11 14:29:35 -07:00
Ian Webster	7e7e3ead29	add telemetry support	2023-06-11 14:10:36 -07:00
Ian Webster	0c2f5d4c8c	Add support for rouge NLP metric	2023-06-10 15:37:24 -07:00
Ian Webster	d2b093ab1e	Add webhook assert type	2023-06-10 13:16:38 -07:00
Ian Webster	1e4df7e760	pass context object with vars in custom assert function	2023-06-10 10:15:51 -07:00
Ian Webster	df94f024e5	make assertionFromString support new assertion types	2023-06-09 23:49:54 -07:00
Ian Webster	cc5fef1b1f	make handling of not- assertions generic	2023-06-09 23:32:34 -07:00
Ian Webster	8061bf0990	Add many new assertion types (#30)	2023-06-09 23:14:48 -07:00
Ian Webster	b9eede737a	Add support for named prompts (#28)	2023-06-09 11:15:18 -07:00
MentalGear	086a53c495	chore: add native ts-node compatibility (#25) ... by removing file extensions from import statements (file extension on import statements are optional)	2023-06-08 17:20:53 -07:00
Ian Webster	0fd9542046	Add support for var arrays (#21)	2023-06-04 18:46:19 -07:00
Ian Webster	fef0ea4503	fix cache test	2023-05-30 12:11:26 -04:00
Ian Webster	46fddad1b2	add test for readTests	2023-05-30 10:48:18 -04:00
Ian Webster	df900c33e1	add support for google sheets loading	2023-05-30 10:33:30 -04:00
Ian Webster	bf81260b44	Simplify API and add support for unified test suite definition (#14)	2023-05-30 06:02:49 -07:00
Ian Webster	34fa1d1282	Improve caching and cache all requests by default (#17) ... * Improve caching and cache requests by default * disable cache in test * add tests * update readme * improve cache configurability * Report cache stats	2023-05-28 06:56:55 -07:00
Ian Webster	fdb2365121	Add glob support (#13)	2023-05-27 15:24:17 -07:00
MentalGear	d71fa1062d	feat: rename eval: to fn: for custum js fnc tests (#11)	2023-05-26 17:10:03 -07:00
Ian Webster	3d74acd634	Add support for grading by semantic similarity (#7)	2023-05-22 09:42:29 -07:00
Ian Webster	9d46fbb6bb	fix tests	2023-05-22 08:43:59 -07:00