alihan/llm-promptfoo
1
0
Fork 0
mirror of https://github.com/promptfoo/promptfoo.git synced 2023-08-15 01:10:51 +03:00
Code Issues Packages Projects Releases Wiki Activity
343 Commits 14 Branches 40 Tags
main
Commit Graph

61 Commits

Author SHA1 Message Date
dependabot[bot]
4bb5415b0a Bump certifi from 2023.5.7 to 2023.7.22 in /examples/langchain-python (#104) 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2023.5.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2023.05.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 13:00:47 -07:00
Ian Webster
1b2eb8970f Update requirements.txt of langchain example 2023-08-09 08:21:25 -07:00
Ian Webster
9f3213c407 formatting 2023-08-08 11:37:27 -07:00
Ian Webster
bdd1deadb5 fix error with provider map parsing 2023-08-07 16:15:03 -07:00
Ian Webster
747d6f3434 Add assertions in webview (#100) 2023-08-07 16:14:55 -07:00
Paul
2e40da26bf testing providers w/ same model diff params (#83) 2023-08-07 12:15:12 -07:00
Ian Webster
50a5c75943 Add postprocessing example to function call test case. #95 2023-08-07 09:44:58 -07:00
Ian Webster
aca0821924 Theories -> Scenarios and load scenarios from default config 2023-07-27 22:49:22 -07:00
Skylertodd
2187908b44 Promptfoo Theories (#89) 2023-07-27 22:46:33 -07:00
Ian Webster
5aff43afcf Add support for native function ApiProviders and Assertions (#93) 2023-07-27 07:34:34 -07:00
Ian Webster
c8a8bb25cf Add support for loading test cases from file/directory path (#88) 2023-07-23 21:46:38 -07:00
Ian Webster
445a90788b bump example python package version 2023-07-21 20:16:02 -07:00
Ian Webster
9b7e40d947 formatting 2023-07-21 09:58:23 -07:00
Ian Webster
b3900a165c checkin llama gpt comparison example 2023-07-20 09:44:39 -07:00
Ian Webster
57fa43f038 replicate improvements 2023-07-19 09:54:08 -07:00
Ian Webster
6a3af6b1f4 ReplicateProvider implementation (#75) 2023-07-19 09:45:43 -07:00
Ian Webster
fd949b7ddb Add colab notebook example 2023-07-14 22:39:34 -07:00
Ian Webster
7e4a1f770d yaml formatting 2023-07-14 21:53:15 -07:00
Ian Webster
51c254fe3a Add support for YAML-formatted OpenAI prompts (#67) 2023-07-13 17:44:36 -07:00
Ian Webster
af994ffbc7 formatting 2023-07-13 17:27:38 -07:00
Ian Webster
dc543b1e0e bump langchain version in example 2023-07-09 22:57:09 -07:00
Ian Webster
f5b2d050f5 update langchain example to compare with gpt-4 2023-07-07 09:06:49 -07:00
Ian Webster
db3a962d6a Improve support for running external scripts as providers (#55) 
* run external scripts with `exec:` provider prefix
* escape ansi characters
* support line breaks in results
 2023-07-06 23:12:22 -07:00
Ian Webster
1e2c3205d8 formatting 2023-06-15 17:30:08 -07:00
Ian Webster
b9f5ce0813 fix node package example import path 2023-06-15 16:42:22 -07:00
Ian Webster
47976bb6a4 Add support for OpenAI functions and custom provider args (#34) 2023-06-15 16:33:55 -07:00
Ian Webster
f5e0e38518 Add multishot example. Closes #38 2023-06-15 16:30:49 -07:00
Ian Webster
62561b552e Fix reference parsing for yaml files 2023-06-10 11:28:19 -07:00
Ian Webster
0fd9542046 Add support for var arrays (#21) 2023-06-04 18:46:19 -07:00
Ian Webster
dc10ffad84 show assertion template example 2023-05-31 10:22:47 -04:00
Ian Webster
46fddad1b2 add test for readTests 2023-05-30 10:48:18 -04:00
Ian Webster
df900c33e1 add support for google sheets loading 2023-05-30 10:33:30 -04:00
Ian Webster
bf81260b44 Simplify API and add support for unified test suite definition (#14) 2023-05-30 06:02:49 -07:00
Ian Webster
34fa1d1282 Improve caching and cache all requests by default (#17) 
* Improve caching and cache requests by default

* disable cache in test

* add tests

* update readme

* improve cache configurability

* Report cache stats
 2023-05-28 06:56:55 -07:00
Ian Webster
fdb2365121 Add glob support (#13) 2023-05-27 15:24:17 -07:00
MentalGear
d71fa1062d feat: rename eval: to fn: for custum js fnc tests (#11) 2023-05-26 17:10:03 -07:00
Ian Webster
67fd82e0b3 fix typo in jest example 2023-05-24 10:19:01 -07:00
Ian Webster
3833768a13 Create README.md 2023-05-23 11:00:47 -07:00
Ian Webster
00d9aa2926 Add jest integration example 2023-05-23 10:05:10 -07:00
Ian Webster
b1d98ab747 bail on esm pain 2023-05-22 22:57:05 -07:00
Ian Webster
398f4b0ebe improve word wrapping in cli output 2023-05-22 07:40:14 -07:00
Ian Webster
f9662b54fc Add promptfooconfig.js to simple-cli example 2023-05-19 17:14:43 -07:00
Ian Webster
a83765fe7c Browser UI for eval results (#5) 2023-05-13 08:04:29 -07:00
Ian Webster
644abf9a08 update grading prompt example 2023-05-07 11:09:40 -07:00
Ian Webster
d28e605005 Ability for LLM to grade its own outputs (#4) 
Adds a --grader option. Expected values with "grade:" prefix send outputs to an LLM for evaluation.
 2023-05-07 09:28:42 -07:00
Ian Webster
e390678aad Ability to set OPENAI_API_HOST. #3 2023-05-06 23:55:34 -07:00
Ian Webster
2c3f489519 Add test results within the matrix, instead of in a column 2023-05-06 21:55:21 -07:00
Ian Webster
c6a3a5914c Add init command 2023-05-03 17:28:06 -07:00
Ian Webster
4f6b6e2a1a Add custom provider example 2023-05-03 16:39:02 -07:00
Ian Webster
a86d534412 Add support for expected value 2023-05-03 12:05:26 -07:00