Andreas Koepf
27cdd25548
add gemma-3-27b & qwq-32b configs
2025-03-15 20:47:51 +01:00
Andreas Koepf (aider)
f373881c06
feat: Add fallback to first non-None model answer when best_answer is unset
2025-03-15 16:52:50 +01:00
Andreas Köpf
bfa5f8078b
Eval N completions per prompt ( #374 )
...
* feat: Add support for generating multiple completions per prompt
* feat: Track best and mean scores for multiple completions per prompt
* feat: Add checkpoint and resume functionality to evaluation script
2025-03-15 16:39:36 +01:00
Oliver Stanley
bd13b1b92a
Fix chain sum veRL example for latest veRL ( #371 )
...
* fixes for latest verl
* add balance_batch cofg
* 1 -> 2 gpu
* tweaks
* also add raw ids to server script
2025-03-14 20:15:54 +01:00
Adefioye
8a0cacc054
Add jugs curriculum ( #369 )
2025-03-14 18:04:33 +01:00
Andreas Koepf
8c12fe86e2
bump version
2025-03-14 16:21:34 +01:00
Andreas Koepf
79ba55aae3
update gallery
2025-03-14 16:18:43 +01:00
Zafir Stojanovski
f72ecaf411
circuit logic curriculum ( #368 )
2025-03-14 16:17:00 +01:00
Rich Jones
2efe577238
Add curricula status to gallery ( #367 )
...
* Add curricula status to gallery
* lint
2025-03-14 16:16:26 +01:00
Rich Jones
e7d05d6510
GoL-Halt Curricula ( #366 )
...
* GoL-Halt Curricula
* trivial
2025-03-14 16:15:45 +01:00
Zafir Stojanovski
d33c667c3d
feat(env): Propositional Logic Curriculum ( #365 )
...
* propositional logic curriculum
* lint
* difficulty meta
2025-03-14 16:12:39 +01:00
Zafir Stojanovski
41f3ef876c
time intervals curriculum ( #363 )
2025-03-14 16:11:55 +01:00
Oliver Stanley
a71994ad03
add rush hour curriculum ( #362 )
...
* add rush hour curriculum
* add to __init__.py
2025-03-14 16:11:23 +01:00
Oliver Stanley
b5651e5e2c
add word ladder curriculum ( #361 )
...
* add word ladder curriculum
* add to __init__.py
2025-03-14 16:10:52 +01:00
Oliver Stanley
8f8bd9d756
add sudoku curriculum ( #360 )
...
* sudoku curriculum
* add test
* add to __init__.py
2025-03-14 16:09:49 +01:00
joesharratt1229
80ac138455
added family relatinship curriculum ( #359 )
2025-03-14 16:08:49 +01:00
joesharratt1229
fa7d8e66b3
added boxnet generator and curr ( #364 )
2025-03-14 08:48:57 +01:00
Adefioye
5bfe730c3b
Simple equations curriculum ( #340 )
...
* Add curriculum to simple_equations and unit tests
* Add difficulty to metadata
2025-03-13 21:17:14 +01:00
joesharratt1229
6354ca5d35
added rearc curr ( #358 )
2025-03-13 21:14:03 +01:00
Zafir Stojanovski
099ea88402
feat(env): Rubiks Cube Curriculum ( #357 )
2025-03-13 21:12:32 +01:00
joesharratt1229
d603d8b72b
added aiw curric ( #356 )
...
added metadata
2025-03-13 21:10:52 +01:00
Zafir Stojanovski
93793f5416
feat(env): Needle in Haystack Curriculum ( #355 )
...
* needle haystack curriculum
2025-03-13 21:09:45 +01:00
Adefioye
adea7a255e
Add gol curriculum ( #354 )
...
* Add gol curriculum
* Add difficulty
* Make levels of grid size of x and y be valid
2025-03-13 21:09:09 +01:00
Zafir Stojanovski
fa2b04f4de
feat(env): Advanced Geometry Curriculum ( #350 )
...
* advanced geometry curriculum
* metadata
---------
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com >
2025-03-13 21:07:49 +01:00
Zafir Stojanovski
ee001f38a4
fraction simplification curriculum ( #349 )
2025-03-13 21:05:50 +01:00
Zafir Stojanovski
db3868150f
lcm curriculum ( #348 )
2025-03-13 21:05:06 +01:00
Zafir Stojanovski
9fcc277101
color cube rotation curriculum ( #347 )
2025-03-13 21:04:34 +01:00
Adefioye
ec3e414a8c
Cryptarithm curriculum ( #346 )
...
* Add curriculum for cryptarithm
* Add difficulty to metadata
2025-03-13 21:03:57 +01:00
Adefioye
4ec1154b47
Add curriculum to ab dataset ( #345 )
...
* Add curriculum to ab dataset
* Add difficulty to metadata
2025-03-13 21:03:02 +01:00
Zafir Stojanovski
596de511f0
feat(env): Prime Factorization ( #344 )
...
* prime factorization
* lint
2025-03-13 21:02:20 +01:00
Zafir Stojanovski
c81036004f
maze curriculum ( #343 )
2025-03-13 21:01:48 +01:00
Zafir Stojanovski
cb7ae5c58e
simple geometry curriculum ( #342 )
2025-03-13 21:00:05 +01:00
Zafir Stojanovski
7a4b8fc5a8
number formatting curriculum ( #341 )
2025-03-13 20:57:43 +01:00
joesharratt1229
3984d7cdfb
fixed geom bug ( #339 )
2025-03-13 20:55:15 +01:00
Rich Jones
fe0f9ae114
Fix Rush Hour ( #336 )
...
* think it fixes rush hour
* explain walls
2025-03-13 20:42:40 +01:00
Adefioye
cad255e31b
Add polynomial multiplication curriculum ( #338 )
...
* Add polynomial multiplication curriculum and unit tests
* Add difficulty metadata
* Add curriculum class for complex_arithmetic and polynomial_multiplication
2025-03-12 00:03:46 +01:00
Andreas Köpf
c8e77d21a7
fix: Improve error logging and preserve full model response in eval process ( #337 )
2025-03-12 00:01:49 +01:00
joesharratt1229
be75c3aa5f
added intermediate integration ( #334 )
2025-03-11 23:57:51 +01:00
Zafir Stojanovski
aa6ccf1946
number filtering curriculum ( #333 )
2025-03-11 23:56:06 +01:00
Oliver Stanley
f14662e213
Add a few new CodeI/O samples, resolve numeric answer scoring bug ( #332 )
...
* add handful of codeio samples
* scoring fix
2025-03-11 23:55:33 +01:00
vncntt
c3c6cc8051
gcd curriculum ( #331 )
2025-03-11 08:25:24 +01:00
Rich Jones
126eecc798
fix dice ( #330 )
2025-03-11 08:24:32 +01:00
Andreas Koepf
770255b608
fix pre-commit
2025-03-11 08:18:55 +01:00
joesharratt1229
b497e35fb8
added self reference curr ( #329 )
2025-03-11 00:23:26 +01:00
joesharratt1229
54074b17ef
Added zebra curriculum ( #328 )
...
* added zebra curriculum
* added metadata
2025-03-11 00:22:54 +01:00
Zafir Stojanovski
f204a848d9
spell backward curriculum ( #327 )
...
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com >
2025-03-11 00:22:28 +01:00
Zafir Stojanovski
a23c8c3d4e
sentence reordering curriculum ( #326 )
2025-03-11 00:21:41 +01:00
joesharratt1229
3c39cbda40
added sokoban dataset ( #325 )
2025-03-11 00:21:03 +01:00
joesharratt1229
91b3347d53
added quantum lock curriculum ( #324 )
2025-03-11 00:20:24 +01:00
joesharratt1229
e9944149bd
added tsumego curric ( #323 )
2025-03-11 00:19:55 +01:00
