16 Commits

Author SHA1 Message Date
magmueller
0b61750a1c Ability to read files 2025-02-04 22:57:06 -08:00
Sean Gu
c2b1b98d20 fix: Execute initial actions when rerun history 2025-01-27 21:00:08 +08:00
magmueller
7711946199 End of file line 2025-01-19 17:16:53 -08:00
magmueller
e7edb94932 Update gitignore 2025-01-19 17:15:56 -08:00
magmueller
7358fa1a03 How to start added 2 points 2025-01-19 10:42:57 -08:00
magmueller
3e2a333d75 Ignore gifs 2024-12-18 20:00:50 +01:00
magmueller
89c6f4fc10 Ignore AgentHistoryList 2024-12-05 10:13:16 +01:00
magmueller
0fa9c93f1e Less agents parallel 2024-12-03 16:20:25 +01:00
magmueller
2a077e1900 Ignore update 2024-11-30 13:32:54 +01:00
magmueller
8512f2e42d Update how we import buildDomTree 2024-11-29 08:12:48 +01:00
magmueller
0f9fd1c78d Include private examples in git ignore 2024-11-24 11:47:13 +01:00
magmueller
60fbd16730 When page not loaded return empty state - fixes break for not loading page 2024-11-23 15:39:47 +01:00
Gregor Žunič
89c63fdd63 Added custom actions registry and fixed extraction layer (#20)
* Validator

* Test mind2web

* Cleaned up logger

* Pytest logger

* Cleaned up logger

* Disable flag for human input

* Multiple clicks per button

* Multiple clicks per button

* More structured system prompt

* Fields with description

* System prompt example

* One logger

* Cleaner logging

* Log step in step function

* Fix critical clicking error - wrong argument used

* Improved thought process of agent

* Improve system prompt

* Remove human input message

* Custome action registration

* Pydantic model for custom actions

* Pydantic model for custome output

* Runs through, model outputs functions, but not called yet

* Work in progress - description for custome actions

* Description works, but schema not yet

* Model can call the right action - but is not executed

* Seperate is_controller_action  and is_custom_action

* Works! Model can call custom function

* Use registry for action, but result is not feed back to model

* Include result in messages

* Works with custom function - but typing is not correct

* Renamed registry

* First test cases

* Captcha tests

* Pydantic for tests

* Improve prompts for multy step

* System prompt structure

* Handle errors like validation error

* Refactor error handling in agent

* Refactor error handling in agent

* Improved logging

* Update view

* Fix click parameter to index

* Simplify dynamic actions

* Use run instead of step

* Rename history

* Rename AgentService to Agent

* Rename ControllerService to Controller

* Pytest file

* Rename get state

* Rename BrowserService

* reversed dom extraction recursion to while

* Rename use_vision

* Rename use_vision

* reversed dom tree items and made browser less anoying

* Renaming and fixing type errors

* Renamed class names for agent

* updated requirements

* Update prompt

* Action registration works for user and controller

* Fix done call by returning ActionResult

* Fix if result is none

* Rename AgentOutput and ActionModel

* Improved prompt Passes 6/8 tests from test_agent_actions

* Calculate token cost

* Improve display

* Simplified logger

* Test function calling

* created super simple xpath extraction algo

* Tests logging

* tiny fixes to dom extraction

* Remove test

* Dont log number of clicks

* Pytest file

* merged per element js checks

* Check if driver is still open

* super fast processing

* fixed agent planning and stuff

* Fix example

* Fix example

* Improve error

* Improved error correction

* New line for step

* small type error fixes

* Test for pydantic

* Fix line

* Removed sample

* fixed readme and examples

---------

Co-authored-by: magmueller <mamagnus00@gmail.com>
2024-11-15 21:42:02 +01:00
Gregor Žunič
219e8ee085 processing 2.0 2024-11-02 18:43:35 +01:00
magmueller
c608b53640 First conversations 2024-11-02 13:03:28 +01:00
Gregor Žunič
eeb91764d1 first repo 2024-10-31 17:00:29 +01:00