833 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Jing Hua	ebb5f1cbc5	add mouse cursor to screenshot	2023-11-30 17:31:46 +08:00
Timothyxxx	e52ba2ab13	Fix the width and height of vm, make agent perform more accurate	2023-11-30 12:10:41 +08:00
Timothyxxx	ecb62d7eb4	Resolve fixed todos	2023-11-29 20:23:59 +08:00
Timothyxxx	80b148793d	Initialize visual components such as SAM for assistance	2023-11-29 20:22:48 +08:00
Timothyxxx	3d0d9d7758	Run through gpt_4v agent pipeline	2023-11-29 20:21:57 +08:00
Timothyxxx	28c6edd6b3	Add data marker tool	2023-11-28 00:40:04 +08:00
Timothyxxx	8470264884	Initialize GPT-4v agent, and prompt for current observation space	2023-11-28 00:38:22 +08:00
Timothyxxx	7ab3799d30	Remove figure for space	2023-11-28 00:31:19 +08:00
Timothyxxx	0d9a03454e	Update with iamge processing tool for marking the click point	2023-11-28 00:30:05 +08:00
Timothyxxx	054f545942	Initialize GPT-4v agent, and prompt for current observation space	2023-11-28 00:23:50 +08:00
Timothyxxx	3fb1dd42d3	README fix	2023-11-27 02:05:26 +08:00
Timothyxxx	eefef4d42d	Add heuristic compression of action series on key/scroll/mouse down&up	2023-11-27 02:04:39 +08:00
Timothyxxx	e7fb85aa2f	Merge remote-tracking branch 'origin/main'	2023-11-27 00:35:51 +08:00
Jing Hua	21c323a255	ducktrack converter	2023-11-27 00:35:40 +08:00
Timothyxxx	8272e93953	Add DuckTrack as initial annotation tool; Initial multimodal test	2023-11-27 00:34:57 +08:00
Timothyxxx	8c0525c20e	Adapt for Windows os; Refine README	2023-11-27 00:29:09 +08:00
Jing Hua	6dee58252e	add key down key up	2023-11-26 23:54:43 +08:00
Jing Hua	ddd1da2d78	remove old code	2023-11-26 23:48:20 +08:00
David Chang	857f73f61d	ver Nov23rd ... trying to set up the environment	2023-11-24 20:39:08 +08:00
Jing Hua	203fae5798	update screenshot command	2023-11-11 02:38:16 +08:00
Jing Hua	a8aebf5d15	mouse and keyboard controllers for windows and linux	2023-11-08 09:22:43 +08:00
Jing Hua	8bc4459f41	update readme	2023-11-02 12:10:47 +08:00
Jing Hua	9620c75cf2	finish reset step	2023-11-02 12:09:36 +08:00
Jing Hua	fd3787ee28	controllers	2023-11-02 11:10:55 +08:00
Jing Hua	824a9068e5	update readme	2023-11-02 11:08:31 +08:00
Jing Hua	a3706004d0	update readme	2023-11-02 09:44:02 +08:00
Jing Hua	4c17a41b0d	delete old files	2023-11-02 09:35:59 +08:00
Jing Hua	b3da09a860	gym interface	2023-10-30 00:28:33 +08:00
Jing Hua	1aecea0763	update readme	2023-10-29 22:19:40 +08:00
Timothyxxx	f975adc804	Add todos and roadmap	2023-10-29 01:04:25 +08:00
Jing Hua	d01a3b44f6	add more actions	2023-10-25 18:26:13 +08:00
Jing Hua	a7144824d2	add controller	2023-10-24 02:26:24 +08:00
Jing Hua	14a4f5c008	First commit	2023-10-16 09:50:38 +08:00