Commit Graph

47 Commits

Author SHA1 Message Date
Magnus Müller
b678f27a3e Migration from highlight index to backendnode id 2025-10-21 21:02:28 -07:00
Magnus Müller
9caa248316 Merge code7 branch changes 2025-10-21 18:55:54 -07:00
mertunsall
09c27fc2cc dont default 2025-08-30 21:08:26 +02:00
mertunsall
b177f3166e better exception handling from browser events -> LLM 2025-08-30 20:50:42 +02:00
Nick Sweeting
140346b99e fix clicking and typing in iframes 2025-08-19 18:15:04 -07:00
Nick Sweeting
76512fbfdd fix Failed to build DOM tree: cannot reuse already awaited coroutine 2025-08-16 17:58:45 -07:00
Nick Sweeting
8d86832095 fix SwitchTab by using TargetID for all tab_ids 2025-08-15 16:45:27 -07:00
Nick Sweeting
7a0889cf20 formatting 2025-08-12 04:14:53 -07:00
Nick Sweeting
d1e569eb31 raise BrowserError exceptions instead of returning success=False 2025-08-12 04:14:53 -07:00
Nick Sweeting
328a4b5be4 Merge branch 'feature/new-dom-extraction-layer' into local-remote-split 2025-08-05 03:12:48 -07:00
Nick Sweeting
032f314a41 Merge branch 'main' into local-remote-split 2025-08-04 19:44:05 -07:00
Gregor Žunič
286b5ec801 Merge commit '90fa344b599d55c2984bd926161acfecbb681d8c' into feature/new-dom-extraction-layer 2025-08-02 14:34:20 -07:00
Gregor Žunič
0aa3d54233 Merge commit '9b882faa10c54be5a870cc5d8abd6ddb19d0682f' into feature/new-dom-extraction-layer 2025-08-02 11:41:28 -07:00
mertunsall
9bd0036223 separate agent history and state 2025-08-02 15:48:49 +01:00
Nick Sweeting
2b12eb6542 wip 2025-07-31 03:31:32 -07:00
Nick Sweeting
ae479d6330 add storage state events and show recent events summary to llm 2025-07-31 00:54:53 -07:00
Nick Sweeting
09b99c8afc tell the LLM when page loading was aborted so it knows to use wait tool 2025-07-29 13:16:46 -07:00
Nick Sweeting
c86cd96274 Merge branch 'main' into claude-issue-2497-20250721-0633 2025-07-24 22:20:15 -07:00
Cursor Agent
0778166109 Add PDF viewer detection and handling in browser state summary
Co-authored-by: mailmertunsal <mailmertunsal@gmail.com>
2025-07-24 15:44:19 +00:00
claude[bot]
ba12148b19 refactor: consolidate PLACEHOLDER_4PX_SCREENSHOT constant
- Move 4px placeholder screenshot constant to browser_use.browser.views
- Update all references to use the single definition
- Fix all lint errors and formatting issues

Co-authored-by: Nick Sweeting <pirate@users.noreply.github.com>
2025-07-21 22:28:01 +00:00
Gregor Žunič
7bf17064f6 started migration to pure cdp 2025-07-11 14:19:22 +02:00
Gregor Žunič
779196d74c nicer types 2025-07-10 22:28:53 +02:00
Gregor Žunič
8a2eda1828 wip, started migrating to pure cdp 2025-07-09 12:22:21 +02:00
Daniel T.
c5a3c0cf2e Refines dynamic page state calculations
Replaces static page statistics with dynamic calculations for
viewport and scroll position metrics. Simplifies code logic in
scroll handling by removing unnecessary defaults for page
scrolling. Improves readability and reliability of page
scroll operations by requiring explicit num_pages parameter.

Enhances page-view consistency and user interaction handling.
2025-07-07 21:22:12 +02:00
Daniel T.
92bd24f1a0 Standardizes string quotes and cleans up whitespace
Replaces double quotes with single quotes for consistency in
string declarations. Removes unnecessary trailing whitespace
across various files to improve code readability and maintain
a uniform style.
2025-07-07 18:57:30 +02:00
Daniel T.
bf878db2ab Enhances page info in browser state
Adds comprehensive page information including viewport and total
page dimensions, detailed scroll position, and page statistics.
Updates message prompts to utilize this data, improving content
presentation and potential user actions.

Relates to enhanced scrolling feature.
2025-07-07 18:47:50 +02:00
Nick Sweeting
a17ee667d8 dont repr screenshot in terminal 2025-05-22 07:01:10 -07:00
Nick Sweeting
86ec6a78b0 new BrowserSession and BrowserProfile design, renamed BrowserState to BrowserStateSummary and more 2025-05-20 02:31:54 -07:00
Nick Sweeting
23252192a0 remove non-functional group tabs code 2025-05-04 19:40:27 +08:00
Nick Sweeting
2be4ba4f70 more pyupgrade changes 2025-05-02 20:50:21 +08:00
Nick Sweeting
fb6fa259a8 apply ruff safe fixes 2025-03-28 18:11:36 -07:00
Nick Sweeting
d95527ab27 Merge branch 'main' into enhancement/add_tabGroups_chrome 2025-03-25 16:34:08 -07:00
Nick Sweeting
73369f910c fix discovery of elements that have both shadow and non shadow child nodes 2025-03-24 14:28:00 -07:00
SwapnilSonker
92fe9212a9 #544 issue resolved , used chrometabs api 2025-02-08 22:48:46 +05:30
magmueller
e7c49bfcac Error handling for restricted domains 2025-01-23 16:53:23 -08:00
magmueller
66ea4a3509 Restric urls 2025-01-23 15:44:14 -08:00
magmueller
2bd6e20953 Include how many pixels are above / below 2025-01-20 10:12:12 -08:00
magmueller
60fbbcae93 Split HistoryTreeProcessor in views and service 2024-12-03 09:26:17 +01:00
magmueller
c581a1b251 First draft multi-action runs 2024-12-01 21:00:22 +01:00
magmueller
9dfc1f3faf Seperate class for browserhistory - fixed type errors 2024-11-28 23:59:49 +01:00
magmueller
519881a7b3 Rerun works 2024-11-28 21:10:25 +01:00
magmueller
de7e38b75c History is saved and loaded 2024-11-28 19:37:18 +01:00
Gregor Žunič
829ed138fd clicking version - still missing iframe/shadow elements penetration 2024-11-25 21:45:05 +01:00
Gregor Žunič
b0c390f2c0 fixed multi tab management, clicking timeouts, general bugfixes 2024-11-20 17:33:49 +01:00
Gregor Žunič
155f49acc4 started the switch from selenium to playwright 2024-11-18 20:38:00 +01:00
Gregor Žunič
89c63fdd63 Added custom actions registry and fixed extraction layer (#20)
* Validator

* Test mind2web

* Cleaned up logger

* Pytest logger

* Cleaned up logger

* Disable flag for human input

* Multiple clicks per button

* Multiple clicks per button

* More structured system prompt

* Fields with description

* System prompt example

* One logger

* Cleaner logging

* Log step in step function

* Fix critical clicking error - wrong argument used

* Improved thought process of agent

* Improve system prompt

* Remove human input message

* Custome action registration

* Pydantic model for custom actions

* Pydantic model for custome output

* Runs through, model outputs functions, but not called yet

* Work in progress - description for custome actions

* Description works, but schema not yet

* Model can call the right action - but is not executed

* Seperate is_controller_action  and is_custom_action

* Works! Model can call custom function

* Use registry for action, but result is not feed back to model

* Include result in messages

* Works with custom function - but typing is not correct

* Renamed registry

* First test cases

* Captcha tests

* Pydantic for tests

* Improve prompts for multy step

* System prompt structure

* Handle errors like validation error

* Refactor error handling in agent

* Refactor error handling in agent

* Improved logging

* Update view

* Fix click parameter to index

* Simplify dynamic actions

* Use run instead of step

* Rename history

* Rename AgentService to Agent

* Rename ControllerService to Controller

* Pytest file

* Rename get state

* Rename BrowserService

* reversed dom extraction recursion to while

* Rename use_vision

* Rename use_vision

* reversed dom tree items and made browser less anoying

* Renaming and fixing type errors

* Renamed class names for agent

* updated requirements

* Update prompt

* Action registration works for user and controller

* Fix done call by returning ActionResult

* Fix if result is none

* Rename AgentOutput and ActionModel

* Improved prompt Passes 6/8 tests from test_agent_actions

* Calculate token cost

* Improve display

* Simplified logger

* Test function calling

* created super simple xpath extraction algo

* Tests logging

* tiny fixes to dom extraction

* Remove test

* Dont log number of clicks

* Pytest file

* merged per element js checks

* Check if driver is still open

* super fast processing

* fixed agent planning and stuff

* Fix example

* Fix example

* Improve error

* Improved error correction

* New line for step

* small type error fixes

* Test for pydantic

* Fix line

* Removed sample

* fixed readme and examples

---------

Co-authored-by: magmueller <mamagnus00@gmail.com>
2024-11-15 21:42:02 +01:00
Gregor Žunič
68201df624 src -> browser_use 2024-11-06 18:18:00 +01:00