Commit Graph

5812 Commits

Author SHA1 Message Date
Magnus Müller
da94635e05 remove-efficiency-rules (#2877)
Auto-generated PR for branch: remove-efficiency-rules
    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Streamlined efficiency guidelines and action docs to make multi‑action
steps safer and clearer. Fixed URL detection by stripping emails before
matching, and improved scroll guidance to prefer larger jumps and omit
index for full‑page scrolling.

- **Bug Fixes**
- Remove emails from task text before URL matching to prevent false
positives.

- **Refactors**
- Simplified efficiency_guidelines in all prompts; added clear do/don’t
for chaining and updated recommended combos (including scroll +
extract).
- Clarified click_element_by_index and input_text descriptions to only
allow indices from the current browser_state.
- Updated scroll action docs to favor high num_pages (e.g., 10) and to
omit index when scrolling the entire page.

<!-- End of auto-generated description by cubic. -->
2025-08-30 18:44:37 -07:00
Magnus Müller
1596f25a5a System prompt efficiency_guidelines 2025-08-30 18:39:10 -07:00
Magnus Müller
30b2d631f9 Merge commit 'cb63d579bd61c4239125780d1d658d2eada36a1b' into remove-efficiency-rules 2025-08-30 18:33:56 -07:00
Magnus Müller
f17a13b308 Typo 2025-08-30 18:33:30 -07:00
Magnus Müller
889657a273 Linter 2025-08-30 18:30:49 -07:00
Magnus Müller
84c644aedb Patch extension with whitelist 2025-08-30 18:27:21 -07:00
Magnus Müller
f50cc45b51 Works to remove cookies 2025-08-30 18:14:59 -07:00
Mert Unsal
cb63d579bd Improve extract structured data input, output, and LLM calls (#2880)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Improves the extract_structured_data tool to return cleaner, more
accurate results by normalizing page markdown, enforcing input limits,
and using a clearer system+user prompt. This makes extractions more
consistent and reduces noisy or invented outputs.

- **Refactors**
- Normalizes markdown: removes artifacts, demarkdowns inline links,
compresses excessive newlines, and trims whitespace.
- Enforces size limits: truncates content at MAX_CHAR_LIMIT and
annotates when truncated.
- Uses structured prompting: sends SystemMessage + UserMessage with
strict rules (page-only facts, list all items, non-conversational
output, note if data is missing).
- Clarifies tool usage: updated action description to discourage using
it for interactive elements and to gate link extraction via
extract_links.
- Simplifies memory handling: only stores small results
(MAX_MEMORY_LENGTH).

<!-- End of auto-generated description by cubic. -->
2025-08-31 03:06:56 +02:00
mertunsall
ca05521b0c try to prevent overusage of extract_structured_data 2025-08-31 02:52:50 +02:00
mertunsall
87ae7f3976 hotfix 2025-08-31 02:25:08 +02:00
mertunsall
84ebfee661 improve extract structured data 2025-08-31 02:21:17 +02:00
Mert Unsal
c06fb84f37 Minor documentation changes for consistency. (#2843)
Minor documentation changes for consistency according to issue :
https://github.com/browser-use/browser-use/issues/2810 @sauravpanda
    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Standardized the max_steps description across the server schema and docs
with consistent wording and punctuation. Cleaned up the real-browser
guide by merging duplicate notes and clarifying that we use DuckDuckGo
because Google blocks this approach.

<!-- End of auto-generated description by cubic. -->
2025-08-31 00:51:22 +02:00
Mert Unsal
157d0804cb Merge branch 'main' into correction-in-docs 2025-08-31 00:50:59 +02:00
Mert Unsal
203febb67d Update gif.py (#2847)
fixes os related font error at gif creation
    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Fixes GIF creation crashes on some OSes by adding a safe fallback when
loading fonts. If loading a TrueType font fails, we now use the existing
regular font.

- **Bug Fixes**
- Catch OSError/AttributeError from ImageFont.truetype and fall back to
regular_font.
- Prevents failures when regular_font has no .path or the font cannot be
loaded.

<!-- End of auto-generated description by cubic. -->
2025-08-31 00:49:20 +02:00
Mert Unsal
e07ad3bc32 Merge branch 'main' into os-font-fix 2025-08-31 00:48:49 +02:00
Mert Unsal
ad7a3d6a84 better exception handling from browser events -> LLM (#2879)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Introduce BrowserError with short- and long-term memory to carry
structured context from browser events to the LLM, replacing brittle
string parsing. Improves guidance after failures (clicks, dropdowns,
uploads) and makes errors more actionable.

- **New Features**
- Added BrowserError with short_term_memory (one-shot guidance) and
long_term_memory (persistent context).
- Dropdown actions return structured memory; on selection failure,
return available options for the LLM to choose from.
- Tools map BrowserError to ActionResult via handle_browser_error,
setting include_extracted_content_only_once when needed.

- **Refactors**
- Replaced <llm_error_msg>-based parsing with structured errors across
click, input, upload, navigation, tabs, scroll, and send keys.
- Standardized error messages and improved timeout handling; registry
now surfaces TimeoutError as a clear RuntimeError.
- Safer file upload validation; common invalid path and missing element
cases return ActionResult instead of crashing.
- Removed automatic get_dropdown_options during click failures to avoid
hidden side effects.
  - Updated dependency: bubus >= 1.5.6.

<!-- End of auto-generated description by cubic. -->
2025-08-30 22:13:02 +02:00
Mert Unsal
0388404396 Merge branch 'main' into mert/better_browser_exceptions 2025-08-30 22:05:17 +02:00
Gregor Žunič
f0f3ac165c Fixed <frame> elements (#2878)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Add full support for <frame> elements alongside <iframe> so framed
content is detected, indexed, and serialized correctly. Also cleans up
SimplifiedNode JSON to avoid duplicated children and large payloads.

- **Bug Fixes**
- Handle FRAME everywhere IFRAME is handled: interactive detection,
visibility/offset, and serialization (adds |FRAME| marker).
- Traverse frame content via content_document.children_nodes and include
nodes when appropriate (fixes include check).
- Recursively remove children_nodes and shadow_roots from original_node
JSON, including nested content_document.

<!-- End of auto-generated description by cubic. -->
2025-08-30 13:04:38 -07:00
mertunsall
890a535f9e reenable returning dropdown option when clicked on a select element. 2025-08-30 22:04:16 +02:00
mertunsall
b60dc35ff0 reactivate the hack for now 2025-08-30 21:52:18 +02:00
mertunsall
0498d0803c fix more tests 2025-08-30 21:39:18 +02:00
mertunsall
9f8a635910 try to catch timeout errors too 2025-08-30 21:37:12 +02:00
mertunsall
1abee767cb hotfix 2025-08-30 21:09:20 +02:00
mertunsall
09c27fc2cc dont default 2025-08-30 21:08:26 +02:00
Mert Unsal
c695bb05e4 Update browser_use/tools/registry/service.py
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2025-08-30 21:06:36 +02:00
mertunsall
4ce3fe2d04 fix tests 2025-08-30 21:01:37 +02:00
mertunsall
afd7c46ccf fix bug from merge 2025-08-30 21:01:23 +02:00
Mert Unsal
93d808ad91 Merge branch 'main' into mert/better_browser_exceptions 2025-08-30 20:53:05 +02:00
mertunsall
b177f3166e better exception handling from browser events -> LLM 2025-08-30 20:50:42 +02:00
Gregor Žunič
b141034ff3 Merge branch 'main' into c/better-frameset-detection 2025-08-30 11:15:58 -07:00
Magnus Müller
89af16361f Add timeouts 2025-08-30 11:04:14 -07:00
Magnus Müller
799fe88aa5 New extension 2025-08-30 11:03:55 -07:00
Magnus Müller
2c40bd23a8 Fix url preload 2025-08-30 10:42:25 -07:00
Magnus Müller
b9ab642d0c Update efficiency guidelines to include new scrolling instructions for extracting structured data. Clarified usage of the scroll action to enhance clarity and efficiency in multi-step processes. 2025-08-30 10:39:50 -07:00
Magnus Müller
3d8f793406 Enhance efficiency guidelines by clarifying action chaining restrictions and emphasizing the importance of clear goals per step. Added instructions to avoid multiple state changes in a single action sequence. 2025-08-30 10:24:26 -07:00
Magnus Müller
89cb9ef954 More efficient scroll instruction 2025-08-30 10:00:28 -07:00
Magnus Müller
d97462e49a New efficiency guidelines 2025-08-30 09:54:40 -07:00
Magnus Müller
b638958bb5 Revert "Remove efficiency_guidelines"
This reverts commit 781f2131ee.
2025-08-30 09:46:48 -07:00
Magnus Müller
3dac24ddec Update docstrings for click_element_by_index and input_text actions to clarify usage restrictions regarding browser_state indices. 2025-08-30 01:01:03 -07:00
Magnus Müller
781f2131ee Remove efficiency_guidelines 2025-08-30 01:00:48 -07:00
Magnus Müller
7de77187b7 time_execution_async import (#2875)
Auto-generated PR for: time_execution_async import
2025-08-29 22:32:23 -07:00
Magnus Müller
8e9937a168 Merge branch 'main' into timeexecutionasync-import 2025-08-29 22:32:16 -07:00
Magnus Müller
aad4b932bf time_execution_async import 2025-08-29 22:32:05 -07:00
Magnus Müller
959397cd16 make highlights sync (#2874)
Auto-generated PR for: make highlights sync
    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Switches highlight rendering from async parallel to synchronous
sequential to avoid PIL ImageDraw thread-safety issues. This stabilizes
overlays and prevents intermittent rendering glitches.

- Bug Fixes
- Process elements sequentially in create_highlighted_screenshot; make
process_element_highlight synchronous.
- Remove asyncio usage and related code; update docstrings accordingly.

<!-- End of auto-generated description by cubic. -->
2025-08-29 22:26:05 -07:00
Magnus Müller
12725a3ed5 Merge branch 'main' into make-highlights-sync 2025-08-29 22:25:56 -07:00
Magnus Müller
25823ce670 make highlights sync 2025-08-29 22:25:32 -07:00
Magnus Müller
e1ab411e6f highlight in parallel (#2873)
Auto-generated PR for: highlight in parallel
2025-08-29 22:18:12 -07:00
Magnus Müller
147e9e7a46 Merge branch 'main' into highlight-in-parallel 2025-08-29 22:18:05 -07:00
Magnus Müller
ce0876adb7 highlight in parallel 2025-08-29 22:17:50 -07:00
Magnus Müller
5ad515816a Generate python highlights for browser screenshot (#2705)
Replaced JavaScript-based highlighting with fast Python image processing
and parallelized DOM building and screenshot capture.

---
[Slack
Thread](https://browser-use.slack.com/archives/D092QUQD6KA/p1755470146356559?thread_ts=1755470146.356559&cid=D092QUQD6KA)

<a
href="https://cursor.com/background-agent?bcId=bc-276bdb3b-198a-483a-af73-f2a9a3efd4f8">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg">
<source media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg">
<img alt="Open in Cursor" src="https://cursor.com/open-in-cursor.svg">
  </picture>
</a>
<a
href="https://cursor.com/agents?id=bc-276bdb3b-198a-483a-af73-f2a9a3efd4f8">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg">
<source media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg">
    <img alt="Open in Web" src="https://cursor.com/open-in-web.svg">
  </picture>
</a>


    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Replaced JavaScript-based element highlights with fast Python image
processing on screenshots. DOM build and screenshot now run in parallel
for lower latency and no in-page side effects.

- **New Features**
- Added python_highlights.py to draw color-coded bounding boxes and
short text labels on screenshots using PIL.
- Applies highlights after a clean screenshot using the DOM
selector_map; uses CDP viewport info with safe defaults.

- **Refactors**
- Parallelized DOM build and screenshot via asyncio and switched to
_build_dom_tree_without_highlights and _capture_clean_screenshot.
- Added graceful fallbacks on errors (minimal DOM state, original
screenshot).

<!-- End of auto-generated description by cubic. -->
2025-08-29 21:54:31 -07:00