Commit Graph

4120 Commits

Author SHA1 Message Date
Alexander Yue
2a095a6c5c Merge branch 'main' into fix-docs-favicon 2025-07-05 20:59:41 -07:00
Alexander Yue
0877d56f68 fix-screenshot-test (#2305)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Changed the screenshot test to start browser sessions one at a time
instead of in parallel to avoid Playwright semaphore errors.

<!-- End of auto-generated description by cubic. -->
2025-07-05 20:57:30 -07:00
Alezander9
faf06f8edb remove broken example file 2025-07-05 20:55:12 -07:00
Alezander9
7b5ea73b4d fix ruff errors 2025-07-05 20:48:37 -07:00
Alezander9
522f55f81e fix-screenshot-test 2025-07-05 20:24:27 -07:00
Alexander Yue
b1109a4fe2 Merge branch 'main' into fix-docs-favicon 2025-07-05 20:02:56 -07:00
Magnus Müller
12af77ed9c fix: handle eval import errors (#2302)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Improved error handling for missing optional imports in eval by raising
clear ImportError messages with install instructions.

<!-- End of auto-generated description by cubic. -->
2025-07-05 21:51:26 +02:00
reformedot
c3949b8c92 chore: linting 2025-07-05 17:08:43 +02:00
reformedot
88c4f3ff12 fix: handle eval import errors 2025-07-05 17:07:42 +02:00
Magnus Müller
cbf7681510 feat: added support for Browserbase and Hyperbrowser as available browsers in the eval (#2301)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Added support for Browserbase and Hyperbrowser as remote browser options
in the eval service.

- **New Features**
- Users can now select Browserbase or Hyperbrowser with the `--browser`
flag.
- Added environment variable checks and session creation logic for both
services.

<!-- End of auto-generated description by cubic. -->
2025-07-05 17:00:55 +02:00
reformedot
fc34224692 fix: fix PR issues 2025-07-05 16:54:09 +02:00
reformedot
8c5672b1ef feat: added support for Browserbase and Hyperbrowser as available browsers in the eval
fix: revert example change
2025-07-05 16:20:24 +02:00
os1rix
7d3416936c fix docs icon 2025-07-05 13:43:02 +03:00
Mert Unsal
243de84d5d fixing detection of interactive elements in Shadow DOM (#2294)
fixing detection of interactive elements in Shadow DOM.
fixing by
1. Avoiding  Early viewport check
2. Use the maximum valid value in zIndex to ensure the element is not
blocked by overlapping elements.

**fixing: [Interaction Issue: Shadow DOM in axeptio cookie banner
#2276](https://github.com/browser-use/browser-use/issues/2276)**

### debug

#### Shadow DOM host element: id is "odkjkuv".


![image](https://github.com/user-attachments/assets/84695360-9b52-48f5-9ea1-7b9914b2c676)

#### Shadow DOM host element:

![image](https://github.com/user-attachments/assets/220a6856-dfef-4113-be98-87244424896c)

#### **window size on the moment**:

![image](https://github.com/user-attachments/assets/80b77711-85f0-4701-b281-ffcf7299016e)



#### Before fixing: Nothing is highlighted in following picture.

![img_v3_02nt_041fa406-69ea-4fab-a2f5-780c8ec7e82g](https://github.com/user-attachments/assets/3244ff42-08ae-4efa-8890-b871bae567a2)

#### After fixing: There are 10 elements be highlighted in following
picture.

![img_v3_02nt_b5b58c7f-1a35-457b-97c2-aabb0318da9g](https://github.com/user-attachments/assets/ed3711a1-12d3-4d9b-bc08-e73f000e9de2)

    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Fixed detection of interactive elements inside Shadow DOM so they are
now correctly highlighted.

- **Bug Fixes**
- Skipped early viewport checks for Shadow DOM nodes to avoid missing
elements.
- Set zIndex to the maximum valid value to prevent highlights from being
blocked.

<!-- End of auto-generated description by cubic. -->
2025-07-05 10:51:12 +02:00
Mert Unsal
3d77ef0e1e Merge branch 'main' into hot-fix/fix-highlight-interative-elements-in-shadow-dom 2025-07-05 10:46:51 +02:00
Mert Unsal
3b8f4ff7cf eval feat (#2291)
refactor: simplify email query parameter description and enhance action
result messages

- Updated the description of the `query` parameter in
`GetRecentEmailsParams` for clarity.
- Refined the action result messages for better readability and
consistency, including specific query details in the long-term memory
response.
    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Simplified the Gmail email query parameter description and improved
action result messages for clearer feedback.

- **Refactors**
  - Shortened and clarified the `query` parameter description.
- Updated action result messages to include query details and improve
readability.

<!-- End of auto-generated description by cubic. -->
2025-07-05 10:41:50 +02:00
Mert Unsal
38dee5e627 Merge branch 'main' into mert/improve_gmail_actions 2025-07-05 10:40:17 +02:00
Mert Unsal
f31747adcf Add basic support for JSON, CSV, and reading PDFs. (#2296)
Improved filesystem error handling and added better validation for
filenames
Updated model names everywhere to gpt-4.1
Added an example of writing into csv
2025-07-05 10:35:19 +02:00
mertunsall
220f0bc994 update models to gpt-4.1 2025-07-05 10:32:49 +02:00
mertunsall
3124ddcfe3 add excel sheet example 2025-07-05 10:15:53 +02:00
mertunsall
7a6644c1c3 fix example 2025-07-05 10:15:24 +02:00
Mert Unsal
cfad5738b8 Merge branch 'main' into mert/improve_fs 2025-07-05 09:56:23 +02:00
Nicolas
b2dc5dbcb0 Merge branch 'main' into hot-fix/fix-highlight-interative-elements-in-shadow-dom 2025-07-05 12:21:58 +08:00
Alexander Yue
0debd997ef make eval service fetch rotating auth info from server (#2263)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
The eval service now fetches rotating authentication info from the
server and injects it into tasks that require login credentials.

- **New Features**
- Added server call to fetch auth distribution if any tasks specify
required auth keys.
- Injects relevant login credentials into task descriptions for agent
use.

<!-- End of auto-generated description by cubic. -->
2025-07-04 20:48:23 -07:00
Alexander Yue
50355fb8d5 Merge branch 'main' into auth-distribution 2025-07-04 20:01:38 -07:00
Mert Unsal
1c5a96c668 Merge branch 'main' into mert/improve_fs 2025-07-05 02:07:30 +02:00
mertunsall
304b065d59 more security 2025-07-05 02:07:11 +02:00
mertunsall
f3e1a59642 improve 2025-07-05 01:51:00 +02:00
mertunsall
550a2a2552 feat: enhance file handling capabilities with JSON and CSV support
- Added new file types: JsonFile and CsvFile to the file system.
- Updated read_file method to handle external files with .json and .csv extensions.
- Modified write_file action to allow .json and .csv file extensions in addition to .md and .txt.
- Introduced pypdf dependency for PDF file handling.

This update improves the flexibility of file operations within the application.
2025-07-05 01:36:04 +02:00
Magnus Müller
817599adab feat: multiple browser support for the eval script (#2295)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Added support for selecting different browsers (local, Anchor Browser,
Brightdata) in the eval script using a new --browser argument.

- **New Features**
- Users can choose the browser backend with --browser (local,
anchor-browser, brightdata, browser-use).
- Updated environment variable handling and logging for new browser
options.

<!-- End of auto-generated description by cubic. -->
2025-07-05 01:14:03 +02:00
Alezander9
1a3ac38197 Merge remote-tracking branch 'upstream/main' into auth-distribution 2025-07-04 15:20:54 -07:00
reformedot
3dbaea1729 fix: improved anchor browser session creation 2025-07-04 23:43:15 +02:00
reformedot
8754e22ce3 feat: added browser arg to the eval script 2025-07-04 23:40:12 +02:00
Magnus Müller
2c564009f7 Remove unused anchor navigation argument from eval service 2025-07-04 21:37:18 +02:00
Magnus Müller
775d5981fc feat: add support for anchor browser when running evals (#1878)
<!-- This is an auto-generated description by cubic. -->

## Summary by cubic
Added support for using Anchor Browser when running evals, allowing
remote browser sessions if an API key is set.

- **New Features**
- Detects ANCHOR_BROWSER_API_KEY and connects to Anchor Browser
automatically.
  - Falls back to local browser if Anchor Browser is unavailable.

<!-- End of auto-generated description by cubic. -->
2025-07-04 21:29:38 +02:00
Magnus Müller
18fe7620de Remove memory logging 2025-07-04 21:21:15 +02:00
Magnus Müller
74b4bfd363 Merge remote-tracking branch 'origin/main' into feat/evals-anchor-support 2025-07-04 21:13:09 +02:00
mingzhong.li
ab83e505b5 fix: fixing detection of interactive elements in shadow DOM; The getBoundingClientRect() of the Shadow DOM host element may return width/height = 0 2025-07-05 00:32:03 +08:00
mertunsall
b9262aaebe make it MORE SIMPLE 2025-07-04 17:29:22 +02:00
mertunsall
f80bf95260 add gmail connection only for tasks that have OTP 2025-07-04 17:21:18 +02:00
mertunsall
bd0dfd8f11 fix 2025-07-04 17:16:34 +02:00
mertunsall
45dc441b53 even simpler. 2025-07-04 16:57:23 +02:00
mertunsall
a60a2a37f9 simplify 2025-07-04 16:43:47 +02:00
mingzhong.li
1fccd380a4 comment: update comment in index.js 2025-07-04 22:22:44 +08:00
mingzhong.li
5a11b8f1cd fix: fixing highlighted elements not show,Make sure to use the maximum valid value in zIndex to avoid being blocked by other elements 2025-07-04 21:45:40 +08:00
mertunsall
d8a08f088e fix error in not initializing controller correctly 2025-07-04 14:42:32 +02:00
mertunsall
ad92564af4 refactor: simplify email query parameter description and enhance action result messages
- Updated the description of the `query` parameter in `GetRecentEmailsParams` for clarity.
- Refined the action result messages for better readability and consistency, including specific query details in the long-term memory response.
2025-07-04 14:09:22 +02:00
Mert Unsal
8bd62df8a7 feat: added gmail api integration to get OTPs and email (#2259)2
## Overview
Adds Gmail integration to automatically retrieve 2FA verification codes
during web automation, eliminating manual intervention for two-factor
authentication flows.

## Key Features
- **Automatic 2FA Code Extraction**: Smart regex patterns extract
verification codes from Gmail with context-aware prioritization
- **Flexible Authentication**: Supports both OAuth2 file-based auth
(development) and direct access tokens (production)
- **Agent Actions**: 3 new actions - `find_2fa_codes`,
`get_recent_emails`, `authenticate_gmail`
- **Retry Logic**: Progressive wait times handle email delivery delays
(10s, 20s, 30s)

## Usage
```python
from browser_use.integrations.gmail import register_gmail_actions

controller = Controller()
register_gmail_actions(controller)  # File-based auth
# or register_gmail_actions(controller, access_token="token")  # Production

# Agent automatically handles 2FA:
# 1. Login with credentials → 2FA prompt → find_2fa_codes → code extracted → login complete
```

## Technical Details
- **Files**: New `browser_use/integrations/gmail/` module with service,
actions, and examples
- **Dependencies**: Added Google API client libraries to
`pyproject.toml`
- **Security**: Read-only Gmail access, credentials gitignored, OAuth2
compliant
- **Backward Compatible**: Zero breaking changes, graceful degradation
without Gmail

## Impact
Solves the biggest friction point in web automation - manual 2FA code
entry. Enables fully autonomous login flows for services requiring email
verification.

    
<!-- This is an auto-generated description by cubic. -->
---

## Summary by cubic
Added Gmail API integration to automatically fetch 2FA codes and recent
emails, enabling agents to handle email-based authentication without
manual steps.

- **New Features**
  - Smart extraction of 2FA codes from Gmail using regex and context.
- Supports both OAuth2 file-based and direct access token
authentication.
- Three new agent actions: find_2fa_codes, get_recent_emails,
authenticate_gmail.
  - Retry logic for delayed email delivery.
  - Example script for setup and usage.

- **Dependencies**
  - Added Google API client libraries to project dependencies.

<!-- End of auto-generated description by cubic. -->
2025-07-04 13:53:47 +02:00
Saurav Panda
fd5adf4080 Merge branch 'main' into 2fa_gmail_integration 2025-07-04 04:30:01 -07:00
Saurav Panda
abdb8efa9e refc: removed authenticate function from action and update ActionResult for gmail integrations 2025-07-04 04:27:18 -07:00