`contextdir.chdir` is not thread-safe, but the formatting runs
synchronously in a single thread, so it's fine.
Testing: Made some changes to various `.rs`, `.py`, and `.toml` files,
and ran `../mach fmt` from a few different subdirectories to ensure it
picks up the files to reformat.
Fixes: #42472
Signed-off-by: pralkarz <pralkarz@tuta.com>
Skip push hooks for try command. This speeds up try for me, which
normally means waiting for clippy a second time. The commit step is
already using `--no-verify`.
Testing: tested this change locally and it works
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
This helps buddies from MI5, MI6, NSA etc. to run tests behind proxy.
Testing: Tested locally behind proxy. For some reason, I already set
`127.0.0.1` and `localhost` in `no_proxy`, but it does not work.
Fixes: #42321
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
All other browsers use a single configuration for their browser
invocations on WPT. Servo historically had two: servo and servodriver.
Now that we run WPT on Servo GitHub CI with Webdriver using the
servodriver, we can align our configuration with theirs.
The existing "servo" configuration is renamed to "servo_legacy" and
"servodriver" is then renamed to "servo". This way, we preserve the
"servo" product name as defined on wpt.fyi, but we do use its webdriver
configuration.
Since webdriver is not fully working yet for debugging purposes, we keep
the "servo_legacy" configuration now. In the future, once the debugging
story has improved, we can remove "servo_legacy".
All in all, this ensures that both on local, Servo GitHub CI and on
wpt.fyi we all use the exact same configuration. I tested this locally
by running the following test:
```
./mach test-wpt tests/wpt/tests/css/css-overflow/scrollbar-gutter-dynamic-004.html
```
This does times out with the servo binary and works with the servodriver
binary.
Running the servo_legacy configuration is done via the `--servo-legacy`
flag:
```
./mach test-wpt tests/wpt/mozilla/tests/mozilla/caption.html --servo-legacy
```
Fixes#40751
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This bumps stylo to allow configuring the stack size of the stylo
threads, which fixes crashes with asan and debug builds.
Testing: Manual testing with `--with-asan` and with the debug profile.
Fixes: #40814
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Switch to using our own dockerfile, and install the required
dependencies into our image.
In follow-ups we can build the docker image in CI, so that users can
download a prebuilt docker image to save time.
Testing: Manually tested by opening servo in the devcontainer and
running `./mach build`
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This allows us to reuse the list without python, e.g. from a
shell-script. This is useful to build docker image layers, without
requiring python / uv / mach first.
Future changes will split the list into multiple text files, so we can
install only required subsets (e.g. skipping gstreamer) and a workflow
to automatically sync changes to the book.
Testing: `./mach bootstrap` still works.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Signed-off-by: Jonathan Schwender <55576758+jschwe@users.noreply.github.com>
This change finishes the big rename associated with the old
`compositing` crates. Long ago, these crates managed a compositor, like
you might find in a traditional web engine. These days, compositing is
done in WebRender so the name has stopped making much sense. Various
structs inside the crates have already been renamed and this is the
final big change necessary for the rename
Testing: This is just a rename so existing tests should cover it.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Probably we should use a proper linter like `shellcheck`, but in the
meantime make the lint slightly smarter, so that it doesn't complaint
about e.g. `$1`. This is especially a problem when adding scripts which
use `awk`, since our lint is quite stupid and doesn't understand `''`
strings.
This fixes linting, for a simple new script introduced in
https://github.com/servo/servo/pull/41775
Testing: ./mach test-tidy still passes.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Instead of attempting to manage the virtual environment ourselves, use
`uv` to manage the installation of dependencies.
Since we still have dependencies coming from upstream wpt, we use
`[tool.setuptool]` in our pyproject.toml to ensure that `uv` dynamically
installs our dependencies according to the requirements.txt files.
Additionally, this PR also reverts `--no-project` usage. `--no-project`
was added as a temporary workaround in
https://github.com/servo/servo/pull/37741.
It's not 100% clear to me what exactly the issue was, but
[apparently](https://github.com/servo/servo/pull/37741#pullrequestreview-2985666234)
the issue caused the build to break.
Removing the arg seems to work fine, except that we get a warning about
a missing `requiress-python` value in `pyproject.toml`.
Apparently it is good practice to specify the requirement as `>=` in th
pyroject, and lock the exact version via `uv pin` (which writes to
`.python_version`, where we already pin 3.11.
Testing: Should be covered by existing tests, which compile code on all
platforms.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This change moves the upload and attestation steps to the nightly
workflow. This results in overall slightly more code, but reduces the
amount of code that has access to extra privileges.
Additionally, this refactoring will allow to easily change the target
repo for the nightly release, i.e. allowing us to reuse the nightly
workflow for our monthly releases in the servo/servo repo (in a
follow-up PR).
We change upload_nightly to get an additional `packages` parameter, that
we use instead of a hardcoded table of expected packages per platform.
`download-artifact` doesn't preserve the whole directory structure, so
it would be a bit cumbersome to preserve that.
Since we anyway only use the upload script in CI, we can just directly
specify the packages in the CI, which also makes things slightly more
flexibel if we want to add new packages or platforms.
There are still more improvements to come, specifically also to the
`upload_nightly` script, with the goal to minimize the amount of code
that needs to be trusted.
Testing: Manually triggering the [release
workflow](https://github.com/servo/servo/actions/runs/20996682863/job/60358012404)
via workflow-dispatch.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Instead of relying on the contents of the already-parsed arguments,
which may have tried to convert URL arguments to paths, parse URLs from
their original place in `sys.argv`. This is a workaround to the fact
that we extend the internal WPT update command to accept URLs of GitHub
Actions. This is a custom Servo extension that has no equivalent to
the WPT upstream command.
Testing: There aren't really tests for this level of the Servo command
line tools.
Fixes: #41728
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
The versions of `cargo-deny` older than 0.18.6 have a bug which causes
the executions of `cargo-deny check` to prematurely fail when reading
the advisory db
(https://github.com/EmbarkStudios/cargo-deny/issues/804).
This error is ignored by `test-tidy` since the error message doesn't
have the expected JSON fields, causing `test-tidy` to succeed even when
there are valid issues in `deny.toml` or `Cargo.lock`.
So upgrade the `cargo-deny` version installed by `mach` to be the latest
version and ensure that at least the version with the fix is installed
on the system. Also fix the `test-tidy` code to always fail when the
exit code from `cargo-deny` is non-zero.
This patch also updates `deny.toml` to include exceptions to allow
`./mach test-tidy` to pass. Some of these need to be investigated
separately from this change.
Fixes#41845.
Fixes#38945.
Testing: Tested locally on NixOS.
---------
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
The book was changed, so we need to update the link to the new location.
Testing: Only changes comment. Reviewers can review by following the new
link.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
I never forget that day: @dependabot updates aws-lc-rs, and
cross-compilation Servo from Windows fails. We were left with two
options:
1. Get a local clone of aws-lc-rs, and modify it in a hacky way
2. Use Git Bash, but still has to set up MSYS2 etc.
Later, we've been restricted since #36070 to only use option 2 due to
restrictions imposed here. But the system is innocent, and it is
aws-lc-rs's responsibility to fix the rules. Still, @d-desiatkin and I
believed that day would come, when we would return justice to the
victim.
Freedom is better served late than never. It is now working normally
since aws-lc-rs has fixed the rules.
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
The variable is not used anywhere and `_platform_bootstrap_gstreamer`
raises an error telling the user to install gstreamer via their package
manager on Linux.
Hence, we can just remove it.
Testing: Trivial, not required.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Nowadays Servo automatically loads system default proxy. It has enabled
my NSA colleagues to visit external networks without modifying transport
layer, whereas `test-wpt` has been failing after it.
We disable the proxy unless users know what they are doing.
Testing: Tested in different physical locations with different proxy
networks.
Fixes: #41440
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
This new lint checks for some easy mistakes to make with the annotation
format.
Testing: New unit tests added.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
This just fixes a typo:
`download_run_resultsa_and_then_run_update` ->
`download_run_results_and_then_run_update`
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
This change allows `./mach update-wpt` to accept a URL to a GitHub
Action run. If a URL is passed, it will attempt to download the stable
unexpected results from the run and update the expected test results.
Note that this currently requires having the `gh` command-line tool
installed and authenticated. Although you can download artifacts without
logging in via the web interface, doing this via the API requires an
access token.
Testing: This change adds a unit test for the regex that matches GitHub
Action URLs.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Properly handle empty commit messages when processing commits during WPT
export. We shouldn't be landing commits with empty messages into Servo,
but sometimes when a PR is in process, the body is empty. In those
cases, this change avoids an error during job execution.
Signed-off-by: chenura999 <chenuraoshada396@gmail.com>
Implements #40502
Adds `--yes` flag to mach bootstrap command which confirms installation
of all required packages.
Unlike `--force` flag (which also does that) it does NOT reinstall all
packages but only confirms them (if required at all).
Test on Ubuntu 24.04.2 LTS by removing safe development packages like
`libgstrtspserver-1.0-dev libges-1.0-dev gstreamer1.0-tools` and run
`mach bootstrap` twice to observe if `--yes` flags automatically install
all dependencies.
With this flag all dependencies were installed with any interaction but
removing some dependencies and running `./mach bootstrap` prompted me to
confirm `apt-get install` action.
---------
Signed-off-by: MCozhusheck <mackozuszek@gmail.com>
This PR introduces a new storage coordination thread, intended to serve
as the central point for managing all current and future storage
endpoints in Servo.
In addition to the new coordination thread, this PR also lays the
infrastructure required to develop a parallel, next-generation IndexedDB
implementation under the indexeddb_next feature flag living on a
separate branch.
Testing: Unit and WPT tests continue to pass
---------
Signed-off-by: Jan Varga <jvarga@igalia.com>
On NixOS, dynamically linked binaries from the Python venv (installed
via uv/pip) cannot run directly because they expect standard Linux
library paths that don't exist on NixOS.
This commit adds FHS wrappers that allow these binaries to run in an
FHS-compatible environment at runtime, without patching them.
I considered using the nixpkgs versions of ruff and pyrefly directly, or
overriding their derivations to match the versions in requirements.txt.
However, decided against it because:
- Version mismatches between nixpkgs and requirements.txt caused type
checking incompatibilities (pyrefly 0.34.0 vs 0.23.1)
- Building these tools from source in nix is slow and adds significant
time to nix-shell initialization (both are rust packages that take quite
some time to build)
Testing: just improvements to the NixOS development environment, no test
needed
**Before:**
```
➜ servo (main) ✔ nix-shell
➜ servo (main) ✔ ./mach fmt
Could not start dynamically linked executable: /home/dyego/coding/random/servo/.venv/bin/ruff
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld
➜ servo (main) ✔ ./mach test-tidy
➤ Checking config file (./servo-tidy.toml)...
➤ Checking directories for correct file extensions...
Could not start dynamically linked executable: ruff
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld
Error running mach:
['test-tidy']
The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.
If filing a bug, please include the full output of mach, including this error
message.
The details of the failure are as follows:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
File "/home/dyego/coding/random/servo/python/servo/testing_commands.py", line 322, in test_tidy
tidy_failed = tidy.scan(not all_files, not no_progress, github_annotations)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dyego/coding/random/servo/python/tidy/tidy.py", line 919, in scan
for error in errors:
File "/home/dyego/coding/random/servo/python/tidy/tidy.py", line 401, in check_ruff_lints
for error in json.loads(e.output):
^^^^^^^^^^^^^^^^^^^^
File "/nix/store/2g9b898aq9kmizmhmhbdip5mixrc5wrk-python3-3.11.14/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/2g9b898aq9kmizmhmhbdip5mixrc5wrk-python3-3.11.14/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/2g9b898aq9kmizmhmhbdip5mixrc5wrk-python3-3.11.14/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
➜ servo (main) ✔
```
(note that the `JSONDecodeError` is because we're trying to parse the
"NixOS cannot run dynamically linked executables intended..." string as
JSON)
**Now:**
```
➜ servo (fix-nix-mach) ✔ nix-shell
➜ servo (fix-nix-mach) ✔ ./mach fmt
➜ servo (fix-nix-mach) ✔ ./mach test-tidy
➤ Checking config file (./servo-tidy.toml)...
➤ Checking directories for correct file extensions...
➤ Checking type annotations in python files ...
➤ Skipping WPT lint checks, because no relevant files changed.
➤ Running `cargo-deny` checks...
➤ Checking formatting of Rust files...
➤ Checking formatting of python files...
➤ Checking formatting of toml files...
✅ test-tidy reported no errors.
➜ servo (fix-nix-mach) ✔
```
Signed-off-by: Dyego Aurélio <dyegoaurelio@gmail.com>
The current user stylesheet use in our tests is a hack that hides real
issues like #40419. Any test that requires ahem.ttf needs to include the
stylesheet like any other webpage.
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
The current logic treats the explict version of release and dev build
specification i.e the `--profile release` and `--profile dev` options as
custom profiles. This breaks other logic that explictly check for
release and dev profile types, like the android build.
Normalize the `--profile` argument so that the strings `release` and
`dev` are treated correctly as `BuildType.Kind.RELEASE` and
`BuildType.Kind.DEV` respectively.
Fixes: #40752
Testing: Tested manually that building android with `--profile release`
creates the correct directory structure.
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Support for sending build notifications was removed in #37818, but the
dependency in `requirements.txt` was not removed. This PR fixest that.
Testing: This change just removes an unsued build dependency, so no
testing is necessary.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
This should help identify flaky unit tests, since codecov will gather
statistics of tests (of the last 60 days) and allow us to easily
identify flaky unit-tests. [Test page on
codecov](https://app.codecov.io/github/servo/servo/tests) based on an
uploaded report from a try run.
Additionally add a catch-all parameter for `test-unit` which is passed
through to the `cargo nextest` invocation, useful for e.g. stressing a
test via `--stress-count`.
Testing: Manually tested with [try
run](https://github.com/servo/servo/actions/runs/18529823800)
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Similar to about:license in firefox. Generated with `cargo-about`.
A few open questions:
- When/how should it be updated (the command I used was `cargo about
generate etc/about.hbs > resources/resource_protocol/license.html`, it
takes ~20 seconds)
- How should `about.toml` be kept in sync with `deny.toml`
- What about licenses for packages that are not crates (i.e. gstreamer)
Fixes: #40266
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
- Remove unused accepted Copyright line `See
http://rust-lang.org/COPYRIGHT`. This URL now yields a 404, and we don't
seem to use this line anywhere in the project so we can remove it. Note:
We should probably use a regex anyway, so we can match the whole
required expression, but thats a different topic.
- Remove unused `licenses_dep_toml` table. We use cargo deny for
checking the licenses of our rust dependencies.
Testing: Changing tidy is covered by `./mach test-tidy` in CI
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
The main change is running `hdc shell power-shell setmode 602`, which
sets the device to performance mode, increasing the screen timeout and
manually waking the device up.
We add a helper class for that, so that we can use the `with` syntax to
undo these changes at the end of scope.
We also add a small HDC wrapper class, which slightly simplifies and
abstracts over some common functionality like running commands or making
a screenshot.
Testing: This is a CI script change.
Fixes: Speculative fix for speedometer sometimes timing out on HOS and
the screenshot showing a locked device screen.
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Removed `setup.py`, `setup.cfg`, `mach.egg-info/`, and `PKG-INFO`. I
don't see these being used anywhere and mach isn't packaged by us. I
checked the firefox source tree and couldn't find them:
45fd8b48b2/python/mach.
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
In a fresh container skipping the apt-update will cause only 4 packages
to be installed and the rest filtered. Also add a debug print to inform
the user if packages were skipped.
Testing: Manually tested
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
`mach bootstrap` no longer installs Wireshark, as it isn't useful for
people not working directly on DevTools support. The linux runner that
runs `mach test-scripts` now installs it manually, skipping the security
prompt.
Testing: Manual test of `mach test-scripts`
Fixes: #40503
Signed-off-by: eri <eri@igalia.com>
Rename the `--scan` argument to `-w/--write-file` and the `--use`
argument to `-r/--read-file`. This is more aligned with `tshark`'s
syntax, and I think it is a more intuitive naming scheme.
Remove the `--filter` and `--range` arguments. They are very easily
replaced by more powerful tools like `grep` and `jq`. It seems
unnecessary to have them in this script (specially when the most useful
thing it does is exporting the capture as NDJSON for other tools to
process). This fixes an issue with the last message not being exported.
Change the default port to `6080`. This matches the current information
[in the
book](https://book.servo.org/hacking/using-devtools.html#connecting-to-servo)
on how to run Servo with DevTools enabled.
Testing: Checked with `math test-scripts`
---------
Signed-off-by: eri <eri@igalia.com>
An implementation detail of `./mach try` is that it creates a git commit
containing information about the try configuration. Adding `--no-verify`
skips running git pre-commit hooks, which could cause the commit to
fail. This is useful for draft / work-in-progress branches, where you
quickly want to commit something and test CI works, before fixing lints
/ formatting etc.
Testing: Manually tested with `./mach try`.
Fixes: #40475
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This is a more reasonable default as the detection of running one or
more tests is a bit inconsistent. It also makes the behavior of the
command a bit more predictable IMO.
Testing: This was tested manually as there is no tests for this
layer of the test runner
Fixes: #40407.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Add a `--coverage` flag to `./mach build` and a `./mach coverage-report`
command to use `cargo llvm-cov` to generate a coverage report from the
raw profiles.
The workflow is:
```
./mach build --coverage [--profile <cargo_profile>]
# Or test-wpt or test-devtools
./mach run --coverage [--profile <cargo_profile>]
# Note, that coverage-report needs to know the cargo build profile.
./mach coverage-report [--profile <cargo_profile>] [optional parameters for cargo-llvm-cov]
```
According to the LLVM documentation on source based coverage, the
optimization profile should not influence the accuracy of the coverage
profile, so we can gather coverage data from optimized builds.
Note that `./mach test-devtools --coverage` will not produce any
coverage profiles yet, since the test runner kills the servo binary,
which prevents writing the profile data at shutdown.
The same problem also affects `test-wpt` with `servodriver`, which will
be fixed by https://github.com/servo/servo/pull/40455.
Testing: Manually tested. A CI workflow to test wpt coverage will be
added in a follow-up PR.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
cargo places the shared library artifact under
`target/<target_triple>/<profile>/libservoshell.so`. The openharmony
prefix, is something we do in mach for the hap, and is not relevant for
the .so (essentially a copy-paste mistake).
Can be compared by looking at the job run of
https://github.com/servo/servo/actions/runs/19125758002/job/54655560361
which failed due to the wrong path.
Testing: Not tested
Fixes: #40467
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
