Commit Graph

51 Commits

Author SHA1 Message Date
Samson
cd92a17c5e ci: Update remaining GitHub actions to versions using Node.js 20 (#31450)
* Update sccache action

* Update baptiste0928/cargo-install

* Update github-script

* Update actions/setup-java

* Update nick-fields/retry
2024-02-29 08:24:38 +00:00
Martin Robinson
2946fa83ec ci: Switch to actions/checkout@v4 (#31379)
This change also removes the use of `fetch-depth: 2` where it isn't
necessary. These were likely copy-and-pasted from other workflows.
2024-02-19 16:23:53 +00:00
Martin Robinson
1cc546c4fc ci: Switch to version 4 of GitHub artifact actions (#31357)
* ci: Switch to version 4 of GitHub artifact actions

This switches to version 4 of the GitHub artifact actions, which
requires producing a single artifact per job and adding merge steps. In
addition, the names of artifacts are standardized:

- Build: <profile>-binary-<platform>
- Full WPT results (only on failure): wpt-full-logs-<platform>-<layout>
- Filtered WPT results (only on failure): wpt-filtered-logs-<platform>-<layout>

* Delete merged build timings and combine with Result job

* Always archives logs even after test failures

* Correct the name of the log files for WPT import
2024-02-17 16:59:43 +00:00
Martin Robinson
a3ce1f2636 ci: Make dispatch-workflow.yml GitHub job names a bit friendlier (#31340)
This makes the names match the ones used in the workflows themselves.
2024-02-13 16:27:11 +00:00
Samson
b10875956a Delete result job in leaf workflows (#31191) 2024-01-27 00:07:04 +00:00
Martin Robinson
0d240b8713 deps: Raise the Python requirement to 3.10 (#31088)
This will allow us to use the pip's new dependency resolver.
2024-01-15 11:56:15 +00:00
Martin Robinson
7fa4ea9740 Upgrade media / GStreamer / GLib (#30750)
- Upgrade the version of GStreamer for Windows

This upgrades the Windows build to use the most recent version of
GStreamer. This is necessary to upgrade our GStreamer dependency.

- Stop shipping GStreamer binaries on Linux

The binary bundle of GStreamer that we package is not used to compile --
only to run layout tests. It's too old for the APIs that we are using
(as evidenced by needed 1.18 for WebRTC) and nowadays Linux
distributions carry a new version so it's unecessary for our build
machines. No longer using this binary bundle will allow us to upgrade
our GStreamer dependency -- which now has stricter checks that we
are using at least version 1.18.

- Upgrade media to use newer versions of GStreamer / GLib dependencies
2024-01-05 08:01:58 +00:00
Samson
1f0f50b22b Customizable wpt options and profile in workflows (#30912)
* wpt -> wpt-sync

* Add profile configuration option

* Add wpt (for custom tests selection) to CI

* Renaming of wpt-* options

* fixup! do renames also in scripts

* fixup! profile

* Fix try labels
2023-12-22 12:49:39 +00:00
Martin Robinson
f2882879d8 Fix the try build when pushing to branches (#30903)
Make it so that all try builds go through try.yml and pass
workflow_call arguments as expected to subsequent workflows.

Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>
2023-12-21 15:12:54 +00:00
Samson
1105eb66e9 Use os version in taplo cache-key & setuptools 65 in py3.8 (#30848) 2023-12-11 15:06:43 +00:00
Samson
3a5634a20c Update linux.yml (#30846) 2023-12-08 18:14:40 +00:00
Corey Farwell
117d59d393 Replace virtualenv with Python's built-in venv (#30377)
* Replace virtualenv with Python's built-in venv.

* Apply Delan's suggestions and make a couple small fixes

- Fix a tidy warning about directories that don't exist
- Use shutil instead of the redundant get_exec_path
- Miscellaneous cleanups

* Fix typo in environment variable

* fix bug where pip still tries to the wrong site-packages

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Delan Azabani <dazabani@igalia.com>
2023-12-07 08:18:30 +00:00
Samson
604979e367 Replace script_plugins with a clippy like rustc driver (named crown) (#30508)
* Remove script_plugins

* Use crown instead of script_plugins

* crown_is_not_used

* Use crown in command base

* bootstrap crown

* tidy happy

* disable sccache

* Bring crown in tree

* Install crown from tree

* fix windows ci

* fix warning

* fix mac

libscript_plugins.dylib is not available anymore

* Update components/script/lib.rs

Co-authored-by: Martin Robinson <mrobinson@igalia.com>

* Update for nightly-2023-03-18

Mostly just based off https://github.com/servo/servo/pull/30630

* Always install crown

it's slow only when there is new version

* Run crown test with `mach test-unit`

* Small fixups; better trace_in_no_trace tests

* Better doc

* crown in config.toml

* Fix tidy for real

* no sccache on rustc_wrapper

* document rustc overrides

* fixup of compiletest

* Make a few minor comment adjustments

* Fix a typo in python/servo/platform/base.py

Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>

* Proper test types

* Ignore tidy on crown/tests

---------

Co-authored-by: Martin Robinson <mrobinson@igalia.com>
2023-12-01 15:50:52 +00:00
Mukilan Thiyagarajan
81a38bde8c ci: do not set LIBCLANG_PATH for nightly build (#30792)
This environment variable was added when we moved
to Ubuntu 22.04 and it is not needed for nightly
builds which we have now switched to 20.04

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-11-28 09:49:17 +00:00
Mukilan Thiyagarajan
d10688b5ef ci: use ubuntu 20.04 for nightly builds on linux (#30787)
Ubuntu 22.04 has a newer glibc (2.34) which means builds
from there won't run on systems with older glibc, most
notably the wpt.fyi taskcluster runners which use 20.04
as the docker base image.

This is a temporary workaround until wpt upgrades to 22.04

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-11-27 11:55:44 +00:00
Samson
96d37d3785 Add --production option to mach (#30707)
* --prod(uction) mach argument

* Use profile in workflows instead of production

* Use profiles in unit tests

* ups

* Build (${{ inputs.profile }})
2023-11-10 10:38:33 +00:00
Martin Robinson
fe7bdb7e4b Remove support for triggering try from comments (#30712)
Triggering from labels means that we have less actions running and less
false job failures spamming project members. Plus, we have more
flexibility with labels rather than the backward compatibility we have
set up for bors comments.
2023-11-09 10:31:49 +00:00
Martin Robinson
e0c0b65e2a Inherit secrets in the main workflow (#30708)
This is an attempt to fix results in the intermittent tracker.
2023-11-08 14:23:54 +00:00
Samson
fd5f04c18e Use runner provided python 2023-11-03 16:41:26 +00:00
Mukilan Thiyagarajan
b268936308 ci: use ubuntu 22.04 on linux runners (#30162)
* ci: use ubuntu 22.04 on linux runners

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

* Add env fix

* update expectations for tests under ubuntu-22.04

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>

---------

Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
2023-11-03 05:42:44 +00:00
Samson
c2af95d2fc Unit tests are becoming more flaky, so retry them (#30682)
* Unit tests are becoming more flaky, so retry them

* Timeouts everywhere

* Limit windows test threads for unit tests
2023-11-02 15:23:04 +00:00
Samson
31ca45e589 Replace environmental cargo_profile___ with ${{}} (#30658) 2023-11-02 05:57:08 +00:00
Martin Robinson
3fde61f2e5 Fix the try build (#30642) 2023-10-29 16:35:58 +00:00
Delan Azabani
a3d2f0c586 Enable debug assertions for all builds other than official releases (#30509)
* Run main and try jobs with debug assertions

* use single quotes in workflow expressions

* set force-debug-assertions in main.yml

* set force-debug-assertions as part of decision job

* fix typo in MachCommands.build

* fix more hardcoded profile names

* fix tidy

* split cargo_profile_option on windows

* Fix running servoshell and unit tests through a symlink

* rename steps to make them less confusing

* fix more hardcoded cargo profile options

* fix missing inputs in linux-wpt and mac-wpt

* make filename an inherent method of Resource

* rework release-with-debug-assertions profile to production profile

* rework resource logic to eliminate std_test_override

* set production flag in nightly release builds

* clean up servobuild.example and windows.yml

* oops forgot to check in embedder_traits/build.rs

* fix mach test-unit behaviour through symlink

* unit tests only need current_dir and ancestors

* fix macOS package smoketest breakage

* expect css/css-color/currentcolor-003 to crash under layout 2013

* fix more references to {force,release-with}-debug-assertions

* fix local build failures under --profile production
2023-10-26 08:22:14 +00:00
Martin Robinson
fdcbe613ac Allow trigger try workfows using labels (#30383)
This change adds and alternate method for triggering try changes.
Instead of comments, changes are triggered via applying labels to pull
requests. The action will remove the label from the request and start
the requested jobs.

This will require creating at least a few labels:

 - T-full
 - T-linux-wpt-2013
 - T-linux-wpt-2020
 - T-macos
 - T-windows

More labels can be added as we support more configurations.

The good thing about this change is that try jobs against the actual
branch in the pull request instead of the master branch. This means
that changes to CI can be tested (unlike for comment processing).

One bit caveat with this change is that when adding multiple labels, a
CI job is triggered for each. Only one real build will run for each
label, but whether or more try jobs is triggered is a race condition.
The first CI job to successfully remove the label will actually trigger
the job. If the same job removes two compatible labels, then they can
share a build (for instance two types of WPT linux jobs). If not there
will be two. Note that this is at least as efficient as the current
behavior.
2023-10-23 11:57:36 +00:00
Samson
7caac9790d Enforce formatting of TOML files (#30128)
* Fmt all toml files

* bootstrap taplo

* enforce toml formatting with taplo

* Install taplo in CI using cargo-install action
2023-09-19 08:57:27 +00:00
Martin Robinson
d3203976dc Revert "Combine all try workflows (#30096)" (#30149)
This reverts commit fed3491f23.
2023-08-21 17:00:14 +00:00
Samson
85a37210c2 Name OS builds (#30145) 2023-08-21 06:08:14 +00:00
Martin Robinson
fed3491f23 Combine all try workflows (#30096)
There are currently two ways to run try. One is to push to the `try` or
`try-*` branches and the other is to trigger a workflow via GitHub
comment. This change combines these methods into one workflow. In
addition, WPT results are reported together rather than separately and
filtered results for all WPT tests are bundled together in the same
artifact.
2023-08-20 09:43:02 +00:00
Martin Robinson
1d79f5dad2 Make the --release/--dev more consistent and less surprising (#30091)
There were some issues with the way that the `--release` and `--dev`
arguments were handled in mach commands.

 - Not all commands accepted them in the same way. For instance `./mach
   test-wpt` didn't really accept them at all.
 - If you did not pass either of them, mach would try to guess which
   build you meant. This guess was often quite surprising as it wasn't
   printed and it depended on the state of the your target directory,
   which is difficult to remember.
 - The `dev` profile is colloquially called a "debug" profile and some
   commands accepted `-d` or `--debug...` like arguments, but `--debug`
   with `./mach run` meant run in a debugger. It was easy to mix this
   up.

This change:

 - Centralizes where build type argument processing happens. Now it the
   same shared decorator in CommandBase.
 - Uses a `BuildType` enum instead of passing around two different
   booleans. This reduces the error checking for situations where both
   are true.
 - Be much less clever about guessing what build to use. Now if you
   don't specify a build type, `--dev` is chosen. I think this behavior
   matches cargo.
 - Makes it so that `./mach test-wpt` accepts the exact same arguments
   and has the same behavior as other commands. In addition, the suite
   correct for `test-wpt` is removed. There are only two suites now and
   it's quite unlikely that people will confuse WPT tests for rust unit
   tests.
2023-08-14 10:21:29 +00:00
Martin Robinson
51fa6c7e18 Checkout the PR when triggering try from PR comments (#30069)
Fixes #30067.
2023-08-04 07:06:43 +00:00
Martin Robinson
9fa67685f4 Remove unnecessary join()s from GitHub Actions (#30040)
`contains()` works on arrays as well as strings, so the `join` is
unnecessary when trying to detect job statuses.
2023-07-28 05:26:19 +00:00
Martin Robinson
d5202a4a98 Enable the GitHub merge queue (#29989)
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
2023-07-13 07:45:48 +00:00
Samson
3242ad2ca3 Split the WPT workflow from the Linux one
This allows running tests for only one layout system at a time. This was
originally part of #29950, but was lost in the shuffle.
2023-07-06 20:36:07 +02:00
Martin Robinson
d31cdb682f Make the choice of layout runtime setting
Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>
2023-07-06 14:49:24 +02:00
Martin Robinson
69c830a6e0 When using "try-<platform>" trigger unit tests
There is currently no way to trigger unit tests using the bors-servo try
infrastructure. This change makes it so that using try-<platform>
triggers unit tests for that platform. This will be especially useful
when testing changes for Windows before trying to land them.
2023-07-04 11:36:16 +02:00
Martin Robinson
ae9a9e1d17 Combine all script tests into test-scripts
Remove:
    - tidy self test support from `./mach test`
    - `./mach test-idl`

Adds a `./mach test-scripts` command that is responsible for running all
Python script tests. Run this during the CI to catch regressions in
changes to scripts. The WebIDL tests are still *very* slow and there are
from Gecko, so only run them when "-a" is passed meaning all tests.
2023-06-29 19:09:33 +02:00
Martin Robinson
e679e1f793 Collect build timings on GitHub CI
This will help to investigate slow builds on GitHub.
2023-06-23 15:14:40 +02:00
Mukilan Thiyagarajan
49aa0b6e06 Use Python 3.9 on linux workflows
It looks like the dist-packages of python 3.8
on the runner has an older version of ssl package
which is now causing nightly builds to fail.
2023-05-18 19:44:37 +05:30
sagudev
219afcc9a8 WPT results aggregating based on layout 2023-04-27 09:05:02 +02:00
sagudev
b4c465e6a2 Run wpt only on try-wpt,try-wpt-2020 branches 2023-04-27 08:50:16 +02:00
sagudev
200f90cf9f Add try-wpt-2020 runner 2023-04-25 09:53:01 +02:00
sagudev
8bed625d28 Use defined layout for wpt test 2023-04-25 09:53:01 +02:00
Mukilan Thiyagarajan
f1ba708cf7 Upload nightly builds to Github Releases
This change extends the `mach upload-nightly` command to
publish the nightly builds for all platforms as GH Release
assets.

The GH releases are made on a separate repository so
that we can persist older nightly builds without having
to accumulate git tags for them.

Some design tradeoffs in this approach are:
1. To allow the 'latest' link from servo.org to remain stable,
the release assets are named 'servo-latest.{ext}' instead of
containing the release tag/date.
2. The release is created as draft and published atomically
when all platforms have been built successfully. This allows
us to link to the 'latest' alias from servo.org while
gauranteeing that it contains builds for all platforms.
The other option here would be to have code in servo.org UI
that uses GH API to find the most recent release with a
successful build for a given platform.
3. The tags in the nightly repo are all based on the same
commit that has no relation to servo code base.

Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
2023-04-12 21:02:07 +05:30
sagudev
2f3364a291 Remove if guard in linux workflow 2023-04-06 20:05:36 +02:00
sagudev
76e9c68ac7 Nightly workflow using platform workflows 2023-04-06 18:43:33 +02:00
sagudev
ec5b646b50 Package in platform workflows 2023-04-06 16:19:08 +02:00
sagudev
7d0865c12b WPT agregated results also as github check 2023-04-03 09:13:04 +02:00
sagudev
91de41a70a Sccache in actions (linux & mac) 2023-04-02 07:18:51 +02:00
sagudev
8959c5754d WPT sync using abstract workflows 2023-03-30 15:49:46 +02:00