This PR fixes error handling in Servo's HTTP fetch implementations to
properly handle content decompression failures (e.g., bad gzip content)
by treating them as network errors as defined in WPT.
- Fixed some previously failing tests in bad-gzip-body.any.js that now
properly reject when consuming bodies with bad gzip content
- Several fetch metadata tests now pass correctly
---------
Signed-off-by: araya <araya@araya.dev>
Rather than having each callside specifying the relevant
information from the GlobalScope, do this via a trait instead.
This would have saved us quite a bit of test debugging
since we would often forget to set relevant information
from the global context for a request.
Now, in the future when we need additional information from
the globalscope for a request, we only need to update this
method to make that happen.
Previously it would also sometimes use `document`, but
calling the relevant information on either `document` or
`globalscope` doesn't matter, since the `globalscope`
defers to the value from the `document` anyways.
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
The specification isn't complete here and thus we
were missing WPT coverage. Spec will be updated.
Also, we are correctly throwing the QuotaExceededError with relevant
information, while Chrome sets both
fields to `null`. Therefore, the test now allows for both (since
providing relevant information is good for the developer and recommended
in the spec [1]).
[1]: https://webidl.spec.whatwg.org/#ref-for-quotaexceedederror%E2%91%A2
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This implements waiting for pending preloads, where the preload request
is still fetching the result when the second "real" request is started.
It is
implemented by storing responses in the `SharedPreloadedResources`
which is communicated via `PreloadId` send to the `CoreResourceManager`.
Part of #35035
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
Replace NetworkError::Internal with structured enum variants
- Adds
UnsupportedScheme,CorsViolation,ConnectionFailure,Timeout,RedirectError,InvalidMethod,ResourceError,SecurityBlock,MixedContent,CacheError,InvalidPort,
LocalDirectoryError, variants in NetworkError enum.
- Refactored the usage of NetworkError::Internal(String) to use the
appropriate new variant
Testing: Changes does not require test.
Fixes: https://github.com/servo/servo/issues/36434
---------
Signed-off-by: Uthman Yahaya Baba <uthmanyahayababa@gmail.com>
Signed-off-by: Usman Yahaya Baba <91813795+uthmaniv@users.noreply.github.com>
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Co-authored-by: Tim van der Lippe <tvanderlippe@gmail.com>
Replace `allow` with `expect` lints for `unused`, `unsafe_code`,
`dead_code`, and `non_upper_case_globals`.
Testing: So far just check it compiled on `x86_64-linux` on NixOS. Need
to use the module `system.fontconfig.enable = true;` I think in my NixOS
config.
Part of: #40383
Searching `allow\(.*\)` for `.rs` files shows the following. for
(total_results:total_files) went from `707:386` to `675:368`, a
reduction of `32:18`.
How many files is too many files per PR? I feel like the 20-30 I have is
too big.
---------
Signed-off-by: Wayne Van Son <waynevanson@gmail.com>
In preparation for adding preload support for all requests, we need to
add relevant client information to all these entrypoints. Additionally,
for links we now also set the referrer correctly and initialize
documents. All of which are required to start loading preload
information when processing requests.
Part of #35035
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
We now keep track of deferred fetches in globalscope
rather than in document loader. This matches what
the spec expects with its concept of a fetchgroup.
Cancellations are still present in a document loader
and don't match yet all places where we perform fetches.
Therefore, these will leave in document loader for now
until we rearranged the fetch record tracking in the
fetch group (which is a separate field from deferred
fetches).
Fixes#41286
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
For all upcoming and already-processed deferred fetches, we now
implement a quota. We don't support policy headers yet to configure the
quota, hence we use the default quota. That's why we have some tests
failing again.
It involves computing which document to compute the quota for, as well
as computing total length of requests. Additionally, the DevTools
computation for headers now uses the same logic for headers as deferred
fetches.
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This implements the required plumbing for the keep-alive flag on
requests. The flag already exists on the request object, but now also
exists on the builder itself.
The flag is then passed into a canceller. While we could make canceller
optional, in many cases it isn't and would require a whole lot more
changes. To follow the spec more closely, opted to put the keep_alive
flag on the canceller as well.
Note that this doesn't pass any extra fetch-later tests since we are not
correctly unloading iframe documents. That will be fixed in a follow-up
PR.
Part of #41230
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This is the first part of preparation for implementing the keepalive
flag for requests. It requires implementation of the concept of a task
group, which we already sort of have with `DocumentLoader`. Therefore,
we currently already cancel any loads, which should be updated to
ignored kept-alive requests.
However, termination also requires storing of deferred fetches, which
are also kept-alive. Therefore, to distinguish the two, we need to store
them separately as stated with the TODO.
In a follow-up, we can then skip canceling keep-alive requests for both
`navigator.sendBeacon` (which aren't deferred) and `fetchLater` (which
are deferred).
Part of #41230
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Use the algorithms introduced in #41109 in more places.
Follow more closely the spec when executing classic scripts, introducing
the concepts of rethrowing and muting errors.
Muting errors is not actually implemented, and will be done in a
followup.
Testing: More tests start passing
Fixes#34199Fixes#27260Fixes#15188
---------
Signed-off-by: Gae24 <96017547+Gae24@users.noreply.github.com>
Changed some allow to expects and removed the unfulfilled expectations.
Testing: Refactor
Part of: #40383
Signed-off-by: anonmiraj <nabilmalek48@gmail.com>
The goal of this change is to prevent having to copy so much data out of
listeners when a fetch completes, which will be particularly important
for off-the-main thread parsing of CSS (see #22478). This change has
pros and cons:
Pros:
- This makes the design of the `FetchResponseListener` a great deal
simpler.
They no longer individually store a dummy `ResourceFetchTiming` that is
only replaced right before `process_response_eof`.
- The creation of the `Arc<Mutex<FetchResponseListener>>` in the
`NetworkListener` is abstracted away from clients and now they just
pass the `FetchResponseListener` to the fetch methods in the global.
Cons:
- Now each `FetchResponseListener` must explicitly call `submit_timing`
instead of having the `NetworkListener` do it. This is arguably a bit
easier to follow in the code.
- Since the internal data of the `NetworkListener` is now an
`Arc<Mutex<Option<FetchResponseListener>>>`, when the fetching code
needs to share state with the `NetworkListener` it either needs to
share an `Option` or some sort of internal state. In one case I've
stored the `Option` and in another case, I've stored a new inner
shared value.
Testing: This should not change observable behavior and is thus covered
by existing tests.
Fixes: #22550
---------
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
`FetchReponseListener` has traditionally lived in `net` even though it
is only used in `script` currently. Because of the two way dependency,
it has also use a lot of templating to implement something pretty basic
(call methods on a trait object).
This change moves the trait to `script` and removes several levels of
templating, making the code quite a bit shorter and easier to
understand.
This change is preparation for fixing #22550 and implementing
off-the-main-thread CSS parsing.
Testing: This should not change any behavior so is covered by existing
tests.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Moves interfaces defined by the performance spec to the
`script/dom/performance/` module from `script/dom/`.
Testing: Just a refactor shouldn't need any testing
Fixes: Partially #38901
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com>
This aligns the request object more with the specification,
since the spec now has a `traversable_for_user_prompts` and
a separate field for the client. Before, they were present
in the same enum.
In doing so, new structs are added that are all required in
the new spec. With this we can add support for preloaded
resources in this client, which are only populated when
we have an applicable Global.
Since the spec moved things around a bit, it now has a
dedicated method to populate the client from the request.
Unfortunately none of the WPT preload tests pass, since
the requests are received out-of-order. The specification
requires us to wait for that to settle, but I haven't figured
out yet how to do that. Given that this PR is already quite
large, opted to do that in a follow-up.
Part of #35035
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
In preparation of adding a client to a request, we first need to add a
policy container to all requests. Some where missing and relying on
fallbacks (that need to be removed).
For worklets this adds the appropriate policy container from the
containing global, but since that's still behind a pref, no tests are
affected.
For font contexts I tried to pass through the relevant information, but
it is a tad too much. Once fontcontext knows more information about its
containing global (as the TODO also points to referrer), we should clean
this up.
Part of #35035
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Does not all tests pass because of a mismatch in microtask timing. The
promises are resolved/rejected in the wrong order.
Part of #34866
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
The first step for aborting fetch calls. It only
has the case where the signal was already aborted
prior to fetch starting.
Part of #34866
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
While trying to figure out what the status of this implementation was, I
added steps and comments to
see what we are missing. Also updated some links,
since I couldn't find an implementation of
`window.fetch`, since the spec URL was pointing
to the chapter instead of the algorithm.
Part of #34866
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
In single process mode, there is a race condition on the initialization
of the global fetch thread: once initialized the global fetch thread
will always use a given core resource thread, and this will be
determined by the component who first initializes it. For example, if
the canvas paint thread first does an async fetch, then this will set
the public core resource as used for all future fetches, including those
coming from a pipeline in private mode.
In multi-process mode, there is a race condition per window event-loop:
the first pipeline to use the fetch will set the core resource thread
for all others.
To ensure the fetch thread uses the correct core resource thread(private
vs public), we need to
pass the core resource thread to each fetch thread operation for which
is it needed.
Testing: It should not break existing fetch WPT tests. The race
condition is not something that can be tested reliably, but it seems to
be based on solid logic.
Fixes: follow-up from
https://github.com/servo/servo/pull/38421/files#r2248950924
---------
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
Any CSP violations happening when loading a worker should be reported
on the global of the document that creates the worker. Since workers
run in different threads, we can't pass in this parent global into
the worker global scope. Instead, we need to send a message to the
parent event loop to report it on the correct global.
Part of https://github.com/servo/servo/issues/4577
Fixes https://github.com/servo/servo/issues/37027
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
The scripted_caller only has information if the context is coming
from a script. If an element fetch listener processes CSP
violations, then this information doesn't exist. Instead, we should
use the global URL and the line number. WPT tests don't appear
to expect a column number, as they are all zero. Not all elements
are updated, as I am not actually sure all of them need it.
The source position remains an Option, since there are also code
paths that don't correspond to element or script sources. Maybe
in the future we can always determine the source position, but
let's take small steps towards that.
Part of #4577
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
This refactoring moves various CSP-related methods away from GlobalScope
and Document into a dedicated entrypoint. It also reduces the amount of
imports of the CSP crate, so that types are consolidated into this one
entrypoint. That way, we control how CSP code interacts with the script
crate.
For reviewing purposes, I split up the refactoring into separate
distinct commits that all move 1 method(group) into the new file.
Testing: no change in behavior, only a build improvement + code cleanup
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Signed-off-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com>
All logic is implemented in `report_csp_violations` to avoid
pulling in various element-logic into SecurityManager.
Update the `icon-blocked.sub.html` WPT test to ensure that
the document is the correct target (verified in Firefox and Chrome).
Fixes#36806
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
It also updates the FetchResponseListener to process CSP violations to
ensure that iframe elements (amongst others) properly generate the CSP
events. These iframe elements are used in the Trusted Types tests
themselves and weren't propagating the violations before.
However, the tests themselves are still not passing since they also use
Websockets, which currently aren't using the fetch machinery itself.
That is fixed as part of [1].
[1]: https://github.com/servo/servo/issues/35028
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Josh Matthews <josh@joshmatthews.net>
Propagate through documents a flag that represents if any of the
ancestor navigables has a potentially trustworthy origin.
The "potentially trustworthy origin" concept appears to have gotten
confused in a couple of places and we were instead testing if a URL had
"potentially trustworthy" properties.
The main test for the ancestor navigables is
[mixed-content/nested-iframes](https://github.com/web-platform-tests/wpt/blob/master/mixed-content/nested-iframes.window.js)
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix#36108
<!-- Either: -->
- [X] There are tests for these changes
---------
Signed-off-by: Sebastian C <sebsebmc@gmail.com>
* Add doc comments to RequestBuilder fields/methods
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Implement Request::cryptographic_nonce_metadata
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Implement HTMLOrSVGElement::nonce
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Set request cryptographic nonce metadata for link elements
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Set request's cryptographic nonce when fetching scripts
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Forward request nonce to rust-content-security-policy
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Update WPT expectations
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
---------
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
`EmbedderMsg` was previously paired with an implicit
`Option<WebViewId>`, even though almost all variants were either always
`Some` or always `None`, depending on whether there was a `WebView
involved.
This patch adds the `WebViewId` to as many `EmbedderMsg` variants as
possible, so we can call their associated `WebView` delegate methods
without needing to check and unwrap the `Option`. In many cases, this
required more changes to plumb through the `WebViewId`.
Notably, all `Request`s now explicitly need a `WebView` or not, in order
to ensure that it is passed when appropriate.
Signed-off-by: Delan Azabani <dazabani@igalia.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Instead of creating an IPC channel for every fetch, allow cancelling
fetches based on the `RequestId` of the original request. This requires
that `RequestId`s be UUIDs so that they are unique between processes
that might communicating with the resource process.
In addition, the resource process loop now keeps a `HashMap` or `Weak`
handles to cancellers and cleans them up.
This allows for creating mutiple `FetchCanceller`s in `script` for a
single fetch request, allowing integration of the media and video
elements to integrate with the `Document` canceller list -- meaning
these fetches also get cancelled when the `Document` unloads.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
No longer hide errors while queueing tasks on the main thread. This
requires creating two types of `TaskSource`s: one for the main thread
and one that can be sent to other threads. This makes queueing a bit
more efficient on the main thread and more importantly, no longer hides
task queue errors.
Fixes#25688.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* fix: add source browsing ctx id to request when initiate navigation
Signed-off-by: Jason Tsai <git@pews.dev>
* chore: clippy
Signed-off-by: Jason Tsai <git@pews.dev>
* Update components/net/http_loader.rs
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Signed-off-by: Jason Tsai <git@pews.dev>
* chore: apply suggestions
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Signed-off-by: Jason Tsai <git@pews.dev>
* chore: fix naming
Signed-off-by: Jason Tsai <git@pews.dev>
* refactor: set request browsing ctx id on pre page load
Signed-off-by: Jason Tsai <git@pews.dev>
---------
Signed-off-by: Jason Tsai <git@pews.dev>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Instead of creating a type for each `TaskSource` variety have each `TaskSource`
hold the same kind of sender (this was inconsistent before, but each
sender was effectively the same trait object), a pipeline, and a
`TaskSourceName`. This elminates the need to reimplement the same
queuing code for every task source.
In addition, have workers hold their own `TaskManager`. This allows just
exposing the manager on the `GlobalScope`. Currently the `TaskCanceller`
is different, but this will also be eliminated in a followup change.
This is a the first step toward having a shared set of `Sender`s on
`GlobalScope`.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Implement PolicyContainer
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
* implement small parts of fetch that interact with policy container
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
* fix: allow policy container's csp list to be unset
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
* fix: use the correct default policy when parsing from a token
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
---------
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
