Commit Graph

358 Commits

Author SHA1 Message Date
Jamie Mansfield
781a571669 LibWeb/Fetch: Handle edge cases in 'get, decode, and split'
See:
- https://github.com/whatwg/fetch/pull/1769
- https://github.com/whatwg/fetch/commit/3153e5e

(cherry picked from commit 84351dfa51dc8bd0046c3a9ec3e574c58fe9f790)
2024-11-10 10:46:01 -05:00
sideshowbarker
7959c42f78 LibWeb/Fetch: Set HTTP status code on cached responses
This change causes HTTP status codes to be set on cached HTTP responses.

Otherwise, without this change, no status codes at all are set on cached
HTTP responses — which causes all cached responses to default to being
loaded/served with a 200 status code. And as a result of that, if the
cached response is from a 30x redirect, then without this change, when
that cached 30x response is loaded, we don’t follow the redirect —
because we see a 200 status, rather than the expected/original 30x.

Fixes https://github.com/LadybirdBrowser/ladybird/issues/863

Note that this change also reverts the temporary workaround added in
https://github.com/LadybirdBrowser/ladybird/commit/f735c464d3f
(https://github.com/LadybirdBrowser/ladybird/pull/899).

(cherry picked from commit 23da1752b50568f2c49b1c63c2777ddffddaf6f5)
2024-11-07 22:17:03 -05:00
Jelle Raaijmakers
6264698077 LibWeb: Check presence of WWW-Authenticate header in fetch response
If a HTTP 401 response we get does not contain a `WWW-Authenticate`
header, we should not trigger the logic to ask the user for credentials
and retry the request.

This part is hinted at in a TODO / 'Needs testing' remark in the spec
but needs to be fleshed out. Raised an upstream issue to do so:

  https://github.com/whatwg/fetch/issues/1766

This fixes login forms triggering an infinite fetch loop when providing
incorrect credentials.

Co-Authored-By: Victor Tran <vicr12345@gmail.com>
(cherry picked from commit e7984a77116d47fde150f81f6e18cae6aaa147ad)
2024-11-07 22:17:03 -05:00
Jamie Mansfield
a10ae0a1e4 LibWeb/Fetch: Add a constant for the keepalive maximum size
(cherry picked from commit 35047de1d8ce57bdab12d47044abff36145fca30)
2024-11-07 22:17:03 -05:00
Jamie Mansfield
d51f03809b LibWeb/Fetch: Correct check for X-Method-Override
Previously the code was checking for X-Method.

See:
 - http://wpt.live/fetch/api/basic/request-forbidden-headers.any.html
(cherry picked from commit 5151f94d393ca6e30a2ca7a34665e2cf6e1f5af9)
2024-11-07 22:17:03 -05:00
Jamie Mansfield
c896b8bf3d LibWeb/Fetch: Add Set-Cookie as a forbidden header name
See:
 - https://github.com/whatwg/fetch/commit/50d77e6
 - http://wpt.live/fetch/api/request/request-headers.any.html
(cherry picked from commit f88e7bc5ee5577c260b1ea263a02b2d5f68e532b)
2024-11-07 22:17:03 -05:00
sideshowbarker
7b2a1208b0 LibWeb/Fetch: Don’t cache redirects
This change disables caching for 301, 302, 303, 307, and 308 responses.
This is just for now, ad-hoc — not adhering to any particular spec.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/863

(cherry picked from commit f735c464d3fe02ac43a1fa46c82ae9a3bb5de8b1)
2024-11-07 22:17:03 -05:00
Jamie Mansfield
a49db64dac LibWeb: Make preferred languages configurable
This also changes fetch to use the preferred languages for the
Accept-Language header.

(cherry picked from commit 2ca8fd1832462c05fdec16a1de73494820597140)
2024-11-05 11:41:15 -05:00
Andreas Kling
09a3dcb9c9 LibWeb: Implement very basic in-memory HTTP caching
This patch adds a simple in-memory HTTP cache to each WebContent
process.

It's currently off by default (turn it on with --enable-http-cache)
since the validation logic is lacking and incomplete.

(cherry picked from commit a3c8e60710451c1325f6718b253e8d1ee2029a30)
2024-11-05 11:22:07 -05:00
Andreas Kling
197076ed7b LibWeb: Make HTTP response directive parsing more robust
(cherry picked from commit 47169db4529d1310d8802b3b60bb97cf926374a7)
2024-11-05 11:22:07 -05:00
Jamie Mansfield
21d9eb17c2 LibWeb/Fetch: Use correct header for prefetch requests
See:
 - https://github.com/whatwg/fetch/commit/a5560d2
(cherry picked from commit 9ce727d3155628fb0135a99fcc80aeff6c158fa2)
2024-11-04 13:40:07 -05:00
Jamie Mansfield
1af28439d8 LibWeb/Fetch: Add missing fetch step for preloaded resources
This is currently no-op and a FIXME exists to implement the "consume a
preloaded resource" AO.

(cherry picked from commit aee77b975cdd2efddb57a4648c84d4e208cf50c9)
2024-11-04 13:40:07 -05:00
Jamie Mansfield
f1052ca896 LibWeb: Support sending DNT header with requests
(cherry picked from commit fb2032697986f0048bf84ba03f60b5d851333c06)
2024-11-01 12:00:07 -04:00
Shannon Booth
3b2bf6534d LibWeb: Don't propogate small OOMs from URLSearchParams
Made easier now that URL percent encode after encoding is also not
throwing any errors. This simplfies a bunch of error handling.

(cherry picked from commit df4739d7ced4159deb2b3e40ba6a1a08b7e7dd5b)
2024-10-19 15:26:29 -04:00
Shannon Booth
db4bab8041 LibURL: Make percent_encode return a String
This simplifies a bunch of places which were needing to error check and
convert from a ByteString to String.

(cherry picked from commit 84a7fead0eefd967d4319f4d71c0a0ca3095d2d1)
2024-10-16 23:56:40 -04:00
Shannon Booth
1308cab372 LibURL+LibWeb: Do not percent decode in password/username getters
Doing it is not part of the spec. Whenever needed, the spec will
explicitly percent decode the username and password.

This fixes some URL WPT tests.

(cherry picked from commit f511c0b441a591bc85f409242229c7b295e118e4)
2024-10-15 12:08:50 -04:00
Andreas Kling
eacf504b30 LibWeb: Make HTML::Task IDs a sequential, distinct numeric type
This also fixes a bug where task IDs were being deallocated from the
wrong IDAllocator. I don't know if it was actually possible to cause any
real trouble with that mistake, nor do I know how to write a test for
it, but this makes the bug go away.

(cherry picked from commit 08d60d7521915b4f73fd3bb40a1aa159f4f31eb1)
2024-10-11 09:27:51 -04:00
kleines Filmröllchen
a3077203fe AK: Don't implicitly convert Optional<T&> to Optional<T>
C++ will jovially select the implicit conversion operator, even if it's
complete bogus, such as for unknown-size types or non-destructible
types. Therefore, all such conversions (which incur a copy) must
(unfortunately) be explicit so that non-copyable types continue to work.
2024-09-14 13:30:27 +02:00
Kenneth Myhra
d15dc82b15 LibWeb: Append fetch record to client's fetch group when request is a
...subresource.

(cherry picked from commit a744a9ebe7c3ff8134d9e529b4d0436ae5ed5f02)
2024-07-28 07:31:26 -04:00
Kemal Zebari
ddefb5a822 LibWeb: Implement Blob::bytes()
Implements https://w3c.github.io/FileAPI/#dom-blob-bytes.

(cherry picked from commit c5f1e478838092dcf6e4ad8ee0bfef32a47e2d68)
2024-07-28 07:29:31 -04:00
Kenneth Myhra
1d4e673e60 LibWeb: Implement formData() method steps for x-www-form-urlencoded
The Response interface of the Fetch API can now parse form urlencoded
bodies when Content-Type is set to 'application/x-www-form-urlencoded'.

(cherry picked from commit b8fa572c6742c0f1f63da0f63c8b86835a86988d)
2024-07-23 13:06:46 -04:00
mobounya
219c40a4ba LibWeb: Conform HTTP-network-or-cache to specs
Implement step 8.10 in '4.5. HTTP-network-or-cache fetch' from the
fetch specs.

(cherry picked from commit 9c93630d02c5e86130bfb45f9d94ba6e5c6c3ba5)
2024-07-21 16:54:54 -04:00
mobounya
e4fa0e7f63 LibWeb: Implement fetch record from the fetch spec
Implement fetch record concept from the '2.4. Fetch groups' in
the fetch specs.

(cherry picked from commit 8d38a1326e5a3cfe4ebbb0bf3d12c8307a6fbb3c)
2024-07-21 16:54:54 -04:00
circl
8998afc9c3 LibWeb/Fetch: Pass error from ResourceLoader into network_error
(cherry picked from commit 4e6eb35520def1b1488af6a800b03c007e56b9dd)
2024-07-14 19:57:14 -04:00
Jamie Mansfield
6e9f9b5efc LibWeb: Implement EmbedderPolicy struct
(cherry picked from commit 190a41971569ad90c26662d7e7e850a61c55c8dd)
2024-07-14 16:45:35 -04:00
circl
98a5fff68b LibWeb: Restrict fetching file: and resource: URLs to internal pages
They are now blocked on pages which:
- Don't have an opaque origin (should be only user-initiated or about:)
- Aren't other file: pages
- Aren't other resource: pages

(cherry picked from commit 1f3285eb0410ff5c902e148932205d9e4b7fbd9b)
2024-06-26 23:11:35 +02:00
Andreas Kling
c9a1eebcb9 Everywhere: Add HTTP::HeaderMap and use for response headers
Instead of using a HashMap<ByteString, ByteString, CaseInsensitive...>
everywhere, we now encapsulate this in a class.

Even better, the new class also allows keeping track of multiple headers
with the same name! This will make it possible for HTTP responses to
actually retain all their headers on the perilous journey from
RequestServer to LibWeb.

(cherry picked from commit e636851481eabdf00953573a5eb459ee52feeacc)

Updated various SerenityOS components to make it build.

Fetch: Make sure we iterate over HeaderMap's headers()

This fixes a build failure when built with CMake option
'-DENABLE_ALL_THE_DEBUG_MACROS=ON'.

(cherry picked from commit c51d01bea712d75f9b2cd700be942935044e49b4)
2024-06-10 12:01:57 +02:00
Jamie Mansfield
4dfb52f133 LibWeb: Implement should block mixed content response to request 2024-06-08 14:07:54 -04:00
Jamie Mansfield
e59bccd25e LibWeb: Upgrade mixed requests to potentially trustworthy URLs
(if appropriate)
2024-06-08 14:07:54 -04:00
Jamie Mansfield
7814dad6cf LibWeb: Implement should block mixed content request 2024-06-08 14:07:54 -04:00
Jamie Mansfield
e7b752eb2d LibWeb: Implement "Set request’s referrer policy on redirect" AO 2024-06-08 14:07:38 -04:00
Jamie Mansfield
a5d298ff7a LibWeb/Fetch: Update "HTTP-redirect fetch" algorithm to latest spec
The spec and implementation's comments had diverged a little, this
brings them in line :)
2024-06-08 14:07:38 -04:00
Jamie Mansfield
aa627454da LibWeb/Fetch: Update "HTTP fetch" algorithm to latest spec comments
The spec and implementation's comments had diverged a little, this
brings them in line :)
2024-06-05 05:04:11 +01:00
Jamie Mansfield
bf8c6f939e LibWeb: Move NetworkPartitionKey into a header
This is used elsewhere, such as the HTMLLinkElement.
2024-06-05 05:02:55 +01:00
Jamie Mansfield
2b5c3886d9 LibWeb: Add from_string function for request priority 2024-06-05 05:02:55 +01:00
Jamie Mansfield
295c4ef51a LibWeb/Fetch: Use MimeType in DataURL 2024-06-02 19:55:53 +02:00
Matthew Olsson
9ea6ab0ad4 LibJS+LibWeb: Fix a ton of JS_CELL-like macro issues 2024-05-30 09:29:20 -06:00
Timothy Flynn
6056428cb5 LibWeb: Support unbuffered fetch requests
Supporting unbuffered fetches is actually part of the fetch spec in its
HTTP-network-fetch algorithm. We had previously implemented this method
in a very ad-hoc manner as a simple wrapper around ResourceLoader. This
is still the case, but we now implement a good amount of these steps
according to spec, using ResourceLoader's unbuffered API. The response
data is forwarded through to the fetch response using streams.

This will eventually let us remove the use of ResourceLoader's buffered
API, as all responses should just be streamed this way. The streams spec
then supplies ways to wait for completion, thus allowing fully buffered
responses. However, we have more work to do to make the other parts of
our fetch implementation (namely, Body::fully_read) use streams before
we can do this.
2024-05-26 18:29:24 +02:00
Lucas CHOLLET
b30c361b08 LibWeb/Fetch: Implement logic to process a response from HTTP's cache
Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-23 13:25:29 -04:00
Lucas CHOLLET
fcbc6123bb LibWeb/Fetch: Add freshness notion to HTTP::Response
Defined in RFC9111 "HTTP Caching".

Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2024-05-23 13:25:29 -04:00
Andrew Kaster
c05f296014 LibWeb: Start adding infrastructure for an HTTP Cache 2024-05-23 13:25:29 -04:00
Andrew Kaster
a40b331f39 LibWeb: Update steps of HTTP-network-or-cache-fetch per the latest spec
One step was added since we last visited this AO.
2024-05-23 13:25:29 -04:00
Kenneth Myhra
76418f3ffa LibWeb: Implement the concept incrementally read a body 2024-05-20 16:57:52 -04:00
Kenneth Myhra
50f454fdd5 LibWeb: Enqueue body chunks in Fetch::extract_body()
Co-authored-by: Timothy Flynn <trflynn89@pm.me>
2024-05-20 16:57:52 -04:00
Kenneth Myhra
f119ac1a9d LibWeb: Integrate TransformStream into fetch_response_handover() 2024-05-20 16:57:52 -04:00
Timothy Flynn
b5ba60f1d1 LibWeb: Change Fetch's ProcessBodyError to accept a plain JS value
This callback is meant to be triggered by streams, which does not always
provide a WebIDL::DOMException. Pass a plain value instead. Of all the
users of this callback, only one actually uses the value, and already
converts the DOMException to a plain value.
2024-05-20 16:57:52 -04:00
Jamie Mansfield
3438293e7b LibWeb/Fetch: Share a conditional in fetch response handover
See:
- https://github.com/whatwg/fetch/commit/aaada1f
2024-05-19 16:25:50 +02:00
Jamie Mansfield
951fbb1837 LibWeb/Fetch: Expose a minimised Content-Type to Resource Timing
See:
- https://github.com/whatwg/fetch/commit/931cd06
2024-05-19 16:25:50 +02:00
Jamie Mansfield
08e4cf1f3b LibWeb/Fetch: Implement Body::bytes()
See:
- https://github.com/whatwg/fetch/commit/1085f4f
2024-05-19 16:25:50 +02:00
Shannon Booth
53eb9af42f LibWeb: Use URL's 'blob URL entry' for blob fetches
Performing a lookup in the blob URL registry does not work in the case
of a web worker - as the registry is not shared between processes.
However - the URL itself passed to a worker has the blob attached to it,
which we can pull out of the URL on a fetch.
2024-05-12 15:46:29 -06:00