Once a cache entry is not fresh, we now remove it from the memory cache.
We will avoid handling revalidation from within WebContent. Instead, we
will just forward the request to RequestServer, where the disk cache
will handle revalidation for itself if needed.
We currently disable the disk cache because the WPT runner will run more
than one RequestServer process at a time. The SQLite database does not
handle this concurrent read/write access well.
We will now enable the disk cache with a per-process database. This is
needed to ensure that WPT Fetch cache tests are sufficiently handled by
RequestServer.
We currently set the response time to when the cache entry writer is
created. This is more or less the same as the request start time, so
this is not correct.
This was a regression from 5384f84550.
That commit changed when the writer was created, but did not move the
setting of the response time to match.
We now set the response time to when the HTTP response headers have been
received (again), which matches how Chromium behaves:
https://source.chromium.org/chromium/chromium/src/+/refs/tags/144.0.7500.0:net/url_request/url_request_job.cc;l=425-433
If we have the response for a non-Range request in the memory cache, we
would previously use it in reply to Range requests. Similar to commit
878b00ae61f998a26aad7f50fae66cf969878ad6, we are just punting on Range
requests in the HTTP caches for now.
A future commit will format memory cache debug messages similarly to the
disk cache messages. To make it easy to read them both at a glance when
both debug flags are turned on, let's add a prefix to these messages.
We currently do not handle responses for range requests at all in our
HTTP caches. This means if we issue a request for a range of bytes=1-10,
that response will be served to a subsequent request for a range of
bytes=10-20. This is obviously invalid - so until we handle these
requests, just don't cache them for now.
If the cURL request completes with anything other than CURLE_OK, we must
not keep the cache entry. For example, if the server's connection closes
while transferring data, we receive CURLE_PARTIAL_FILE. We don't want
this cache entry to be treated as valid in a subsequent request.
The in-memory HTTP Fetch cache currently keeps the realm which created
each cache entry alive indefinitely. This patch migrates this cache to
LibHTTP, to ensure it is completely unaware of any JS objects.
Now that we are not interacting with Fetch response objects, we can no
longer use Streams infrastructure to pipe the response body into the
Fetch response. Fetch also ultimately creates the cache response once
the HTTP response headers have arrived. So the LibHTTP cache will hold
entries in a pending list until we have received the entire response
body. Then it is moved to a completed list and may be used thereafter.
No need to duplicate this in LibWeb.
In doing so, this also fixes an apparent bug for SWR handling in LibWeb.
We were previously deciding if we were in the SWR lifetime with:
stale_while_revalidate > current_age
However, the SWR lifetime is meant to be an additional time on top of
the freshness lifetime:
freshness_lifetime + stale_while_revalidate > current_age
This directive allows our disk cache to serve stale responses for a time
indicated by the directive itself, while we revalidate the response in
the background.
Issuing requests that weren't initiated by a client is a new thing for
RequestServer. In this implementation, we associate the request with
the client that initiated the request to the stale cache entry. This
adds a "background request" mode to the Request object, to prevent us
from trying to send any of the revalidation response over IPC.
We were returning the incorrect result when upgrading a cache entry to
have exclusivity on must-revalidate requests. This could result in the
entry being read and updated at the same time, especially if the server
returned a non-304 response.
There are a couple of remaining RFC 9111 methods in LibWeb's Fetch, but
these are currently directly tied to the way we store GC-allocated HTTP
response objects. So de-coupling that is left as a future exercise.
We currently have two ongoing implementations of RFC 9111, HTTP caching.
In order to consolidate these, this patch moves the implementation from
RequestServer to LibHTTP for re-use within LibWeb.
