Add support for temporary storage via a new config option
`temporary_storage`
and a corresponding command-line argument `--temporary-storage`.
When enabled, client storage uses a storage directory
(e.g. `clientstorage/temporary/<uuid>`) instead of the shared default
location.
This can be used to provide isolation between concurrent servo
instances.
This is especially useful for WPT runs, where multiple Servo instances
may
execute in parallel and would otherwise share the same storage, leading
to
cross-test interference.
Based on that, this PR also updates the WPT runner to enable temporary
storage
by default.
Testing: Manual testing and a full try run.
Signed-off-by: Jan Varga <jvarga@igalia.com>
Add the Storage Standard WebIDL for NavigatorStorage and StorageManager,
wire navigator.storage on Window and Worker, and implement persisted(),
persist(), and estimate().
Testing: covered by WP test.
part of #39100fixes#39101
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
This removes some complains that clippy will have in 1.95.
As this is mostly just match guards, it doesn't update MSRV.
Testing: This is equivalent exchanges, so current WPT would find
anything.
---------
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Integrate client storage into indexeddb for the creation and deletion of
databases.
Testing: Existing WPT tests.
Fixes: None
---------
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
Changes all the uses of `ServoUrl` to `ImmutableOrigin` in
`WebStorageThreadMsg`
Testing: Adds a test case as described in the issue.
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title></title>
</head>
<body>
<main>
<h1>Tab and Program crashes!</h1>
<iframe srcdoc="
<body>
<p>Hello Web users! <br/> I am an iframe, <br /> I set a key and then retreive it. </p>
<script>
for (let i = 0; i < 1000; i++) {
window.localStorage.setItem('truth', 'Servo is awesome! 🌐')
const example = window.localStorage.getItem('truth')
console.log(example === 'Servo is awesome! 🌐')
}
</script>
</body>
"></iframe>
</main>
</body>
</html>
```
Fixes: #44001
---------
Signed-off-by: Niya Gupta <niyabits@disroot.org>
Indexeddb keys need to remain sorted in the database, so that key range
queries can be made via sql. This allows for key range querying without
having to load each key into memory. The solution is to serialize keys
with a custom encoding that sorts them according to the spec. The
implementation is taken from firefox.
Testing: WPT, and unit tests
Fixes: None
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Removed the ```.unwrap()``` in ```add_new_environment``` causing the
storage thread to panic on SQLite failures
Testing: Ran ```./mach test-wpt tests/wpt/tests/webstorage/```
Result;
```
▶ TIMEOUT [expected OK] /webstorage/storage_local_setitem_quotaexceedederr.window.html
Ran 53 tests finished in 63.9 seconds.
• 52 ran as expected.
• 1 tests timed out unexpectedly
```
Fixes: #43880
---------
Signed-off-by: Sabb <sarafaabbas@gmail.com>
Fix object-store key range handling
Testing: key range handling wpt test fixed.
Fixes: part of #40983
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
Abort pending upgrade requests when a worker closes, correctly roll back
newlyy created databases by deleting backend state on old_version == 0.
Testing: IndexedDB/worker-termination-aborts-upgrade.window.js.ini test
pass.
part of https://github.com/servo/servo/issues/40983
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
This PR considers the following constraints:
- Resources must be available when building servo via a published
crates.io package (i.e. no `../../../resources/<file>` file references).
- Minimal setup when writing tests (`nextest` spawns each test in its
own process, so we don't want to explicitly initialize the resource
handler for every `#[test]` fn)
- Use local resources when developing locally
- Support loading the resources from a proper resource directory if the
embedder wishes so, including via a custom mechanism, not necessarily as
files
(File) Resources that are only accessed from servoshell are out of scope
of this PR, since it mainly focusses on unblocking publishing `libservo`
to crates.io.
Baking the resources into the binary by default simplifies the setup a
lot. We already supported that before, but only for testing purposes and
explicitly not for production builds.
Using [`inventory`](https://crates.io/crates/inventory) adds a simple
way for the embedder to replace the default baked in resources, while
also keeping the test usage of baked in resources simple.
rippy.png is also referenced from image_cache - We simply duplicate it,
since the image is small, to avoid adding unnecessarily complex
solutions like adding a dedicated crate.
Testing: Covered by existing tests. [mach try
full](https://github.com/jschwe/servo/actions/runs/23811669469)
Fixes: Part of #43145
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
Implement the missing IDBObjectStore.name duplicate name checks so
renaming an object store to an existing store now throw ConstraintError
Testing: more indexeddb WPT test passed.
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
indexeddb: make put() overwrite existing SQLite records
Testing: more indexeddb test should pass.
Fixes: #43707
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
Follow up of #43526. This addresses Nico's comment:
https://github.com/servo/servo/pull/43526#issuecomment-4104953308
- `bluetooth_traits` -> `servo_bluetooth_traits`
- `base` -> `servo_base`
- `bluetooth` -> `servo_bluetooth`
- `background_hang_monitor` -> `servo_background_hang_monitor`
Testing: This should not change any behaviour.
---------
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
Add further missing repository keys / descriptions, which I missed in
#43451 due to a suboptimal grep (assuming that rust-version.workspace is
always the last item).
Additionally fix crate self-references, which caused cargo-publish to
fail, due to it trying to fetch the crate from crates.io.
When specifying the current crate in `[dev-dependencies]` to enable a
test feature or similar, apparently one should not use `workspace =
true` and instead use `path`.
This requires extending the previously added `tidy` check, to allow
`path` dependencies in this specific case outside of the workspace
Cargo.toml.
Testing: Covered by existing tests
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
RustRover flags unused dependencies gray in Cargo.toml, which works well
except for macro / derive related crates (false-positives). Based on
that I removed the flagged crates (after double checking with grep based
search).
There is one weird case in `storage`, where `tokio` was added as a
dependency with multiple feature flags enabled, without tokio actually
being used. Since the same feature combination is present in `net`,
probably this is a remnant of migrating storage from net.
Testing: If CI passes, this is fine. I additionally also searched for
references of the packages I removed in the respective crate (I hope I
didn't forget any, but should be 90% good).
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
In order to prepare for publishing this PR does the following steps (see
commits):
- Move all `path` dependencies to the workspace Cargo.toml, and
reference that.
- Move all path dependencies in the workspace Cargo.toml into a
dedicated section, to make bumping version numbers easier later.
- Add the version requirement. Note that we currently only version bump
servoshell. There was agreement to version everything with the same
version as servoshell, but that be done in a follow-up. The diff is
already large enough as is.
- Add a tidy lint to catch `path` usages outside the root Cargo.toml. I
switched to [`tomllib`] (which was added to the python stdlib in 3.11),
since the third-party `toml` library failed to parse Cargo.toml files
with `workspace.version` (did not like the `.` in a `key`).
[`tomllib`]: https://docs.python.org/3/library/tomllib.html
Testing: Should be covered by regular CI testing.
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This PR makes public and private storage thread groups independent.
previously, `new_storage_threads()` returned two `StorageThreads`
handles, but both handles were backed by the same underlying storage
worker group. this meant Servo carried a public/private distinction
through the API and wiring layers, while both sides still talked to the
same storage backend threads.
Testing: Added shutdown_storage_group test, and should not effect WTP
test.
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
The frontend connection metadata was not consistently synchronized with
backend object store state during open/upgrade, and upgrade abort did
not fully restore the connection object store set in all cases.
Testing: More indexeddb tests should pass.
part of https://github.com/servo/servo/issues/40983
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
This is a preparation for publishing to crates.io. Changes include:
- Add `servo-` prefixes to avoid name collisions on crates.io
- Use `-` instead of `_` in package names.
- Rename the crates to their original names in Cargo.toml,
to keep the diff minimal
- Rename `media` to `servo-media-thread` to avoid name collision with
`servo-media` (originally from the media repository).
This is an outcome of the previous discussion at [#general > Switch
remaining git dependencies to
crates.io](https://servo.zulipchat.com/#narrow/channel/263398-general/topic/Switch.20remaining.20git.20dependencies.20to.20crates.2Eio/with/576336288)
Testing: This should be mostly covered by our CI, but some amount of
breakage is to be expected, since some package names could still be
referenced from scripts which are not tested or run in CI. [mach try
run](https://github.com/jschwe/servo/actions/runs/22502945949)
---------
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This changes does three things:
- Updates all specification links to reflect version 3 of the
specification
- Renames the `IDBDatabase::close` member to
`IDBDatabase::close_pending` to better match the specification
- Adds a `OpenRequest::processed` member, again to better match the
specification
Testing: This should not change behavior, so is covered by existing
IndexedDB WPT tests.
Fixes: Part of #40983
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
Indexeddb: transaction lifecycle
Testing: more Indexeddb test should pass
part of #40983
---------
Signed-off-by: Taym Haddadi <haddadi.taym@gmail.com>
#42442 triggered a bunch of intermittent panics due to DBs being cleaned
up while tests were still trying to use them. This is very difficult to
test reliably, but replacing calls to unwrap with sending error values
to the script thread is clearly an improvement.
Testing: Some intermittent crashes now turn into intermittent timeouts.
---------
Signed-off-by: Josh Matthews <josh@joshmatthews.net>
Co-authored-by: Martin Robinson <mrobinson@igalia.com>
`local_storage_origins` is ~2-3kb of memory, which is probably large
enough to be included in the report, given that it likely can grow much
more. A much bigger memory user is sqlite3, but I'm still working on
finding a way to extract memory usage for it.
Testing: Manual
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Retrieve already stored indexes from the backend when querying for other
object store data.
Also removes the extraneous blocking IPC call when creating an
`IDBObjectStore` object by batching all the info into a single struct.
Testing: WPT
Fixes: #42438
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Implements `createIndex` and `deleteIndex`, they have already been
implemented in the backend.
Testing: WPT
Fixes: Partially #38100
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Implement the full connection lifecycle, from opening and potentially
upgrading a database, waiting on existing connections to close, firing
the versionchange and blocked events, and updating pending open requests
when connections close.
Testing: WPT
Fixes: Part of https://github.com/servo/servo/issues/40983
---------
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
Because of the deprecation of bincode we are switching to postcard
(which will already be used by ipc-channel).
This changes all usages to postcard.
Sadly there are still some dependencies using bincode, so we are
probably increasing the binary size.
Currently we do not modify the deny.toml because we have dependencies
that depend on bincode.
Testing: Compilation should be enough for test.
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Bug that would result in local storage changes being overwritten by
session storage ones.
Testing: Added a unit test
Fixes: #41324
---------
Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
Move the message handling related to opening new connections to the
factory, and store open request on it. This is a first step towards
tracking connections and allowing two-way messaging regarding them.
Also the very beginnings of tracking connections on the backend.
Testing: WPT
Fixes: Part of https://github.com/servo/servo/issues/40983
---------
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
Track open requests by their ID, so that specific requests can be
aborted instead of aborting all request for a given db name. This is
important because worker and windows at the same origin could be opening
databases and aborting their opening, but this should not affect other
scopes at the origin.
Testing: Existing WPT test suites.
Fixes: None, but part of https://github.com/servo/servo/issues/40983
---------
Signed-off-by: gterzian <2792687+gterzian@users.noreply.github.com>
localStorage entries are identified via their associated origins found
for the
given sites. This brings localStorage in line with cookies and
sessionStorage,
which were already handled by the API.
Both public and private browsing contexts are included in the clearing
operation.
The necessary support and testing has been added to the storage crate to
clear
localStorage data.
Testing: New unit tests and a new integration test have been added.
---------
Signed-off-by: Jan Varga <jvarga@igalia.com>
sessionStorage entries are identified via their associated origins and
removed
across all browsing contexts. This brings sessionStorage in line with
cookies,
which were already handled by the API.
Both public and private browsing contexts are included in the clearing
operation.
The necessary support has been added to the storage crate to clear
WebStorage
data.
Testing: A new integration test has been added.
Signed-off-by: Jan Varga <jvarga@igalia.com>
There would be a follow up to bump all tokio dependencies together with
bot, as we see three separate PRs today. What a waste of electricity.
Testing: If it builds and binary size stays same as previous, it works.
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
Changed most #[allow]s to #[expect]s, mainly for
clippy::too_many_arguments
Removed unfulfilled expectations
This is my first opensource contribution, so please let me know if
anything should
be done differently.
Testing: Refactor
Part of: #40838
---------
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
Signed-off-by: TimurBora <timurborisov5561@gmail.com>
Co-authored-by: Tim van der Lippe <TimvdLippe@users.noreply.github.com>
- Update sea-query 1.0.0-rc.26 -> 1.0.0-rc.29
- Update sea-query-rusqlite 0.8.0-rc.14 -> 0.8.0-rc.15
- Move both to the root so that dependabot can update them.
- Create a group for dependabot rules.
Testing: It builds.
Fixes: #41573
Signed-off-by: Euclid Ye <yezhizhenjiakang@gmail.com>
