A new event `PlayerEvent::DurationChanged` has been added to the media
engine to simplify handling changes in the duration of a media stream
(sometimes the duration could be inaccurate or even determined by the
media engine with significant delay after the initial metadata was
ready).
This new event eliminates the need to check conditions for two
consecutive `MetadataUpdated` events (the initial metadata and the
metadata with the changed duration) during the loading phase.
servo-media:
- Add new `PlayerEvent::DurationChanged` event
(https://github.com/servo/media/pull/461)
Testing: No expected changes in test results
Fixes: https://github.com/servo/servo/issues/40626
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
The two are semantically equivalent, but `find_map` is more concise.
Testing: Covered by existing tests
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
Follow the HTML specification and split the previous `playback position`
to `current playback position` (a time on the media timeline) and
`official playbac position` (an approximation of the current playback
position that is kept stable while scripts are running) which must be
set to `the current playback position` any time the user agent provides
a stable state.
See https://html.spec.whatwg.org/multipage/#current-playback-position
See https://html.spec.whatwg.org/multipage/#official-playback-position
Note that at this point, there are no difference between the `current`
and `official` playback positions (as the time on the media timeline),
except the `seek` case.
Testing: Improvements in the following tests
-
html/semantics/embedded-content/media-elements/seeking/seek-to-max-value.htm
-
html/semantics/embedded-content/media-elements/seeking/seek-to-negative-time.htm
Fixes: https://github.com/servo/servo/issues/34496
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Following the HTML specification, a new internal method,
`ended_playback`, has been added to differentiate it from the `ended`
attribute:
- `ended_playback`: playback has ended for the `forward/backward`
directions
- `ended` attribute: playback has ended for the `forward` direction
See https://html.spec.whatwg.org/multipage/#ended-playback
Added descriptions for the steps of the `Play`, `Pause`,
`internal_play_steps` (new), `internal_pause_steps`,
`notify_about_playing`, `change_ready_state` methods.
Testing: No expected changes in test results.
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
The `seek` method isn't explicitly exposed in the API, but is called
from various places (`currentTime`, `fastSeek`, by the `media metadata`
event) and doesn't itself create new DOM objects, so the `canGC`
argument can be omitted.
It was previously required because calling the `Seekable` method creates
a new `TimeRanges` object, but it will now be replaced by the internal
`seekable` method to avoid interaction with the JavaScript engine (and
potential garbage collector).
The `earlyest possible position` method has been changed to match the
specification and use `seekable` instead of `played`. See
https://html.spec.whatwg.org/multipage/#earliest-possible-position
Testing: No expected changes in tests
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Updated the `PlayerEvent::PositionChanged/SeekDone` events to support
precise position timestamps (u64 -> f64).
Also set precise time value (seconds with fractional part) for the
HTMLMediaElement `duration` from media metadata to sync with
`PlayerEvent` changes.
servo-media:
- Pass precise position timestamp within PlayerEvent
(https://github.com/servo/media/pull/460)
Testing: No expected changes in test results
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
The goal of this change is to prevent having to copy so much data out of
listeners when a fetch completes, which will be particularly important
for off-the-main thread parsing of CSS (see #22478). This change has
pros and cons:
Pros:
- This makes the design of the `FetchResponseListener` a great deal
simpler.
They no longer individually store a dummy `ResourceFetchTiming` that is
only replaced right before `process_response_eof`.
- The creation of the `Arc<Mutex<FetchResponseListener>>` in the
`NetworkListener` is abstracted away from clients and now they just
pass the `FetchResponseListener` to the fetch methods in the global.
Cons:
- Now each `FetchResponseListener` must explicitly call `submit_timing`
instead of having the `NetworkListener` do it. This is arguably a bit
easier to follow in the code.
- Since the internal data of the `NetworkListener` is now an
`Arc<Mutex<Option<FetchResponseListener>>>`, when the fetching code
needs to share state with the `NetworkListener` it either needs to
share an `Option` or some sort of internal state. In one case I've
stored the `Option` and in another case, I've stored a new inner
shared value.
Testing: This should not change observable behavior and is thus covered
by existing tests.
Fixes: #22550
---------
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
For some specific cases (one of them see below) where are requirement to
identify the reason of the attribute mutation (by whom/why it was
changed - by cloning, parser or directly), so the following `reason`
parameter was added to `AttributeMutation::Set` option.
Note that for the HTMLMediaElement the internal `muted` state should be
set to true when the element is created and the element has a `muted`
content attribute specified (should be done once on the first attribute
mutation caused by parser or cloning).
See https://html.spec.whatwg.org/multipage/#dom-media-muted
Testing: Improvements in the following tests
-
html/semantics/embedded-content/media-elements/user-interface/muted.html
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Follow the specification and allow to render media controls (expose a
user interface to the user) without any HTMLMediaElement `readyState`
precondition.
The media controls be shown for the following examples without specified
media source (e.g. without `src` attribute).
```
<audio controls></audio>
<video controls>/<video>
```
See https://html.spec.whatwg.org/multipage/media.html#user-interface
For media controls UI don't define root `div` element's style `width`
property from the video natural size (`videoWidth/Height` IDL attributes
which return `0` if `readyState` is `HAVE_NOTHING`).
When the `media` element is adopted between different documents need
adopt the media controls id, so media controls UI
will keep access to the whilelist of media controls identifiers via
`document.servoGetMediaControls(id)` API.
Testing: Improvements in the following tests
-
html/rendering/replaced-elements/embedded-content-rendering-rules/audio-controls-00*
-
html/semantics/the-button-element/command-and-commandfor/on-audio-behavior.tentative.html
-
html/semantics/the-button-element/command-and-commandfor/on-audio-invalid-behavior.tentative.html
Note that `Invoker Commands` API is not yet supported so these `command
and commandfor` tests should fail, but due to previous bug with delayed
media controls shown up. The invoker `onClick` function send actions
with mouse events
(move/up/down) to the test driver at the position of the button element
(via getBoundingClientRect), but before test driver
will handle these actions the mediadata is received and media controls
are shown up with "play" button at this position.
So the media control "play" button is clicked instead of the button, and
it triggers media element `pause` state change.
See https://open-ui.org/components/invokers.explainer/
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Adding an optional message to be attached to a IndexSizeError.
The enum definition of IndexSize is now `IndexSize(Option<String>)`.
Future PRs should probably add more appropriate messages to some of the
`IndexSize(None)`s.
Testing: Just a refactor
Fixes: Partially #39053
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com>
`FetchReponseListener` has traditionally lived in `net` even though it
is only used in `script` currently. Because of the two way dependency,
it has also use a lot of templating to implement something pretty basic
(call methods on a trait object).
This change moves the trait to `script` and removes several levels of
templating, making the code quite a bit shorter and easier to
understand.
This change is preparation for fixing #22550 and implementing
off-the-main-thread CSS parsing.
Testing: This should not change any behavior so is covered by existing
tests.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
Removes some unneeded lints, especially `#[allow(unsafe_code)]`.
Testing: Refactor
Part of: #40383
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com>
The HTMLMediaElement should stop playback of any previously playing
media resource on running media element loading algorithm or before
attempt to load next source child.
This step will reset media player (including release of the underlying
media resources and media frames from the video renderer).
The closure for marshalling the incoming GLPLayer messages from ROUTER
("router-proxy") thread to script thread will use
`Weak<MediaFrameRenderer>` instead of `Trusted<HTMLMediaElement>` to
break circular references dependency (decouple ownership).
The media frame renderer will track the media player id to be able
ignore async locking render call with new frame after playback of any
previously playing media was stopped.
Testing: No WPT test expectation changes
Fixes (partially): https://github.com/servo/servo/issues/40243
Fixes (partially): https://github.com/servo/servo/issues/40265
Fixes: https://github.com/servo/servo/issues/37173
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Servo has a lot of comments like this:
```rust
// https://example-spec.com/#do-the-thing
fn do_the_thing() {}
```
and I keep turning these into doc comments whenever I'm working close to
one of them. Doing so allows me to hover over a function call in an IDE
and open its specification without having to jump to the function
definition first. This change fixes all of these comments at once.
This was done using `find components -name '*.rs' -exec perl -i -0777
-pe 's|^([ \t]*)// (https?://.*)\n\1(fn )|\1/// <$2>\n\1$3|mg' {} +`.
Note that these comments should be doc comments even within trait `impl`
blocks, because rustdoc will use them as fallback documentation when the
method definition on the trait does not have documentation.
Testing: Comments only, no testing required
Preparation for https://github.com/servo/servo/pull/39552
---------
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
Moves interfaces defined by the performance spec to the
`script/dom/performance/` module from `script/dom/`.
Testing: Just a refactor shouldn't need any testing
Fixes: Partially #38901
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com>
When changing the 'volume' attribute of media element, should also call
`set_volume()` api of underlying player if it is valid value.
Testing: No Test is required, as this should not affect any WPT Test
Fixes: N/A
---------
Signed-off-by: rayguo17 <rayguo17@gmail.com>
Signed-off-by: TIN TUN AUNG <62133983+rayguo17@users.noreply.github.com>
Co-authored-by: Euclid Ye <yezhizhenjiakang@gmail.com>
The 'media' element has multiple media data processing steps for media
source loading and requires properly handling any media network and
decoding errors between different components:
- element (resource selection algorithm)
- fetch context/listener (async response)
- media backend (async playback events)
The fetch listener will stop processing any actions (response/chunk/eof)
in case if current fetch request was cancelled (due to a
network/decoding error or it was aborted by user).
Also send EOS event to media backend ONLY once the entire media resource
has been fetched so media pipeline will not generate decoding (playback)
error for fetched empty sources (eg. "about:blank" or "data:,").
See
https://html.spec.whatwg.org/multipage/#media-data-processing-steps-list
Testing: No changes in WPT test expectations
Fixes: https://github.com/servo/servo/issues/39759 ("about:blank")
Fixes: https://github.com/servo/servo/issues/38980 ("data:,")
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
Refactored Seekable and Seek method in HtmlMediaElement, adding CanGC
argument on all its calls.
Testing: No tests needed
Fixes: #39944
---------
Signed-off-by: Domenico Rizzo <domenico.rizzo@gmail.com>
This implements LazyDOMString (from now on DOMString) as outlined in
https://github.com/servo/servo/issues/39479.
Constructing from a *mut JSString we keep the in a
RootedTraceableBox<Heap<*mut JSString>> and transform
the string into a rust string if necessary via the `make_rust_string`
method.
Methods used in script are implemented on this string. Currently we
transform the string at all times.
But in the future more efficient implementations are possible.
We implement the safety critical sections in a separate module
DOMStringInner which allows simple constructors, `make_rust_string` and
the `bytes` method.
This method returns the new type `EncodedBytes` which contains the
reference to the underlying string in either format.
Testing: WPT tests still seem to work, so this should test this
functionality.
---------
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Follow the HTML implemetation and support selection of the 'media'
element loading source from 'source' children elements with validation
processing over their 'media/src/type' attributes.
See https://html.spec.whatwg.org/multipage/#concept-media-load-algorithm
To handle loading with multiple 'source' children was introduced source
children pointer to track the remaining unprocessed 'source' elements
after the resource selection algorithm was invoked.
Testing: Changes in the following tests
with improvements:
-
html/semantics/embedded-content/media-elements/loading-the-media-resource/*
- webgl/tests/conformance/textures/misc/texture-video-transparent.html
with regressions (after enabled 'source' type attribute validation):
- content-security-policy/media-src/media-src*
- mixed-content/gen/top\*audio|video-tag*
- resource-timing/initiator-type/*
Fixes: https://github.com/servo/servo/issues/21481
Fixes: https://github.com/servo/servo/issues/34127
Signed-off-by: Andrei Volykhin <andrei.volykhin@gmail.com>
This updates all Rc that were ignored for malloc_size_of to use
conditional_malloc_size_of, unless the type in the Rc itself doesn't
support malloc_size.
Regular expressions used to search for all occurrences:
```
ignore_malloc_size_of = "Rc.*"
ignore_malloc_size_of = "Arc.*"
```
There are a couple left since they have nested Rc, which I don't know
how to fix.
To be able to define these, several new implementations were added to
`malloc_size_of/lib.rs` as well as
`HashMapTracedValues`.
Testing: if it compiles, it's safe
Signed-off-by: Tim van der Lippe <tvanderlippe@gmail.com>
This is part of the future work of implementing LazyDOMString as
outlined in https://github.com/servo/servo/issues/39479.
We use str() method or direct implementations on DOMString for these
methods. We also change some types.
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Testing: This is essentially just renaming a method and a type and
should not change functionality.
Signed-off-by: Narfinger <Narfinger@users.noreply.github.com>
Regular shadow roots can never be detached once they are created.
However we were specifically detaching shadow roots from media elements
when they were disconnected. This is actually one of the very few
aspects of shadow roots that predate the implementation work I did
earlier this year.
I'm not sure why we ever did this. Maybe its for efficiency reasons,
because keeping the shadow tree around is not necessary when the media
element is not connected. But I can't imagine this yields any benefits,
especially since you would have to reconstruct the shadow tree when the
media element is re-connected (as is the case in the crash we observe).
For simplicities sake, I have completely removed this functionality.
Doing so ends up simplifying the code quite a bit.
Testing: This change adds a new crashtest
Fixes https://github.com/servo/servo/issues/36722
---------
Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
Creating elements by directly calling their interface constructors leads
to some state not being intialized correctly (see #39285). It is also
not in line with the specifications as many of them refer to the
[`create an element`][1] algorithm when an element needs to be created,
which directly maps to `Element::create` in the script crate.
So, switch all such places where elements are created by script to use
`Element::create`.
[1]: https://dom.spec.whatwg.org/#concept-create-element
Testing: Existing WPT tests.
Fixes: #39285
Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
Implements document's active sandboxing flags. These are currently
populated only from CSP-derived sandboxing flags for a new document,
when defined in the CSP.
Testing: 1 new pass, and some new wpt's are added to test points in the
spec where these flags influence behaviour.
Signed-off-by: Shane Handley <shanehandley@fastmail.com>
