Instead of using a custom paintable to draw the controls for video and
audio elements, we build them out of plain old HTML elements within a
shadow root.
This required a few hacks in the previous commits in order to allow a
replaced element to host children within a shadow root, but it's
fairly self-contained.
A big benefit is that we can drive all the UI updates off of plain old
DOM events (except the play button overlay on videos, which uses the
video element representation), so we can test our media and input event
handling more thoroughly. :^)
The control bar visibility is now more similar to how other browsers
handle it. It will show upon hovering over the element, but if the
cursor is kept still for more than a second, it will hide again. While
dragging, the controls remain visible, and will then hide after the
mouse button is released.
The icons have been redesigned from scratch, and the mute icon now
visualizes the volume level along with indicating the mute state.
Publish new video frames to an ExternalContentSource, and switch
VideoPaintable from draw_scaled_immutable_bitmap to
draw_external_content.
Because DrawExternalContent reads the latest bitmap at replay time,
frame-only updates (no timeline or control change) now call
set_needs_display(InvalidateDisplayList::No) — skipping display list
rebuilds entirely. This addresses problem 2 from the previous commit.
This saves us from having our own color conversion code, which was
taking up a fair amount of time in VideoDataProvider. With this change,
we should be able to play high resolution videos without interruptions
on machines where the CPU can keep up with decoding.
In order to make this change, ImmutableBitmap is now able to be
constructed with YUV data instead of an RBG bitmap. It holds onto a
YUVData instance that stores the buffers of image data, since Skia
itself doesn't take ownership of them.
In order to support greater than 8 bits of color depth, we normalize
the 10- or 12-bit color values into a 16-bit range.
The `Bitmap` type was referring to to its internal pixel format by a
name that represents the order of the color components as they are layed
out in memory. Contrary, the `Color` type was using a naming that where
the name represents the order of the components from most to least
significant byte when viewed as a unsigned 32bit integer. This is
confusing as you have to keep remembering which mental model to use
depending on which code you work with.
To unify the two, the naming of RGBA-like colors in the `Color` type has
been adjusted to match the one from the Bitmap type. This seems to be
generally in line with how web APIs think about these types:
* `ImageData.pixelFormat` can be `rgba-8unorm` backed by a
`Uint8ClamedArray`, but there is no pixel format backed by a 32bit
unsigned type.
* WebGL can use format `RGBA` with type `UNSIGNED_BYTE`, but there is no
such format with type `UNSIGNED_INT`.
Additionally, it appears that other browsers and browser-adjacent
libraries also think similarly about these types:
* Firefox:
https://github.com/mozilla-firefox/firefox/blob/main/gfx/2d/Types.h
* WebKit:
https://github.com/WebKit/WebKit/blob/main/Source/WebCore/platform/graphics/PixelFormat.h
* Skia:
https://chromium.googlesource.com/skia/+/refs/heads/main/include/core/SkColorType.h
This has the not so nice side effect that APIs that interact with these
types through 32bit unsigned integers now have the component order
inverted due to little-endian byte order. E.g. specifying a color as hex
constant needs to be done as `0xAABBGGRR` if it is to be treated as
RGBA8888.
We could alleviate this by providing endian-independent APIs to callers.
But I suspect long-term we might want to think differently about bitmap
data anyway, e.g. to better support HDR in the future. However, such
changes would be more involved than just unifying the naming as done
here. So I considered that out of scope for now.
With this commit, all PlaybackManager can do is autoplay a file from
start to finish, with no pausing or seeking functionality.
All audio playback functionality has been removed from HTMLMediaElement
and HTMLAudioElement in anticipation of PlaybackManager taking that
over, for both audio-only and audio/video.
The extra representations of a video element are unnecessary to specify
in VideoPaintable, as the playback system itself should take care of
all the details of the representation as specified.
PaintContext dates back to a time when display lists didn't exist and it
truly represented "paint context". Renaming it to better align with its
current role.
Instead of trying to manually determine which parts of a bitmap fall
within the box of the `<img>` element, just draw the whole bitmap and
let Skia clip the draw-area to the correct rectangle.
This fixes a bug where the entire bitmap was squashed into the rectangle
of the image box instead of being clipped.
With this change, image rendering is now correct enough to import some
of the WPT tests for object-fit and object-position. To get some good
coverage I have imported all tests for the `<img>` tag. I also wanted to
import a subset of the tests for the `<object>` tag, since those are
passing as well now. Unfortunately, they are flaky for unknown reasons.
This is the second attempt at this bugfix. The prior one was e055927ead
and broke image rendering whenever the page was scrolled. It has
subsequently been reverted in 16b14273d1. Hopefully this time it is not
horribly broken.
Instead of trying to manually determine which parts of a bitmap fall
within the box of the `<img>` element, just draw the whole bitmap and
let Skia clip the draw-area to the correct rectangle.
This fixes a bug where the entire bitmap was squashed into the rectangle
of the image box instead of being clipped.
With this change, image rendering is now correct enough to import some
of the WPT tests for object-fit and object-position. To get some good
coverage I have imported all tests for the `<img>` tag. I also wanted to
import a subset of the tests for the `<object>` tag, since those are
passing as well now. Unfortunately, they are flaky for unknown reasons.
This reduces the number of `.cpp` files that need to be recompiled when
one of the below header files changes as follows:
Painting/Command.h: 1030 -> 61
Painting/DisplayList.h: 1030 -> 60
Painting/DisplayListRecorder.h: 557 -> 59
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
Now that the heap has no knowledge about a JavaScript realm and is
purely for managing the memory of the heap, it does not make sense
to name this function to say that it is a non-realm variant.
