DecodedImageFrame now owns decoded bitmap pixels directly, so the
separate ImmutableBitmap wrapper no longer carries useful semantics.
Remove the class and pass decoded image frames or bitmaps at the
boundaries where pixels are actually required.
The Skia image cache now keys off DecodedImageFrame, matching the
display-list commands that paint decoded images. Video frames stay
owned by LibMedia, with the explicit YUV-to-bitmap conversion living
at HTMLVideoElement's decoded-frame entry point for canvas and WebGL
callers.
Decoded image data should not continue to traffic in ImmutableBitmap now
that the bitmap wrapper is being retired. Introduce DecodedImageFrame as
the paintable decoded-image unit and store a Bitmap plus ColorSpace in
it directly.
Thread the new frame type through decoded image data, display-list
image commands, filters, canvas drawImage, patterns, WebGL texture
upload, and CSS/SVG image consumers. ImmutableBitmap remains only at
the legacy boundaries that still need it, such as HTML video snapshots
and callers that explicitly ask for a bitmap snapshot.
This keeps color-space ownership with the decoded frame while making
the expensive or legacy ImmutableBitmap path explicit at the few call
sites that still need it.
Decoded video frames should own their planar YUV data and color space
directly. Keeping that storage behind ImmutableBitmap gave a
still-image abstraction media-specific behavior and made calls like
bitmap() potentially allocate and convert a whole video frame.
Move YUV ownership into Media::VideoFrame, where the lifetime naturally
follows media playback, and remove the YUV-backed mode from
ImmutableBitmap. This commit intentionally keeps the visible Web paint
path on ExternalContentSource by converting the current frame back to
an ImmutableBitmap where Web still expects one.
Callers that need pixels now ask the frame to convert explicitly. That
preserves behavior for canvas and bitmap consumers while making the
expensive YUV-to-pixel path visible at the call site instead of
hiding it behind ImmutableBitmap::bitmap().
The display queue used TimedImage even though the media pipeline is
selecting decoded video frames. That naming hid the real object being
handed from LibMedia to Web and kept the queue interface coupled to
bitmap-style painting.
Rename the wrapper to TimedVideoFrame and pass ref-counted VideoFrame
objects through the provider and display sink. Web still reads the
ImmutableBitmap from the frame for painting in this commit, so rendered
output and conversion behavior stay the same while the playback-facing
interfaces become frame-shaped.
This tightens the implementation of video element sizing to the spec by
implementing two spec concepts:
- The media resource's natural width and height, and
- The video element's natural width and height.
The element's natural dimensions change based on the representation,
which has many inputs, so update checks are triggered from many
locations.
The resize event is fired when the media resource's natural dimensions
change, and the layout is invalidated if the element's natural
dimensions change.
Tests for a few important resize triggers have been added.
Previously, the LibWeb bindings generator would output multiple per
interface files like Prototype/Constructor/Namespace/GlobalMixin
depending on the contents of that IDL file.
This complicates the build system as it means that it does not know
what files will be generated without knowledge of the contents of that
IDL file.
Instead, for each IDL file only generate a single Bindings/<IDLFile>.h
and Bindings/<IDLFile>.cpp.
These are the same code, so we may as well move them up the chain. This
becomes useful in a later commit, where it will be used to rewrite
inline-flow to inline-block for layout of shadow DOM.
This saves us from having our own color conversion code, which was
taking up a fair amount of time in VideoDataProvider. With this change,
we should be able to play high resolution videos without interruptions
on machines where the CPU can keep up with decoding.
In order to make this change, ImmutableBitmap is now able to be
constructed with YUV data instead of an RBG bitmap. It holds onto a
YUVData instance that stores the buffers of image data, since Skia
itself doesn't take ownership of them.
In order to support greater than 8 bits of color depth, we normalize
the 10- or 12-bit color values into a 16-bit range.
With this commit, all PlaybackManager can do is autoplay a file from
start to finish, with no pausing or seeking functionality.
All audio playback functionality has been removed from HTMLMediaElement
and HTMLAudioElement in anticipation of PlaybackManager taking that
over, for both audio-only and audio/video.
Before this change, we were going through the chain of base classes for
each IDL interface object and having them set the prototype to their
prototype.
Instead of doing that, reorder things so that we set the right prototype
immediately in Foo::initialize(), and then don't bother in all the base
class overrides.
This knocks off a ~1% profile item on Speedometer 3.
12c6ac78e2 with fixed mistake when cache
slot is copied instead of being referenced:
```cpp
auto cache =
box.cached_intrinsic_sizes().min_content_height.ensure(width);
```
while it should've been:
```cpp
auto& cache =
box.cached_intrinsic_sizes().min_content_height.ensure(width);
```
This change moves intrinsic sizes cache from
LayoutState, which is local to current layout run,
to layout nodes, so it could be reused between
layout runs. This optimization is possible because
we can guarantee that these measurements will
remain unchanged unless the style of the element
or any of its descendants changes.
For now, invalidation is implemented simply by
resetting cache on whole ancestors chain once we
figured that element needs layout update.
The case when layout is invalidated by DOM's
structural changes is covered by layout tree
invalidation that drops intrinsic sizes cache
along with layout nodes.
I measured improvement on couple websites:
- Mail list on GMail 28ms -> 6ms
- GitHub large code page 47ms -> 36ms
- Discord chat history 15ms -> 8ms
(Time does not include `commit()`)
This reduces the number of `.cpp` files that need to be recompiled when
one of the below header files changes as follows:
CSS/ComputedProperties.h: 1113 -> 49
CSS/ComputedValues.h: 1120 -> 209
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
We currently have 2 virtual methods to inform DOM::Element subclasses
when an attribute has changed, one of which is spec-compliant. This
patch removes the non-compliant variant.
Now that the heap has no knowledge about a JavaScript realm and is
purely for managing the memory of the heap, it does not make sense
to name this function to say that it is a non-realm variant.
