Previously, both mask and clip-path were rendered to separate mutable
Gfx::Bitmap objects which forced CPU rasterization. They were then
combined using a CPU pixel-by-pixel operation before being returned
as an ImmutableBitmap.
Instead of including mask in the final bitmap as already rasterized
images, we now use display lists which opens opportunity to utilize
GPU if available.
Bitmap::apply_mask() and ApplyMaskBitmap display list command are no
longer used and have been removed.
This moves filter resolution from display list recording time to
resolve_paint_properties(), caching the resolved values in CSS pixels
on PaintableBox. Device pixel conversion is now deferred until paint
time via to_gfx_filter().
This follows the existing pattern used for other paint-only properties
like box shadows and border radii.
Reuse existing paintables during relayout to reduce GC allocation
pressure. Each paintable subclass implements reset_for_relayout()
to clear state before reuse.
This method was initially introduced to calculate the total paint area
including effects like box-shadows that paint outside the border box.
It was used in StackingContext for sizing bitmaps when painting
elements with opacity or transforms, and later for clip-path bounds.
This functionality is no longer needed as stacking context painting
and clip-path handling have been refactored to use different
approaches (AccumulatedVisualContext now handles clip-path).
When delegating mouse events to iframes, coordinate transformations
were not accounting for scroll offsets. This caused clicks inside
iframes to be incorrectly positioned when the parent page was scrolled.
Remove the now-obsolete ClipFrame infrastructure:
- Delete ClipFrame.h and ClipFrame.cpp
- Remove assign_clip_frames() from ViewportPaintable
- Remove enclosing_clip_frame and own_clip_frame from PaintableBox
- Remove m_clip_state HashMap from ViewportPaintable
Clip handling is now fully managed through AccumulatedVisualContext
nodes with ClipData.
Integrate AccumulatedVisualContext with display list recording and
playback. This is the main commit of the refactoring that delivers the
architectural improvements enabled by AccumulatedVisualContext.
Recording changes:
Each display list command now stores a single
RefPtr<AccumulatedVisualContext> instead of separate scroll_frame_id
and ClipFrame. The recorder simply captures the current accumulated
context when appending commands.
The before_paint()/after_paint() hooks that pushed/popped scroll frame
IDs are replaced by directly setting accumulated_visual_context on the
recorder before painting each element.
Playback changes:
The display list player now uses LCA (Lowest Common Ancestor) based
traversal to switch between visual contexts efficiently. When
transitioning from context A to context B:
1. Find the LCA of A and B in the context tree
2. Pop (restore) states back to the LCA depth
3. Push (save + apply) states from LCA down to B
This approach minimizes redundant save/restore operations. For example,
when rendering siblings that share a common scroll container, the
player keeps that scroll state applied and only switches the divergent
parts of their context chains.
Key deletions:
- Remove translate_by() from all 45 display list commands - commands
are now immutable
- Remove transform/perspective fields from PushStackingContext -
transforms are tracked via AccumulatedVisualContext
- Remove push_scroll_frame_id()/pop_scroll_frame_id() from
DisplayListRecorder
- Remove before_paint()/after_paint() hooks from Paintable
- Merge ApplyOpacity, ApplyCompositeAndBlendingOperator, ApplyFilter
into single ApplyEffects command
Stacking context painting changes:
The StackingContext::paint() method is significantly simplified.
Instead of building a PushStackingContextParams struct with transform
matrices and pushing/popping stacking contexts, it now:
1. Sets the accumulated visual context (which already contains
transforms)
2. Applies effects (opacity, blend mode, filters) if needed
3. Applies clip path if needed
4. Paints the content
5. Restores state
The visual state management that was interleaved throughout the
painting code is now handled uniformly by the context tree.
Introduce AccumulatedVisualContext, a tree structure that tracks the
cumulative visual state (scroll offsets, clip regions, transforms,
perspective) for each paintable box.
Motivation:
Before this change, visual state was fragmented across multiple
mechanisms:
- ClipFrame: Tracked clip rectangles, each storing its own
enclosing_scroll_frame_id to handle scroll offset adjustments
- scroll_frame_id: Passed separately to each display list command
- PushStackingContext: Stored transform matrices directly in the command
- Every display list command implemented translate_by() (45 methods
total) to allow scroll offset adjustment during playback
This fragmentation led to:
- Complex, error-prone coordinate transformation logic scattered
throughout the codebase
- Commands being mutated during playback to apply scroll offsets
- Duplicate logic between painting and hit testing for coordinate
transformations
Solution:
AccumulatedVisualContext builds a tree where each node represents a
single visual operation:
- ScrollData: A scroll frame with its ID
- ClipData: A clip rectangle with optional border radii
- TransformData: A 4x4 transform matrix with its origin
- PerspectiveData: A perspective projection matrix
Each PaintableBox stores a reference to its accumulated context node.
The tree structure naturally captures the parent-child relationships,
so traversing from any node to the root gives the complete chain of
visual transformations.
Benefits this enables (in subsequent commits):
- Display list commands become immutable - no more translate_by()
- Single RefPtr<AccumulatedVisualContext> replaces separate
scroll_frame_id and ClipFrame on commands
- LCA-based tree traversal during playback for efficient save/restore
- transform_point_for_hit_test() provides coordinate transformation for
hit testing using the same structure
Add ElementResizeAction to Page (maybe there's a better place). It's
just a mousemove delegate that updates styles on the target element.
Add ChromeMetrics for zoom-invariant chrome like scrollbar thumb
thickness, resize gripper size, paddings, etc. It's not user-stylable
but separates basic concerns in a way that a visually gifted
designer unlike myself can adjust to taste.
These values are pre-divided by zoom factor so that PaintableBox can
continue using device_pixels_per_css_pixel calls as normal.
The adjusted metrics are computed on demand from Page multiple times
per paint cycle, which is not ideal but avoids lifetime management and
atomics. Maybe someone with more surety about the painting flow control
can improve this, but it won't be a huge win. If profiling shows
this slowing paints, then Ladybird is in good shape.
Update PaintableBox to draw the resize gripper and deconflict
the scrollbars. Set apropriate cursors for scrollbars and gripper in
mousemove. We override EventHandler's cursor handling because nothing
should ever come between a man and his resize gripper.
Chrome metrics use the CSSPixels class. This is good because it's
broadly compatible but bad because they're actually different units
when zoom is not 1.0. If that's a problem, we could make a new type
or just use double.
This adds visit_edges(Cell::Visitor&) methods to various helper structs
that contain GC pointers, and makes sure they are called from owning
GC-heap-allocated objects as needed.
These were found by our Clang plugin after expanding its capabilities.
The added rules will be enforced by CI going forward.
The Transformation class wasn't really accomplishing anything. It still
had to store StyleValues, so it was basically the same as
TransformationStyleValue, with extra steps to convert from one to the
other. So... let's just use TransformationStyleValue instead!
Apart from moving code around, the behavior has changed a bit. We now
actually acknowledge unresolvable parameters and return an error when
we try to produce a matrix from them. Previously we just skipped over
them, which was pretty wrong. This gets us an extra pass in the
typed-om test.
We also get some slightly different results with our transform
serialization, because we're not converting to CSSPixels and back.
This function used layout node pointer to check if it's corresponding to
viewport. There is no need for that, since `is_viewport_paintable()`
does exactly the same check without going through layout node.
Before this change, you could only scroll the current hovered scroll
container, even if it was at the beginning or end and thus having no
effect.
Now, if it doesn't update, it will not be classed as handled and will
move onto the next scroll container.
The overlay shown for the node hovered in the inspector is painted as
part of the normal tree traversal of all paintables. This works well in
most cases, but falls short in specific scenarios:
* If the hovered node or one of its ancestors establishes a stacking
context and there is another element that establishes a stacking
context close by or overlapping it, the overlay and especially the
tooltip can become partially hidden behind the second element. Ditto
for elements that act as if they established a stacking context.
* If the hovered node or one of its ancestors involves clipping, the
clip is applied to the overlay and espicially the tooltip. This can
cause them to be partially invisible.
* Similarly, if the hovered node or one of its ancestors has a defined
mask, the mask is applied to the overlay, often making it mostly
invisible.
* No overlays are shown for SVG nodes because they are painted
differently from HTML documents.
Some of these problems may be fixable with the current system. But some
seem like they fundamentally cannot work fully when the overlays are
painted as part of the regular tree traversal.
Instead we pull out painting the overlay as a separate pass executed
after the tree traversal. This way we ensure that the overlays are
always painted last and therefore on top of everything else. This also
makes sure that the overlays are unaffected by clips and masks. And
since overlay painting is independent from painting the actual elements,
it just works as well.
However we need to be careful, because we still need to apply some of
the steps of the tree traversal to get the correct result. Namely we
need to apply scroll offsets and transforms. To do so, we collect all
ancestors of the hovered node and apply those as if we were in the
normal tree traversal.
The debug option 'Show Line Box Borders' and the inspector overlay for
text nodes are conceptually similar. However they use two different
code paths. This commits unifies both to use the same code.
Previously, we were collapsing whitespace in Layout::TextNode and then
passed the resulting string for further processing through ChunkIterator
-> InlineLevelIterator -> InlineFormattingContext -> LineBuilder ->
LineBoxFragment -> PaintableFragment. Our painting tree is where we deal
with things like range offsets into the underlying text nodes, but since
we modified the original string, the offsets were wrong.
This changes the way we generate fragments:
* Layout::TextNode no longer collapses whitespace as part of its
stored "text for rendering", but moves this logic to ChunkIterator
which splits up this text into separate views whenever whitespace
needs to be collapsed.
* Layout::LineBox now only extends the last fragment if its end offset
is equal to the new fragment's start offset. Otherwise, there's a
gap caused by collapsing whitespace and we need to generate a
separate fragment for that in order to have a correct start offset.
Some tests need new baselines because of the fixed start offsets.
Fixes#566.
PaintContext dates back to a time when display lists didn't exist and it
truly represented "paint context". Renaming it to better align with its
current role.
The faux position we created here is adjusted by the device pixel ratio
later on, which would invoke integer overflow on screens with a DPR
greater than 1.
Instead of creating special data for a mouse move event, let's just add
an explicit leave event handler.
Until now, every paint phase of every PaintableBox injected its own
clipping sequence into the display list:
```
before_paint: Save
AddClipRect (1)
...clip rectangles for each containing block with clip...
AddClipRect (N)
paint: ...paint phase items...
after_paint: Restore
```
Because we ran that sequence for every phase of every box, Skia had to
rebuild clip stack `paint_phases * paintable_boxes` times. Worse,
usually most paint phases contribute no visible drawing at all, yet we
still had to emit clipping items because `before_paint()` has no way to
know that in advance.
This change takes a different approach:
- Clip information is now attached as metadata `ClipFrame` to each
DisplayList item.
- `DisplayListPlayer` groups consecutive commands that share a
`ClipFrame`, applying the clip once at the start of the group and
restoring it once at the end.
Going from 10 ms to 5 ms in rasterization on Discord might not sound
like much, but keep in mind that for 60fps we have 16 ms per frame and
there is a lot more work besides display list rasterization we do in
each frame.
* https://discord.com/channels/1247070541085671459/1247090064480014443
- DisplayList items: 81844 -> 3671
- rasterize time: 10 ms -> 5 ms
- record time: 5 ms -> 3 ms
* https://github.com/LadybirdBrowser/ladybird
- DisplayList items: 7902 -> 1176
- rasterize time: 4 ms -> 4 ms
- record time: 3 ms -> 2 ms
Initially ClippableAndScrollable was introduced, because we had
PaintableBox and InlinePaintable and both wanted to share clipping and
scrolling logic. Now, when InlinePaintable is gone, we could inline
ClippableAndScrollable implementation into PaintableBox.
Previously, we always applied the enclosing clip rectangle for all paint
phases except overlays, and the own clip rectangle for the background
and foreground phases. The problem is that applying a clip rectangle
means emitting an AddClipRect display list item for each clip rectangle
in the containing block. With this change, we choose whether to include
the own clip based on the paint phase and this way avoid emitting
AddClipRect for enclosing clip rectangles twice.
This way we can reuse the logic between PaintableWithLines and
PaintableBox. It also introduces the .is_positioned() check for the
children of a PaintableWithLines, which makes sure to skip positioned
child nodes since those are handled by the StackingContext.
We were calculating the scroll delta based on the last known mouse
position, even if that position ventured far beyond the scroll bar's
rect. This caused weird behavior such as scrolling when the mouse was
clearly not on the scrollbar.
This reworks the scrollbar logic to use the mouse's absolute position
instead of a delta, and to always calculate and set the absolute scroll
offset. This makes it much easier to reason about the scrollbar's
position, plus we don't need the last mouse position anymore.
As far as I can tell, our scroll bar behavior now closely resembles
Firefox' behavior.
Currently, compute_scrollbar_data does not adjust the position of the
scrollbar thumb based on the actual scroll offset. This is because we
perform this offset in most cases inside the display list executor, in
order to allow us to avoid recomputing the display list.
However, there are cases where we do want the thumb rect with an offset
inside PaintableBox. We currently use scroll_thumb_rect to perform that
computation.
In an upcoming patch, we will need both this offset thumb rect and the
scrollbar gutter rect. So this patch moves the computation of the offset
to compute_scrollbar_data, performed behind an optional parameter.
