Andreas Kling 355fb6b825 LibWeb: Stream Rust CSS tokenizer tokens over FFI ...

Avoid building a temporary Rust token vector before calling back into
C++. The tokenizer now invokes the callback as each token is produced,
while borrowing the already-filtered input for source slices.

Reserve an initial C++ token capacity from the input size so the common
path avoids repeated growth while appending the converted tokens.

With this change, the Rust CSS tokenizer is now ~1.3x faster than the
C++ CSS tokenizer at churning through all the https://vercel.com/ CSS.

2026-05-03 17:22:17 +02:00

12 KiB

Raw Blame History

View Raw