Create parser-free script and module materializers for decoded cache
blobs. Cached functions create SFDs without Rust compile inputs and
attach their precompiled executable immediately, while declaration
metadata is populated from decoded records.
Treat cache blobs as external input from the HTTP disk cache. Run
bytecode validation unconditionally before fixing up cache pointers, and
reject decoded source ranges or metadata indices that would be
out-of-bounds during C++ materialization.
Report executable validation failures as parser errors so callers can
reject corrupt sidecars and fall back to source compilation. LibJS tests
cover corrupt top-level bytecode, declaration bytecode, and declaration
source spans.
Store a SHA-256 fingerprint of the decoded source text in each bytecode
cache blob, and require callers to provide the expected fingerprint when
validating or decoding a blob.
This rejects sidecars for stale HTTP cache entries whose URL and request
headers still match but whose source body has been replaced. Bytecode
cache tests cover the mismatched-source rejection path.
Expose an owned decoded bytecode cache handle through RustIntegration.
This lets C++ callers keep validated metadata and executable records in
Rust-owned cache structures without invoking the parser.
Extend the js bytecode cache validation mode to create and free the FFI
handle so blob generation exercises the ownership path.
Record whether each bytecode cache blob contains a classic script or a
module, and pass that type through the serializer call sites.
Require validation callers to provide the expected program type so
script and module sidecars cannot be reused for the wrong loader.
Add a versioned Rust bytecode cache writer for fully compiled programs.
The blob records executable bytecode, metadata tables, source maps,
exception handlers, nested function bytecode, and class blueprints
without materializing GC objects.
Expose the serialized blob through RustIntegration as an owned
ByteBuffer so Web-facing callers can store it as HTTP cache data.
Keep fetched script and module compilation on the latency-sensitive path
limited to top-level code and eager direct IIFE compilation before
returning bytecode to the main thread.
Add a separate full off-thread compile entry point for bytecode cache
generation. Cache jobs can use it to compile every nested function after
the execution path has already been unblocked.
Add Rust and C++ integration points for cloning lazy function compile
payloads, compiling them to GC-free bytecode off the main thread, and
materializing the result later on the main thread.
The cloned payload lets background compilation race with lazy
main-thread compilation without sharing AST ownership between threads.
Compiled function artifacts recursively include nested functions, so
materialization can discard the corresponding AST subtree.
Split Rust program compilation so code generation and assembly finish
before the main thread materializes GC-backed executable objects. The
new CompiledProgram handle owns the parsed program, generator state, and
bytecode until C++ consumes it on the main thread.
Wire WebContent script fetching through that handle for classic scripts
and modules. Syntax-error paths still return ParsedProgram, so existing
error reporting stays in place. Successful fetches now do top-level
codegen on the thread pool before deferred_invoke hands control back to
the main thread.
Executable creation, SharedFunctionInstanceData materialization, module
metadata extraction, and declaration data extraction still run on the
main thread where VM and GC access is valid.
The proposal has not seemed to progress for a while, and there is
a open issue about module imports which breaks HTML integration.
While we could probably make an AD-HOC change to fix that issue,
it is deep enough in the JS engine that I am not particularly
keen on making that change.
Until other browsers begin to make positive signals about shipping
ShadowRealms, let's remove our implementation for now.
There is still some cleanup that can be done with regard to the
HTML integration, but there are a few more items that need to be
untangled there.
Delete AST.cpp, AST.h, ASTDump.cpp, ScopeRecord.h, and the dead
get_builtin(MemberExpression const&) from Builtins.cpp.
Extract ImportEntry and ExportEntry into a new ModuleEntry.h,
since they are data types used by the module system, not AST
node types.
Inline ModuleRequest's sorting constructor and
SourceRange::filename().
Remove the dead annex_b_function_declarations field from
EvalDeclarationData, which was only populated by the C++ parser.
Delete Parser.cpp/h and ScopeCollector.cpp/h, now that all parsing
goes through the Rust pipeline.
Port test262-runner to use RustIntegration::parse_program() for its
fast parse-only check instead of the C++ Parser.
Add parsed_program_has_errors() and free_parsed_program() to the
RustIntegration public API for parse-only use cases.
Port all remaining users of the C++ Parser/Lexer/Generator to
use the Rust pipeline instead:
- Intrinsics: Remove C++ fallback in parse_builtin_file()
- ECMAScriptFunctionObject: Remove C++ compile() fallback
- NativeJavaScriptBackedFunction: Remove C++ compile() fallback
- EventTarget: Port to compile_dynamic_function
- WebDriver/ExecuteScript: Port to compile_dynamic_function
- LibTest/JavaScriptTestRunner.h: Remove Parser/Lexer includes
- FuzzilliJs: Remove unused Parser/Lexer includes
Also remove the dead Statement-based template instantiation of
async_block_start/async_function_start.
Remove PipelineComparison.cpp/h and all LIBJS_COMPARE_PIPELINES
support from RustIntegration.cpp. This includes:
- The compare_pipelines_enabled() function
- All comparison blocks in compile_script/eval/module/function
- The pair_shared_function_data() helper
- The m_cpp_comparison_sfd field on SharedFunctionInstanceData
The Rust pipeline has been validated extensively through comparison
testing and no longer needs the side-by-side verification harness.
Replace the BytecodeFactory header with cbindgen.
This will help ensure that types and enums and constants are kept in
sync between the C++ and Rust code. It's also a step in exporting more
Rust enums directly rather than relying on magic constants for
switch statements.
The FFI functions are now all placed in the JS::FFI namespace, which
is the cause for all the churn in the scripting parts of LibJS and
LibWeb.
Add compile_parsed_module() to RustIntegration, which takes a
RustParsedProgram and a SourceCode (from parse_program with
ProgramType::Module) and compiles it on the main thread with GC
interaction.
Rewrite compile_module() to use the new split functions internally.
Add SourceTextModule::parse_from_pre_parsed() and
JavaScriptModuleScript::create_from_pre_parsed() to allow creating
module scripts from a pre-parsed RustParsedProgram.
This prepares the infrastructure for off-thread module parsing.
Create a SourceCode on the main thread (performing UTF-8 to UTF-16
conversion), then submit parse_program() to the ThreadPool for
Rust parsing on a worker thread. This unblocks the WebContent event
loop during external script loading.
Add Script::create_from_parsed() and
ClassicScript::create_from_pre_parsed() factory methods that take a
pre-parsed RustParsedProgram and a SourceCode, performing only the
GC-allocating compile step on the main thread.
Falls back to synchronous parsing when the Rust pipeline is
unavailable (LIBJS_CPP=1 or LIBJS_COMPARE_PIPELINES=1).
Expose the Rust parse/compile split to C++ callers:
- parse_program(): takes raw UTF-16 data and a ProgramType
parameter (Script or Module). No GC interaction, thread-safe.
- compile_parsed_script(): takes a pre-parsed RustParsedProgram
and a SourceCode, checks for errors, and calls
rust_compile_parsed_script(). Returns a ScriptResult.
Rewrite compile_script() to use the split path internally. The
pipeline comparison logic now gets the AST dump from the
ParsedProgram before compilation consumes it.
Implement a complete Rust reimplementation of the LibJS frontend:
lexer, parser, AST, scope collector, and bytecode code generator.
The Rust pipeline is built via Corrosion (CMake-Cargo bridge) and
linked into LibJS as a static library. It is gated behind a build
flag (ENABLE_RUST, on by default except on Windows) and two runtime
environment variables:
- LIBJS_CPP: Use the C++ pipeline instead of Rust
- LIBJS_COMPARE_PIPELINES=1: Run both pipelines in lockstep,
aborting on any difference in AST or bytecode generated.
The C++ side communicates with Rust through a C FFI layer
(RustIntegration.cpp/h) that passes source text to Rust and receives
a populated Executable back via a BytecodeFactory interface.
