Implement a complete Rust reimplementation of the LibJS frontend:
lexer, parser, AST, scope collector, and bytecode code generator.
The Rust pipeline is built via Corrosion (CMake-Cargo bridge) and
linked into LibJS as a static library. It is gated behind a build
flag (ENABLE_RUST, on by default except on Windows) and two runtime
environment variables:
- LIBJS_CPP: Use the C++ pipeline instead of Rust
- LIBJS_COMPARE_PIPELINES=1: Run both pipelines in lockstep,
aborting on any difference in AST or bytecode generated.
The C++ side communicates with Rust through a C FFI layer
(RustIntegration.cpp/h) that passes source text to Rust and receives
a populated Executable back via a BytecodeFactory interface.
Functions created via new Function() cannot assume that unresolved
identifiers refer to global variables, since they may be called in
an arbitrary scope. Pass a flag through the scope collector analysis
to suppress the global identifier optimization in this case.
When the parser speculatively tries to parse an arrow function
expression, encountering `this` inside a default parameter value
like `(a = this)` propagates uses_this flags to ancestor function
scopes via set_uses_this(). If the arrow attempt fails (no =>
follows), these flags were left behind, incorrectly marking
ancestor scopes as using `this`.
Fix this by saving and restoring the uses_this and
uses_this_from_environment flags on all ancestor function scopes
around speculative arrow function parsing.
The parser was not calling add_declaration() for using declarations in
for-loop initializers, causing the scope collector to miss them. This
meant identifiers declared with `using` in for-loops were incorrectly
resolved as globals instead of local variables.
When parsing object expressions like {x: 42}, the parser was eagerly
registering the identifier "x" in the scope collector even though it's
only used as a property key and not a variable reference.
Fix this by deferring the registration until we know we're dealing with
a shorthand property ({x}), where the identifier is actually a variable
reference that needs to be captured by the scope collector.
The short-circuit evaluation in `is_invalid || !consume_exponent()`
skipped the consume_exponent() call when is_invalid was already true.
This meant that for inputs like `1._1e2`, the exponent part would not
be consumed and would instead be lexed as separate tokens.
Add the ability to dump AST and bytecode to a String instead of only
to stdout/stderr. This is done by adding an optional StringBuilder
output sink to ASTDumpState, and a new dump_to_string() method on
both ASTNode and Bytecode::Executable.
These will be used for comparing output between compilation pipelines.
The scope collector uses HashMaps for identifier groups and variables,
which means their iteration order is non-deterministic. This causes
local variable indices and function declaration instantiation (FDI)
bytecode to vary between runs.
Fix this by sorting identifier group keys alphabetically before
assigning local variable indices, and sorting vars_to_initialize by
name before emitting FDI bytecode.
Also make register allocation deterministic by always picking the
lowest-numbered free register instead of whichever one happens to be
at the end of the free list.
This is preparation for bringing in a new source->bytecode pipeline
written in Rust. Checking for regressions is significantly easier
if we can expect identical output from both pipelines.
There's now an ENABLE_RUST CMake option (on by default).
Install Rust via rustup in devcontainer scripts, document the
requirement in build instructions, and add Cargo's target/ directory
to .gitignore.
In the benchmark added here, fmt's dragonbox is ~3x faster than our own
Ryu implementation (1197ms for dragonbox vs. 3435ms for Ryu).
Daniel Lemire recently published an article about these algorithms:
https://lemire.me/blog/2026/02/01/converting-floats-to-strings-quickly/
In this article, fmt's dragonbox implementation is actually one of the
slower ones (with the caveat that some comments note that the article is
a bit out-of-date). I've gone with fmt here because:
1. It has a readily available recent version on vcpkg.
2. It provides the methods we need to actually convert a floating point
to decimal exponential form.
3. There is an ongoing effort to replace dragonbox with a new algorithm,
zmij, which promises to be faster.
4. It is one of the only users of AK/UFixedBigInt, so we can potentially
remove that as well soon.
5. Bringing in fmt opens the door to replacing a bunch of AK::format
facilities with fmt as well.
When the computed significand lands exactly on 10 ^ (precision - 1), the
value sits right on a power-of-10 boundary where two representations are
possible. For example, consider 1e-21. The nearest double value to 1e-21
is actually slightly less than 1e-21. So with `toPrecision(16)`, we must
choose between:
exponent=-21 significand=1000000000000000 -> 1.000000000000000e-21
exponent=-22 significand=9999999999999999 -> 9.999999999999999e-22
The spec dictates that we must pick the value that is closer to the true
value. In this case, the second value is actually closer.
The arithmetic here nearly exactly matches that of toPrecision from
commit cf180bd4da.
The only difference is test262 contains tests for which our exponent
estimate is off-by-1. We now handle this by detecting this inaccuracy.
adjusting the exponent, and recomputing the significand.
This will be needed by Number.prototype.toExponential. It will also need
some changes, so moving it up ahead of time will make that diff more
practical to read.
For `export default (class Name { })`, two things were wrong:
The parser extracted the class expression's name as the export's
local binding name instead of `*default*`. Per the spec, this is
`export default AssignmentExpression ;` whose BoundNames is
`*default*`, not the class name.
The bytecode generator had a special case for ClassExpression that
skipped emitting InitializeLexicalBinding for named classes.
These two bugs compensated for each other (no crash, but wrong
behavior). Fix both: always use `*default*` as the local binding
name for expression exports, and always emit InitializeLexicalBinding
for the `*default*` binding.
Rework our hash functions a bit for significant better performance:
* Rename int_hash to u32_hash to mirror u64_hash.
* Make pair_int_hash call u64_hash instead of multiple u32_hash()es.
* Implement MurmurHash3's fmix32 and fmix64 for u32_hash and u64_hash.
On my machine, this speeds up u32_hash by 20%, u64_hash by ~290%, and
pair_int_hash by ~260%.
We lose the property that an input of 0 results in something that is not
0. I've experimented with an offset to both hash functions, but it
resulted in a measurable performance degradation for u64_hash. If
there's a good use case for 0 not to result in 0, we can always add in
that offset as a countermeasure in the future.
Our previous implementation produced incorrect results for values near
the limits of double precision. This new implementation avoid floating-
point arithmetic entirely by:
1. Decomposing the double value into its exact binary form.
2. Computing the formulas from the spec using bigints.
3. Using Ryu to calculate the decimal exponent.
When a statement in a switch case body doesn't produce a result (e.g.
a variable declaration), we were incorrectly resetting the completion
value to undefined. This caused the completion value of preceding
expression statements to be lost.
Per step 13 of ScriptEvaluation in the ECMA-262 spec, the script body
should only be evaluated if GlobalDeclarationInstantiation returned a
normal completion.
This can't currently be triggered since we always create fresh Script
objects, but if we ever start reusing cached executables across
evaluations, this would prevent a subtle bug where the script body
runs despite GDI failing.
Both return paths from parse_import_statement() were missing a call to
consume_or_insert_semicolon(), causing explicit semicolons to be left
unconsumed and parsed as spurious EmptyStatements.
The result of parsing an identifier cannot change. It is not cheap to do
so, so let's cache the result.
This is hammered in a few test262 tests. On my machine, this reduces the
runtime of each test/staging/sm/Date/dst-offset-caching-{N}-of-8.js by
0.3 to 0.5 seconds. For example:
dst-offset-caching-1-of-8.js: Reduces from 1.2s -> 0.9s
dst-offset-caching-3-of-8.js: Reduces from 1.5s -> 1.1s
These are String from the outset, so this patch is almost entirely just
changing function parameter types. This will allow us to cache time zone
parse results without invoking any extra allocations.
When a nested function (arrow or function expression) inside a default
parameter expression captures a name that also has a body var
declaration, the capture must propagate to the parent scope. Otherwise,
the outer scope optimizes the binding to a local register, making it
invisible to GetBinding at runtime.
When a function has parameter expressions (default values), body var
declarations that shadow a name referenced in a default parameter
expression must not be optimized to local variables. The default
expression needs to resolve the name from the outer scope via the
environment chain, not read the uninitialized local.
We now mark identifiers referenced during formal parameter parsing
with an IsReferencedInFormalParameters flag, and skip local variable
optimization for body vars that carry both this flag and IsVar (but
not IsForbiddenLexical, which indicates parameter names themselves).
When emitting block declaration instantiation, we were not calling
set_local_initialized() after writing block-scoped function
declarations to local variables via Mov. This caused unnecessary
ThrowIfTDZ checks to be emitted when those locals were later read.
Block-scoped function declarations are always initialized at block
entry (via NewFunction + Mov), so TDZ checks for them are redundant.
When hex, octal, or binary integer literals overflow u64, we used to
fall back to UINT64_MAX. This produced incorrect results for any value
larger than 2^64.
Fix this by accumulating the value as a double digit-by-digit when
the u64 parse fails.
When parsing class elements, after consuming the `static` keyword, the
parser would unconditionally consume `async` if it appeared next. This
meant that for `static async()`, where `async` is the method name (not
a modifier), the `async` token was consumed too early and its source
position was lost.
Fix this by applying the same lookahead check used for top-level async
detection: only consume `async` as a modifier if the following token is
not `(`, `;`, `}`, or preceded by a line terminator. This lets `async`
be parsed as a property key with its correct source position.
When an identifier was registered and its group already existed but
had no declaration_kind set, we failed to propagate it. This caused
var declarations to lose their annotation in AST dumps when the
identifier was referenced before its declaration.
Move the duplicated ThrowIfTDZ emission logic from three places in
ASTCodegen.cpp into a single Generator::emit_tdz_check_if_needed()
helper. This handles both argument TDZ (which requires a Mov to
empty first) and lexically-declared variable TDZ uniformly.
This avoids emitting some unnecessary ThrowIfTDZ instructions.
ScopeCollector::add_declaration() was adding var declarations to the
top-level scope's m_var_declarations once per bound identifier and once
more after the for_each_bound_identifier loop - so a `var a, b, c`
would be added 4 times instead of 1.
The Script constructor iterates m_var_declarations and expands each
entry's bound identifiers, resulting in O(N²) work for a single var
statement with N declarators.
Running the Emscripten-compiled version of ScummVM with a 32,174-
declarator var statement, this produced over 1 billion entries,
consuming 14+ GB of RAM and blocking the event loop for 35+ seconds.
After this fix, this drops down to 200 MB and just short of 200ms.
The find_source_record lambda was doing a reverse linear scan through
the entire source map for every instruction emitted, resulting in
quadratic behavior. This was catastrophic for large scripts like
Octane/mandreel.js, where compile() dominated the profile at ~30s.
Since both source map entries and instruction iteration are ordered by
offset, replace the per-instruction scan with a forward cursor that
advances in lockstep with instruction emission.
The compile() function was adding source map entries for all
instructions in a block upfront, before processing assembly-time
optimizations (Jump-to-next-block elision, Jump-to-Return/End inlining,
JumpIf-to-JumpTrue/JumpFalse conversion). When a Jump was skipped,
its phantom source map entry remained at the offset where the next
block's first instruction would be placed, causing binary_search to
find the wrong source location for error messages.
Fix by building source map entries inline with instruction emission,
ensuring only actually-emitted instructions get entries. For blocks
with duplicate source map entries at the same offset (from rewind in
fuse_compare_and_jump), the last entry is used.
Add ThisExpression handling to the expression_identifier() helper used
for base_identifier in bytecode instructions. This makes PutById and
GetById emit base_identifier:this when the base is a this expression.
Previously, the function only handled a single level of member access,
producing strings like "<object>.isWall" for chained expressions like
"graphSet[j][k].isWall". Now it recurses through nested member
expressions, identifiers, string/numeric literals, and `this`.
When MemberExpression::generate_bytecode calls emit_load_from_reference,
it only uses the loaded_value and discards the reference operands. For
computed member expressions (e.g. a[0]), this was generating an
unnecessary Mov to save the property register for potential store-back.
Add a ReferenceMode parameter to emit_load_from_reference. When LoadOnly
is passed, the computed property path skips the register save and Mov.
Per AssignmentRestElement and AssignmentElement in the specification,
the DestructuringAssignmentTarget reference must be evaluated before
iterating or stepping the iterator. We were doing it in the wrong
order, which caused observable differences when the target evaluation
has side effects, and could lead to infinite loops when the iterator
never completes.
Add Generator::emit_evaluate_reference() to evaluate a member
expression's base and property into ReferenceOperands without performing
a load or store, then use the pre-evaluated reference for the store
after iteration completes.
