Commit Graph

4 Commits

Author SHA1 Message Date
Andreas Kling
30f108ba36 LibJS: Remove C++ lexer, use Rust tokenizer for syntax highlighting
Delete Lexer.cpp/h and Token.cpp, replacing all tokenization with a
new rust_tokenize() FFI function that calls back for each token.

Rewrite SyntaxHighlighter.cpp and js.cpp REPL to use the Rust
tokenizer. The token type and category enums in Token.h now mirror
the Rust definitions in token.rs.

Move is_syntax_character/is_whitespace/is_line_terminator helpers
into RegExpConstructor.cpp as static functions, since they were only
used there.
2026-03-19 21:55:10 -05:00
Andrew Kaster
f06bd0303f LibJS: Use enum for retrieving well known symbols from C++ to Rust 2026-03-19 09:48:32 +01:00
Andrew Kaster
5d43707896 LibJS: Directly use LiteralValueKind enum across FFI boundary 2026-03-19 09:48:32 +01:00
Andrew Kaster
92e4c20ad5 LibJS: Generate FFI header using cbindgen instead of hand-rolling
Replace the BytecodeFactory header with cbindgen.

This will help ensure that types and enums and constants are kept in
sync between the C++ and Rust code. It's also a step in exporting more
Rust enums directly rather than relying on magic constants for
switch statements.

The FFI functions are now all placed in the JS::FFI namespace, which
is the cause for all the churn in the scripting parts of LibJS and
LibWeb.
2026-03-17 20:49:50 -05:00