Skip to content

🎲 test: load stdlib in panic-fuzz and add lexer-panic proptest#135

Merged
timfennis merged 1 commit into
masterfrom
feature/expand-fuzz-coverage
May 19, 2026
Merged

🎲 test: load stdlib in panic-fuzz and add lexer-panic proptest#135
timfennis merged 1 commit into
masterfrom
feature/expand-fuzz-coverage

Conversation

@timfennis
Copy link
Copy Markdown
Owner

Summary

  • Stdlib in panic-fuzz: tests/proptest/tests/panic.rs now builds a FunctionRegistry, registers the stdlib, and feeds (name, StaticType) pairs to the analyser scope plus matching Vec<VmValue> (same iteration order so global slots line up) to Vm::new. Random programs that reference built-ins (print, len, …) now reach the analyser/compiler/VM instead of being rejected up front. Registry is rebuilt per case (Rc<NativeFunction> is !Send); release-mode cost is negligible — 1M cases in ~66s.
  • Lexer-panic proptest: new tests/proptest/tests/lexer_panic.rs mirrors the panic.rs structure (regression-replay default test + #[ignore]d fuzz test) but generates random String inputs. Char strategy is weighted toward lexer-trigger bytes (", \, #, r, digit prefixes/suffixes, operators, comment starts) plus full unicode for multi-byte offset coverage.
  • Number::pow exponent guard: first 1M-case run with stdlib loaded shrank to (-2) ^ i128::MAX, which calls num::pow::Pow on a multi-billion-bit magnitude and never returns (the author had flagged this inline in int.rs:120 but no guard existed). Now bails with BinaryOperatorError("exponent too large to compute") when the exponent magnitude exceeds 2^32 bits.
  • Regressions: tests/functional/programs/900_bugs/bug0020_pow_huge_exponent.ndc pins the user-visible error, and the proptest seed in panic.regressions guards against the underlying hang.

Notes for reviewers

  • 32-bit exponent ceiling is generous (~4 billion); any realistic exponent is orders of magnitude below this.
  • Lexer fuzzer found nothing at 100k cases — currently functions as a regression net rather than a bug source. The setup is in place to escalate (PROPTEST_CASES, MAX_INPUT_LEN).
  • The two test files share helpers (timeout/quiet panic hook); kept them duplicated for now since each test crate file is its own binary. Happy to dedup into a common/ module if preferred.

🤖 Generated with Claude Code

Wires the stdlib `FunctionRegistry` into the panic-fuzz analyser scope
and VM globals so randomly-generated programs that reference built-ins
(`print`, `len`, etc.) reach the analyser/compiler/VM instead of being
rejected up front. Adds a parallel lexer-only proptest seeded with
random strings.

The expanded fuzzer immediately found `2 ^ i128::MAX` hanging
`Number::pow` via `num::pow::Pow` on a multi-billion-bit magnitude.
Guarded with an early-return error when the exponent magnitude exceeds
2^32 bits, plus an `expect-error` regression test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ee19d660a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ndc_core/src/num.rs
@timfennis timfennis marked this pull request as draft May 7, 2026 14:28
@timfennis timfennis marked this pull request as ready for review May 19, 2026 08:31
@timfennis timfennis merged commit 2f77ea6 into master May 19, 2026
1 check passed
@timfennis timfennis deleted the feature/expand-fuzz-coverage branch May 19, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant