DataExtract-15

DataExtract-15 is an official BenchLocal Bench Pack for deterministic LLM data extraction. It measures whether a model can pull the right fields from practical text inputs and return clean, scorable extracted data.

A Bench Pack is an installable benchmark package that runs inside the BenchLocal desktop app. BenchLocal provides the shared app experience for provider setup, model selection, sampling controls, run histories, and side-by-side comparison across benchmark packs.

This repository contains the benchmark source: extraction scenarios, scoring logic, methodology, a BenchLocal adapter, and a CLI runner for local development.

Run With BenchLocal

1. Download BenchLocal from the latest BenchLocal release.
2. Open BenchLocal and install DataExtract-15 from the official Bench Pack registry.
3. Add one or more models, select DataExtract-15, and start a run.

Bench Pack Structure

lib/                    Benchmark core, scoring, and model runtime
benchlocal/             Thin BenchLocal SDK adapter
cli/                    Non-UI runner
benchlocal.pack.json  Static install/discovery manifest
METHODOLOGY.md          Published benchmark methodology

BenchLocal Adapter

• benchlocal/index.ts is the only place that imports @benchlocal/sdk.
• lib/ stays framework-agnostic and is shared by the CLI and BenchLocal.
• benchlocal.pack.json is the canonical Bench Pack metadata manifest used for install, inspection, and runtime metadata.
• Per-pack default sampling belongs on the manifest, not on global host settings.

Development

• BenchLocal build: npm run build:benchlocal
• CLI runner: npm run cli
• Methodology: METHODOLOGY.md

Validation

npm run typecheck
npm run build:benchlocal