Benchmark · Software
SWE-Forward
Checks that AI’s code survives the next release.
- What it measures
- Code fixes that survive the next release
- Scored by
- A simulated future of the codebase
- Headline figure
- Admission-checked tasks
- Domain
- Software
- Applies to
- AI coding tools
- Access
- On request
How it is scored
Simulated futures
A frozen patch is run against the codebase’s next releases.
What it tests
Code that lasts
- The AI’s fix is frozen, then we simulate the codebase’s future. Short-sighted code loses.
- A task is admitted only when a known-good fix and a known-bad fix score differently.
Experts behind it
Practitioners define what correct means for this benchmark. Paid, remote roles.
Other benchmarks