Bug-fix task pack from private repositories with hidden tests, 12 firms

Summary

8,460 coding tasks built from real bug reports and the pull requests that fixed them in the private codebases of 12 US software companies. Each task gives an agent the issue text and the repository at the base commit, then scores it with the tests the original fix added, which fail before the fix and pass after. Only repositories that have never been public are used, which keeps the tasks out of public training data. Secrets and identities are scrubbed and every task is checked to run in a clean container.

Collected to order. We collect it from US companies for your order. Scope, volume and timing are confirmed in writing before you sign.

Pricing on request

Request scope and pricing

At a glance

Planned scope. Volume, sources and timing are confirmed in writing before you sign.

Records: 8,460tasks
Size: 15.2 GB
Tokens, estimated: 3.8B
Years: 2018 to 2026
Firms: 12

Owner profile: 12 US software companies, 25 to 220 staff each, with Python, TypeScript, Go and Java codebases
Industry: Software
Data type: Evaluation and task packs
Headcount: 25 to 220 staff per firm
Region: United States
Languages: English
Formats: Source tree, JSONL
Updated: 2026-10-11

Sources

System	Kind	Records
GitHub	Code	380 repositories
GitLab	Code	145 repositories
Jira	Projects and wiki	8,460 linked bug reports

Good for

evaluating coding agents on bug fixes no model has seen in training
reinforcement learning with test-based rewards in real application code
measuring how agents read an issue, find the right files and write a minimal patch
comparing agent patches with the patch the team actually merged

Fields

Field	Type	Description
task_id	string	Task id
firm_code	string	Pool member code
language	enum	Python, TypeScript, Go or Java
issue_text	string	The bug report as filed, de-identified
repo_snapshot	string	Path to the repository bundle at the base commit
base_commit	string	Commit the agent starts from
fail_to_pass	array<string>	Tests that fail before the fix and pass after
pass_to_pass_count	int	Existing tests that must keep passing
gold_patch_lines	int	Lines changed by the merged fix, which is held back from agents
difficulty	enum	under 15 min, 15 min to 1 h, 1 to 4 h or over 4 h for a staff engineer

Example record

{
  "task_id": "BF-F09-01277",
  "firm_code": "F09",
  "language": "Python",
  "issue_text": "CSV import treats a trailing delimiter as an extra empty column and shifts every value after it. Reported by [PERSON_2] after [ACCOUNT_57] uploaded an inventory file and quantities landed in the SKU column.",
  "repo_snapshot": "repos/F09/[REPO_4].bundle",
  "base_commit": "4be09d2c",
  "fail_to_pass": [
    "tests/importers/test_csv.py::test_trailing_delimiter_ignored",
    "tests/importers/test_csv.py::test_quoted_empty_last_field"
  ],
  "pass_to_pass_count": 412,
  "gold_patch_lines": 18,
  "difficulty": "15 min to 1 h"
}

One record, de-identified. Placeholders such as [PERSON_12] and [COMPANY_4] stay the same across the whole dataset, so threads still read correctly.

Licenses available

Exclusive
One buyer. Nobody else can license it during the exclusive term, usually 2 years.
Non-exclusive
Licensed to more than one buyer.

What ships with it

Datasheet: what it is, what it is good for, verified size, sources, years, format and limits.
Provenance file: the owner shown by code, staff notice and consent status, exclusions, mapped to the EU AI Act Article 53 training-data summary and California AB 2013.
Manifest with a hash for every file.
QC report and de-identification report.
Your own copy, with unique canary records.

How it is de-identified

Default exclusions removed, files flattened to text, identifiers found in three layers and replaced with consistent placeholders, and the result measured against a 99% bar for each identifier type in each source.

The full method