Sample

PubMed Citation Metadata and Refusal Calibration

A public JSONL preview is available below. Review it alongside the package scope and limitations.

Download public sample

CapabilityPubMed citation metadata grounding and refusal calibration

Designationmixed

Version2026-08-31

Updated2026-09-01

SourcesNCBI PubMed E-utilities

FormatsJSONL records, JSONL eval, JSONL SFT, JSONL observed-failure DPO, held-out eval JSONL

Product typeproduct family

Pricing modelcomponent pricing

500Proof rowsPubMed-backed proof evaluation.

32.2%Proof accuracy161 correct out of 500 source-aware graded rows.

67.8%Non-correct305 refusal or uncertainty responses and 34 substantive incorrect responses.

9,573Remediation rowsValidated PubMed metadata and refusal-calibration records.

970Held-out rowsUntouched validation rows with zero proof or remediation overlap.

339DPO pairsRejected answers are real observed proof-run failures.

Pricing

PubMed Metadata Proof Evalproof eval

$1,299 reference price

500 actual rows

500-row PubMed-backed proof evaluation with source-aware deterministic grading.

Request package

PubMed Metadata Remediationremediation training

$6,900 reference price

9,573 actual rows

Validated remediation rows targeting observed refusal, metadata identity, and retraction/correction failures.

Request package

PubMed Held-Out Validationheld out validation

$1,500 reference price

970 actual rows

Untouched held-out rows excluded from proof and remediation PMIDs, DOIs, source records, and source URLs.

Request package

PubMed Metadata Calibration Packfull improvement pack

$8,500 reference price

11,043 actual rows

500-row proof eval, 9,573 remediation rows, and 970 held-out validation rows. No model improvement result is included.

Request package

PubMed expansioncustom dataset

Custom quote

Any larger PubMed package requires separate source-supply and validation review.

Request package

PubMed proof evaluationproof eval

500 rows

source_aware_graded

Source-aware result: 161 correct, 34 substantive incorrect, and 305 refusal or uncertainty responses.

PubMed remediation recordsremediation

9,573 rows

validated

9,145 unique PMIDs, 9,122 unique DOIs, and 2,467 unique journals.

PubMed held-out validationheld out

970 rows

validated

970 unique PMIDs, 920 unique DOIs, and 553 unique journals.

PubMed observed-failure DPO poolremediation

339 rows

proof_derived_observed_failures

305 refusal-derived pairs and 34 substantive wrong-answer-derived pairs. No fabricated rejected responses.

Public PubMed samplesample

75 rows

public_preview

Public-safe preview. It does not include remediation or held-out datasets in full.

Schema fields

question, verified_answer, accepted_answers, pmid, doi, title, journal, publication_date, authors, publication_types, task_family, source_urls

Limitations

The proof result is specific to one completed browser-assisted model run and source-aware deterministic grading.
The 67.8% non-correct result is driven primarily by refusal or uncertainty.
Most non-correct rows were unnecessary refusal or uncertainty, not fabricated biomedical claims.
No post-training improvement has been measured yet.
The remediation set contains 9,573 rows, not 10,000 rows.
The held-out set contains 970 rows, not 1,000 rows.
PubMed metadata is authoritative for indexed publication metadata. Full-text rights and downstream redistribution rights require separate review.

Discuss this archived dataset