Back to All datasetsSample
PubMed Citation Metadata and Refusal Calibration
A public JSONL preview is available below. Review it alongside the package scope and limitations.
Download public sample500Proof rowsPubMed-backed proof evaluation.
32.2%Proof accuracy161 correct out of 500 source-aware graded rows.
67.8%Non-correct305 refusal or uncertainty responses and 34 substantive incorrect responses.
9,573Remediation rowsValidated PubMed metadata and refusal-calibration records.
970Held-out rowsUntouched validation rows with zero proof or remediation overlap.
339DPO pairsRejected answers are real observed proof-run failures.
Pricing
PubMed Metadata Proof Evalproof eval
$1,299 reference price
500 actual rows
500-row PubMed-backed proof evaluation with source-aware deterministic grading.
Request packagePubMed Metadata Remediationremediation training
$6,900 reference price
9,573 actual rows
Validated remediation rows targeting observed refusal, metadata identity, and retraction/correction failures.
Request packagePubMed Held-Out Validationheld out validation
$1,500 reference price
970 actual rows
Untouched held-out rows excluded from proof and remediation PMIDs, DOIs, source records, and source URLs.
Request packagePubMed Metadata Calibration Packfull improvement pack
$8,500 reference price
11,043 actual rows
500-row proof eval, 9,573 remediation rows, and 970 held-out validation rows. No model improvement result is included.
Request packagePubMed expansioncustom dataset
Custom quote
Any larger PubMed package requires separate source-supply and validation review.
Request packagePubMed proof evaluationproof eval
500 rows
source_aware_graded
Source-aware result: 161 correct, 34 substantive incorrect, and 305 refusal or uncertainty responses.
PubMed remediation recordsremediation
9,573 rows
validated
9,145 unique PMIDs, 9,122 unique DOIs, and 2,467 unique journals.
PubMed held-out validationheld out
970 rows
validated
970 unique PMIDs, 920 unique DOIs, and 553 unique journals.
PubMed observed-failure DPO poolremediation
339 rows
proof_derived_observed_failures
305 refusal-derived pairs and 34 substantive wrong-answer-derived pairs. No fabricated rejected responses.
Public PubMed samplesample
75 rows
public_preview
Public-safe preview. It does not include remediation or held-out datasets in full.
Schema fields
question, verified_answer, accepted_answers, pmid, doi, title, journal, publication_date, authors, publication_types, task_family, source_urls
Limitations
- The proof result is specific to one completed browser-assisted model run and source-aware deterministic grading.
- The 67.8% non-correct result is driven primarily by refusal or uncertainty.
- Most non-correct rows were unnecessary refusal or uncertainty, not fabricated biomedical claims.
- No post-training improvement has been measured yet.
- The remediation set contains 9,573 rows, not 10,000 rows.
- The held-out set contains 970 rows, not 1,000 rows.
- PubMed metadata is authoritative for indexed publication metadata. Full-text rights and downstream redistribution rights require separate review.
Discuss this archived dataset