Back to All datasetsSample
Clinical Trial Outcomes Verification
4 real rows from the existing reference export, unedited. Review the schema and provenance here; confirm the specific package and delivery terms before purchasing.
47Legacy diagnostic rowsExisting published clinical-source verification export.
500Source-aware proof rowsFresh ClinicalTrials.gov API v2 proof rows graded against source truth.
9.0%Proof accuracy45 correct, 4 substantive incorrect, and 451 unnecessary refusal or uncertainty responses.
10,000Remediation rowsTargeted refusal-calibration and ClinicalTrials.gov fact-grounding rows.
1,000Held-out rowsUntouched held-out validation rows isolated from proof and remediation NCT IDs.
455Observed-failure DPO pairsRejected responses are actual proof-run refusals or wrong answers.
Pricing
Legacy diagnostic export is fixed-price. Expanded proof, remediation, held-out, and DPO package should be quoted as a clinical refusal-calibration system.
Legacy clinical-trial exportdiagnostic
47 rows
published
Existing clinical-source verification product.
ClinicalTrials.gov proof evalproof eval
500 rows
source_aware_graded
Source-aware result: 45 correct, 4 substantive incorrect, and 451 unnecessary refusal or uncertainty responses.
ClinicalTrials.gov remediationremediation
10,000 rows
validated
Built to teach supported answers when ClinicalTrials.gov contains the fact and calibrated abstention when the source is insufficient.
ClinicalTrials.gov held-out validationheld out
1,000 rows
validated
Untouched held-out validation with zero proof or remediation NCT overlap.
Limitations
- The 91.0% non-correct rate should not be described as a hallucination rate. Source-aware grading shows the dominant behavior was unnecessary refusal or uncertainty.
- No training or post-training held-out improvement result is claimed yet.
- True abstention examples are naturally source-backed but scarcer than answerable ClinicalTrials.gov facts in the current source pool.
Row 1 of 47
- row_id
- CT01
- entity
- Kisunla (donanemab-azbt) — Eli Lilly
- claimed_value
- FDA approved Kisunla (donanemab-azbt) for early symptomatic Alzheimer's disease on July 2, 2024, based on Phase 3 TRAILBLAZER-ALZ 2 trial data.
- claimed_value_source_url
- https://investor.lilly.com/news-releases/news-release-details/lillys-kisunlatm-donanemab-azbt-approved-fda-treatment-early
- registry_value
- openFDA drugsfda.json confirms application BLA761248 (sponsor: Eli Lilly and Company), original ('ORIG') submission with submission_status 'AP' (approved) and submission_status_date 2024-07-02 — an exact date match. Cross-checked live against FDA's Drugs@FDA HTML database (accessdata.fda.gov/scripts/cder/daf) -- a separate, legacy FDA system from the openFDA REST API: its 'Original Approvals' table for BLA761248 lists Action Date 07/02/2024, ORIG-1, Approval -- an exact independent match.
- registry_value_source_url
- https://www.accessdata.fda.gov/scripts/cder/daf/index.cfm?event=overview.process&ApplNo=761248
- agreement
- match
- explanation
- Verified live via scripts/verification/clinicaltrials.mjs getDrugApprovalHistory('KISUNLA'): drugsfda.json's ORIG submission for BLA761248 shows submission_status_date '20240702', exactly matching the press release's July 2, 2024 approval date. Sponsor name is recorded as 'ELI LILLY CO', consistent with the claim. Second method added this pass: fetched Drugs@FDA's own overview page for ApplNo=761248 live and parsed its approvals table directly, not just re-reading drugsfda.json -- Drugs@FDA is a distinct, older FDA web application from the modern openFDA REST API, so this is a genuinely separate confirmation of the same date, not a duplicate read of the same underlying record.
- grading_mode
- exact
- difficulty
- easy
- tags
[
"fda-approval",
"alzheimers",
"kisunla",
"2020s"
]
- model_tested
- claude-sonnet-5
- model_question
- On what date did the FDA approve Eli Lilly's Kisunla (donanemab-azbt) for early symptomatic Alzheimer's disease?
- model_response
- The FDA approved Kisunla (donanemab-azbt) on July 2, 2024, for adults with early symptomatic Alzheimer's disease (mild cognitive impairment or mild dementia stage).
- model_verdict
- correct
- model_grading_note
- Model states July 2, 2024 approval date for Kisunla, exactly matching verified registry data.
Row 2 of 47
- row_id
- CT02
- entity
- Donanemab — TRAILBLAZER-ALZ 2 (NCT04437511)
- claimed_value
- TRAILBLAZER-ALZ 2 met its primary endpoint: in the low/medium tau population, donanemab showed a 35.1% slowing of disease progression on the iADRS scale versus placebo.
- claimed_value_source_url
- https://www.neurologylive.com/view/fda-approves-eli-lilly-donanemab-early-symptomatic-ad
- registry_value
- ClinicalTrials.gov's posted results for NCT04437511 show, for the 'Intermediate (Low-medium) Tau Population' iADRS primary outcome: LS mean change difference 3.25 (95% CI 1.883-4.618), p<0.001. Raw group means (fetched separately) are -6.02 (donanemab) vs -9.27 (placebo) in this population — a relative difference of 1-(6.02/9.27) = 35.1%, matching the claim exactly.
- registry_value_source_url
- https://clinicaltrials.gov/study/NCT04437511?tab=results
- agreement
- match
- explanation
- Confirmed by calling getStudyByNctId('NCT04437511') and getPrimaryOutcomeResults('NCT04437511') against the real, live registry, then independently cross-checked against the peer-reviewed JAMA publication (Sims et al. 2023, DOI 10.1001/jama.2023.13239) via scripts/verification/openalex.mjs — its OpenAlex-reconstructed abstract reports the identical LSM values (-6.02 vs -9.27, diff 3.25, p<.001) and explicitly cites 'Trial Registration: ClinicalTrials.gov Identifier: NCT04437511'. Note: checked via scripts/verification/pubmed-oa.mjs whether this JAMA article (PMID 37459141, PMCID PMC10352931) is in the commercially-safe PMC Open Access subset — it is not (isOpenAccess: false) — so only OpenAlex's separately CC0-licensed abstract index was used, never PMC full text.
- grading_mode
- numeric
- difficulty
- hard
- tags
[
"clinical-trial",
"alzheimers",
"donanemab",
"iadrs"
]
- model_tested
- claude-sonnet-5
- model_question
- In the TRAILBLAZER-ALZ 2 trial, what percentage slowing of disease progression on the iADRS scale did donanemab show relative to placebo in the low/medium tau population?
- model_response
- I don't have confident recall of the exact percentage from TRAILBLAZER-ALZ 2's low/medium tau subgroup. I recall the trial's headline result was about a 35% slowing of decline on iADRS in the low/medium tau population (with a smaller effect, around 22%, in the combined population), but I'm not fully certain of that precise figure and would recommend verifying against the primary publication (Sims et al., JAMA 2023) rather than relying on my memory.
- model_verdict
- hedged_correctly
- model_grading_note
- Model explicitly flags uncertainty about the exact figure ('not fully certain... would recommend verifying') while still landing on the correct approximate value (~35%), an appropriate hedge rather than a confident fabrication.
Row 3 of 47
- row_id
- CT03
- entity
- Paxlovid (nirmatrelvir/ritonavir) — Pfizer, EPIC-HR (NCT04960202)
- claimed_value
- FDA granted traditional (full) approval to Paxlovid for high-risk adults on May 25, 2023, based on the EPIC-HR trial.
- claimed_value_source_url
- https://www.biopharmadive.com/news/pfizer-study-results-show-paxlovid-benefit-less-clear-in-lower-risk-patient/625549/
- registry_value
- openFDA drugsfda.json shows application NDA217188 (sponsor: Pfizer), ORIG submission approved (AP) with submission_status_date 2023-05-25 — an exact date match. NCT04960202 (EPIC-HR) is COMPLETED with hasResults=true. Cross-checked live against Drugs@FDA's HTML database for NDA217188: Action Date 05/25/2023, ORIG-1, Approval -- an exact independent match to the openFDA date.
- registry_value_source_url
- https://www.accessdata.fda.gov/scripts/cder/daf/index.cfm?event=overview.process&ApplNo=217188
- agreement
- match
- explanation
- getDrugApprovalHistory('PAXLOVID') returned NDA217188 approved 2023-05-25, matching the full-approval date reported in coverage of the EUA-to-full-approval transition. This is distinct from Paxlovid's original Dec 2021 Emergency Use Authorization. Second method added this pass: Drugs@FDA (a separate system/pipeline from openFDA's REST API), fetched live, independently confirms the same 05/25/2023 date.
- grading_mode
- exact
- difficulty
- easy
- tags
[
"fda-approval",
"covid-19",
"paxlovid",
"2020s"
]
- model_tested
- claude-sonnet-5
- model_question
- On what date did the FDA grant traditional (full) approval to Paxlovid for high-risk adults, based on the EPIC-HR trial?
- model_response
- The FDA granted full (traditional) approval to Paxlovid (nirmatrelvir/ritonavir) for high-risk adults on May 25, 2023.
- model_verdict
- correct
- model_grading_note
- Model states May 25, 2023 full-approval date for Paxlovid, exactly matching verified registry data.
Row 4 of 47
- row_id
- CT04
- entity
- Paxlovid — EPIC-HR trial (NCT04960202)
- claimed_value
- In the final EPIC-HR analysis, Paxlovid reduced the risk of COVID-19-related hospitalization or death by 89% relative to placebo.
- claimed_value_source_url
- https://www.statnews.com/2021/12/14/pfizers-covid-pill-remains-89-effective-in-final-analysis-company-says/
- registry_value
- ClinicalTrials.gov's posted primary outcome (mITT population, hospitalization-or-death through Day 28) shows 0.752% (nirmatrelvir/ritonavir, n=671) vs 6.888% (placebo, n=647), p<0.0001. Relative risk reduction computed from these registry-posted rates: 1-(0.752/6.888) = 89.1%, matching the claimed 89% figure. Independently confirmed by the peer-reviewed publication (Hammond et al., NEJM 2022, DOI 10.1056/NEJMoa2118542, confirmed real via OpenAlex): its abstract states the risk of progression to severe Covid-19 was '89% lower... than the risk with placebo,' explicitly citing 'ClinicalTrials.gov number, NCT04960202.'
- registry_value_source_url
- https://clinicaltrials.gov/study/NCT04960202?tab=results
- agreement
- match
- explanation
- getPrimaryOutcomeResults('NCT04960202') returned the exact per-arm event rates used to independently recompute the relative risk reduction (89.1%) rather than trusting Pfizer's press-release arithmetic directly — it lands within rounding of the claimed 89%. Second method added this pass: the NEJM publication states the identical 89% figure and cites the same NCT ID, independent of both the registry's raw mITT percentages and Pfizer's press materials. Checked via pubmed-oa.mjs: PMID 35172054 (PMCID PMC8908851) carries no commercially-safe OA license detected, so only OpenAlex's separately CC0-licensed abstract was used, never PMC full text.
- grading_mode
- numeric
- difficulty
- hard
- tags
[
"clinical-trial",
"covid-19",
"paxlovid",
"efficacy"
]
- model_tested
- claude-sonnet-5
- model_question
- In the final analysis of Pfizer's EPIC-HR trial, by what percentage did Paxlovid reduce the risk of COVID-19-related hospitalization or death compared to placebo?
- model_response
- In the final (updated/full) analysis of EPIC-HR, Paxlovid reduced the risk of COVID-19-related hospitalization or death by approximately 86% compared to placebo among high-risk, unvaccinated adults treated within 5 days of symptom onset (the earlier interim analysis had reported 89%).
- model_verdict
- hallucinated
- model_grading_note
- Ground truth confirms the final EPIC-HR analysis showed an 89% (89.1%) relative risk reduction, but the model inverts this, stating the final analysis was 86% and the interim was 89%.