{"id":"public-SECP00043","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its revenue reported under us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax?","verified_answer":"364357000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2026 Q3, period end 2026-06-27, reported value 364357000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00044","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2018 with period end 2018-09-29, what was its revenue reported under us-gaap:Revenues?","verified_answer":"265595000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000320193-18-000145, CIK 0000320193, us-gaap:Revenues, fiscal 2018 FY, period end 2018-09-29, reported value 265595000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019318000145/0000320193-18-000145-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/Revenues.json"],"task_family":"revenue","failure_types":[],"reference_model":"chatgpt-web","model_response":"265595000000","verdict":"correct","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00045","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2017 with period end 2017-09-30, what was its revenue reported under us-gaap:SalesRevenueNet?","verified_answer":"229234000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000320193-17-000070, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2017 FY, period end 2017-09-30, reported value 229234000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019317000070/0000320193-17-000070-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"revenue","failure_types":[],"reference_model":"chatgpt-web","model_response":"229234000000","verdict":"correct","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00046","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2016 with period end 2016-09-24, what was its revenue reported under us-gaap:SalesRevenueNet?","verified_answer":"215639000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001628280-16-020309, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2016 FY, period end 2016-09-24, reported value 215639000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000162828016020309/0001628280-16-020309-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"revenue","failure_types":[],"reference_model":"chatgpt-web","model_response":"215639000000","verdict":"correct","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00047","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2018 with period end 2018-06-30, what was its revenue reported under us-gaap:SalesRevenueNet?","verified_answer":"202695000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-18-000100, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2018 Q3, period end 2018-06-30, reported value 202695000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019318000100/0000320193-18-000100-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00068","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its revenue reported under us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax?","verified_answer":"331839000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2026 FY, period end 2026-06-30, reported value 331839000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00069","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its revenue reported under us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax?","verified_answer":"281724000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2025 FY, period end 2025-06-30, reported value 281724000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00070","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its revenue reported under us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax?","verified_answer":"241832000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2026 Q3, period end 2026-03-31, reported value 241832000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00071","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2010 with period end 2010-06-30, what was its revenue reported under us-gaap:Revenues?","verified_answer":"62484000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-10-171791, CIK 0000789019, us-gaap:Revenues, fiscal 2010 FY, period end 2010-06-30, reported value 62484000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312510171791/0001193125-10-171791-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Revenues.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00072","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q2 fiscal 2011 with period end 2010-12-31, what was its revenue reported under us-gaap:Revenues?","verified_answer":"36148000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-11-015947, CIK 0000789019, us-gaap:Revenues, fiscal 2011 Q2, period end 2010-12-31, reported value 36148000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312511015947/0001193125-11-015947-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Revenues.json"],"task_family":"revenue","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00048","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"101464000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:NetIncomeLoss, fiscal 2026 Q3, period end 2026-06-27, reported value 101464000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00076","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"133749000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:NetIncomeLoss, fiscal 2026 FY, period end 2026-06-30, reported value 133749000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00077","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"101832000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:NetIncomeLoss, fiscal 2025 FY, period end 2025-06-30, reported value 101832000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00078","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"97983000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:NetIncomeLoss, fiscal 2026 Q3, period end 2026-03-31, reported value 97983000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00116","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-K filing for fiscal 2025 with period end 2025-12-31, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"77670000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001018724-26-000004, CIK 0001018724, us-gaap:NetIncomeLoss, fiscal 2025 FY, period end 2025-12-31, reported value 77670000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/0001018724-26-000004-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00117","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"135281000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001018724-26-000026, CIK 0001018724, us-gaap:NetIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 135281000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000026/0001018724-26-000026-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00143","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Meta Platforms, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"42621000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-26-050705, CIK 0001326801, us-gaap:NetIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 42621000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1326801/000162828026050705/0001628280-26-050705-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001326801/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00163","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Alphabet Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its net income reported under us-gaap:NetIncomeLoss?","verified_answer":"174771000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001652044-26-000071, CIK 0001652044, us-gaap:NetIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 174771000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1652044/000165204426000071/0001652044-26-000071-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001652044/us-gaap/NetIncomeLoss.json"],"task_family":"net_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00049","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"122432000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:OperatingIncomeLoss, fiscal 2026 Q3, period end 2026-06-27, reported value 122432000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00079","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"155237000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:OperatingIncomeLoss, fiscal 2026 FY, period end 2026-06-30, reported value 155237000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00080","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"128528000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:OperatingIncomeLoss, fiscal 2025 FY, period end 2025-06-30, reported value 128528000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00081","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"114634000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:OperatingIncomeLoss, fiscal 2026 Q3, period end 2026-03-31, reported value 114634000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00118","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-K filing for fiscal 2025 with period end 2025-12-31, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"79975000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001018724-26-000004, CIK 0001018724, us-gaap:OperatingIncomeLoss, fiscal 2025 FY, period end 2025-12-31, reported value 79975000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/0001018724-26-000004-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00119","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"51313000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001018724-26-000026, CIK 0001018724, us-gaap:OperatingIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 51313000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000026/0001018724-26-000026-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00144","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Meta Platforms, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"41647000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-26-050705, CIK 0001326801, us-gaap:OperatingIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 41647000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1326801/000162828026050705/0001628280-26-050705-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001326801/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00164","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Alphabet Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its operating income reported under us-gaap:OperatingIncomeLoss?","verified_answer":"80466000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001652044-26-000071, CIK 0001652044, us-gaap:OperatingIncomeLoss, fiscal 2026 Q2, period end 2026-06-30, reported value 80466000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1652044/000165204426000071/0001652044-26-000071-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001652044/us-gaap/OperatingIncomeLoss.json"],"task_family":"operating_income","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00050","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivities?","verified_answer":"116996000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:NetCashProvidedByUsedInOperatingActivities, fiscal 2026 Q3, period end 2026-06-27, reported value 116996000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/NetCashProvidedByUsedInOperatingActivities.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00051","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2016 with period end 2016-09-24, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations?","verified_answer":"65824000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001628280-16-020309, CIK 0000320193, us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations, fiscal 2016 FY, period end 2016-09-24, reported value 65824000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000162828016020309/0001628280-16-020309-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/NetCashProvidedByUsedInOperatingActivitiesContinuingOperations.json"],"task_family":"cash_flow","failure_types":[],"reference_model":"chatgpt-web","model_response":"65824000000","verdict":"correct","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00052","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2015 with period end 2015-09-26, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations?","verified_answer":"81266000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-15-356351, CIK 0000320193, us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations, fiscal 2015 FY, period end 2015-09-26, reported value 81266000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000119312515356351/0001193125-15-356351-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/NetCashProvidedByUsedInOperatingActivitiesContinuingOperations.json"],"task_family":"cash_flow","failure_types":[],"reference_model":"chatgpt-web","model_response":"81266000000","verdict":"correct","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00053","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q1 fiscal 2017 with period end 2016-12-31, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations?","verified_answer":"27056000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-17-000717, CIK 0000320193, us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations, fiscal 2017 Q1, period end 2016-12-31, reported value 27056000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000162828017000717/0001628280-17-000717-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/NetCashProvidedByUsedInOperatingActivitiesContinuingOperations.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00082","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivities?","verified_answer":"182935000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:NetCashProvidedByUsedInOperatingActivities, fiscal 2026 FY, period end 2026-06-30, reported value 182935000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetCashProvidedByUsedInOperatingActivities.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00083","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivities?","verified_answer":"136162000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:NetCashProvidedByUsedInOperatingActivities, fiscal 2025 FY, period end 2025-06-30, reported value 136162000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetCashProvidedByUsedInOperatingActivities.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00084","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivities?","verified_answer":"127494000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:NetCashProvidedByUsedInOperatingActivities, fiscal 2026 Q3, period end 2026-03-31, reported value 127494000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetCashProvidedByUsedInOperatingActivities.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00085","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2017 with period end 2017-06-30, what was its net cash provided by operating activities reported under us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations?","verified_answer":"39507000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001564590-17-014900, CIK 0000789019, us-gaap:NetCashProvidedByUsedInOperatingActivitiesContinuingOperations, fiscal 2017 FY, period end 2017-06-30, reported value 39507000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000156459017014900/0001564590-17-014900-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/NetCashProvidedByUsedInOperatingActivitiesContinuingOperations.json"],"task_family":"cash_flow","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00056","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its total assets reported under us-gaap:Assets?","verified_answer":"383266000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:Assets, fiscal 2026 Q3, period end 2026-06-27, reported value 383266000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00094","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its total assets reported under us-gaap:Assets?","verified_answer":"758376000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:Assets, fiscal 2026 FY, period end 2026-06-30, reported value 758376000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00095","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its total assets reported under us-gaap:Assets?","verified_answer":"619003000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:Assets, fiscal 2025 FY, period end 2025-06-30, reported value 619003000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00096","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its total assets reported under us-gaap:Assets?","verified_answer":"694228000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:Assets, fiscal 2026 Q3, period end 2026-03-31, reported value 694228000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00126","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-K filing for fiscal 2025 with period end 2025-12-31, what was its total assets reported under us-gaap:Assets?","verified_answer":"818042000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001018724-26-000004, CIK 0001018724, us-gaap:Assets, fiscal 2025 FY, period end 2025-12-31, reported value 818042000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/0001018724-26-000004-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00127","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Amazon.com, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total assets reported under us-gaap:Assets?","verified_answer":"1095689000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001018724-26-000026, CIK 0001018724, us-gaap:Assets, fiscal 2026 Q2, period end 2026-06-30, reported value 1095689000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000026/0001018724-26-000026-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00148","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Meta Platforms, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total assets reported under us-gaap:Assets?","verified_answer":"449956000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-26-050705, CIK 0001326801, us-gaap:Assets, fiscal 2026 Q2, period end 2026-06-30, reported value 449956000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1326801/000162828026050705/0001628280-26-050705-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001326801/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00168","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Alphabet Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total assets reported under us-gaap:Assets?","verified_answer":"921983000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001652044-26-000071, CIK 0001652044, us-gaap:Assets, fiscal 2026 Q2, period end 2026-06-30, reported value 921983000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1652044/000165204426000071/0001652044-26-000071-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001652044/us-gaap/Assets.json"],"task_family":"assets","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00057","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-06-27, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"275746000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0000320193-26-000020, CIK 0000320193, us-gaap:Liabilities, fiscal 2026 Q3, period end 2026-06-27, reported value 275746000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019326000020/0000320193-26-000020-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00097","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2026 with period end 2026-06-30, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"315989000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:Liabilities, fiscal 2026 FY, period end 2026-06-30, reported value 315989000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00098","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-K filing for fiscal 2025 with period end 2025-06-30, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"275524000000","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:Liabilities, fiscal 2025 FY, period end 2025-06-30, reported value 275524000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"numeric_exact"}
{"id":"public-SECP00099","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Microsoft Corporation's SEC 10-Q filing for Q3 fiscal 2026 with period end 2026-03-31, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"279861000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001193125-26-191507, CIK 0000789019, us-gaap:Liabilities, fiscal 2026 Q3, period end 2026-03-31, reported value 279861000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526191507/0001193125-26-191507-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00149","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Meta Platforms, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"188735000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-26-050705, CIK 0001326801, us-gaap:Liabilities, fiscal 2026 Q2, period end 2026-06-30, reported value 188735000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1326801/000162828026050705/0001628280-26-050705-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001326801/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00169","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Alphabet Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"281503000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001652044-26-000071, CIK 0001652044, us-gaap:Liabilities, fiscal 2026 Q2, period end 2026-06-30, reported value 281503000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1652044/000165204426000071/0001652044-26-000071-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001652044/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00185","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to NVIDIA Corporation's SEC 10-Q filing for Q2 fiscal 2027 with period end 2026-07-26, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"91288000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001045810-26-000075, CIK 0001045810, us-gaap:Liabilities, fiscal 2027 Q2, period end 2026-07-26, reported value 91288000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1045810/000104581026000075/0001045810-26-000075-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001045810/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00205","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Tesla, Inc.'s SEC 10-Q filing for Q2 fiscal 2026 with period end 2026-06-30, what was its total liabilities reported under us-gaap:Liabilities?","verified_answer":"61005000000","verification_summary":"Verified against SEC EDGAR, 10-Q, accession 0001628280-26-049270, CIK 0001318605, us-gaap:Liabilities, fiscal 2026 Q2, period end 2026-06-30, reported value 61005000000 USD.","source_urls":["https://www.sec.gov/Archives/edgar/data/1318605/000162828026049270/0001628280-26-049270-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001318605/us-gaap/Liabilities.json"],"task_family":"liabilities","failure_types":["wrong_numeric_value"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_exact"}
{"id":"public-SECP00038","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2023, what was the fiscal period end date for the us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax annual revenue fact?","verified_answer":"2023-09-30","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000320193-23-000106, CIK 0000320193, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2023 FY, period end 2023-09-30.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019323000106/0000320193-23-000106-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"fiscal_year_dates","failure_types":[],"reference_model":"chatgpt-web","model_response":"2023-09-30","verdict":"correct","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00039","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2018, what was the fiscal period end date for the us-gaap:Revenues annual revenue fact?","verified_answer":"2018-09-29","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000320193-18-000145, CIK 0000320193, us-gaap:Revenues, fiscal 2018 FY, period end 2018-09-29.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019318000145/0000320193-18-000145-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/Revenues.json"],"task_family":"fiscal_year_dates","failure_types":[],"reference_model":"chatgpt-web","model_response":"2018-09-29","verdict":"correct","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00040","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2017, what was the fiscal period end date for the us-gaap:SalesRevenueNet annual revenue fact?","verified_answer":"2017-09-30","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000320193-17-000070, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2017 FY, period end 2017-09-30.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000032019317000070/0000320193-17-000070-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"fiscal_year_dates","failure_types":[],"reference_model":"chatgpt-web","model_response":"2017-09-30","verdict":"correct","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00041","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2016, what was the fiscal period end date for the us-gaap:SalesRevenueNet annual revenue fact?","verified_answer":"2016-09-24","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001628280-16-020309, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2016 FY, period end 2016-09-24.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000162828016020309/0001628280-16-020309-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"fiscal_year_dates","failure_types":[],"reference_model":"chatgpt-web","model_response":"2016-09-24","verdict":"correct","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00042","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to Apple Inc.'s SEC 10-K filing for fiscal 2015, what was the fiscal period end date for the us-gaap:SalesRevenueNet annual revenue fact?","verified_answer":"2015-09-26","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-15-356351, CIK 0000320193, us-gaap:SalesRevenueNet, fiscal 2015 FY, period end 2015-09-26.","source_urls":["https://www.sec.gov/Archives/edgar/data/320193/000119312515356351/0001193125-15-356351-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/SalesRevenueNet.json"],"task_family":"fiscal_year_dates","failure_types":[],"reference_model":"chatgpt-web","model_response":"2015-09-26","verdict":"correct","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00062","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to MICROSOFT CORP's SEC 10-K filing for fiscal 2026, what was the fiscal period end date for the us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax annual revenue fact?","verified_answer":"2026-06-30","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-26-323660, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2026 FY, period end 2026-06-30.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"fiscal_year_dates","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00063","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to MICROSOFT CORP's SEC 10-K filing for fiscal 2025, what was the fiscal period end date for the us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax annual revenue fact?","verified_answer":"2025-06-30","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0000950170-25-100235, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, fiscal 2025 FY, period end 2025-06-30.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"fiscal_year_dates","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00064","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"According to MICROSOFT CORP's SEC 10-K filing for fiscal 2010, what was the fiscal period end date for the us-gaap:Revenues annual revenue fact?","verified_answer":"2010-06-30","verification_summary":"Verified against SEC EDGAR, 10-K, accession 0001193125-10-171791, CIK 0000789019, us-gaap:Revenues, fiscal 2010 FY, period end 2010-06-30.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312510171791/0001193125-10-171791-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/Revenues.json"],"task_family":"fiscal_year_dates","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"date_exact"}
{"id":"public-SECP00103","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using MICROSOFT CORP's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2025 to fiscal 2026? Round to two decimal places.","verified_answer":"17.79%","verification_summary":"Verified against SEC EDGAR, accession 0001193125-26-323660, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2026-06-30, calculated from SEC values 331839000000 and 281724000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00132","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using AMAZON COM INC's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"12.38%","verification_summary":"Verified against SEC EDGAR, accession 0001018724-26-000004, CIK 0001018724, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-12-31, calculated from SEC values 716924000000 and 637959000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/0001018724-26-000004-index.htm","https://www.sec.gov/Archives/edgar/data/1018724/000101872425000004/0001018724-25-000004-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00189","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using NVIDIA CORP's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2021 to fiscal 2022? Round to two decimal places.","verified_answer":"61.40%","verification_summary":"Verified against SEC EDGAR, accession 0001045810-22-000036, CIK 0001045810, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2022-01-30, calculated from SEC values 26914000000 and 16675000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1045810/000104581022000036/0001045810-22-000036-index.htm","https://www.sec.gov/Archives/edgar/data/1045810/000104581021000010/0001045810-21-000010-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001045810/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00247","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using NETFLIX INC's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"15.85%","verification_summary":"Verified against SEC EDGAR, accession 0001065280-26-000034, CIK 0001065280, us-gaap:Revenues, period end 2025-12-31, calculated from SEC values 45183036000 and 39000966000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1065280/000106528026000034/0001065280-26-000034-index.htm","https://www.sec.gov/Archives/edgar/data/1065280/000106528025000044/0001065280-25-000044-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001065280/us-gaap/Revenues.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00277","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using Broadcom Inc.'s SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"23.87%","verification_summary":"Verified against SEC EDGAR, accession 0001730168-25-000121, CIK 0001730168, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-11-02, calculated from SEC values 63887000000 and 51574000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1730168/000173016825000121/0001730168-25-000121-index.htm","https://www.sec.gov/Archives/edgar/data/1730168/000173016824000139/0001730168-24-000139-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001730168/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00295","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using VISA INC.'s SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"11.34%","verification_summary":"Verified against SEC EDGAR, accession 0001403161-25-000089, CIK 0001403161, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-09-30, calculated from SEC values 40000000000 and 35926000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1403161/000140316125000089/0001403161-25-000089-index.htm","https://www.sec.gov/Archives/edgar/data/1403161/000140316124000058/0001403161-24-000058-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001403161/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00314","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using Mastercard Inc's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2020 to fiscal 2021? Round to two decimal places.","verified_answer":"26.38%","verification_summary":"Verified against SEC EDGAR, accession 0001141391-22-000023, CIK 0001141391, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2021-12-31, calculated from SEC values 29845000000 and 23616000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1141391/000114139122000023/0001141391-22-000023-index.htm","https://www.sec.gov/Archives/edgar/data/1141391/000114139121000018/0001141391-21-000018-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001141391/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00326","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using JPMORGAN CHASE & CO's SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"2.75%","verification_summary":"Verified against SEC EDGAR, accession 0001628280-26-008131, CIK 0000019617, us-gaap:Revenues, period end 2025-12-31, calculated from SEC values 182447000000 and 177556000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/19617/000162828026008131/0001628280-26-008131-index.htm","https://www.sec.gov/Archives/edgar/data/19617/000001961725000270/0000019617-25-000270-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000019617/us-gaap/Revenues.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00339","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"Using BANK OF AMERICA CORP /DE/'s SEC-filed revenue values, what was the year-over-year percentage change from fiscal 2024 to fiscal 2025? Round to two decimal places.","verified_answer":"11.00%","verification_summary":"Verified against SEC EDGAR, accession 0000070858-26-000157, CIK 0000070858, us-gaap:Revenues, period end 2025-12-31, calculated from SEC values 113097000000 and 101887000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/70858/000007085826000157/0000070858-26-000157-index.htm","https://www.sec.gov/Archives/edgar/data/70858/000007085825000139/0000070858-25-000139-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000070858/us-gaap/Revenues.json"],"task_family":"yoy_calculations","failure_types":["wrong_numeric_value","calculation_error","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"hard","grading_mode":"numeric_percentage_tolerance_0.01"}
{"id":"public-SECP00102","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did MICROSOFT CORP's revenue change from fiscal 2025 to fiscal 2026? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001193125-26-323660, CIK 0000789019, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2026-06-30, calculated from SEC values 331839000000 and 281724000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/0001193125-26-323660-index.htm","https://www.sec.gov/Archives/edgar/data/789019/000095017025100235/0000950170-25-100235-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000789019/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":[],"reference_model":"chatgpt-web","model_response":"increased","verdict":"correct","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00131","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did AMAZON COM INC's revenue change from fiscal 2024 to fiscal 2025? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001018724-26-000004, CIK 0001018724, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-12-31, calculated from SEC values 716924000000 and 637959000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/0001018724-26-000004-index.htm","https://www.sec.gov/Archives/edgar/data/1018724/000101872425000004/0001018724-25-000004-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001018724/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":[],"reference_model":"chatgpt-web","model_response":"increased","verdict":"correct","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00188","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did NVIDIA CORP's revenue change from fiscal 2021 to fiscal 2022? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001045810-22-000036, CIK 0001045810, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2022-01-30, calculated from SEC values 26914000000 and 16675000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1045810/000104581022000036/0001045810-22-000036-index.htm","https://www.sec.gov/Archives/edgar/data/1045810/000104581021000010/0001045810-21-000010-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001045810/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":[],"reference_model":"chatgpt-web","model_response":"increased","verdict":"correct","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00246","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did NETFLIX INC's revenue change from fiscal 2024 to fiscal 2025? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001065280-26-000034, CIK 0001065280, us-gaap:Revenues, period end 2025-12-31, calculated from SEC values 45183036000 and 39000966000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1065280/000106528026000034/0001065280-26-000034-index.htm","https://www.sec.gov/Archives/edgar/data/1065280/000106528025000044/0001065280-25-000044-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001065280/us-gaap/Revenues.json"],"task_family":"period_comparisons","failure_types":[],"reference_model":"chatgpt-web","model_response":"increased","verdict":"correct","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00276","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did Broadcom Inc.'s revenue change from fiscal 2024 to fiscal 2025? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001730168-25-000121, CIK 0001730168, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-11-02, calculated from SEC values 63887000000 and 51574000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1730168/000173016825000121/0001730168-25-000121-index.htm","https://www.sec.gov/Archives/edgar/data/1730168/000173016824000139/0001730168-24-000139-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001730168/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":[],"reference_model":"chatgpt-web","model_response":"increased","verdict":"correct","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00294","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did VISA INC.'s revenue change from fiscal 2024 to fiscal 2025? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001403161-25-000089, CIK 0001403161, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2025-09-30, calculated from SEC values 40000000000 and 35926000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1403161/000140316125000089/0001403161-25-000089-index.htm","https://www.sec.gov/Archives/edgar/data/1403161/000140316124000058/0001403161-24-000058-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001403161/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00313","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did Mastercard Inc's revenue change from fiscal 2020 to fiscal 2021? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001141391-22-000023, CIK 0001141391, us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax, period end 2021-12-31, calculated from SEC values 29845000000 and 23616000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/1141391/000114139122000023/0001141391-22-000023-index.htm","https://www.sec.gov/Archives/edgar/data/1141391/000114139121000018/0001141391-21-000018-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0001141391/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json"],"task_family":"period_comparisons","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"categorical_exact"}
{"id":"public-SECP00325","dataset":"sec-edgar-financial-grounding-sample","domain":"finance","question":"How did JPMORGAN CHASE & CO's revenue change from fiscal 2024 to fiscal 2025? Answer with increased, decreased, or stayed the same.","verified_answer":"increased","verification_summary":"Verified against SEC EDGAR, accession 0001628280-26-008131, CIK 0000019617, us-gaap:Revenues, period end 2025-12-31, calculated from SEC values 182447000000 and 177556000000.","source_urls":["https://www.sec.gov/Archives/edgar/data/19617/000162828026008131/0001628280-26-008131-index.htm","https://www.sec.gov/Archives/edgar/data/19617/000001961725000270/0000019617-25-000270-index.htm","https://data.sec.gov/api/xbrl/companyconcept/CIK0000019617/us-gaap/Revenues.json"],"task_family":"period_comparisons","failure_types":["wrong_date","temporal_confusion"],"reference_model":"chatgpt-web","model_response":"unknown","verdict":"incorrect","difficulty":"medium","grading_mode":"categorical_exact"}