← All datasets
Sample

Code-Reality Verification Dataset

4 real rows pulled directly from the full dataset, unedited — the actual field structure and content a buyer receives. The full purchase includes all 42 rows.

Row 1 of 42
row_id
CR1
ecosystem
npm
package
lodash
version
4.17.21
claim
lodash 4.17.21's `_.chunk(array, size)` function actually accepts a third `guard` parameter, used internally to support lodash's iteratee-shorthand convention (e.g. being safely passed to `_.map`).
verdict
true
actual_finding
Confirmed against the real published npm tarball: package/chunk.js line 30 defines `function chunk(array, size, guard) {`, matching the claimed three-parameter signature exactly. SECOND METHOD (dynamic execution, this session): actually ran `npm install lodash@4.17.21` and executed the real code in Node 24.18.0 — `_.chunk(['a','b','c','d'], 2)` returns `[['a','b'],['c','d']]` as expected, confirming the function is live and callable with this signature.
source_url
https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz
difficulty
easy
grading_mode
judge
tags
[
  "npm",
  "lodash",
  "parameter-signature",
  "method-existence"
]
verification_method
static_source_inspection+dynamic_execution
model_tested
claude-sonnet-5
model_question
In lodash 4.17.21, what parameters does the `_.chunk` function accept, and does it include a third parameter?
model_response
`_.chunk(array, [size=1])` — it takes an array to split and an optional size for each chunk (defaulting to 1). It does not have a third parameter in lodash 4.17.21.
model_verdict
hallucinated
model_grading_note
Model states _.chunk has no third parameter, directly contradicting the confirmed ground truth that chunk(array, size, guard) has a third guard parameter.
Row 2 of 42
row_id
CR2
ecosystem
npm
package
lodash
version
4.17.21
claim
lodash 4.17.21's `_.debounce(func, wait, options)` takes a third `options` object argument (for `leading`/`trailing`/`maxWait`).
verdict
true
actual_finding
Confirmed: package/debounce.js defines `function debounce(func, wait, options) {`, matching the claimed signature. SECOND METHOD (dynamic execution, this session): installed lodash 4.17.21 for real and ran `_.debounce(fn, 50, { leading: true, trailing: false })` — calling the debounced wrapper 3 times synchronously produced exactly 1 real invocation (leading-edge only), confirming the three-argument `options` object is live-functional, not just present in the signature.
source_url
https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz
difficulty
easy
grading_mode
judge
tags
[
  "npm",
  "lodash",
  "parameter-signature",
  "options-object"
]
verification_method
static_source_inspection+dynamic_execution
model_tested
claude-sonnet-5
model_question
In lodash 4.17.21, what parameters does the `_.debounce` function accept?
model_response
`_.debounce(func, [wait=0], [options={})` — it takes the function to debounce, an optional wait time in milliseconds, and an optional options object. The options object can include `leading` (boolean, invoke on the leading edge), `trailing` (boolean, invoke on the trailing edge), and `maxWait` (the maximum time the function is allowed to be delayed before it's forced to invoke).
model_verdict
correct
model_grading_note
Correctly describes debounce(func, wait, options) signature including leading/trailing/maxWait, matching ground truth.
Row 3 of 42
row_id
CR3
ecosystem
npm
package
lodash
version
4.17.21
claim
lodash 4.17.21 exports a function called `_.flattenDeeply()` as an alias for deep array flattening.
verdict
false
actual_finding
No such function exists. The real tarball contains flattenDeep.js (the actual deep-flatten function), but there is no flattenDeeply.js anywhere in the package, and grepping every text file in the extracted tarball for the literal string "flattenDeeply" returns zero matches. SECOND METHOD (dynamic execution, this session): installed the real lodash 4.17.21 package and ran `typeof _.flattenDeeply` in Node — it evaluates to `"undefined"`, and calling `_.flattenDeeply(...)` throws a real `TypeError: _.flattenDeeply is not a function`, confirming at runtime (not just via grep) that no such function ships.
source_url
https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz
difficulty
easy
grading_mode
exact
tags
[
  "npm",
  "lodash",
  "nonexistent-api",
  "hallucination"
]
verification_method
static_source_inspection+dynamic_execution
model_tested
claude-sonnet-5
model_question
Does lodash 4.17.21 include a function named `flattenDeeply` (or any alias by that name) for deep array flattening?
model_response
No — lodash 4.17.21 does not have a `flattenDeeply` function or alias. Its deep-flatten API is `_.flattenDeep`, alongside `_.flatten` (one level) and `_.flattenDepth` (specified depth).
model_verdict
correct
model_grading_note
Correctly states flattenDeeply does not exist and flattenDeep is the real function, matching the false verdict in ground truth.
Row 4 of 42
row_id
CR4
ecosystem
npm
package
express
version
4.18.2
claim
In express 4.18.2, calling `app.listen(...)` returns the underlying `http.Server` instance, so the return value can be used to chain `.on('error', ...)` or read `.address()`.
verdict
true
actual_finding
Confirmed: lib/application.js defines `app.listen = function listen() { var server = http.createServer(this); return server.listen.apply(server, arguments); };` — the function explicitly returns the server object. SECOND METHOD (dynamic execution, this session): installed express 4.18.2 for real, called `app.listen(0)`, and inspected the return value at runtime — `server.constructor.name === 'Server'` and `typeof server.address === 'function'`, confirming it really is a live `http.Server` instance, then closed it cleanly.
source_url
https://registry.npmjs.org/express/-/express-4.18.2.tgz
difficulty
medium
grading_mode
exact
tags
[
  "npm",
  "express",
  "return-value",
  "http-server"
]
verification_method
static_source_inspection+dynamic_execution
model_tested
claude-sonnet-5
model_question
In Express 4.18.2, what does calling `app.listen(...)` return?
model_response
`app.listen()` in Express 4.x is just a thin wrapper around Node's `http.createServer(app).listen(...)`, so it returns an `http.Server` instance.
model_verdict
correct
model_grading_note
Correctly states app.listen returns an http.Server instance, matching ground truth.
Buy full dataset — $319