← All datasets
Sample

Language & Runtime Semantics Verification

4 real rows pulled directly from the full dataset, unedited — the actual field structure and content a buyer receives. The full purchase includes all 40 rows.

Row 1 of 40
row_id
CX01
claim
Python's sorted() and list.sort() are stable — elements that compare equal on the sort key retain their original relative order.
claim_source_url
https://docs.python.org/3/howto/sorting.html#sort-stability-and-complexity
verification_method
python3
verified_value
[(0, 'b'), (0, 'd'), (1, 'a'), (1, 'c')]
verification_source_url
executed directly
agreement
match
explanation
Ran, right now (Python 3.12.3): sorted([(1,'a'),(0,'b'),(1,'c'),(0,'d')], key=lambda x: x[0]). Real output: [(0, 'b'), (0, 'd'), (1, 'a'), (1, 'c')]. The two key=0 elements ('b' then 'd') and the two key=1 elements ('a' then 'c') both keep their original input order after sorting purely by the first tuple element — direct empirical confirmation of the stability guarantee CPython's docs state (Timsort, adaptive stable merge sort, guaranteed since Python 2.3).
model_tested
claude-sonnet-5
model_question
When Python's sorted() or list.sort() is applied using a key that produces ties, what happens to the relative order of the tied elements compared to their order in the original input?
model_response
Python's sort is stable, so tied elements (equal keys) retain their original relative order from the input — this is guaranteed by the language, not an implementation detail.
model_verdict
correct
model_grading_note
Matches the verified stability guarantee exactly - tied elements retain original relative order.
grading_mode
exact
difficulty
medium
tags
[
  "python",
  "sorting",
  "stability",
  "timsort"
]
Row 2 of 40
row_id
CX02
claim
JavaScript's Array.prototype.sort() is guaranteed stable by the ECMAScript spec (as of ES2019) — modern engines including V8/Node actually implement this, not just claim to.
claim_source_url
https://tc39.es/ecma262/#sec-array.prototype.sort
verification_method
node
verified_value
[[0,"b"],[0,"d"],[1,"a"],[1,"c"]]
verification_source_url
executed directly
agreement
match
explanation
Ran, right now (Node v24.18.0, V8's TimSort-based implementation): the JS analog of the Python test above — [[1,'a'],[0,'b'],[1,'c'],[0,'d']].sort((a,b)=>a[0]-b[0]). Real output: [[0,"b"],[0,"d"],[1,"a"],[1,"c"]]. Same result pattern as CX01: elements with equal sort keys keep their original relative order. Worth noting explicitly: sort stability was only a MUST in the spec starting with ES2019 (TC39); earlier V8 versions (pre-V8 7.0 / pre-Node 11) used an unstable sort for arrays longer than 10 elements, so this specific guarantee is itself version-dependent — see CX02's companion changelog rows for exact version claims.
model_tested
claude-sonnet-5
model_question
According to the ECMAScript specification, is JavaScript's Array.prototype.sort() required to be a stable sort, and do modern engines like V8/Node actually implement it that way?
model_response
Yes — the ECMAScript spec has required Array.prototype.sort() to be stable since ES2019. Before that it was implementation-defined, but all major modern engines (V8/Node, SpiderMonkey, JavaScriptCore) now implement it as a stable sort in practice (V8 has used TimSort for this since around Node 11/V8 7.0).
model_verdict
correct
model_grading_note
Matches record precisely: stable as of ES2019, implemented via TimSort in V8 since ~Node 11/V8 7.0.
grading_mode
exact
difficulty
medium
tags
[
  "javascript",
  "sorting",
  "stability",
  "ecmascript"
]
Row 3 of 40
row_id
CX03
claim
JavaScript Array.prototype.sort() called with no comparator sorts elements as strings by default, so [10, 1, 2].sort() produces [1, 10, 2] rather than numeric order.
claim_source_url
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
verification_method
node
verified_value
[1,10,2]
verification_source_url
executed directly
agreement
match
explanation
Ran, right now (Node v24.18.0): [10, 1, 2].sort() with no comparator argument. Real output: [1,10,2] — confirms the claim exactly: 10 sorts before 2 because the default comparator converts each element to a string ('10' < '2' lexicographically, since '1' < '2') rather than comparing numerically. This is a genuinely common real-world bug source (MDN's own docs warn about it explicitly) and a frequent silent miscoding an LLM can produce if it assumes .sort() is numeric by default the way it usually is in other languages.
model_tested
claude-sonnet-5
model_question
When JavaScript's Array.prototype.sort() is called with no comparator function, how does it order the elements of an array, and what would calling it on [10, 1, 2] produce?
model_response
Array.prototype.sort() with no comparator converts elements to strings and sorts them by UTF-16 code unit values (lexicographic/dictionary order), not numeric order. So [10, 1, 2] becomes ["10", "1", "2"] as strings, sorted lexicographically to ["1", "10", "2"], giving the result [1, 10, 2].
model_verdict
correct
model_grading_note
Matches verified output [1,10,2] and correctly explains string/lexicographic default sort behavior.
grading_mode
exact
difficulty
easy
tags
[
  "javascript",
  "sorting",
  "gotcha"
]
Row 4 of 40
row_id
CX04
claim
Python's // floor-division operator rounds toward negative infinity, so -7 // 2 equals -4 (not -3, which is what truncation toward zero would give).
claim_source_url
https://docs.python.org/3/reference/expressions.html#binary-arithmetic-operations
verification_method
python3
verified_value
-4 3 3
verification_source_url
executed directly
agreement
match
explanation
Ran, right now (Python 3.12.3): print(-7 // 2, 7 // 2, -7 // -2). Real output: "-4 3 3" (space-separated: -7//2, 7//2, -7//-2). -7 // 2 = -4, confirming floor (round-toward-negative-infinity) semantics — -3.5 floors to -4, not truncates to -3. Positive-operand case (7//2=3) and same-sign-negative case (-7//-2=3) shown alongside for contrast, since floor and truncation only diverge when exactly one operand is negative and the division isn't exact.
model_tested
claude-sonnet-5
model_question
In Python, which direction does the // floor-division operator round toward for negative results, and what is the result of -7 // 2?
model_response
Python's `//` floor-division rounds toward negative infinity (not toward zero), so it always rounds down. Thus `-7 // 2` equals `-4` (since -3.5 rounds down to -4).
model_verdict
correct
model_grading_note
Matches verified -7 // 2 = -4 and correctly states floor/round-toward-negative-infinity semantics.
grading_mode
numeric
difficulty
easy
tags
[
  "python",
  "division",
  "arithmetic"
]
Buy full dataset — $309