Skip to content
LDF

Evidence at a glance

Every dot pair below is one model answering the same questions twice: once with the data as compact JSON, once as LDF v0.1. The chart is built from reports/evidence/evidence.json, the table the selector ships with, so it shows exactly what ldf-mcp --model and the Claude Code hook decide from.

50%55%60%65%70%75%80%85%90%95%100%Opus 5.5 · 200 recordsOpus 5.5 · 200 records, flat users: LDF 65/65, JSON 65/65 (+0 / -0), LDF at 0.61 of JSON's tokens; accuracy rule: passesflat users0.61×Opus 5.5 · 200 records, nested orders: LDF 94/95, JSON 94/95 (+1 / -1), LDF at 0.65 of JSON's tokens; accuracy rule: passesnested orders0.65×Opus 5.5 · 200 records, sparse events: LDF 78/80, JSON 78/80 (+2 / -2), LDF at 0.80 of JSON's tokens; accuracy rule: passessparse events0.80×Opus 5.5 · 200 records, mixed types: LDF 57/57, JSON 57/57 (+0 / -0), LDF at 0.73 of JSON's tokens; accuracy rule: passesmixed types0.73×Opus 5.5 · 1,000 recordsOpus 5.5 · 1,000 records, flat users: LDF 59/64, JSON 64/64 (+0 / -5), LDF at 0.61 of JSON's tokens; accuracy rule: failsflat users0.61×Opus 5.5 · 1,000 records, nested orders: LDF 66/68, JSON 66/68 (+2 / -2), LDF at 0.65 of JSON's tokens; accuracy rule: passesnested orders0.65×Opus 5.5 · 1,000 records, sparse events: LDF 57/60, JSON 56/60 (+3 / -2), LDF at 0.80 of JSON's tokens; accuracy rule: passessparse events0.80×Opus 5.5 · 1,000 records, mixed types: LDF 52/52, JSON 52/52 (+0 / -0), LDF at 0.73 of JSON's tokens; accuracy rule: passesmixed types0.73×Sonnet 5.5 · 200 recordsSonnet 5.5 · 200 records, flat users: LDF 64/65, JSON 62/65 (+2 / -0), LDF at 0.61 of JSON's tokens; accuracy rule: passesflat users0.61×Sonnet 5.5 · 200 records, nested orders: LDF 80/95, JSON 87/95 (+3 / -10), LDF at 0.65 of JSON's tokens; accuracy rule: failsnested orders0.65×Sonnet 5.5 · 200 records, sparse events: LDF 77/80, JSON 78/80 (+1 / -2), LDF at 0.80 of JSON's tokens; accuracy rule: passessparse events0.80×Sonnet 5.5 · 200 records, mixed types: LDF 57/57, JSON 57/57 (+0 / -0), LDF at 0.73 of JSON's tokens; accuracy rule: passesmixed types0.73×Sonnet 5.5 · 1,000 recordsSonnet 5.5 · 1,000 records, flat users: LDF 64/64, JSON 64/64 (+0 / -0), LDF at 0.61 of JSON's tokens; accuracy rule: passesflat users0.61×Sonnet 5.5 · 1,000 records, nested orders: LDF 57/68, JSON 65/68 (+0 / -8), LDF at 0.65 of JSON's tokens; accuracy rule: failsnested orders0.65×Sonnet 5.5 · 1,000 records, sparse events: LDF 54/60, JSON 59/60 (+1 / -6), LDF at 0.80 of JSON's tokens; accuracy rule: failssparse events0.80×Sonnet 5.5 · 1,000 records, mixed types: LDF 52/52, JSON 52/52 (+0 / -0), LDF at 0.73 of JSON's tokens; accuracy rule: passesmixed types0.73×Haiku 5.5 · 200 recordsHaiku 5.5 · 200 records, flat users: LDF 87/89, JSON 87/89 (+2 / -2), LDF at 0.61 of JSON's tokens; accuracy rule: passesflat users0.61×Haiku 5.5 · 200 records, nested orders: LDF 124/135, JSON 130/135 (+1 / -7), LDF at 0.65 of JSON's tokens; accuracy rule: failsnested orders0.65×Haiku 5.5 · 200 records, sparse events: LDF 108/115, JSON 106/115 (+6 / -4), LDF at 0.78 of JSON's tokens; accuracy rule: passessparse events0.78×Haiku 5.5 · 200 records, mixed types: LDF 79/82, JSON 81/82 (+0 / -2), LDF at 0.72 of JSON's tokens; accuracy rule: passesmixed types0.72×DeepSeek · 200 recordsDeepSeek · 200 records, flat users: LDF 61/89, JSON 59/89 (+7 / -5), LDF at 0.61 of JSON's tokens; accuracy rule: passesflat users0.61×DeepSeek · 200 records, nested orders: LDF 84/135, JSON 84/135 (+6 / -6), LDF at 0.65 of JSON's tokens; accuracy rule: passesnested orders0.65×DeepSeek · 200 records, sparse events: LDF 60/115, JSON 70/115 (+5 / -15), LDF at 0.78 of JSON's tokens; accuracy rule: failssparse events0.78×DeepSeek · 200 records, mixed types: LDF 82/82, JSON 82/82 (+0 / -0), LDF at 0.72 of JSON's tokens; accuracy rule: passesmixed types0.72×
Table view
model, sizedatasetJSONLDF+ / -ptokensaccuracy rule
Opus 5.5 · 200 recordsflat users65/6565/65+0 / -01.000.61passes
Opus 5.5 · 200 recordsnested orders94/9594/95+1 / -11.000.65passes
Opus 5.5 · 200 recordssparse events78/8078/80+2 / -21.000.80passes
Opus 5.5 · 200 recordsmixed types57/5757/57+0 / -01.000.73passes
Opus 5.5 · 1,000 recordsflat users64/6459/64+0 / -50.060.61fails
Opus 5.5 · 1,000 recordsnested orders66/6866/68+2 / -21.000.65passes
Opus 5.5 · 1,000 recordssparse events56/6057/60+3 / -21.000.80passes
Opus 5.5 · 1,000 recordsmixed types52/5252/52+0 / -01.000.73passes
Sonnet 5.5 · 200 recordsflat users62/6564/65+2 / -00.500.61passes
Sonnet 5.5 · 200 recordsnested orders87/9580/95+3 / -100.090.65fails
Sonnet 5.5 · 200 recordssparse events78/8077/80+1 / -21.000.80passes
Sonnet 5.5 · 200 recordsmixed types57/5757/57+0 / -01.000.73passes
Sonnet 5.5 · 1,000 recordsflat users64/6464/64+0 / -01.000.61passes
Sonnet 5.5 · 1,000 recordsnested orders65/6857/68+0 / -80.010.65fails
Sonnet 5.5 · 1,000 recordssparse events59/6054/60+1 / -60.130.80fails
Sonnet 5.5 · 1,000 recordsmixed types52/5252/52+0 / -01.000.73passes
Haiku 5.5 · 200 recordsflat users87/8987/89+2 / -21.000.61passes
Haiku 5.5 · 200 recordsnested orders130/135124/135+1 / -70.070.65fails
Haiku 5.5 · 200 recordssparse events106/115108/115+6 / -40.750.78passes
Haiku 5.5 · 200 recordsmixed types81/8279/82+0 / -20.500.72passes
DeepSeek · 200 recordsflat users59/8961/89+7 / -50.770.61passes
DeepSeek · 200 recordsnested orders84/13584/135+6 / -61.000.65passes
DeepSeek · 200 recordssparse events70/11560/115+5 / -150.040.78fails
DeepSeek · 200 recordsmixed types82/8282/82+0 / -01.000.72passes

How to read it:

  • Close dots mean the format makes no difference to that model on that data, and LDF saves the tokens on the right.
  • An orange dot well left of the blue one is a loss. The selector’s rule is at least 50 questions, at most 3 points below JSON and not significantly worse (sign test); a model that fails it on some shape of data keeps getting JSON there.
  • Held-out rows use records and questions that were not seen when the layouts were chosen. Stage B rows (Haiku, DeepSeek) use the same 200 records as the design work.

Grading is strict: an answer with the right value but without its JSON quotes, or wrapped in an object, counts as wrong. Most of Sonnet’s losses on nested orders are ids answered without quotes; the reports say so cell by cell: