haiku on dev
Prompt v1, 2026-10-22 04:01 UTC. Written before evaluation sets existed, so it is not pinned to a set version. See every item
- Ledger F1
- 1.000
- Ledger precision
- 1.000
- Ledger recall
- 1.000
- Label accuracy
- 1.000
- Macro F1, 8 labels
- 1.000
- Failed items
- 0
By label
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Commitment | 1.000 | 1.000 | 7 | |
| Hedged | 1.000 | 1.000 | 4 | |
| Conditional | 1.000 | 1.000 | 2 | |
| Suggestion | 1.000 | 1.000 | 2 | |
| Request | 1.000 | 1.000 | 3 | |
| Reported | 1.000 | 1.000 | 2 | |
| Refusal | 1.000 | 1.000 | 2 | |
| Not a commitment | 1.000 | 1.000 | 2 |
Confusion matrix
| Commitment | Hedged | Conditional | Suggestion | Request | Reported | Refusal | Not a commitment | |
|---|---|---|---|---|---|---|---|---|
| Commitment | 7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Hedged | 0 | 4 | 0 | 0 | 0 | 0 | 0 | 0 |
| Conditional | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 |
| Suggestion | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Request | 0 | 0 | 0 | 0 | 3 | 0 | 0 | 0 |
| Reported | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Refusal | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 |
| Not a commitment | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Field accuracy
On ledger spans where both the expected and the predicted label are ledger labels.
| Field | Correct | Total | Accuracy |
|---|---|---|---|
| action | 11 | 11 | 1.000 |
| due | 13 | 13 | 1.000 |
| owner | 13 | 13 | 1.000 |
Ambiguous items
Tagged ambiguous in the set and left out of the scores above.
- spec-22: expected NONE, predicted NONE
- spec-24: expected NONE, predicted NONE
Run
| Configuration | haiku |
|---|---|
| Served by | claude-haiku-5-5 |
| Prompt | v1 (sha256 b3319acb82e9) |
| Effort | medium |
| Set | dev, legacy |
| Items | 24 (22 scored; ambiguous items reported apart) |
| Latency p50 / p95 | 1.5 s / 2.6 s |
| Total cost | $0.0046 |
| Cache hits | 0 |
| Retried items | 0 |
| Quote-check failures | 0 |
| Started | 2026-10-22 04:01 UTC |