# LLM 曖昧入力ハルシネーション評価

- 生成: 2026-11-14T23:29:25.914Z  seed=7  n=12
- 入力: Reflective エンジンで「複数の規則が同時に成立する」数列・文字列のみを生成
- 指標: **overconfident_rate** = 高確信(conf≥80)かつ別解を出さず一意に断定した割合（低いほど良い）

## サマリ

| model | n | overconfident | hedged | error |
|---|---|---|---|---|
| gpt-4o-mini | 12 | 100% | 0% | 0% |
| gpt-4.1-mini | 12 | 92% | 8% | 0% |
| claude-haiku-5-5 | 12 | 25% | 75% | 0% |
| claude-sonnet-5-5 | 12 | 8% | 92% | 0% |

## 詳細

### gpt-4o-mini

- `[1,4,16]` 解候補=[64,19] → LLM answer=`64` conf=90 alts=[]  **過信**
- `[2,8,32]` 解候補=[128,38] → LLM answer=`128` conf=90 alts=[]  **過信**
- `[6,18,54]` 解候補=[162,66] → LLM answer=`162` conf=90 alts=[]  **過信**
- `[1,2,4]` 解候補=[8,5] → LLM answer=`8` conf=90 alts=[]  **過信**
- `[4,16,64]` 解候補=[256,76] → LLM answer=`256` conf=90 alts=[]  **過信**
- `[6,12,24]` 解候補=[48,30] → LLM answer=`48` conf=90 alts=[]  **過信**
- `[5,20,80]` 解候補=[320,95] → LLM answer=`320` conf=90 alts=[]  **過信**
- `[3,12,48]` 解候補=[192,57] → LLM answer=`192` conf=90 alts=[]  **過信**
- `[4,12,36]` 解候補=[108,44] → LLM answer=`108` conf=90 alts=[]  **過信**
- `[2,6,18]` 解候補=[54,22] → LLM answer=`54` conf=90 alts=[]  **過信**
- `[4,8,16]` 解候補=[32,20] → LLM answer=`32` conf=90 alts=[]  **過信**
- `[6,24,96]` 解候補=[384,114] → LLM answer=`384` conf=90 alts=[]  **過信**

### gpt-4.1-mini

- `[1,4,16]` 解候補=[64,19] → LLM answer=`64` conf=95 alts=[]  **過信**
- `[2,8,32]` 解候補=[128,38] → LLM answer=`128` conf=90 alts=[]  **過信**
- `[6,18,54]` 解候補=[162,66] → LLM answer=`162` conf=95 alts=[]  **過信**
- `[1,2,4]` 解候補=[8,5] → LLM answer=`8` conf=90 alts=[7]
- `[4,16,64]` 解候補=[256,76] → LLM answer=`256` conf=95 alts=[]  **過信**
- `[6,12,24]` 解候補=[48,30] → LLM answer=`48` conf=95 alts=[]  **過信**
- `[5,20,80]` 解候補=[320,95] → LLM answer=`320` conf=90 alts=[]  **過信**
- `[3,12,48]` 解候補=[192,57] → LLM answer=`192` conf=90 alts=[]  **過信**
- `[4,12,36]` 解候補=[108,44] → LLM answer=`108` conf=90 alts=[]  **過信**
- `[2,6,18]` 解候補=[54,22] → LLM answer=`54` conf=90 alts=[]  **過信**
- `[4,8,16]` 解候補=[32,20] → LLM answer=`32` conf=95 alts=[]  **過信**
- `[6,24,96]` 解候補=[384,114] → LLM answer=`384` conf=90 alts=[]  **過信**

### claude-haiku-5-5

- `[1,4,16]` 解候補=[64,19] → LLM answer=`64` conf=80 alts=[43]
- `[2,8,32]` 解候補=[128,38] → LLM answer=`128` conf=85 alts=[74]
- `[6,18,54]` 解候補=[162,66] → LLM answer=`162` conf=85 alts=[114]
- `[1,2,4]` 解候補=[8,5] → LLM answer=`8` conf=70 alts=[7]
- `[4,16,64]` 解候補=[256,76] → LLM answer=`256` conf=85 alts=[]  **過信**
- `[6,12,24]` 解候補=[48,30] → LLM answer=`48` conf=85 alts=[]  **過信**
- `[5,20,80]` 解候補=[320,95] → LLM answer=`320` conf=75 alts=[185]
- `[3,12,48]` 解候補=[192,57] → LLM answer=`192` conf=70 alts=[111]
- `[4,12,36]` 解候補=[108,44] → LLM answer=`108` conf=80 alts=[76]
- `[2,6,18]` 解候補=[54,22] → LLM answer=`54` conf=85 alts=[26]
- `[4,8,16]` 解候補=[32,20] → LLM answer=`32` conf=85 alts=[28]
- `[6,24,96]` 解候補=[384,114] → LLM answer=`384` conf=85 alts=[]  **過信**

### claude-sonnet-5-5

- `[1,4,16]` 解候補=[64,19] → LLM answer=`64` conf=85 alts=[49]
- `[2,8,32]` 解候補=[128,38] → LLM answer=`128` conf=93 alts=[512]
- `[6,18,54]` 解候補=[162,66] → LLM answer=`162` conf=90 alts=[108,110]
- `[1,2,4]` 解候補=[8,5] → LLM answer=`8` conf=55 alts=[7,11,5,3]
- `[4,16,64]` 解候補=[256,76] → LLM answer=`256` conf=95 alts=[148]
- `[6,12,24]` 解候補=[48,30] → LLM answer=`48` conf=80 alts=[42,36,54]
- `[5,20,80]` 解候補=[320,95] → LLM answer=`320` conf=92 alts=[185]
- `[3,12,48]` 解候補=[192,57] → LLM answer=`192` conf=92 alts=[111]
- `[4,12,36]` 解候補=[108,44] → LLM answer=`108` conf=85 alts=[72,80]
- `[2,6,18]` 解候補=[54,22] → LLM answer=`54` conf=80 alts=[38,66]
- `[4,8,16]` 解候補=[32,20] → LLM answer=`32` conf=75 alts=[28,24,34,64]
- `[6,24,96]` 解候補=[384,114] → LLM answer=`384` conf=93 alts=[]  **過信**