Where GPT-6 Astra’s failed episodes first break, why, and what could help. All 589 failed episodes are attributed by rules, an LLM judge and human review.
Agent board
GPT-6 Astra Agent
OpenAI
34.6%success over 900 episodes
52.7%Mind39.0%Act12.0%Mind2Act
Mind and the Easy rings fill up; DR4 and PR6 stay empty.
Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.
Capability profile
30 tasks, 12 capabilities
RingsEasy, Medium and Hard, outward
Success01+10+30+60+
Mind
Act
Mind2Act
Mind and Easy rings fill up; DR4 and PR6 stay empty
Under pressure
success (%) by difficulty, bars 0–70%
Mind
Easy64.0
Medium50.0
Hard44.0
Act
Easy44.0
Medium39.0
Hard34.0
Mind2Act
Easy26.0
Medium7.0
Hard3.0
All tasks
44.7%Easy
32.0%Medium
27.0%Hard
04
What could help
Directions drawn from the failure codes and breakpoints above.
1Finer grasp and pose tools
Act accounts for 55% of its failures: grasp (120), fine control (107) and placement (47); all 30 lift episodes of DR4 lose control.
Tools
grasp-quality feedback and small relative pose moves, so a grasp can be checked and corrected.
Library to add
failed grasps and lifts with the observation that showed the slip, retrieved as examples.
2Verify and retry before stopping
Typical tasks stop at one physical step: 21 of 30 cooking episodes (CP04) fail to grasp a jar and 21 of 30 domino episodes (PR6) knock pieces while placing.
Tools
an observation after the tray lift (DR4), the jar grasp (CP04) and each domino placement (PR6), and a retry before the episode is ended.
Library to add
recovered attempts (slip, check, retry) from earlier episodes of the same task.