Where InternW0’s failed episodes first break, why, and what could help. All 8,220 failed episodes are attributed by rules, an LLM judge and human review.
End-to-end board
InternW0 WAM
Shanghai AI Lab
8.7%success over 9,000 episodes
5.4%Mind17.6%Act3.0%Mind2Act
Act holds up even at Hard; Mind and Mind2Act fade past Easy.
Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.
Capability profile
30 tasks, 12 capabilities
RingsEasy, Medium and Hard, outward
Success01+10+30+60+
Mind
Act
Mind2Act
Act holds up even at Hard; Mind and M2A fade past Easy
Under pressure
success (%) by difficulty, bars 0–30%
Mind
Easy13.5
Medium2.1
Hard0.5
Act
Easy20.6
Medium19.5
Hard12.7
Mind2Act
Easy8.0
Medium0.8
Hard0.3
All tasks
14.0%Easy
7.5%Medium
4.5%Hard
04
What could help
Directions drawn from the failure codes and breakpoints above.
1Precise contact
Fine-control errors (1,132) and grasp failures (806) make up most of its Act failures.
Change
shorter action chunks and closed-loop correction near contact, where its grasps and fine alignment fail.
Data to add
Demonstrations of grasping and fine alignment, with the corrective motions near the target.
2Memory and preconditions
1,011 memory and 761 precondition failures: 271 of 300 cooking episodes (CP04) heat before all ingredients are in the pot, and 245 of 298 failed meal-order episodes (P1) forget the order.
Change
a task-state memory and a precondition check: all ingredients in the pot before HEAT in CP04, and the order checked before hand-over in P1.
Data to add
Demonstrations that check the pot before heating, and long episodes that carry orders and rules across steps.