End-to-end Leaderboard (VLA / WAM)

InternW0

WAMShanghai AI Lab

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall8.68 / 8.67%9.17 / 9.04%8.18 / 8.29%
Mind6.12 / 5.37%6.04 / 5.20%6.20 / 5.53%
Act20.33 / 17.60%20.90 / 18.20%19.75 / 17.00%
Mind2Act4.13 / 3.03%4.88 / 3.73%3.38 / 2.33%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory1.30 / 0.17%1.05 / 0.00%1.55 / 0.33%
E1Duplicate Item Identification0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
E2Object-Code Keypad Entry2.60 / 0.33%2.10 / 0.00%3.10 / 0.67%
PProcess memory1.98 / 1.00%2.95 / 1.67%1.00 / 0.33%
P1Order-Based Meal Assembly0.85 / 0.67%0.78 / 0.67%0.92 / 0.67%
P2Steamer Stack Order Reconstruction3.11 / 1.33%5.13 / 2.67%1.08 / 0.00%
SSpatial memory5.70 / 5.33%6.64 / 6.33%4.76 / 4.33%
S1Rotating Tray Layout Reconstruction1.41 / 0.67%1.96 / 1.33%0.86 / 0.00%
S2Hidden-Zone Navigation10.00 / 10.00%11.33 / 11.33%8.67 / 8.67%
FFeedback-rule learning11.67 / 11.50%8.96 / 8.67%14.38 / 14.33%
F1Unknown Button Mapping and Slider Navigation9.68 / 9.33%7.93 / 7.33%11.43 / 11.33%
F2Color-Block Trial Ordering13.67 / 13.67%10.00 / 10.00%17.33 / 17.33%
MIntegrated memory9.95 / 8.83%10.59 / 9.33%9.30 / 8.33%
M1Workpiece Tracking and Route Reproduction0.29 / 0.00%0.37 / 0.00%0.20 / 0.00%
M2Memory Card Matching19.60 / 17.67%20.81 / 18.67%18.40 / 16.67%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
E2Object-Code Keypad EntryEasy3.40 / 1.00%2.10 / 0.00%4.70 / 2.00%
Medium2.57 / 0.00%2.49 / 0.00%2.64 / 0.00%
Hard1.82 / 0.00%1.70 / 0.00%1.95 / 0.00%
P1Order-Based Meal AssemblyEasy0.30 / 0.00%0.00 / 0.00%0.60 / 0.00%
Medium1.05 / 1.00%2.11 / 2.00%0.00 / 0.00%
Hard1.19 / 1.00%0.23 / 0.00%2.15 / 2.00%
P2Steamer Stack Order ReconstructionEasy5.40 / 3.00%9.20 / 6.00%1.60 / 0.00%
Medium3.17 / 1.00%5.29 / 2.00%1.05 / 0.00%
Hard0.75 / 0.00%0.90 / 0.00%0.60 / 0.00%
S1Rotating Tray Layout ReconstructionEasy3.05 / 2.00%4.90 / 4.00%1.20 / 0.00%
Medium0.60 / 0.00%0.44 / 0.00%0.77 / 0.00%
Hard0.56 / 0.00%0.53 / 0.00%0.60 / 0.00%
S2Hidden-Zone NavigationEasy26.00 / 26.00%28.00 / 28.00%24.00 / 24.00%
Medium3.00 / 3.00%4.00 / 4.00%2.00 / 2.00%
Hard1.00 / 1.00%2.00 / 2.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy24.75 / 24.00%17.20 / 16.00%32.30 / 32.00%
Medium3.22 / 3.00%4.44 / 4.00%2.00 / 2.00%
Hard1.07 / 1.00%2.15 / 2.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy31.00 / 31.00%22.00 / 22.00%40.00 / 40.00%
Medium8.00 / 8.00%4.00 / 4.00%12.00 / 12.00%
Hard2.00 / 2.00%4.00 / 4.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.75 / 0.00%0.90 / 0.00%0.60 / 0.00%
Medium0.11 / 0.00%0.22 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy49.50 / 48.00%50.40 / 48.00%48.60 / 48.00%
Medium8.19 / 5.00%11.08 / 8.00%5.30 / 2.00%
Hard1.12 / 0.00%0.94 / 0.00%1.31 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response36.69 / 31.83%36.93 / 32.33%36.44 / 31.33%
DR1Continuous Conveyor Picking29.51 / 22.67%24.50 / 16.67%34.52 / 28.67%
DR2Light-Guided Button Response44.58 / 32.00%53.91 / 43.33%35.26 / 20.67%
DR3Periodic Gate Crossing and Withdrawal72.67 / 72.67%69.33 / 69.33%76.00 / 76.00%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution9.42 / 8.11%10.22 / 8.78%8.63 / 7.44%
PR1Linear Resistor Adjustment45.70 / 39.67%49.00 / 42.67%42.39 / 36.67%
PR2Rotary Voltage Regulator Setting4.19 / 2.67%3.16 / 1.33%5.22 / 4.00%
PR3Marker Line Tracing0.20 / 0.00%0.40 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and Placement6.33 / 6.33%8.67 / 8.67%4.00 / 4.00%
PR5Oriented Stamp Placement0.11 / 0.00%0.07 / 0.00%0.15 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy42.80 / 33.00%37.40 / 26.00%48.20 / 40.00%
Medium41.82 / 35.00%32.27 / 24.00%51.37 / 46.00%
Hard3.90 / 0.00%3.81 / 0.00%3.99 / 0.00%
DR2Light-Guided Button ResponseEasy51.16 / 34.00%57.06 / 42.00%45.26 / 26.00%
Medium48.40 / 37.00%60.71 / 52.00%36.08 / 22.00%
Hard34.20 / 25.00%43.95 / 36.00%24.44 / 14.00%
DR3Periodic Gate Crossing and WithdrawalEasy74.00 / 74.00%68.00 / 68.00%80.00 / 80.00%
Medium74.00 / 74.00%68.00 / 68.00%80.00 / 80.00%
Hard70.00 / 70.00%72.00 / 72.00%68.00 / 68.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy63.92 / 55.00%72.07 / 64.00%55.76 / 46.00%
Medium41.77 / 35.00%38.23 / 30.00%45.31 / 40.00%
Hard31.40 / 29.00%36.70 / 34.00%26.11 / 24.00%
PR2Rotary Voltage Regulator SettingEasy5.68 / 2.00%4.35 / 0.00%7.02 / 4.00%
Medium3.78 / 3.00%2.94 / 2.00%4.63 / 4.00%
Hard3.11 / 3.00%2.20 / 2.00%4.02 / 4.00%
PR3Marker Line TracingEasy0.60 / 0.00%1.20 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy8.00 / 8.00%10.00 / 10.00%6.00 / 6.00%
Medium11.00 / 11.00%16.00 / 16.00%6.00 / 6.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR5Oriented Stamp PlacementEasy0.15 / 0.00%0.00 / 0.00%0.30 / 0.00%
Medium0.11 / 0.00%0.22 / 0.00%0.00 / 0.00%
Hard0.07 / 0.00%0.00 / 0.00%0.15 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination4.13 / 3.03%4.88 / 3.73%3.38 / 2.33%
CP01Conveyor Meal Assembly3.10 / 1.67%4.32 / 2.67%1.88 / 0.67%
CP02Rotating Cake Decoration0.42 / 0.33%0.09 / 0.00%0.74 / 0.67%
CP03Dynamic Rack Loading0.34 / 0.00%0.37 / 0.00%0.31 / 0.00%
CP04Dual-Arm Cooking0.05 / 0.00%0.00 / 0.00%0.10 / 0.00%
CP05Keyboard Sequence Reproduction0.76 / 0.00%0.79 / 0.00%0.72 / 0.00%
CP06Produce Weighing and Balancing21.14 / 16.00%20.52 / 15.33%21.77 / 16.67%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction13.38 / 11.67%20.77 / 18.67%5.98 / 4.67%
CP09Whiteboard Drawing Reconstruction0.03 / 0.00%0.02 / 0.00%0.05 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance2.06 / 0.67%1.89 / 0.67%2.23 / 0.67%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy5.45 / 3.00%8.90 / 6.00%2.00 / 0.00%
Medium2.97 / 2.00%3.13 / 2.00%2.80 / 2.00%
Hard0.89 / 0.00%0.93 / 0.00%0.84 / 0.00%
CP02Rotating Cake DecorationEasy1.23 / 1.00%0.23 / 0.00%2.23 / 2.00%
Medium0.02 / 0.00%0.03 / 0.00%0.00 / 0.00%
Hard0.01 / 0.00%0.02 / 0.00%0.00 / 0.00%
CP03Dynamic Rack LoadingEasy0.95 / 0.00%1.00 / 0.00%0.90 / 0.00%
Medium0.04 / 0.00%0.08 / 0.00%0.00 / 0.00%
Hard0.03 / 0.00%0.02 / 0.00%0.04 / 0.00%
CP04Dual-Arm CookingEasy0.15 / 0.00%0.00 / 0.00%0.30 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence ReproductionEasy1.20 / 0.00%0.86 / 0.00%1.54 / 0.00%
Medium0.75 / 0.00%1.07 / 0.00%0.42 / 0.00%
Hard0.32 / 0.00%0.43 / 0.00%0.20 / 0.00%
CP06Produce Weighing and BalancingEasy51.40 / 47.00%50.10 / 46.00%52.70 / 48.00%
Medium6.70 / 1.00%6.00 / 0.00%7.40 / 2.00%
Hard5.33 / 0.00%5.46 / 0.00%5.21 / 0.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy28.85 / 27.00%44.00 / 42.00%13.70 / 12.00%
Medium6.72 / 5.00%12.00 / 10.00%1.45 / 0.00%
Hard4.55 / 3.00%6.32 / 4.00%2.78 / 2.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.09 / 0.00%0.05 / 0.00%0.14 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy4.90 / 2.00%4.20 / 2.00%5.60 / 2.00%
Medium0.97 / 0.00%1.25 / 0.00%0.70 / 0.00%
Hard0.32 / 0.00%0.23 / 0.00%0.40 / 0.00%

Efficiency

Avg. task time
19.2 min
Avg. inference time
488.1 ms
Inference hardware
RTX PRO 5000 · 72GB
Mind · Act · Mind2Act
TasksAvg. task time
Mind17.6 min
Act7.4 min
Mind2Act32.5 min

Diagnosis

Where InternW0’s failed episodes first break, why, and what could help. All 8,220 failed episodes are attributed by rules, an LLM judge and human review.

End-to-end board

InternW0 WAM

Shanghai AI Lab

8.7%success over 9,000 episodes
Act holds up even at Hard; Mind and Mind2Act fade past Easy.
ENFP · Goalkeeper
01

Where it breaks

The first step a failed episode could not recover from, and which side failed.

Where failures break

first unrecoverable step

  • perceive5.5%
  • reason27.3%
  • execute37.2%
  • close the loop10.5%
  • episode level19.4%
Which side failed

share of failures

  • 32.8%Mind
  • 37.2%Act
  • 10.5%composition
  • 19.4%other
02

Why it fails

The most frequent failure codes and the tasks where episodes stop.

First failures

most frequent failure codes

  1. A-PRECISIONfine-control error1,132
  2. M-MEMORYforgot or misremembered1,011
  3. A-GRASPgrasp failed or not held806
  4. C-PREREQacted before a precondition held761
Where tasks break

episodes stopping there, three typical tasks

  1. 271/300CP04 heat too early
  2. 288/299E2 misapply the keypad rule
  3. 245/298P1 forget the order
03

What it can do

Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.

Capability profile

30 tasks, 12 capabilities

MindIntegrated memoryIntegratedM1 · Integrated memory · Easy: 0% successM1 · Integrated memory · Medium: 0% successM1 · Integrated memory · Hard: 0% successM1M2 · Integrated memory · Easy: 30–59% successM2 · Integrated memory · Medium: 1–9% successM2 · Integrated memory · Hard: 0% successM2Spatial memorySpatialS1 · Spatial memory · Easy: 1–9% successS1 · Spatial memory · Medium: 0% successS1 · Spatial memory · Hard: 0% successS1S2 · Spatial memory · Easy: 10–29% successS2 · Spatial memory · Medium: 1–9% successS2 · Spatial memory · Hard: 1–9% successS2Process memoryProcessP1 · Process memory · Easy: 0% successP1 · Process memory · Medium: 1–9% successP1 · Process memory · Hard: 1–9% successP1P2 · Process memory · Easy: 1–9% successP2 · Process memory · Medium: 1–9% successP2 · Process memory · Hard: 0% successP2Entity memoryEntityE1 · Entity memory · Easy: 0% successE1 · Entity memory · Medium: 0% successE1 · Entity memory · Hard: 0% successE1E2 · Entity memory · Easy: 1–9% successE2 · Entity memory · Medium: 0% successE2 · Entity memory · Hard: 0% successE2Feedback-rule learningRuleF1 · Feedback-rule learning · Easy: 10–29% successF1 · Feedback-rule learning · Medium: 1–9% successF1 · Feedback-rule learning · Hard: 1–9% successF1F2 · Feedback-rule learning · Easy: 30–59% successF2 · Feedback-rule learning · Medium: 1–9% successF2 · Feedback-rule learning · Hard: 1–9% successF2ActDynamic responseDynamicDR1 · Dynamic response · Easy: 30–59% successDR1 · Dynamic response · Medium: 30–59% successDR1 · Dynamic response · Hard: 0% successDR1DR2 · Dynamic response · Easy: 30–59% successDR2 · Dynamic response · Medium: 30–59% successDR2 · Dynamic response · Hard: 10–29% successDR2DR3 · Dynamic response · Easy: 60% or more successDR3 · Dynamic response · Medium: 60% or more successDR3 · Dynamic response · Hard: 60% or more successDR3DR4 · Dynamic response · Easy: 0% successDR4 · Dynamic response · Medium: 0% successDR4 · Dynamic response · Hard: 0% successDR4Precise executionPrecisionPR1 · Precise execution · Easy: 30–59% successPR1 · Precise execution · Medium: 30–59% successPR1 · Precise execution · Hard: 10–29% successPR1PR2 · Precise execution · Easy: 1–9% successPR2 · Precise execution · Medium: 1–9% successPR2 · Precise execution · Hard: 1–9% successPR2PR3 · Precise execution · Easy: 0% successPR3 · Precise execution · Medium: 0% successPR3 · Precise execution · Hard: 0% successPR3PR4 · Precise execution · Easy: 1–9% successPR4 · Precise execution · Medium: 10–29% successPR4 · Precise execution · Hard: 0% successPR4PR5 · Precise execution · Easy: 0% successPR5 · Precise execution · Medium: 0% successPR5 · Precise execution · Hard: 0% successPR5PR6 · Precise execution · Easy: 0% successPR6 · Precise execution · Medium: 0% successPR6 · Precise execution · Hard: 0% successPR6Mind2ActPredictive decisionPredictCP10 · Predictive decision · Easy: 1–9% successCP10 · Predictive decision · Medium: 0% successCP10 · Predictive decision · Hard: 0% successCP10Sequence and planPlanCP05 · Sequence and plan · Easy: 0% successCP05 · Sequence and plan · Medium: 0% successCP05 · Sequence and plan · Hard: 0% successCP05CP07 · Sequence and plan · Easy: 0% successCP07 · Sequence and plan · Medium: 0% successCP07 · Sequence and plan · Hard: 0% successCP07Feedback adjustmentFeedbackCP04 · Feedback adjustment · Easy: 0% successCP04 · Feedback adjustment · Medium: 0% successCP04 · Feedback adjustment · Hard: 0% successCP04CP06 · Feedback adjustment · Easy: 30–59% successCP06 · Feedback adjustment · Medium: 1–9% successCP06 · Feedback adjustment · Hard: 0% successCP06Prerequisites and repairRepairCP02 · Prerequisites and repair · Easy: 1–9% successCP02 · Prerequisites and repair · Medium: 0% successCP02 · Prerequisites and repair · Hard: 0% successCP02CP08 · Prerequisites and repair · Easy: 10–29% successCP08 · Prerequisites and repair · Medium: 1–9% successCP08 · Prerequisites and repair · Hard: 1–9% successCP08CP09 · Prerequisites and repair · Easy: 0% successCP09 · Prerequisites and repair · Medium: 0% successCP09 · Prerequisites and repair · Hard: 0% successCP09Cumulative progressCumulativeCP01 · Cumulative progress · Easy: 1–9% successCP01 · Cumulative progress · Medium: 1–9% successCP01 · Cumulative progress · Hard: 0% successCP01CP03 · Cumulative progress · Easy: 0% successCP03 · Cumulative progress · Medium: 0% successCP03 · Cumulative progress · Hard: 0% successCP03

RingsEasy, Medium and Hard, outward

Success01+10+30+60+
Mind
Act
Mind2Act

Act holds up even at Hard; Mind and M2A fade past Easy

Under pressure

success (%) by difficulty, bars 0–30%

Mind

  • Easy13.5
  • Medium2.1
  • Hard0.5

Act

  • Easy20.6
  • Medium19.5
  • Hard12.7

Mind2Act

  • Easy8.0
  • Medium0.8
  • Hard0.3

All tasks

  • 14.0%Easy
  • 7.5%Medium
  • 4.5%Hard
04

What could help

Directions drawn from the failure codes and breakpoints above.

  1. 1Precise contact

    Fine-control errors (1,132) and grasp failures (806) make up most of its Act failures.

    Change
    shorter action chunks and closed-loop correction near contact, where its grasps and fine alignment fail.
    Data to add
    Demonstrations of grasping and fine alignment, with the corrective motions near the target.
  2. 2Memory and preconditions

    1,011 memory and 761 precondition failures: 271 of 300 cooking episodes (CP04) heat before all ingredients are in the pot, and 245 of 298 failed meal-order episodes (P1) forget the order.

    Change
    a task-state memory and a precondition check: all ingredients in the pot before HEAT in CP04, and the order checked before hand-over in P1.
    Data to add
    Demonstrations that check the pot before heating, and long episodes that carry orders and rules across steps.