End-to-end Leaderboard (VLA / WAM)

FastWAM

WAMTsinghua University

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall3.07 / 2.94%4.48 / 4.40%1.65 / 1.49%
Mind1.21 / 0.90%1.98 / 1.53%0.44 / 0.27%
Act9.34 / 7.40%13.30 / 10.73%5.39 / 4.07%
Mind2Act0.86 / 0.53%1.32 / 0.93%0.40 / 0.13%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory1.81 / 0.67%1.85 / 0.33%1.76 / 1.00%
E1Duplicate Item Identification2.75 / 1.00%2.28 / 0.00%3.21 / 2.00%
E2Object-Code Keypad Entry0.87 / 0.33%1.43 / 0.67%0.32 / 0.00%
PProcess memory0.19 / 0.17%0.38 / 0.33%0.00 / 0.00%
P1Order-Based Meal Assembly0.38 / 0.33%0.75 / 0.67%0.00 / 0.00%
P2Steamer Stack Order Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
SSpatial memory0.01 / 0.00%0.03 / 0.00%0.00 / 0.00%
S1Rotating Tray Layout Reconstruction0.03 / 0.00%0.05 / 0.00%0.00 / 0.00%
S2Hidden-Zone Navigation0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
FFeedback-rule learning2.05 / 2.00%4.10 / 4.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider Navigation4.10 / 4.00%8.20 / 8.00%0.00 / 0.00%
F2Color-Block Trial Ordering0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
MIntegrated memory1.98 / 1.67%3.54 / 3.00%0.43 / 0.33%
M1Workpiece Tracking and Route Reproduction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card Matching3.96 / 3.33%7.07 / 6.00%0.85 / 0.67%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy5.25 / 3.00%3.30 / 0.00%7.20 / 6.00%
Medium1.54 / 0.00%2.09 / 0.00%0.99 / 0.00%
Hard1.45 / 0.00%1.45 / 0.00%1.45 / 0.00%
E2Object-Code Keypad EntryEasy1.75 / 1.00%2.90 / 2.00%0.60 / 0.00%
Medium0.44 / 0.00%0.73 / 0.00%0.15 / 0.00%
Hard0.43 / 0.00%0.65 / 0.00%0.20 / 0.00%
P1Order-Based Meal AssemblyEasy0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Medium1.05 / 1.00%2.11 / 2.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
P2Steamer Stack Order ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
S1Rotating Tray Layout ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
S2Hidden-Zone NavigationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy11.30 / 11.00%22.60 / 22.00%0.00 / 0.00%
Medium1.00 / 1.00%2.00 / 2.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy11.35 / 10.00%20.40 / 18.00%2.30 / 2.00%
Medium0.33 / 0.00%0.44 / 0.00%0.22 / 0.00%
Hard0.21 / 0.00%0.38 / 0.00%0.04 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response19.75 / 15.83%26.63 / 21.67%12.86 / 10.00%
DR1Continuous Conveyor Picking10.36 / 2.00%12.28 / 2.67%8.43 / 1.33%
DR2Light-Guided Button Response7.22 / 0.00%10.23 / 0.00%4.22 / 0.00%
DR3Periodic Gate Crossing and Withdrawal61.40 / 61.33%84.00 / 84.00%38.80 / 38.67%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution2.41 / 1.78%4.42 / 3.44%0.40 / 0.11%
PR1Linear Resistor Adjustment12.99 / 10.33%25.11 / 20.67%0.86 / 0.00%
PR2Rotary Voltage Regulator Setting1.36 / 0.33%1.17 / 0.00%1.55 / 0.67%
PR3Marker Line Tracing0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR5Oriented Stamp Placement0.04 / 0.00%0.07 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy20.00 / 6.00%23.40 / 8.00%16.60 / 4.00%
Medium7.92 / 0.00%9.24 / 0.00%6.60 / 0.00%
Hard3.15 / 0.00%4.20 / 0.00%2.10 / 0.00%
DR2Light-Guided Button ResponseEasy9.57 / 0.00%12.96 / 0.00%6.18 / 0.00%
Medium7.44 / 0.00%11.04 / 0.00%3.83 / 0.00%
Hard4.67 / 0.00%6.69 / 0.00%2.64 / 0.00%
DR3Periodic Gate Crossing and WithdrawalEasy63.15 / 63.00%76.00 / 76.00%50.30 / 50.00%
Medium76.06 / 76.00%88.00 / 88.00%64.11 / 64.00%
Hard45.00 / 45.00%88.00 / 88.00%2.00 / 2.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy15.27 / 9.00%28.63 / 18.00%1.91 / 0.00%
Medium12.24 / 11.00%23.80 / 22.00%0.67 / 0.00%
Hard11.45 / 11.00%22.90 / 22.00%0.00 / 0.00%
PR2Rotary Voltage Regulator SettingEasy3.89 / 1.00%3.50 / 0.00%4.29 / 2.00%
Medium0.18 / 0.00%0.00 / 0.00%0.36 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR3Marker Line TracingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.22 / 0.00%0.44 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR5Oriented Stamp PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.11 / 0.00%0.22 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination0.86 / 0.53%1.32 / 0.93%0.40 / 0.13%
CP01Conveyor Meal Assembly5.39 / 4.33%9.32 / 8.00%1.45 / 0.67%
CP02Rotating Cake Decoration0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP03Dynamic Rack Loading0.04 / 0.00%0.07 / 0.00%0.00 / 0.00%
CP04Dual-Arm Cooking0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence Reproduction0.01 / 0.00%0.03 / 0.00%0.00 / 0.00%
CP06Produce Weighing and Balancing1.14 / 0.33%1.71 / 0.67%0.57 / 0.00%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction0.33 / 0.00%0.24 / 0.00%0.42 / 0.00%
CP09Whiteboard Drawing Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance1.68 / 0.67%1.82 / 0.67%1.54 / 0.67%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy14.80 / 13.00%26.00 / 24.00%3.60 / 2.00%
Medium0.58 / 0.00%0.67 / 0.00%0.50 / 0.00%
Hard0.79 / 0.00%1.31 / 0.00%0.26 / 0.00%
CP02Rotating Cake DecorationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP03Dynamic Rack LoadingEasy0.10 / 0.00%0.20 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.01 / 0.00%0.02 / 0.00%0.00 / 0.00%
CP04Dual-Arm CookingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence ReproductionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.04 / 0.00%0.07 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP06Produce Weighing and BalancingEasy2.60 / 1.00%4.10 / 2.00%1.10 / 0.00%
Medium0.38 / 0.00%0.40 / 0.00%0.35 / 0.00%
Hard0.43 / 0.00%0.62 / 0.00%0.25 / 0.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy0.10 / 0.00%0.20 / 0.00%0.00 / 0.00%
Medium0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Hard0.81 / 0.00%0.36 / 0.00%1.26 / 0.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy3.70 / 2.00%3.90 / 2.00%3.50 / 2.00%
Medium1.02 / 0.00%1.20 / 0.00%0.85 / 0.00%
Hard0.32 / 0.00%0.37 / 0.00%0.26 / 0.00%

Efficiency

Avg. task time
12.4 min
Avg. inference time
262.1 ms
Inference hardware
RTX PRO 5000 · 72GB
Mind · Act · Mind2Act
TasksAvg. task time
Mind10.9 min
Act4.5 min
Mind2Act21.9 min