End-to-end Leaderboard (VLA / WAM)

X-VLA

VLATsinghua AIR

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall6.17 / 6.96%6.76 / 7.64%5.59 / 6.27%
Mind1.41 / 1.03%2.09 / 1.60%0.73 / 0.47%
Act21.89 / 19.63%23.20 / 21.07%20.58 / 18.20%
Mind2Act0.70 / 0.20%0.87 / 0.27%0.52 / 0.13%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory1.62 / 0.50%1.39 / 0.00%1.86 / 1.00%
E1Duplicate Item Identification0.21 / 0.00%0.09 / 0.00%0.33 / 0.00%
E2Object-Code Keypad Entry3.04 / 1.00%2.70 / 0.00%3.39 / 2.00%
PProcess memory0.01 / 0.00%0.03 / 0.00%0.00 / 0.00%
P1Order-Based Meal Assembly0.03 / 0.00%0.05 / 0.00%0.00 / 0.00%
P2Steamer Stack Order Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
SSpatial memory0.40 / 0.17%0.72 / 0.33%0.08 / 0.00%
S1Rotating Tray Layout Reconstruction0.80 / 0.33%1.43 / 0.67%0.16 / 0.00%
S2Hidden-Zone Navigation0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
FFeedback-rule learning2.50 / 2.50%3.67 / 3.67%1.33 / 1.33%
F1Unknown Button Mapping and Slider Navigation5.00 / 5.00%7.33 / 7.33%2.67 / 2.67%
F2Color-Block Trial Ordering0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
MIntegrated memory2.52 / 2.00%4.66 / 4.00%0.39 / 0.00%
M1Workpiece Tracking and Route Reproduction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card Matching5.05 / 4.00%9.32 / 8.00%0.77 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.28 / 0.00%0.11 / 0.00%0.44 / 0.00%
Hard0.35 / 0.00%0.15 / 0.00%0.55 / 0.00%
E2Object-Code Keypad EntryEasy4.25 / 2.00%3.30 / 0.00%5.20 / 4.00%
Medium3.20 / 1.00%2.64 / 0.00%3.76 / 2.00%
Hard1.67 / 0.00%2.15 / 0.00%1.20 / 0.00%
P1Order-Based Meal AssemblyEasy0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
P2Steamer Stack Order ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
S1Rotating Tray Layout ReconstructionEasy0.45 / 0.00%0.60 / 0.00%0.30 / 0.00%
Medium0.61 / 0.00%1.10 / 0.00%0.11 / 0.00%
Hard1.34 / 1.00%2.60 / 2.00%0.07 / 0.00%
S2Hidden-Zone NavigationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy14.00 / 14.00%20.00 / 20.00%8.00 / 8.00%
Medium1.00 / 1.00%2.00 / 2.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy13.80 / 12.00%26.10 / 24.00%1.50 / 0.00%
Medium1.16 / 0.00%1.65 / 0.00%0.66 / 0.00%
Hard0.19 / 0.00%0.23 / 0.00%0.15 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response26.49 / 22.08%28.60 / 24.33%24.37 / 19.83%
DR1Continuous Conveyor Picking0.86 / 0.00%0.14 / 0.00%1.57 / 0.00%
DR2Light-Guided Button Response29.38 / 12.67%34.88 / 18.00%23.88 / 7.33%
DR3Periodic Gate Crossing and Withdrawal75.70 / 75.67%79.36 / 79.33%72.05 / 72.00%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution18.82 / 18.00%19.60 / 18.89%18.05 / 17.11%
PR1Linear Resistor Adjustment46.07 / 43.33%47.35 / 45.33%44.78 / 41.33%
PR2Rotary Voltage Regulator Setting2.46 / 0.67%2.87 / 1.33%2.06 / 0.00%
PR3Marker Line Tracing0.30 / 0.00%0.60 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and Placement64.00 / 64.00%66.67 / 66.67%61.33 / 61.33%
PR5Oriented Stamp Placement0.10 / 0.00%0.10 / 0.00%0.10 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy0.80 / 0.00%0.00 / 0.00%1.60 / 0.00%
Medium1.14 / 0.00%0.18 / 0.00%2.11 / 0.00%
Hard0.62 / 0.00%0.26 / 0.00%0.99 / 0.00%
DR2Light-Guided Button ResponseEasy37.23 / 15.00%43.96 / 22.00%30.50 / 8.00%
Medium24.83 / 8.00%29.86 / 12.00%19.80 / 4.00%
Hard26.09 / 15.00%30.83 / 20.00%21.34 / 10.00%
DR3Periodic Gate Crossing and WithdrawalEasy77.08 / 77.00%82.00 / 82.00%72.15 / 72.00%
Medium72.00 / 72.00%78.00 / 78.00%66.00 / 66.00%
Hard78.04 / 78.00%78.08 / 78.00%78.00 / 78.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy63.59 / 58.00%67.25 / 64.00%59.94 / 52.00%
Medium45.92 / 44.00%49.83 / 48.00%42.01 / 40.00%
Hard28.68 / 28.00%24.97 / 24.00%32.40 / 32.00%
PR2Rotary Voltage Regulator SettingEasy4.79 / 0.00%4.41 / 0.00%5.17 / 0.00%
Medium1.51 / 1.00%2.19 / 2.00%0.82 / 0.00%
Hard1.10 / 1.00%2.00 / 2.00%0.19 / 0.00%
PR3Marker Line TracingEasy0.90 / 0.00%1.80 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy97.00 / 97.00%98.00 / 98.00%96.00 / 96.00%
Medium1.00 / 1.00%2.00 / 2.00%0.00 / 0.00%
Hard94.00 / 94.00%100.00 / 100.00%88.00 / 88.00%
PR5Oriented Stamp PlacementEasy0.30 / 0.00%0.30 / 0.00%0.30 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination0.70 / 0.20%0.87 / 0.27%0.52 / 0.13%
CP01Conveyor Meal Assembly2.61 / 1.33%3.51 / 2.00%1.71 / 0.67%
CP02Rotating Cake Decoration0.01 / 0.00%0.00 / 0.00%0.03 / 0.00%
CP03Dynamic Rack Loading0.03 / 0.00%0.03 / 0.00%0.03 / 0.00%
CP04Dual-Arm Cooking0.20 / 0.00%0.30 / 0.00%0.10 / 0.00%
CP05Keyboard Sequence Reproduction0.33 / 0.00%0.58 / 0.00%0.08 / 0.00%
CP06Produce Weighing and Balancing2.39 / 0.67%2.78 / 0.67%2.00 / 0.67%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction0.35 / 0.00%0.46 / 0.00%0.24 / 0.00%
CP09Whiteboard Drawing Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance1.07 / 0.00%1.08 / 0.00%1.06 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy6.70 / 4.00%9.20 / 6.00%4.20 / 2.00%
Medium0.78 / 0.00%0.90 / 0.00%0.67 / 0.00%
Hard0.34 / 0.00%0.42 / 0.00%0.26 / 0.00%
CP02Rotating Cake DecorationEasy0.04 / 0.00%0.00 / 0.00%0.07 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP03Dynamic Rack LoadingEasy0.10 / 0.00%0.10 / 0.00%0.10 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP04Dual-Arm CookingEasy0.60 / 0.00%0.90 / 0.00%0.30 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence ReproductionEasy0.04 / 0.00%0.07 / 0.00%0.00 / 0.00%
Medium0.65 / 0.00%1.18 / 0.00%0.12 / 0.00%
Hard0.31 / 0.00%0.49 / 0.00%0.13 / 0.00%
CP06Produce Weighing and BalancingEasy5.60 / 2.00%5.80 / 2.00%5.40 / 2.00%
Medium0.38 / 0.00%0.35 / 0.00%0.40 / 0.00%
Hard1.19 / 0.00%2.18 / 0.00%0.20 / 0.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy0.45 / 0.00%0.80 / 0.00%0.10 / 0.00%
Medium0.28 / 0.00%0.55 / 0.00%0.00 / 0.00%
Hard0.32 / 0.00%0.03 / 0.00%0.62 / 0.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy1.95 / 0.00%2.00 / 0.00%1.90 / 0.00%
Medium0.88 / 0.00%0.85 / 0.00%0.90 / 0.00%
Hard0.39 / 0.00%0.40 / 0.00%0.37 / 0.00%

Efficiency

Avg. task time
16.5 min
Avg. inference time
153.0 ms
Inference hardware
RTX PRO 5000 · 72GB
Mind · Act · Mind2Act
TasksAvg. task time
Mind16.0 min
Act6.5 min
Mind2Act27.2 min