End-to-end Leaderboard (VLA / WAM)

KinRT

VLAIIGroup, Tsinghua

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall7.92 / 6.98%9.06 / 8.09%6.77 / 5.87%
Mind5.65 / 4.40%7.87 / 6.47%3.43 / 2.33%
Act17.39 / 13.60%17.81 / 14.00%16.97 / 13.20%
Mind2Act4.31 / 2.93%5.28 / 3.80%3.34 / 2.07%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory7.25 / 3.67%7.19 / 3.67%7.31 / 3.67%
E1Duplicate Item Identification11.87 / 6.67%11.66 / 6.67%12.08 / 6.67%
E2Object-Code Keypad Entry2.64 / 0.67%2.72 / 0.67%2.55 / 0.67%
PProcess memory9.42 / 8.00%17.10 / 15.33%1.74 / 0.67%
P1Order-Based Meal Assembly1.00 / 0.67%1.09 / 0.67%0.90 / 0.67%
P2Steamer Stack Order Reconstruction17.85 / 15.33%33.12 / 30.00%2.58 / 0.67%
SSpatial memory2.53 / 2.33%2.64 / 2.33%2.43 / 2.33%
S1Rotating Tray Layout Reconstruction0.40 / 0.00%0.61 / 0.00%0.19 / 0.00%
S2Hidden-Zone Navigation4.67 / 4.67%4.67 / 4.67%4.67 / 4.67%
FFeedback-rule learning4.43 / 4.33%4.05 / 4.00%4.82 / 4.67%
F1Unknown Button Mapping and Slider Navigation3.20 / 3.00%3.43 / 3.33%2.97 / 2.67%
F2Color-Block Trial Ordering5.67 / 5.67%4.67 / 4.67%6.67 / 6.67%
MIntegrated memory4.60 / 3.67%8.37 / 7.00%0.83 / 0.33%
M1Workpiece Tracking and Route Reproduction0.10 / 0.00%0.20 / 0.00%0.00 / 0.00%
M2Memory Card Matching9.10 / 7.33%16.55 / 14.00%1.66 / 0.67%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy21.30 / 15.00%20.30 / 14.00%22.30 / 16.00%
Medium6.61 / 1.00%7.39 / 2.00%5.83 / 0.00%
Hard7.70 / 4.00%7.30 / 4.00%8.10 / 4.00%
E2Object-Code Keypad EntryEasy4.25 / 2.00%4.10 / 2.00%4.40 / 2.00%
Medium2.35 / 0.00%2.79 / 0.00%1.91 / 0.00%
Hard1.31 / 0.00%1.27 / 0.00%1.35 / 0.00%
P1Order-Based Meal AssemblyEasy1.45 / 1.00%2.60 / 2.00%0.30 / 0.00%
Medium1.28 / 1.00%0.22 / 0.00%2.33 / 2.00%
Hard0.26 / 0.00%0.45 / 0.00%0.07 / 0.00%
P2Steamer Stack Order ReconstructionEasy28.30 / 27.00%53.80 / 52.00%2.80 / 2.00%
Medium19.59 / 16.00%36.05 / 32.00%3.14 / 0.00%
Hard5.65 / 3.00%9.50 / 6.00%1.80 / 0.00%
S1Rotating Tray Layout ReconstructionEasy0.38 / 0.00%0.45 / 0.00%0.30 / 0.00%
Medium0.72 / 0.00%1.32 / 0.00%0.11 / 0.00%
Hard0.11 / 0.00%0.07 / 0.00%0.15 / 0.00%
S2Hidden-Zone NavigationEasy13.00 / 13.00%12.00 / 12.00%14.00 / 14.00%
Medium1.00 / 1.00%2.00 / 2.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy8.60 / 8.00%10.30 / 10.00%6.90 / 6.00%
Medium1.00 / 1.00%0.00 / 0.00%2.00 / 2.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy14.00 / 14.00%10.00 / 10.00%18.00 / 18.00%
Medium3.00 / 3.00%4.00 / 4.00%2.00 / 2.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.30 / 0.00%0.60 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy21.15 / 18.00%38.80 / 34.00%3.50 / 2.00%
Medium5.76 / 4.00%10.31 / 8.00%1.21 / 0.00%
Hard0.39 / 0.00%0.53 / 0.00%0.26 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response30.32 / 24.75%33.23 / 27.50%27.42 / 22.00%
DR1Continuous Conveyor Picking25.89 / 18.67%32.29 / 24.67%19.49 / 12.67%
DR2Light-Guided Button Response15.93 / 1.00%15.09 / 0.00%16.78 / 2.00%
DR3Periodic Gate Crossing and Withdrawal79.46 / 79.33%85.52 / 85.33%73.41 / 73.33%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution8.78 / 6.17%7.54 / 5.00%10.01 / 7.33%
PR1Linear Resistor Adjustment30.43 / 18.67%31.64 / 20.00%29.23 / 17.33%
PR2Rotary Voltage Regulator Setting3.21 / 0.33%2.72 / 0.00%3.69 / 0.67%
PR3Marker Line Tracing0.10 / 0.00%0.00 / 0.00%0.20 / 0.00%
PR4Cloth Edge Alignment and Placement18.00 / 18.00%10.00 / 10.00%26.00 / 26.00%
PR5Oriented Stamp Placement0.92 / 0.00%0.89 / 0.00%0.94 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy53.30 / 42.00%61.20 / 52.00%45.40 / 32.00%
Medium22.10 / 14.00%33.26 / 22.00%10.93 / 6.00%
Hard2.27 / 0.00%2.40 / 0.00%2.14 / 0.00%
DR2Light-Guided Button ResponseEasy21.85 / 1.00%20.76 / 0.00%22.94 / 2.00%
Medium15.23 / 1.00%14.74 / 0.00%15.73 / 2.00%
Hard10.72 / 1.00%9.78 / 0.00%11.66 / 2.00%
DR3Periodic Gate Crossing and WithdrawalEasy77.30 / 77.00%80.45 / 80.00%74.15 / 74.00%
Medium74.06 / 74.00%82.11 / 82.00%66.00 / 66.00%
Hard87.04 / 87.00%94.00 / 94.00%80.08 / 80.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy54.41 / 38.00%59.20 / 44.00%49.63 / 32.00%
Medium21.04 / 8.00%21.29 / 8.00%20.80 / 8.00%
Hard15.85 / 10.00%14.43 / 8.00%17.26 / 12.00%
PR2Rotary Voltage Regulator SettingEasy6.85 / 0.00%6.47 / 0.00%7.23 / 0.00%
Medium2.00 / 1.00%1.16 / 0.00%2.85 / 2.00%
Hard0.77 / 0.00%0.53 / 0.00%1.00 / 0.00%
PR3Marker Line TracingEasy0.30 / 0.00%0.00 / 0.00%0.60 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy32.00 / 32.00%22.00 / 22.00%42.00 / 42.00%
Medium13.00 / 13.00%8.00 / 8.00%18.00 / 18.00%
Hard9.00 / 9.00%0.00 / 0.00%18.00 / 18.00%
PR5Oriented Stamp PlacementEasy1.50 / 0.00%1.20 / 0.00%1.80 / 0.00%
Medium1.10 / 0.00%1.32 / 0.00%0.88 / 0.00%
Hard0.15 / 0.00%0.15 / 0.00%0.15 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination4.31 / 2.93%5.28 / 3.80%3.34 / 2.07%
CP01Conveyor Meal Assembly10.93 / 8.00%10.79 / 7.33%11.06 / 8.67%
CP02Rotating Cake Decoration0.53 / 0.33%0.82 / 0.67%0.24 / 0.00%
CP03Dynamic Rack Loading0.75 / 0.00%1.03 / 0.00%0.48 / 0.00%
CP04Dual-Arm Cooking0.28 / 0.00%0.35 / 0.00%0.21 / 0.00%
CP05Keyboard Sequence Reproduction0.79 / 0.00%0.74 / 0.00%0.83 / 0.00%
CP06Produce Weighing and Balancing22.88 / 17.00%28.64 / 23.33%17.12 / 10.67%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction4.42 / 2.67%7.95 / 5.33%0.89 / 0.00%
CP09Whiteboard Drawing Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance2.53 / 1.33%2.50 / 1.33%2.56 / 1.33%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy21.80 / 18.00%14.40 / 10.00%29.20 / 26.00%
Medium6.47 / 3.00%10.03 / 6.00%2.90 / 0.00%
Hard4.51 / 3.00%7.94 / 6.00%1.07 / 0.00%
CP02Rotating Cake DecorationEasy1.41 / 1.00%2.38 / 2.00%0.45 / 0.00%
Medium0.08 / 0.00%0.07 / 0.00%0.10 / 0.00%
Hard0.09 / 0.00%0.02 / 0.00%0.16 / 0.00%
CP03Dynamic Rack LoadingEasy1.05 / 0.00%1.30 / 0.00%0.80 / 0.00%
Medium0.98 / 0.00%1.48 / 0.00%0.48 / 0.00%
Hard0.23 / 0.00%0.30 / 0.00%0.16 / 0.00%
CP04Dual-Arm CookingEasy0.45 / 0.00%0.75 / 0.00%0.15 / 0.00%
Medium0.15 / 0.00%0.10 / 0.00%0.20 / 0.00%
Hard0.23 / 0.00%0.19 / 0.00%0.28 / 0.00%
CP05Keyboard Sequence ReproductionEasy1.58 / 0.00%1.05 / 0.00%2.10 / 0.00%
Medium0.62 / 0.00%0.95 / 0.00%0.30 / 0.00%
Hard0.16 / 0.00%0.23 / 0.00%0.08 / 0.00%
CP06Produce Weighing and BalancingEasy42.60 / 36.00%54.80 / 50.00%30.40 / 22.00%
Medium18.33 / 13.00%23.10 / 18.00%13.55 / 8.00%
Hard7.71 / 2.00%8.02 / 2.00%7.40 / 2.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy10.85 / 8.00%19.70 / 16.00%2.00 / 0.00%
Medium1.57 / 0.00%2.65 / 0.00%0.50 / 0.00%
Hard0.84 / 0.00%1.51 / 0.00%0.17 / 0.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy6.40 / 4.00%6.40 / 4.00%6.40 / 4.00%
Medium0.85 / 0.00%0.85 / 0.00%0.85 / 0.00%
Hard0.34 / 0.00%0.26 / 0.00%0.42 / 0.00%

Efficiency

Avg. task time
4.8 min
Avg. inference time
110.9 ms
Inference hardware
RTX PRO 5000 · 72GB
Mind · Act · Mind2Act
TasksAvg. task time
Mind3.5 min
Act3.3 min
Mind2Act7.5 min