End-to-end Leaderboard (VLA / WAM)

G0.5

VLAGalaxea AI

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall6.22 / 5.89%6.67 / 6.40%5.76 / 5.38%
Mind2.80 / 1.93%3.69 / 2.73%1.91 / 1.13%
Act15.97 / 13.43%16.69 / 14.13%15.26 / 12.73%
Mind2Act3.05 / 2.30%3.16 / 2.33%2.94 / 2.27%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory1.67 / 0.00%1.76 / 0.00%1.58 / 0.00%
E1Duplicate Item Identification0.43 / 0.00%0.42 / 0.00%0.43 / 0.00%
E2Object-Code Keypad Entry2.92 / 0.00%3.11 / 0.00%2.72 / 0.00%
PProcess memory2.34 / 1.00%2.82 / 1.33%1.87 / 0.67%
P1Order-Based Meal Assembly0.07 / 0.00%0.09 / 0.00%0.06 / 0.00%
P2Steamer Stack Order Reconstruction4.62 / 2.00%5.55 / 2.67%3.68 / 1.33%
SSpatial memory0.94 / 0.83%1.46 / 1.33%0.42 / 0.33%
S1Rotating Tray Layout Reconstruction0.21 / 0.00%0.25 / 0.00%0.17 / 0.00%
S2Hidden-Zone Navigation1.67 / 1.67%2.67 / 2.67%0.67 / 0.67%
FFeedback-rule learning4.65 / 4.50%6.20 / 6.00%3.10 / 3.00%
F1Unknown Button Mapping and Slider Navigation8.63 / 8.33%12.40 / 12.00%4.87 / 4.67%
F2Color-Block Trial Ordering0.67 / 0.67%0.00 / 0.00%1.33 / 1.33%
MIntegrated memory4.40 / 3.33%6.20 / 5.00%2.60 / 1.67%
M1Workpiece Tracking and Route Reproduction0.03 / 0.00%0.05 / 0.00%0.00 / 0.00%
M2Memory Card Matching8.78 / 6.67%12.35 / 10.00%5.21 / 3.33%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy0.30 / 0.00%0.00 / 0.00%0.60 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.97 / 0.00%1.25 / 0.00%0.70 / 0.00%
E2Object-Code Keypad EntryEasy4.05 / 0.00%4.50 / 0.00%3.60 / 0.00%
Medium2.64 / 0.00%2.79 / 0.00%2.49 / 0.00%
Hard2.06 / 0.00%2.05 / 0.00%2.08 / 0.00%
P1Order-Based Meal AssemblyEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.11 / 0.00%0.11 / 0.00%0.11 / 0.00%
Hard0.11 / 0.00%0.15 / 0.00%0.07 / 0.00%
P2Steamer Stack Order ReconstructionEasy9.40 / 5.00%10.60 / 6.00%8.20 / 4.00%
Medium3.69 / 1.00%5.14 / 2.00%2.25 / 0.00%
Hard0.75 / 0.00%0.90 / 0.00%0.60 / 0.00%
S1Rotating Tray Layout ReconstructionEasy0.22 / 0.00%0.15 / 0.00%0.30 / 0.00%
Medium0.22 / 0.00%0.44 / 0.00%0.00 / 0.00%
Hard0.19 / 0.00%0.15 / 0.00%0.23 / 0.00%
S2Hidden-Zone NavigationEasy5.00 / 5.00%8.00 / 8.00%2.00 / 2.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy25.90 / 25.00%37.20 / 36.00%14.60 / 14.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy2.00 / 2.00%0.00 / 0.00%4.00 / 4.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy21.70 / 19.00%30.70 / 28.00%12.70 / 10.00%
Medium3.64 / 1.00%5.30 / 2.00%1.98 / 0.00%
Hard0.99 / 0.00%1.05 / 0.00%0.94 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response32.80 / 28.08%35.07 / 30.50%30.54 / 25.67%
DR1Continuous Conveyor Picking31.54 / 24.67%32.47 / 25.33%30.61 / 24.00%
DR2Light-Guided Button Response48.67 / 36.67%56.48 / 45.33%40.87 / 28.00%
DR3Periodic Gate Crossing and Withdrawal51.00 / 51.00%51.33 / 51.33%50.67 / 50.67%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution4.75 / 3.67%4.43 / 3.22%5.07 / 4.11%
PR1Linear Resistor Adjustment5.67 / 3.00%6.21 / 4.00%5.12 / 2.00%
PR2Rotary Voltage Regulator Setting4.59 / 1.67%4.91 / 1.33%4.28 / 2.00%
PR3Marker Line Tracing0.22 / 0.00%0.30 / 0.00%0.15 / 0.00%
PR4Cloth Edge Alignment and Placement17.33 / 17.33%14.00 / 14.00%20.67 / 20.67%
PR5Oriented Stamp Placement0.69 / 0.00%1.17 / 0.00%0.20 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy74.90 / 70.00%77.20 / 72.00%72.60 / 68.00%
Medium15.22 / 4.00%15.44 / 4.00%15.00 / 4.00%
Hard4.50 / 0.00%4.76 / 0.00%4.24 / 0.00%
DR2Light-Guided Button ResponseEasy53.14 / 37.00%67.14 / 54.00%39.14 / 20.00%
Medium50.33 / 39.00%53.92 / 42.00%46.74 / 36.00%
Hard42.55 / 34.00%48.37 / 40.00%36.73 / 28.00%
DR3Periodic Gate Crossing and WithdrawalEasy51.00 / 51.00%50.00 / 50.00%52.00 / 52.00%
Medium51.00 / 51.00%52.00 / 52.00%50.00 / 50.00%
Hard51.00 / 51.00%52.00 / 52.00%50.00 / 50.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy11.93 / 6.00%11.35 / 6.00%12.51 / 6.00%
Medium4.49 / 3.00%6.87 / 6.00%2.12 / 0.00%
Hard0.57 / 0.00%0.41 / 0.00%0.74 / 0.00%
PR2Rotary Voltage Regulator SettingEasy10.13 / 4.00%8.95 / 2.00%11.31 / 6.00%
Medium3.23 / 1.00%5.22 / 2.00%1.23 / 0.00%
Hard0.43 / 0.00%0.55 / 0.00%0.31 / 0.00%
PR3Marker Line TracingEasy0.30 / 0.00%0.60 / 0.00%0.00 / 0.00%
Medium0.22 / 0.00%0.00 / 0.00%0.44 / 0.00%
Hard0.15 / 0.00%0.30 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy51.00 / 51.00%42.00 / 42.00%60.00 / 60.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard1.00 / 1.00%0.00 / 0.00%2.00 / 2.00%
PR5Oriented Stamp PlacementEasy1.80 / 0.00%3.00 / 0.00%0.60 / 0.00%
Medium0.11 / 0.00%0.22 / 0.00%0.00 / 0.00%
Hard0.15 / 0.00%0.30 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination3.05 / 2.30%3.16 / 2.33%2.94 / 2.27%
CP01Conveyor Meal Assembly10.48 / 8.00%8.82 / 6.00%12.13 / 10.00%
CP02Rotating Cake Decoration0.03 / 0.00%0.05 / 0.00%0.01 / 0.00%
CP03Dynamic Rack Loading0.20 / 0.00%0.09 / 0.00%0.31 / 0.00%
CP04Dual-Arm Cooking0.05 / 0.00%0.05 / 0.00%0.05 / 0.00%
CP05Keyboard Sequence Reproduction0.07 / 0.00%0.12 / 0.00%0.03 / 0.00%
CP06Produce Weighing and Balancing15.57 / 12.33%19.37 / 16.00%11.77 / 8.67%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction0.28 / 0.00%0.37 / 0.00%0.19 / 0.00%
CP09Whiteboard Drawing Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance3.83 / 2.67%2.75 / 1.33%4.90 / 4.00%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy27.70 / 24.00%22.10 / 18.00%33.30 / 30.00%
Medium1.75 / 0.00%1.90 / 0.00%1.60 / 0.00%
Hard1.98 / 0.00%2.46 / 0.00%1.49 / 0.00%
CP02Rotating Cake DecorationEasy0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.01 / 0.00%0.00 / 0.00%0.02 / 0.00%
CP03Dynamic Rack LoadingEasy0.45 / 0.00%0.10 / 0.00%0.80 / 0.00%
Medium0.04 / 0.00%0.08 / 0.00%0.00 / 0.00%
Hard0.12 / 0.00%0.10 / 0.00%0.14 / 0.00%
CP04Dual-Arm CookingEasy0.07 / 0.00%0.15 / 0.00%0.00 / 0.00%
Medium0.05 / 0.00%0.00 / 0.00%0.10 / 0.00%
Hard0.02 / 0.00%0.00 / 0.00%0.05 / 0.00%
CP05Keyboard Sequence ReproductionEasy0.23 / 0.00%0.38 / 0.00%0.07 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP06Produce Weighing and BalancingEasy33.85 / 30.00%37.80 / 34.00%29.90 / 26.00%
Medium8.75 / 5.00%14.35 / 10.00%3.15 / 0.00%
Hard4.11 / 2.00%5.96 / 4.00%2.27 / 0.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy0.50 / 0.00%0.60 / 0.00%0.40 / 0.00%
Medium0.20 / 0.00%0.30 / 0.00%0.10 / 0.00%
Hard0.14 / 0.00%0.20 / 0.00%0.08 / 0.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy10.20 / 8.00%6.70 / 4.00%13.70 / 12.00%
Medium1.02 / 0.00%1.30 / 0.00%0.75 / 0.00%
Hard0.26 / 0.00%0.26 / 0.00%0.26 / 0.00%

Efficiency

Avg. task time
6.7 min
Avg. inference time
170.3 ms
Inference hardware
RTX PRO 5000 · 72GB
Mind · Act · Mind2Act
TasksAvg. task time
Mind7.5 min
Act4.6 min
Mind2Act8.0 min

Diagnosis

Where G0.5’s failed episodes first break, why, and what could help. All 8,470 failed episodes are attributed by rules, an LLM judge and human review.

End-to-end board

G0.5 VLA

Galaxea AI

5.9%success over 9,000 episodes
Act carries the successes; Mind is nearly empty past Easy.
ENFP · Goalkeeper N, P weak
01

Where it breaks

The first step a failed episode could not recover from, and which side failed.

Where failures break

first unrecoverable step

  • perceive2.9%
  • reason24.7%
  • execute41.4%
  • close the loop6.1%
  • episode level24.8%
Which side failed

share of failures

  • 27.7%Mind
  • 41.4%Act
  • 6.1%composition
  • 24.8%other
02

Why it fails

The most frequent failure codes and the tasks where episodes stop.

First failures

most frequent failure codes

  1. A-GRASPgrasp failed or not held1,324
  2. M-MEMORYforgot or misremembered924
  3. X-STALLstalled or looped866
  4. A-PRECISIONfine-control error863
Where tasks break

episodes stopping there, three typical tasks

  1. 250/300E1 stall before submitting
  2. 210/300P1 forget the order
  3. 202/300CP07 never grasp a piece
03

What it can do

Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.

Capability profile

30 tasks, 12 capabilities

MindIntegrated memoryIntegratedM1 · Integrated memory · Easy: 0% successM1 · Integrated memory · Medium: 0% successM1 · Integrated memory · Hard: 0% successM1M2 · Integrated memory · Easy: 10–29% successM2 · Integrated memory · Medium: 1–9% successM2 · Integrated memory · Hard: 0% successM2Spatial memorySpatialS1 · Spatial memory · Easy: 0% successS1 · Spatial memory · Medium: 0% successS1 · Spatial memory · Hard: 0% successS1S2 · Spatial memory · Easy: 1–9% successS2 · Spatial memory · Medium: 0% successS2 · Spatial memory · Hard: 0% successS2Process memoryProcessP1 · Process memory · Easy: 0% successP1 · Process memory · Medium: 0% successP1 · Process memory · Hard: 0% successP1P2 · Process memory · Easy: 1–9% successP2 · Process memory · Medium: 1–9% successP2 · Process memory · Hard: 0% successP2Entity memoryEntityE1 · Entity memory · Easy: 0% successE1 · Entity memory · Medium: 0% successE1 · Entity memory · Hard: 0% successE1E2 · Entity memory · Easy: 0% successE2 · Entity memory · Medium: 0% successE2 · Entity memory · Hard: 0% successE2Feedback-rule learningRuleF1 · Feedback-rule learning · Easy: 10–29% successF1 · Feedback-rule learning · Medium: 0% successF1 · Feedback-rule learning · Hard: 0% successF1F2 · Feedback-rule learning · Easy: 1–9% successF2 · Feedback-rule learning · Medium: 0% successF2 · Feedback-rule learning · Hard: 0% successF2ActDynamic responseDynamicDR1 · Dynamic response · Easy: 60% or more successDR1 · Dynamic response · Medium: 1–9% successDR1 · Dynamic response · Hard: 0% successDR1DR2 · Dynamic response · Easy: 30–59% successDR2 · Dynamic response · Medium: 30–59% successDR2 · Dynamic response · Hard: 30–59% successDR2DR3 · Dynamic response · Easy: 30–59% successDR3 · Dynamic response · Medium: 30–59% successDR3 · Dynamic response · Hard: 30–59% successDR3DR4 · Dynamic response · Easy: 0% successDR4 · Dynamic response · Medium: 0% successDR4 · Dynamic response · Hard: 0% successDR4Precise executionPrecisionPR1 · Precise execution · Easy: 1–9% successPR1 · Precise execution · Medium: 1–9% successPR1 · Precise execution · Hard: 0% successPR1PR2 · Precise execution · Easy: 1–9% successPR2 · Precise execution · Medium: 1–9% successPR2 · Precise execution · Hard: 0% successPR2PR3 · Precise execution · Easy: 0% successPR3 · Precise execution · Medium: 0% successPR3 · Precise execution · Hard: 0% successPR3PR4 · Precise execution · Easy: 30–59% successPR4 · Precise execution · Medium: 0% successPR4 · Precise execution · Hard: 1–9% successPR4PR5 · Precise execution · Easy: 0% successPR5 · Precise execution · Medium: 0% successPR5 · Precise execution · Hard: 0% successPR5PR6 · Precise execution · Easy: 0% successPR6 · Precise execution · Medium: 0% successPR6 · Precise execution · Hard: 0% successPR6Mind2ActPredictive decisionPredictCP10 · Predictive decision · Easy: 1–9% successCP10 · Predictive decision · Medium: 0% successCP10 · Predictive decision · Hard: 0% successCP10Sequence and planPlanCP05 · Sequence and plan · Easy: 0% successCP05 · Sequence and plan · Medium: 0% successCP05 · Sequence and plan · Hard: 0% successCP05CP07 · Sequence and plan · Easy: 0% successCP07 · Sequence and plan · Medium: 0% successCP07 · Sequence and plan · Hard: 0% successCP07Feedback adjustmentFeedbackCP04 · Feedback adjustment · Easy: 0% successCP04 · Feedback adjustment · Medium: 0% successCP04 · Feedback adjustment · Hard: 0% successCP04CP06 · Feedback adjustment · Easy: 30–59% successCP06 · Feedback adjustment · Medium: 1–9% successCP06 · Feedback adjustment · Hard: 1–9% successCP06Prerequisites and repairRepairCP02 · Prerequisites and repair · Easy: 0% successCP02 · Prerequisites and repair · Medium: 0% successCP02 · Prerequisites and repair · Hard: 0% successCP02CP08 · Prerequisites and repair · Easy: 0% successCP08 · Prerequisites and repair · Medium: 0% successCP08 · Prerequisites and repair · Hard: 0% successCP08CP09 · Prerequisites and repair · Easy: 0% successCP09 · Prerequisites and repair · Medium: 0% successCP09 · Prerequisites and repair · Hard: 0% successCP09Cumulative progressCumulativeCP01 · Cumulative progress · Easy: 10–29% successCP01 · Cumulative progress · Medium: 0% successCP01 · Cumulative progress · Hard: 0% successCP01CP03 · Cumulative progress · Easy: 0% successCP03 · Cumulative progress · Medium: 0% successCP03 · Cumulative progress · Hard: 0% successCP03

RingsEasy, Medium and Hard, outward

Success01+10+30+60+
Mind
Act
Mind2Act

Act carries the successes; Mind nearly empty past Easy

Under pressure

success (%) by difficulty, bars 0–30%

Mind

  • Easy5.6
  • Medium0.2
  • Hard0.0

Act

  • Easy21.9
  • Medium9.8
  • Hard8.6

Mind2Act

  • Easy6.2
  • Medium0.5
  • Hard0.2

All tasks

  • 11.2%Easy
  • 3.5%Medium
  • 2.9%Hard
04

What could help

Directions drawn from the failure codes and breakpoints above.

  1. 1Grasp and hold

    Grasp failures are its most frequent first failure (1,324), and 202 of 300 bridge-building episodes (CP07) never grasp a piece.

    Change
    a grip check before lifting, with re-grasp when the object slips.
    Data to add
    Contact-rich demonstrations of grasping small parts, including slips and the re-grasp that recovers them.
  2. 2Task state and the final step

    924 memory failures and 866 stalls: 210 of 300 meal-order episodes (P1) forget the order and 250 of 300 repeated-item episodes (E1) stall before submitting.

    Change
    a task-state memory that P1 checks before hand-over, and an escape from repeated motions so that E1 reaches its submission.
    Data to add
    Long demonstrations that carry orders and items across steps and end with the P1 hand-over or the E1 submission.