Agent Leaderboard

Gemini 4 Argon

AgentGoogle

Results reported

Results

Score / Success

SuiteMeanCleanRandom
Overall4.36 / 4.67%4.77 / 5.33%3.94 / 4.00%
Mind4.97 / 4.33%5.73 / 5.33%4.21 / 3.33%
Act12.25 / 9.67%13.15 / 10.67%11.36 / 8.67%
Mind2Act0.10 / 0.00%0.11 / 0.00%0.09 / 0.00%

Task results

Score / Success · Averages across Easy, Medium and Hard.

Mind

TaskMeanCleanRandom
EEntity memory8.48 / 6.67%8.08 / 6.67%8.88 / 6.67%
E1Duplicate Item Identification7.00 / 6.67%6.67 / 6.67%7.33 / 6.67%
E2Object-Code Keypad Entry9.96 / 6.67%9.49 / 6.67%10.42 / 6.67%
PProcess memory0.75 / 0.00%0.50 / 0.00%1.00 / 0.00%
P1Order-Based Meal Assembly1.50 / 0.00%1.00 / 0.00%2.00 / 0.00%
P2Steamer Stack Order Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
SSpatial memory5.09 / 5.00%6.67 / 6.67%3.52 / 3.33%
S1Rotating Tray Layout Reconstruction0.18 / 0.00%0.00 / 0.00%0.37 / 0.00%
S2Hidden-Zone Navigation10.00 / 10.00%13.33 / 13.33%6.67 / 6.67%
FFeedback-rule learning5.25 / 5.00%10.00 / 10.00%0.50 / 0.00%
F1Unknown Button Mapping and Slider Navigation7.17 / 6.67%13.33 / 13.33%1.00 / 0.00%
F2Color-Block Trial Ordering3.33 / 3.33%6.67 / 6.67%0.00 / 0.00%
MIntegrated memory5.28 / 5.00%3.40 / 3.33%7.17 / 6.67%
M1Workpiece Tracking and Route Reproduction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card Matching10.56 / 10.00%6.79 / 6.67%14.33 / 13.33%
Results by difficulty
TaskLevelMeanCleanRandom
E1Duplicate Item IdentificationEasy20.00 / 20.00%20.00 / 20.00%20.00 / 20.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard1.00 / 0.00%0.00 / 0.00%2.00 / 0.00%
E2Object-Code Keypad EntryEasy13.00 / 10.00%6.00 / 0.00%20.00 / 20.00%
Medium15.87 / 10.00%21.47 / 20.00%10.27 / 0.00%
Hard1.00 / 0.00%1.00 / 0.00%1.00 / 0.00%
P1Order-Based Meal AssemblyEasy4.50 / 0.00%3.00 / 0.00%6.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
P2Steamer Stack Order ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
S1Rotating Tray Layout ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.55 / 0.00%0.00 / 0.00%1.10 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
S2Hidden-Zone NavigationEasy30.00 / 30.00%40.00 / 40.00%20.00 / 20.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F1Unknown Button Mapping and Slider NavigationEasy11.50 / 10.00%20.00 / 20.00%3.00 / 0.00%
Medium10.00 / 10.00%20.00 / 20.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
F2Color-Block Trial OrderingEasy10.00 / 10.00%20.00 / 20.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M1Workpiece Tracking and Route ReproductionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
M2Memory Card MatchingEasy31.50 / 30.00%20.00 / 20.00%43.00 / 40.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.19 / 0.00%0.38 / 0.00%0.00 / 0.00%

Act

TaskMeanCleanRandom
DRDynamic response14.01 / 10.83%13.28 / 10.00%14.75 / 11.67%
DR1Continuous Conveyor Picking4.07 / 0.00%3.15 / 0.00%4.99 / 0.00%
DR2Light-Guided Button Response11.98 / 3.33%16.63 / 6.67%7.33 / 0.00%
DR3Periodic Gate Crossing and Withdrawal40.00 / 40.00%33.33 / 33.33%46.67 / 46.67%
DR4Ball-on-Tray Transport and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PRPrecise execution11.08 / 8.89%13.07 / 11.11%9.10 / 6.67%
PR1Linear Resistor Adjustment47.98 / 40.00%46.25 / 40.00%49.71 / 40.00%
PR2Rotary Voltage Regulator Setting18.52 / 13.33%32.16 / 26.67%4.87 / 0.00%
PR3Marker Line Tracing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR5Oriented Stamp Placement0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell Ringing0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
DR1Continuous Conveyor PickingEasy7.00 / 0.00%6.00 / 0.00%8.00 / 0.00%
Medium2.64 / 0.00%0.88 / 0.00%4.40 / 0.00%
Hard2.57 / 0.00%2.57 / 0.00%2.57 / 0.00%
DR2Light-Guided Button ResponseEasy12.90 / 0.00%17.40 / 0.00%8.40 / 0.00%
Medium17.48 / 10.00%25.28 / 20.00%9.68 / 0.00%
Hard5.55 / 0.00%7.20 / 0.00%3.90 / 0.00%
DR3Periodic Gate Crossing and WithdrawalEasy40.00 / 40.00%40.00 / 40.00%40.00 / 40.00%
Medium40.00 / 40.00%20.00 / 20.00%60.00 / 60.00%
Hard40.00 / 40.00%40.00 / 40.00%40.00 / 40.00%
DR4Ball-on-Tray Transport and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR1Linear Resistor AdjustmentEasy43.78 / 30.00%55.15 / 40.00%32.42 / 20.00%
Medium38.93 / 30.00%43.61 / 40.00%34.25 / 20.00%
Hard61.23 / 60.00%40.00 / 40.00%82.46 / 80.00%
PR2Rotary Voltage Regulator SettingEasy21.18 / 10.00%33.75 / 20.00%8.61 / 0.00%
Medium12.40 / 10.00%21.13 / 20.00%3.68 / 0.00%
Hard21.97 / 20.00%41.60 / 40.00%2.34 / 0.00%
PR3Marker Line TracingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR4Cloth Edge Alignment and PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR5Oriented Stamp PlacementEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
PR6Domino Chain Bell RingingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%

Mind2Act

TaskMeanCleanRandom
CPMind–action coordination0.10 / 0.00%0.11 / 0.00%0.09 / 0.00%
CP01Conveyor Meal Assembly0.33 / 0.00%0.67 / 0.00%0.00 / 0.00%
CP02Rotating Cake Decoration0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP03Dynamic Rack Loading0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP04Dual-Arm Cooking0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence Reproduction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP06Produce Weighing and Balancing0.58 / 0.00%0.33 / 0.00%0.83 / 0.00%
CP07Bridge Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block Construction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP09Whiteboard Drawing Reconstruction0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball Avoidance0.08 / 0.00%0.08 / 0.00%0.08 / 0.00%
Results by difficulty
TaskLevelMeanCleanRandom
CP01Conveyor Meal AssemblyEasy1.00 / 0.00%2.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP02Rotating Cake DecorationEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP03Dynamic Rack LoadingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP04Dual-Arm CookingEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP05Keyboard Sequence ReproductionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP06Produce Weighing and BalancingEasy1.50 / 0.00%1.00 / 0.00%2.00 / 0.00%
Medium0.25 / 0.00%0.00 / 0.00%0.50 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP07Bridge ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP08Reference-Guided Block ConstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP09Whiteboard Drawing ReconstructionEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
CP10Predictive Ball Return and Red-Ball AvoidanceEasy0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Medium0.00 / 0.00%0.00 / 0.00%0.00 / 0.00%
Hard0.23 / 0.00%0.23 / 0.00%0.23 / 0.00%

Efficiency

Avg. cost per attempt
≈ ¥7.83
Avg. task time
18.0 min
Avg. response time
27.8 s
Mind · Act · Mind2Act
TasksAvg. cost per attemptAvg. task timeAvg. response time
Mind≈ ¥7.4315.2 min26.8 s
Act≈ ¥7.2415.8 min23.8 s
Mind2Act≈ ¥8.8122.9 min32.0 s
Usage totals

Evaluation cost

900 selected official episodes across Clean and Random.

MetricRecorded totalCoverage
Model decisions29,628900 / 900 episodes
API request attempts29,998900 / 900 episodes
Upstream requests29,99829,998 / 29,998 requests
Input tokens688,190,63529,628 / 29,998 requests
Output tokens2,757,40929,628 / 29,998 requests
Cached input tokens (included)209,113,68329,624 / 29,998 requests
Reasoning output tokens (included)522,31129,578 / 29,998 requests
Cumulative API request time231 h 19 min 14 s29,998 / 29,998 requests
Cumulative inference time234 h 36 min 59 s900 / 900 episodes
Cumulative episode time269 h 34 min 17 s900 / 900 episodes
Cumulative run time289 h 41 min 39 s900 / 900 episodes
Estimated API costCNY 7,044.2129,624 / 29,998 requests
Cost basis

Recorded requests with complete input, cached-input and output usage.

CNY per 1M tokens: input 14; cached input 0.7; output 70.

Cost = uncached input × input rate + cached input × cache rate + output × output rate.

API platform billing, 1–10 Nov 2026.