DeepSeek V4.1 Flash
Results reported
Results
Score / Success
| Suite | Mean | Clean | Random |
|---|---|---|---|
| Overall | 6.09 / 6.56% | 5.81 / 6.22% | 6.37 / 6.89% |
| Mind | 12.37 / 11.00% | 10.98 / 9.33% | 13.77 / 12.67% |
| Act | 11.34 / 8.67% | 11.80 / 9.33% | 10.89 / 8.00% |
| Mind2Act | 0.32 / 0.00% | 0.23 / 0.00% | 0.41 / 0.00% |
Task results
Score / Success · Averages across Easy, Medium and Hard.
Mind
| Task | Mean | Clean | Random |
|---|---|---|---|
| EEntity memory | 23.59 / 20.00% | 20.92 / 16.67% | 26.26 / 23.33% |
| E1Duplicate Item Identification | 22.85 / 20.00% | 17.63 / 13.33% | 28.07 / 26.67% |
| E2Object-Code Keypad Entry | 24.33 / 20.00% | 24.22 / 20.00% | 24.44 / 20.00% |
| PProcess memory | 0.18 / 0.00% | 0.00 / 0.00% | 0.37 / 0.00% |
| P1Order-Based Meal Assembly | 0.37 / 0.00% | 0.00 / 0.00% | 0.73 / 0.00% |
| P2Steamer Stack Order Reconstruction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| SSpatial memory | 10.00 / 10.00% | 6.67 / 6.67% | 13.33 / 13.33% |
| S1Rotating Tray Layout Reconstruction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| S2Hidden-Zone Navigation | 20.00 / 20.00% | 13.33 / 13.33% | 26.67 / 26.67% |
| FFeedback-rule learning | 11.18 / 8.33% | 10.13 / 6.67% | 12.23 / 10.00% |
| F1Unknown Button Mapping and Slider Navigation | 22.37 / 16.67% | 20.27 / 13.33% | 24.47 / 20.00% |
| F2Color-Block Trial Ordering | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| MIntegrated memory | 16.92 / 16.67% | 17.17 / 16.67% | 16.67 / 16.67% |
| M1Workpiece Tracking and Route Reproduction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| M2Memory Card Matching | 33.83 / 33.33% | 34.33 / 33.33% | 33.33 / 33.33% |
Results by difficulty
| Task | Level | Mean | Clean | Random |
|---|---|---|---|---|
| E1Duplicate Item Identification | Easy | 63.00 / 60.00% | 46.00 / 40.00% | 80.00 / 80.00% |
| Medium | 3.30 / 0.00% | 4.40 / 0.00% | 2.20 / 0.00% | |
| Hard | 2.25 / 0.00% | 2.50 / 0.00% | 2.00 / 0.00% | |
| E2Object-Code Keypad Entry | Easy | 23.00 / 20.00% | 23.00 / 20.00% | 23.00 / 20.00% |
| Medium | 25.87 / 20.00% | 24.40 / 20.00% | 27.33 / 20.00% | |
| Hard | 24.12 / 20.00% | 25.25 / 20.00% | 23.00 / 20.00% | |
| P1Order-Based Meal Assembly | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 1.10 / 0.00% | 0.00 / 0.00% | 2.20 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| P2Steamer Stack Order Reconstruction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| S1Rotating Tray Layout Reconstruction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| S2Hidden-Zone Navigation | Easy | 50.00 / 50.00% | 40.00 / 40.00% | 60.00 / 60.00% |
| Medium | 10.00 / 10.00% | 0.00 / 0.00% | 20.00 / 20.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| F1Unknown Button Mapping and Slider Navigation | Easy | 37.50 / 30.00% | 9.00 / 0.00% | 66.00 / 60.00% |
| Medium | 16.60 / 10.00% | 28.80 / 20.00% | 4.40 / 0.00% | |
| Hard | 13.00 / 10.00% | 23.00 / 20.00% | 3.00 / 0.00% | |
| F2Color-Block Trial Ordering | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| M1Workpiece Tracking and Route Reproduction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| M2Memory Card Matching | Easy | 81.50 / 80.00% | 83.00 / 80.00% | 80.00 / 80.00% |
| Medium | 20.00 / 20.00% | 20.00 / 20.00% | 20.00 / 20.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
Act
| Task | Mean | Clean | Random |
|---|---|---|---|
| DRDynamic response | 14.51 / 10.00% | 16.05 / 11.67% | 12.98 / 8.33% |
| DR1Continuous Conveyor Picking | 9.97 / 6.67% | 10.36 / 6.67% | 9.59 / 6.67% |
| DR2Light-Guided Button Response | 21.42 / 6.67% | 27.16 / 13.33% | 15.67 / 0.00% |
| DR3Periodic Gate Crossing and Withdrawal | 26.67 / 26.67% | 26.67 / 26.67% | 26.67 / 26.67% |
| DR4Ball-on-Tray Transport and Placement | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| PRPrecise execution | 9.23 / 7.78% | 8.96 / 7.78% | 9.50 / 7.78% |
| PR1Linear Resistor Adjustment | 42.50 / 40.00% | 41.66 / 40.00% | 43.33 / 40.00% |
| PR2Rotary Voltage Regulator Setting | 12.88 / 6.67% | 12.11 / 6.67% | 13.64 / 6.67% |
| PR3Marker Line Tracing | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| PR4Cloth Edge Alignment and Placement | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| PR5Oriented Stamp Placement | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| PR6Domino Chain Bell Ringing | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
Results by difficulty
| Task | Level | Mean | Clean | Random |
|---|---|---|---|---|
| DR1Continuous Conveyor Picking | Easy | 26.00 / 20.00% | 28.00 / 20.00% | 24.00 / 20.00% |
| Medium | 2.20 / 0.00% | 2.64 / 0.00% | 1.76 / 0.00% | |
| Hard | 1.71 / 0.00% | 0.43 / 0.00% | 3.00 / 0.00% | |
| DR2Light-Guided Button Response | Easy | 40.10 / 20.00% | 52.00 / 40.00% | 28.20 / 0.00% |
| Medium | 13.20 / 0.00% | 16.28 / 0.00% | 10.12 / 0.00% | |
| Hard | 10.95 / 0.00% | 13.20 / 0.00% | 8.70 / 0.00% | |
| DR3Periodic Gate Crossing and Withdrawal | Easy | 30.00 / 30.00% | 40.00 / 40.00% | 20.00 / 20.00% |
| Medium | 30.00 / 30.00% | 20.00 / 20.00% | 40.00 / 40.00% | |
| Hard | 20.00 / 20.00% | 20.00 / 20.00% | 20.00 / 20.00% | |
| DR4Ball-on-Tray Transport and Placement | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| PR1Linear Resistor Adjustment | Easy | 43.27 / 40.00% | 60.00 / 60.00% | 26.54 / 20.00% |
| Medium | 43.07 / 40.00% | 24.99 / 20.00% | 61.14 / 60.00% | |
| Hard | 41.16 / 40.00% | 40.00 / 40.00% | 42.31 / 40.00% | |
| PR2Rotary Voltage Regulator Setting | Easy | 11.86 / 0.00% | 13.12 / 0.00% | 10.59 / 0.00% |
| Medium | 15.93 / 10.00% | 3.20 / 0.00% | 28.66 / 20.00% | |
| Hard | 10.84 / 10.00% | 20.00 / 20.00% | 1.68 / 0.00% | |
| PR3Marker Line Tracing | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| PR4Cloth Edge Alignment and Placement | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| PR5Oriented Stamp Placement | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| PR6Domino Chain Bell Ringing | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
Mind2Act
| Task | Mean | Clean | Random |
|---|---|---|---|
| CPMind–action coordination | 0.32 / 0.00% | 0.23 / 0.00% | 0.41 / 0.00% |
| CP01Conveyor Meal Assembly | 1.00 / 0.00% | 0.56 / 0.00% | 1.44 / 0.00% |
| CP02Rotating Cake Decoration | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP03Dynamic Rack Loading | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP04Dual-Arm Cooking | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP05Keyboard Sequence Reproduction | 1.18 / 0.00% | 0.49 / 0.00% | 1.86 / 0.00% |
| CP06Produce Weighing and Balancing | 0.64 / 0.00% | 1.28 / 0.00% | 0.00 / 0.00% |
| CP07Bridge Construction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP08Reference-Guided Block Construction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP09Whiteboard Drawing Reconstruction | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| CP10Predictive Ball Return and Red-Ball Avoidance | 0.41 / 0.00% | 0.00 / 0.00% | 0.82 / 0.00% |
Results by difficulty
| Task | Level | Mean | Clean | Random |
|---|---|---|---|---|
| CP01Conveyor Meal Assembly | Easy | 1.50 / 0.00% | 0.00 / 0.00% | 3.00 / 0.00% |
| Medium | 1.50 / 0.00% | 1.67 / 0.00% | 1.33 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP02Rotating Cake Decoration | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP03Dynamic Rack Loading | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP04Dual-Arm Cooking | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP05Keyboard Sequence Reproduction | Easy | 2.81 / 0.00% | 0.75 / 0.00% | 4.88 / 0.00% |
| Medium | 0.38 / 0.00% | 0.50 / 0.00% | 0.25 / 0.00% | |
| Hard | 0.35 / 0.00% | 0.23 / 0.00% | 0.47 / 0.00% | |
| CP06Produce Weighing and Balancing | Easy | 1.00 / 0.00% | 2.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.50 / 0.00% | 1.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.42 / 0.00% | 0.84 / 0.00% | 0.00 / 0.00% | |
| CP07Bridge Construction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP08Reference-Guided Block Construction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP09Whiteboard Drawing Reconstruction | Easy | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% |
| Medium | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| Hard | 0.00 / 0.00% | 0.00 / 0.00% | 0.00 / 0.00% | |
| CP10Predictive Ball Return and Red-Ball Avoidance | Easy | 0.50 / 0.00% | 0.00 / 0.00% | 1.00 / 0.00% |
| Medium | 0.50 / 0.00% | 0.00 / 0.00% | 1.00 / 0.00% | |
| Hard | 0.23 / 0.00% | 0.00 / 0.00% | 0.47 / 0.00% |
Efficiency
- Avg. cost per attempt
- ≈ ¥8.26
- Avg. task time
- 96.1 min
- Avg. response time
- 35.6 s
Mind · Act · Mind2Act
| Tasks | Avg. cost per attempt | Avg. task time | Avg. response time |
|---|---|---|---|
| Mind | ≈ ¥8.51 | 91.9 min | 38.7 s |
| Act | ≈ ¥6.95 | 93.0 min | 35.4 s |
| Mind2Act | ≈ ¥9.31 | 103.2 min | 33.3 s |
Usage totals
Evaluation cost
900 selected official episodes across Clean and Random.
| Metric | Recorded total | Coverage |
|---|---|---|
| Model decisions | 100,632 | 900 / 900 episodes |
| API request attempts | 107,979 | 900 / 900 episodes |
| Upstream requests | 107,979 | 107,979 / 107,979 requests |
| Input tokens | 32,352,392,211 | 100,631 / 107,979 requests |
| Output tokens | 433,769,839 | 100,631 / 107,979 requests |
| Cached input tokens (included) | 30,977,891,016 | 100,570 / 107,979 requests |
| Reasoning output tokens (included) | 399,793,118 | 96,345 / 107,979 requests |
| Cumulative API request time | 1,066 h 42 min 15 s | 107,979 / 107,979 requests |
| Cumulative inference time | 1,296 h 38 min 30 s | 899 / 900 episodes |
| Cumulative episode time | 1,439 h 10 min 57 s | 899 / 900 episodes |
| Cumulative run time | 1,465 h 29 min 43 s | 899 / 900 episodes |
| Estimated API cost | CNY 7,432.09 | 100,570 / 107,979 requests |
Cost basis
Recorded requests with complete input, cached-input and output usage.
CNY per 1M tokens: input 2; cached input 0.04; output 8.
Cost = uncached input × input rate + cached input × cache rate + output × output rate.
API platform billing, 1–10 Nov 2026.