Document
Mind2Act World
A diagnostic benchmark for embodied mind, physical action, and mind-action composition in robotic manipulation.
Evaluation scale
Case counts are per model, across Clean and Random.
| Suite | Tasks | Levels | Settings | VLA / WAM cases | Agent cases |
|---|---|---|---|---|---|
| Mind | 10 | 3 | 2 | 3,000 | 300 |
| Act | 10 | 3 | 2 | 3,000 | 300 |
| Mind2Act | 10 | 3 | 2 | 3,000 | 300 |
| Total | 30 | 3 | 2 | 9,000 | 900 |
Every task has three levels (Easy, Medium, Hard) and is run in two settings (Clean, Random). An end-to-end model gets 50 cases for each level in each setting, an agent 5.
A robot has to think and act in a loop: what it decides sets the next action, and what that action does changes the next decision. A success rate alone cannot say which part went wrong, the reasoning, the execution or the hand-off between them.
Three suites
Mind tests reasoning and Act tests execution, each on its own. Mind2Act needs both in the same task.
| Suite | Question it asks | Tasks |
|---|---|---|
| Mind | What must the robot remember or infer before it acts? | 10 |
| Act | Can the robot turn the intended action into a valid physical outcome? | 10 |
| Mind2Act | How does execution feedback change what happens next? | 10 |
Design principles
- DiagnosisEach failed case is traced to the step where it went wrong.
- Same conditionsEvery model on a track gets the same cases, instructions and budgets.
- Hidden casesEvaluation seeds are held out from training.
- Physical scoringScores come from where objects end up and what the mechanisms register.
01Benchmark
Tasks & difficulty
30 tasks across Mind, Act and Mind2Act, each with three difficulty levels.
How difficulty increases
A harder level asks more of the same skill, for example more items to remember or more steps to plan. Each task page shows the demo videos, the instruction, the success condition, the scoring rules and the three levels side by side.
| Task category | What increases | Tasks |
|---|---|---|
| EEntity memory | More identities and associations to retain; longer retrieval spans and more complex expressions. | 2 |
| PProcess memory | More accumulated orders or layers to reconstruct, with additional orders and distractors. | 2 |
| SSpatial memory | Faster changes to remembered spatial arrangements and larger navigation grids. | 2 |
| FFeedback-rule learning | Larger action or permutation spaces to infer from limited feedback. | 2 |
| MIntegrated memory | Longer routes with revisits and more identity–position pairs to track. | 2 |
| DRDynamic response | More moving targets, shorter response windows, multiple gates and tighter transport constraints. | 4 |
| PRPrecise execution | Tighter reading and heading tolerances, more complex paths and folds, and longer physical chains. | 6 |
| CPMind–action coordination | More items or remembered steps, additional prerequisites, and tighter feedback-dependent execution. | 10 |
Task index
Mind
- E1Duplicate Item Identification
- E2Object-Code Keypad Entry
- P1Order-Based Meal Assembly
- P2Steamer Stack Order Reconstruction
- S1Rotating Tray Layout Reconstruction
- S2Hidden-Zone Navigation
- F1Unknown Button Mapping and Slider Navigation
- F2Color-Block Trial Ordering
- M1Workpiece Tracking and Route Reproduction
- M2Memory Card Matching
Act
- DR1Continuous Conveyor Picking
- DR2Light-Guided Button Response
- DR3Periodic Gate Crossing and Withdrawal
- DR4Ball-on-Tray Transport and Placement
- PR1Linear Resistor Adjustment
- PR2Rotary Voltage Regulator Setting
- PR3Marker Line Tracing
- PR4Cloth Edge Alignment and Placement
- PR5Oriented Stamp Placement
- PR6Domino Chain Bell Ringing
Mind2Act
- CP01Conveyor Meal Assembly
- CP02Rotating Cake Decoration
- CP03Dynamic Rack Loading
- CP04Dual-Arm Cooking
- CP05Keyboard Sequence Reproduction
- CP06Produce Weighing and Balancing
- CP07Bridge Construction
- CP08Reference-Guided Block Construction
- CP09Whiteboard Drawing Reconstruction
- CP10Predictive Ball Return and Red-Ball Avoidance
02Evaluation
Evaluation protocol
We run every model ourselves, on hidden cases with fixed time budgets, and score the physical result.
Tracks
End-to-end models (VLA/WAM) and agents run on separate tracks, with different case counts and budgets.
Counts are per model. A configuration is one task at one difficulty in one setting.
| Measure | VLA / WAM | Agent |
|---|---|---|
| Action interface | Robot actions | Actions through tools |
| Cases / config. | 50 | 5 |
| Cases / setting | 4,500 | 450 |
| Total cases | 9,000 | 900 |
Cases and environments
A case fixes the task, level, seed, scene and instruction. Evaluation seeds are never used for training. Each case contributes one accepted result, obtained on our machines with versioned code and assets.
Clean and Random
Clean uses the original scene. Random varies materials and lighting and adds physical clutter, while preserving the task goals and scoring rules.


Random varies table materials (23 options), lighting (6 light types) and physical clutter. Lights keep their positions. Each colour channel ranges from 0 to 2 times white, and intensity from 0.8 to 1.2 times its original value.
Clean and Random cases come in pairs: same task objects and poses, robot, camera, physics settings, seed, instruction and scoring. The Random scene of a case is fixed, so every model sees the same one. The leaderboard shows the mean of the two settings; each model page shows both.
Scoring
A successful case scores 100. Other outcomes receive the partial credit defined for that task, or zero. The individual task pages show the success conditions and score values for each difficulty beside the videos.
- Task score
- The mean score over the task’s fixed case set, including zeros and partial credit.
- Success rate
- The percentage of evaluation cases that score 100.
- Suite score
- The mean over the ten tasks in Mind, Act or Mind2Act.
- Overall score
- 0.25 × Mind + 0.25 × Act + 0.5 × Mind2Act
Leaderboard values average Clean and Random. Overall success rate is not weighted: it is the number of successful cases divided by the fixed case count, so Mind, Act and Mind2Act each count for a third.
Only what happens in the scene counts. An object has to rest inside its container; sending the command is not enough. Buttons, keys and levers count when the mechanism itself moves.
Budgets and termination
Control and RGB run at 30 Hz and physics at 120 Hz: one control step is four physics steps, 1/30 s. Budgets are counted in physics steps, so model latency never consumes task time.
| Limit | VLA / WAM | Agent |
|---|---|---|
| Physics budget | 1.2 × demonstration P90 | 4 × VLA / WAM budget |
| Decisions | Unlimited | 150 |
| Wall-clock limit | 2 h | 6 h |
Physics budgets use the 90th-percentile duration of training demonstrations for each task and difficulty, rounded to whole control steps. Clean and Random use the same budget.
An agent case ends when either the physics budget or 150 decisions are used. Rejected and invalid actions count as decisions; network retries do not, with up to 12 retries per decision. Wall-clock limits are unscored safety stops.
Two kinds of ending. Outcome tasks (22) check the success condition after every action, end at 100 when it is met, and otherwise run to the end of the budget and score the final state. Event tasks (8) score mechanism events and end when the prescribed sequence is complete.
Scores reflect final outcomes and task-specific events during execution. Containment follows each task’s rules; buttons, keys and levers count once per press of the mechanism itself.
Validity and integrity
Scores and success rates use the fixed evaluation set. An accepted rerun replaces the result for the same case without changing the case count.
valid- Included in the reported scores and success rates, including zero-score outcomes.
infra_failed- Rendering, asset, process, controller or snapshot failure. The same case is rerun; its accepted result replaces the previous attempt.
operator_cancelled- Stopped by an operator. The attempt is retained, and the case remains in the evaluation set.
Results report the number of valid cases and infrastructure failures.
03Results
Leaderboard
VLA/WAM and agents are ranked separately.
| Metric | Definition |
|---|---|
| Overall score | 0.25 × Mind + 0.25 × Act + 0.5 × Mind2Act |
| Suite score | Mean score across the suite’s ten tasks. |
| Success rate | Percentage of evaluation cases scoring 100. |
| Settings | Mean of Clean and Random. |
04Participation
Release & submission
Code and setup guides are available on GitHub, with training data on Hugging Face. For evaluation inquiries, contact Mind2ActCommittee@outlook.com.
What comes next
Sandbox agent leaderboard. Agents on the current leaderboard run inside our in-house harness. A sandbox setting with its own leaderboard will let participants bring their own agent harness, which runs in an isolated container with access only to the cameras, the robot state and joint-level control. Submission details will follow.
Mind2Act Mini World. A smaller, fully open-source version of the benchmark will make training and evaluation easier to run on your own hardware.