Diagnosis

Profile cards for selected models

A success rate shows how often a model completes a task, but not where it goes wrong. As part of Mind2Act World, we publish model profile cards for selected models with distinctive behaviour. A card locates where a model’s episodes break down, explains why, and points to the capabilities worth improving first.

From model profile card to a better model

  1. Your model

    End-to-endRound 1 of 2

    • Mind
    • Act
    • Mind2Act
    1. 1 · Evaluate

      Three views

      • Mind
      • Act
      • Mind2Act
      Levels
      EasyMediumHard
      Scenes
      CleanRandom

      30 tasks × 3 levels × 2 scenes

    2. 2 · Diagnose In this project

      Model profile card

      Views

      Weak view

      Act

      Points to

      Targeted data

      First breaks at

      1. Perceive
      2. Reason
      3. Execute
      4. Close

      Why

      A-GRASP grasp not held

    3. 3 · What helps In development

      • End-to-endTargeted data
      • AgentExperience library

    4 · Improve and evaluate again In development

    Illustrative example. Evaluation and model profile cards are part of this project; the dashed steps, which feed a diagnosis back into the model, are ongoing work.

    Same score, different problems

    Illustrative comparison: two hypothetical models both score 0% on building a bridge from blocks, for different reasons.

    Score0%both models

    Model A

    Process
    stops at step 1 of 6
    First breaks at
    Execute the first grasp slips
    Why
    A-GRASP grasp not held
    Direction
    Grasp and lift demonstrations

    Model B

    Process
    reaches step 4 of 6
    First breaks at
    Reason places the deck before the piers
    Why
    M-PLAN wrong step order
    Direction
    Planning that checks prerequisites
    1. How far it getsA process score counts the milestones each episode reaches.
    2. Where it first failsThe first step it could not recover from: perceive, reason, execute or close the loop.
    3. Why, and a directionA failure code, its view (Mind, Act or Mind2Act), and a direction for the model's architecture.

    What the dimensions cover

    The card groups results by suite, capability and application domain. Each task is evaluated at Easy, Medium and Hard in Clean and Random scenes, 180 configurations in all.

    Mind

    Remember task information, learn rules from feedback and choose the next operation. Ten tasks, five capabilities.

    • Entity memory E1 E2
    • Process memory P1 P2
    • Spatial memory S1 S2
    • Feedback-rule learning F1 F2
    • Integrated memory M1 M2

    Act

    Execute timely, precise and stable actions under physical constraints. Ten tasks: dynamic response and precise execution.

    Mind2Act

    Use task intent to guide actions and execution feedback to update what comes next. Ten tasks, five capabilities.

    Application domains

    Five real-world settings the tasks are drawn from.

    Model profile cards

    Measured cards for three systems, one per paradigm: an end-to-end VLA, a world action model and an agent. Every failed episode is traced to the first step it could not recover from.

    1. Perceive read the scene
    2. Reason recall and decide
    3. Execute carry out the action
    4. Close the loop check and update

    End-to-end board

    G0.5 VLA

    Galaxea AI

    5.9%success over 9,000 episodes
    Act carries the successes; Mind is nearly empty past Easy.
    ENFP · Goalkeeper N, P weak
    01

    Where it breaks

    The first step a failed episode could not recover from, and which side failed.

    Where failures break

    first unrecoverable step

    • perceive2.9%
    • reason24.7%
    • execute41.4%
    • close the loop6.1%
    • episode level24.8%
    Which side failed

    share of failures

    • 27.7%Mind
    • 41.4%Act
    • 6.1%composition
    • 24.8%other
    02

    Why it fails

    The most frequent failure codes and the tasks where episodes stop.

    First failures

    most frequent failure codes

    1. A-GRASPgrasp failed or not held1,324
    2. M-MEMORYforgot or misremembered924
    3. X-STALLstalled or looped866
    4. A-PRECISIONfine-control error863
    Where tasks break

    episodes stopping there, three typical tasks

    1. 250/300E1 stall before submitting
    2. 210/300P1 forget the order
    3. 202/300CP07 never grasp a piece
    03

    What it can do

    Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.

    Capability profile

    30 tasks, 12 capabilities

    MindIntegrated memoryIntegratedM1 · Integrated memory · Easy: 0% successM1 · Integrated memory · Medium: 0% successM1 · Integrated memory · Hard: 0% successM1M2 · Integrated memory · Easy: 10–29% successM2 · Integrated memory · Medium: 1–9% successM2 · Integrated memory · Hard: 0% successM2Spatial memorySpatialS1 · Spatial memory · Easy: 0% successS1 · Spatial memory · Medium: 0% successS1 · Spatial memory · Hard: 0% successS1S2 · Spatial memory · Easy: 1–9% successS2 · Spatial memory · Medium: 0% successS2 · Spatial memory · Hard: 0% successS2Process memoryProcessP1 · Process memory · Easy: 0% successP1 · Process memory · Medium: 0% successP1 · Process memory · Hard: 0% successP1P2 · Process memory · Easy: 1–9% successP2 · Process memory · Medium: 1–9% successP2 · Process memory · Hard: 0% successP2Entity memoryEntityE1 · Entity memory · Easy: 0% successE1 · Entity memory · Medium: 0% successE1 · Entity memory · Hard: 0% successE1E2 · Entity memory · Easy: 0% successE2 · Entity memory · Medium: 0% successE2 · Entity memory · Hard: 0% successE2Feedback-rule learningRuleF1 · Feedback-rule learning · Easy: 10–29% successF1 · Feedback-rule learning · Medium: 0% successF1 · Feedback-rule learning · Hard: 0% successF1F2 · Feedback-rule learning · Easy: 1–9% successF2 · Feedback-rule learning · Medium: 0% successF2 · Feedback-rule learning · Hard: 0% successF2ActDynamic responseDynamicDR1 · Dynamic response · Easy: 60% or more successDR1 · Dynamic response · Medium: 1–9% successDR1 · Dynamic response · Hard: 0% successDR1DR2 · Dynamic response · Easy: 30–59% successDR2 · Dynamic response · Medium: 30–59% successDR2 · Dynamic response · Hard: 30–59% successDR2DR3 · Dynamic response · Easy: 30–59% successDR3 · Dynamic response · Medium: 30–59% successDR3 · Dynamic response · Hard: 30–59% successDR3DR4 · Dynamic response · Easy: 0% successDR4 · Dynamic response · Medium: 0% successDR4 · Dynamic response · Hard: 0% successDR4Precise executionPrecisionPR1 · Precise execution · Easy: 1–9% successPR1 · Precise execution · Medium: 1–9% successPR1 · Precise execution · Hard: 0% successPR1PR2 · Precise execution · Easy: 1–9% successPR2 · Precise execution · Medium: 1–9% successPR2 · Precise execution · Hard: 0% successPR2PR3 · Precise execution · Easy: 0% successPR3 · Precise execution · Medium: 0% successPR3 · Precise execution · Hard: 0% successPR3PR4 · Precise execution · Easy: 30–59% successPR4 · Precise execution · Medium: 0% successPR4 · Precise execution · Hard: 1–9% successPR4PR5 · Precise execution · Easy: 0% successPR5 · Precise execution · Medium: 0% successPR5 · Precise execution · Hard: 0% successPR5PR6 · Precise execution · Easy: 0% successPR6 · Precise execution · Medium: 0% successPR6 · Precise execution · Hard: 0% successPR6Mind2ActPredictive decisionPredictCP10 · Predictive decision · Easy: 1–9% successCP10 · Predictive decision · Medium: 0% successCP10 · Predictive decision · Hard: 0% successCP10Sequence and planPlanCP05 · Sequence and plan · Easy: 0% successCP05 · Sequence and plan · Medium: 0% successCP05 · Sequence and plan · Hard: 0% successCP05CP07 · Sequence and plan · Easy: 0% successCP07 · Sequence and plan · Medium: 0% successCP07 · Sequence and plan · Hard: 0% successCP07Feedback adjustmentFeedbackCP04 · Feedback adjustment · Easy: 0% successCP04 · Feedback adjustment · Medium: 0% successCP04 · Feedback adjustment · Hard: 0% successCP04CP06 · Feedback adjustment · Easy: 30–59% successCP06 · Feedback adjustment · Medium: 1–9% successCP06 · Feedback adjustment · Hard: 1–9% successCP06Prerequisites and repairRepairCP02 · Prerequisites and repair · Easy: 0% successCP02 · Prerequisites and repair · Medium: 0% successCP02 · Prerequisites and repair · Hard: 0% successCP02CP08 · Prerequisites and repair · Easy: 0% successCP08 · Prerequisites and repair · Medium: 0% successCP08 · Prerequisites and repair · Hard: 0% successCP08CP09 · Prerequisites and repair · Easy: 0% successCP09 · Prerequisites and repair · Medium: 0% successCP09 · Prerequisites and repair · Hard: 0% successCP09Cumulative progressCumulativeCP01 · Cumulative progress · Easy: 10–29% successCP01 · Cumulative progress · Medium: 0% successCP01 · Cumulative progress · Hard: 0% successCP01CP03 · Cumulative progress · Easy: 0% successCP03 · Cumulative progress · Medium: 0% successCP03 · Cumulative progress · Hard: 0% successCP03

    RingsEasy, Medium and Hard, outward

    Success01+10+30+60+
    Mind
    Act
    Mind2Act

    Act carries the successes; Mind nearly empty past Easy

    Under pressure

    success (%) by difficulty, bars 0–30%

    Mind

    • Easy5.6
    • Medium0.2
    • Hard0.0

    Act

    • Easy21.9
    • Medium9.8
    • Hard8.6

    Mind2Act

    • Easy6.2
    • Medium0.5
    • Hard0.2

    All tasks

    • 11.2%Easy
    • 3.5%Medium
    • 2.9%Hard
    04

    What could help

    Directions drawn from the failure codes and breakpoints above.

    1. 1Grasp and hold

      Grasp failures are its most frequent first failure (1,324), and 202 of 300 bridge-building episodes (CP07) never grasp a piece.

      Change
      a grip check before lifting, with re-grasp when the object slips.
      Data to add
      Contact-rich demonstrations of grasping small parts, including slips and the re-grasp that recovers them.
    2. 2Task state and the final step

      924 memory failures and 866 stalls: 210 of 300 meal-order episodes (P1) forget the order and 250 of 300 repeated-item episodes (E1) stall before submitting.

      Change
      a task-state memory that P1 checks before hand-over, and an escape from repeated motions so that E1 reaches its submission.
      Data to add
      Long demonstrations that carry orders and items across steps and end with the P1 hand-over or the E1 submission.

    End-to-end board

    InternW0 WAM

    Shanghai AI Lab

    8.7%success over 9,000 episodes
    Act holds up even at Hard; Mind and Mind2Act fade past Easy.
    ENFP · Goalkeeper
    01

    Where it breaks

    The first step a failed episode could not recover from, and which side failed.

    Where failures break

    first unrecoverable step

    • perceive5.5%
    • reason27.3%
    • execute37.2%
    • close the loop10.5%
    • episode level19.4%
    Which side failed

    share of failures

    • 32.8%Mind
    • 37.2%Act
    • 10.5%composition
    • 19.4%other
    02

    Why it fails

    The most frequent failure codes and the tasks where episodes stop.

    First failures

    most frequent failure codes

    1. A-PRECISIONfine-control error1,132
    2. M-MEMORYforgot or misremembered1,011
    3. A-GRASPgrasp failed or not held806
    4. C-PREREQacted before a precondition held761
    Where tasks break

    episodes stopping there, three typical tasks

    1. 271/300CP04 heat too early
    2. 288/299E2 misapply the keypad rule
    3. 245/298P1 forget the order
    03

    What it can do

    Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.

    Capability profile

    30 tasks, 12 capabilities

    MindIntegrated memoryIntegratedM1 · Integrated memory · Easy: 0% successM1 · Integrated memory · Medium: 0% successM1 · Integrated memory · Hard: 0% successM1M2 · Integrated memory · Easy: 30–59% successM2 · Integrated memory · Medium: 1–9% successM2 · Integrated memory · Hard: 0% successM2Spatial memorySpatialS1 · Spatial memory · Easy: 1–9% successS1 · Spatial memory · Medium: 0% successS1 · Spatial memory · Hard: 0% successS1S2 · Spatial memory · Easy: 10–29% successS2 · Spatial memory · Medium: 1–9% successS2 · Spatial memory · Hard: 1–9% successS2Process memoryProcessP1 · Process memory · Easy: 0% successP1 · Process memory · Medium: 1–9% successP1 · Process memory · Hard: 1–9% successP1P2 · Process memory · Easy: 1–9% successP2 · Process memory · Medium: 1–9% successP2 · Process memory · Hard: 0% successP2Entity memoryEntityE1 · Entity memory · Easy: 0% successE1 · Entity memory · Medium: 0% successE1 · Entity memory · Hard: 0% successE1E2 · Entity memory · Easy: 1–9% successE2 · Entity memory · Medium: 0% successE2 · Entity memory · Hard: 0% successE2Feedback-rule learningRuleF1 · Feedback-rule learning · Easy: 10–29% successF1 · Feedback-rule learning · Medium: 1–9% successF1 · Feedback-rule learning · Hard: 1–9% successF1F2 · Feedback-rule learning · Easy: 30–59% successF2 · Feedback-rule learning · Medium: 1–9% successF2 · Feedback-rule learning · Hard: 1–9% successF2ActDynamic responseDynamicDR1 · Dynamic response · Easy: 30–59% successDR1 · Dynamic response · Medium: 30–59% successDR1 · Dynamic response · Hard: 0% successDR1DR2 · Dynamic response · Easy: 30–59% successDR2 · Dynamic response · Medium: 30–59% successDR2 · Dynamic response · Hard: 10–29% successDR2DR3 · Dynamic response · Easy: 60% or more successDR3 · Dynamic response · Medium: 60% or more successDR3 · Dynamic response · Hard: 60% or more successDR3DR4 · Dynamic response · Easy: 0% successDR4 · Dynamic response · Medium: 0% successDR4 · Dynamic response · Hard: 0% successDR4Precise executionPrecisionPR1 · Precise execution · Easy: 30–59% successPR1 · Precise execution · Medium: 30–59% successPR1 · Precise execution · Hard: 10–29% successPR1PR2 · Precise execution · Easy: 1–9% successPR2 · Precise execution · Medium: 1–9% successPR2 · Precise execution · Hard: 1–9% successPR2PR3 · Precise execution · Easy: 0% successPR3 · Precise execution · Medium: 0% successPR3 · Precise execution · Hard: 0% successPR3PR4 · Precise execution · Easy: 1–9% successPR4 · Precise execution · Medium: 10–29% successPR4 · Precise execution · Hard: 0% successPR4PR5 · Precise execution · Easy: 0% successPR5 · Precise execution · Medium: 0% successPR5 · Precise execution · Hard: 0% successPR5PR6 · Precise execution · Easy: 0% successPR6 · Precise execution · Medium: 0% successPR6 · Precise execution · Hard: 0% successPR6Mind2ActPredictive decisionPredictCP10 · Predictive decision · Easy: 1–9% successCP10 · Predictive decision · Medium: 0% successCP10 · Predictive decision · Hard: 0% successCP10Sequence and planPlanCP05 · Sequence and plan · Easy: 0% successCP05 · Sequence and plan · Medium: 0% successCP05 · Sequence and plan · Hard: 0% successCP05CP07 · Sequence and plan · Easy: 0% successCP07 · Sequence and plan · Medium: 0% successCP07 · Sequence and plan · Hard: 0% successCP07Feedback adjustmentFeedbackCP04 · Feedback adjustment · Easy: 0% successCP04 · Feedback adjustment · Medium: 0% successCP04 · Feedback adjustment · Hard: 0% successCP04CP06 · Feedback adjustment · Easy: 30–59% successCP06 · Feedback adjustment · Medium: 1–9% successCP06 · Feedback adjustment · Hard: 0% successCP06Prerequisites and repairRepairCP02 · Prerequisites and repair · Easy: 1–9% successCP02 · Prerequisites and repair · Medium: 0% successCP02 · Prerequisites and repair · Hard: 0% successCP02CP08 · Prerequisites and repair · Easy: 10–29% successCP08 · Prerequisites and repair · Medium: 1–9% successCP08 · Prerequisites and repair · Hard: 1–9% successCP08CP09 · Prerequisites and repair · Easy: 0% successCP09 · Prerequisites and repair · Medium: 0% successCP09 · Prerequisites and repair · Hard: 0% successCP09Cumulative progressCumulativeCP01 · Cumulative progress · Easy: 1–9% successCP01 · Cumulative progress · Medium: 1–9% successCP01 · Cumulative progress · Hard: 0% successCP01CP03 · Cumulative progress · Easy: 0% successCP03 · Cumulative progress · Medium: 0% successCP03 · Cumulative progress · Hard: 0% successCP03

    RingsEasy, Medium and Hard, outward

    Success01+10+30+60+
    Mind
    Act
    Mind2Act

    Act holds up even at Hard; Mind and M2A fade past Easy

    Under pressure

    success (%) by difficulty, bars 0–30%

    Mind

    • Easy13.5
    • Medium2.1
    • Hard0.5

    Act

    • Easy20.6
    • Medium19.5
    • Hard12.7

    Mind2Act

    • Easy8.0
    • Medium0.8
    • Hard0.3

    All tasks

    • 14.0%Easy
    • 7.5%Medium
    • 4.5%Hard
    04

    What could help

    Directions drawn from the failure codes and breakpoints above.

    1. 1Precise contact

      Fine-control errors (1,132) and grasp failures (806) make up most of its Act failures.

      Change
      shorter action chunks and closed-loop correction near contact, where its grasps and fine alignment fail.
      Data to add
      Demonstrations of grasping and fine alignment, with the corrective motions near the target.
    2. 2Memory and preconditions

      1,011 memory and 761 precondition failures: 271 of 300 cooking episodes (CP04) heat before all ingredients are in the pot, and 245 of 298 failed meal-order episodes (P1) forget the order.

      Change
      a task-state memory and a precondition check: all ingredients in the pot before HEAT in CP04, and the order checked before hand-over in P1.
      Data to add
      Demonstrations that check the pot before heating, and long episodes that carry orders and rules across steps.

    Agent board

    GPT-6 Astra Agent

    OpenAI

    34.6%success over 900 episodes
    Mind and the Easy rings fill up; DR4 and PR6 stay empty.
    ISFJ · Dispatcher
    01

    Where it breaks

    The first step a failed episode could not recover from, and which side failed.

    Where failures break

    first unrecoverable step

    • perceive5.6%
    • reason24.8%
    • execute55.3%
    • close the loop7.1%
    • episode level7.1%
    Which side failed

    share of failures

    • 30.4%Mind
    • 55.3%Act
    • 7.1%composition
    • 7.1%other
    02

    Why it fails

    The most frequent failure codes and the tasks where episodes stop.

    First failures

    most frequent failure codes

    1. A-GRASPgrasp failed or not held120
    2. A-PRECISIONfine-control error107
    3. A-PLACEmisplaced on release47
    4. M-MEMORYforgot or misremembered45
    Where tasks break

    episodes stopping there, three typical tasks

    1. 30/30DR4 lose control in the lift
    2. 21/30CP04 fail to grasp a jar
    3. 21/30PR6 knock pieces while placing
    03

    What it can do

    Success on every task at Easy, Medium and Hard, and how it holds up as tasks get harder.

    Capability profile

    30 tasks, 12 capabilities

    MindIntegrated memoryIntegratedM1 · Integrated memory · Easy: 0% successM1 · Integrated memory · Medium: 0% successM1 · Integrated memory · Hard: 0% successM1M2 · Integrated memory · Easy: 60% or more successM2 · Integrated memory · Medium: 60% or more successM2 · Integrated memory · Hard: 60% or more successM2Spatial memorySpatialS1 · Spatial memory · Easy: 10–29% successS1 · Spatial memory · Medium: 30–59% successS1 · Spatial memory · Hard: 10–29% successS1S2 · Spatial memory · Easy: 60% or more successS2 · Spatial memory · Medium: 60% or more successS2 · Spatial memory · Hard: 60% or more successS2Process memoryProcessP1 · Process memory · Easy: 60% or more successP1 · Process memory · Medium: 10–29% successP1 · Process memory · Hard: 10–29% successP1P2 · Process memory · Easy: 0% successP2 · Process memory · Medium: 0% successP2 · Process memory · Hard: 0% successP2Entity memoryEntityE1 · Entity memory · Easy: 60% or more successE1 · Entity memory · Medium: 30–59% successE1 · Entity memory · Hard: 60% or more successE1E2 · Entity memory · Easy: 60% or more successE2 · Entity memory · Medium: 60% or more successE2 · Entity memory · Hard: 60% or more successE2Feedback-rule learningRuleF1 · Feedback-rule learning · Easy: 60% or more successF1 · Feedback-rule learning · Medium: 60% or more successF1 · Feedback-rule learning · Hard: 60% or more successF1F2 · Feedback-rule learning · Easy: 60% or more successF2 · Feedback-rule learning · Medium: 60% or more successF2 · Feedback-rule learning · Hard: 10–29% successF2ActDynamic responseDynamicDR1 · Dynamic response · Easy: 60% or more successDR1 · Dynamic response · Medium: 60% or more successDR1 · Dynamic response · Hard: 60% or more successDR1DR2 · Dynamic response · Easy: 60% or more successDR2 · Dynamic response · Medium: 60% or more successDR2 · Dynamic response · Hard: 60% or more successDR2DR3 · Dynamic response · Easy: 60% or more successDR3 · Dynamic response · Medium: 60% or more successDR3 · Dynamic response · Hard: 60% or more successDR3DR4 · Dynamic response · Easy: 0% successDR4 · Dynamic response · Medium: 0% successDR4 · Dynamic response · Hard: 0% successDR4Precise executionPrecisionPR1 · Precise execution · Easy: 60% or more successPR1 · Precise execution · Medium: 60% or more successPR1 · Precise execution · Hard: 60% or more successPR1PR2 · Precise execution · Easy: 60% or more successPR2 · Precise execution · Medium: 30–59% successPR2 · Precise execution · Hard: 10–29% successPR2PR3 · Precise execution · Easy: 0% successPR3 · Precise execution · Medium: 0% successPR3 · Precise execution · Hard: 0% successPR3PR4 · Precise execution · Easy: 0% successPR4 · Precise execution · Medium: 0% successPR4 · Precise execution · Hard: 0% successPR4PR5 · Precise execution · Easy: 0% successPR5 · Precise execution · Medium: 0% successPR5 · Precise execution · Hard: 0% successPR5PR6 · Precise execution · Easy: 0% successPR6 · Precise execution · Medium: 0% successPR6 · Precise execution · Hard: 0% successPR6Mind2ActPredictive decisionPredictCP10 · Predictive decision · Easy: 0% successCP10 · Predictive decision · Medium: 0% successCP10 · Predictive decision · Hard: 0% successCP10Sequence and planPlanCP05 · Sequence and plan · Easy: 60% or more successCP05 · Sequence and plan · Medium: 0% successCP05 · Sequence and plan · Hard: 0% successCP05CP07 · Sequence and plan · Easy: 0% successCP07 · Sequence and plan · Medium: 0% successCP07 · Sequence and plan · Hard: 0% successCP07Feedback adjustmentFeedbackCP04 · Feedback adjustment · Easy: 0% successCP04 · Feedback adjustment · Medium: 0% successCP04 · Feedback adjustment · Hard: 0% successCP04CP06 · Feedback adjustment · Easy: 30–59% successCP06 · Feedback adjustment · Medium: 0% successCP06 · Feedback adjustment · Hard: 0% successCP06Prerequisites and repairRepairCP02 · Prerequisites and repair · Easy: 30–59% successCP02 · Prerequisites and repair · Medium: 10–29% successCP02 · Prerequisites and repair · Hard: 0% successCP02CP08 · Prerequisites and repair · Easy: 30–59% successCP08 · Prerequisites and repair · Medium: 0% successCP08 · Prerequisites and repair · Hard: 0% successCP08CP09 · Prerequisites and repair · Easy: 0% successCP09 · Prerequisites and repair · Medium: 0% successCP09 · Prerequisites and repair · Hard: 0% successCP09Cumulative progressCumulativeCP01 · Cumulative progress · Easy: 60% or more successCP01 · Cumulative progress · Medium: 30–59% successCP01 · Cumulative progress · Hard: 30–59% successCP01CP03 · Cumulative progress · Easy: 0% successCP03 · Cumulative progress · Medium: 0% successCP03 · Cumulative progress · Hard: 0% successCP03

    RingsEasy, Medium and Hard, outward

    Success01+10+30+60+
    Mind
    Act
    Mind2Act

    Mind and Easy rings fill up; DR4 and PR6 stay empty

    Under pressure

    success (%) by difficulty, bars 0–70%

    Mind

    • Easy64.0
    • Medium50.0
    • Hard44.0

    Act

    • Easy44.0
    • Medium39.0
    • Hard34.0

    Mind2Act

    • Easy26.0
    • Medium7.0
    • Hard3.0

    All tasks

    • 44.7%Easy
    • 32.0%Medium
    • 27.0%Hard
    04

    What could help

    Directions drawn from the failure codes and breakpoints above.

    1. 1Finer grasp and pose tools

      Act accounts for 55% of its failures: grasp (120), fine control (107) and placement (47); all 30 lift episodes of DR4 lose control.

      Tools
      grasp-quality feedback and small relative pose moves, so a grasp can be checked and corrected.
      Library to add
      failed grasps and lifts with the observation that showed the slip, retrieved as examples.
    2. 2Verify and retry before stopping

      Typical tasks stop at one physical step: 21 of 30 cooking episodes (CP04) fail to grasp a jar and 21 of 30 domino episodes (PR6) knock pieces while placing.

      Tools
      an observation after the tray lift (DR4), the jar grasp (CP04) and each domino placement (PR6), and a retry before the episode is ended.
      Library to add
      recovered attempts (slip, check, retry) from earlier episodes of the same task.