Planner + step follower vs pt50, side by side

BEHAVIOR-1K 2026 challenge, public test instances 301 to 310, one rollout each

The hybrid run puts a timer planner in front of the c_gp1k_999 step follower. The planner decides which step of the task comes next and the follower turns that step into actions. This page compares it with pt50, our current submission, on a fixed set of five tasks.

Results on the 5-task set

Sum of Q over the rollouts that finished

task	hybrid Q	hybrid rollouts	pt50 Q	pt50 rollouts
turning_on_radio	0	10	3.0	10
make_microwave_popcorn	0	10	1.0	10
setting_mousetraps	1.0	6	5.33	10
sweeping_garage	0	8	0	10
clean_a_trumpet	0	10	0	10
total	1.0	44	9.33	50

Six hybrid rollouts were lost to simulator segfaults, four on setting_mousetraps and two on sweeping_garage. They were not rerun, because full marks on all six would still bring the hybrid total to only 7.0, below the 9.33 that pt50 scores.

Takeaway

The planner advances steps at the right times and the robot goes for the right object, but the follower does not complete the steps. On radio 307 it hovers an open hand by the radio and times out at 3225 frames, while pt50 grasps the radio and presses the button in 1168.

We also repeated Yosub's Astra-style closed loop on picking_up_trash instance 311 with three planners. Argon scored 0.333 and 0, Opus scored 0 and 0, and the timer planner scored 0, 0 and 0. That matches his Astra result of 1/3, 0 and 0.

pt50 stays the submission.

Clips

pt50 plays on the left and the planner with the c_gp1k_999 follower on the right, on the same instance and seed. Each side shows two wrist views and the head camera, played at 2x speed, and each label carries that run's Q. The side that finishes first holds its last frame.

turning_on_radio

pt50 3.0, hybrid 0

instance 307, pt50 q 1.00 success, hybrid q 0.00, 2x speedpt50 grasps the radio and presses the button in 1168 frames. The hybrid robot reaches the radio but hovers an open hand beside it until the episode times out at 3225 frames.

instance 308, pt50 q 1.00 success, hybrid q 0.00, 2x speedpt50 picks up the radio and turns it on. The hybrid robot moves its hand to the radio on the table but never closes the grasp.

instance 310, pt50 q 1.00 success, hybrid q 0.00, 2x speedpt50 lifts the radio and turns it on. The hybrid robot targets the same radio but does not finish the grasp.

make_microwave_popcorn

pt50 1.0, hybrid 0

instance 301, pt50 q 1.00 success, hybrid q 0.00, 2x speedpt50 puts the popcorn in the microwave and finishes early. The hybrid robot handles the popcorn bag and works around the microwave but never completes the step.

setting_mousetraps

pt50 5.33, hybrid 1.04 hybrid rollouts lost to segfaults

instance 301, pt50 q 0.83, hybrid q 0.17, 2x speedBoth robots pick mousetraps off the cabinet and set them on the bathroom floor. pt50 places most of them, while the hybrid robot gets only a small part of the task done before time runs out.

instance 302, pt50 q 0.67, hybrid q 0.33, 2x speedBoth robots carry traps from the cabinet to the floor by the toilet. The hybrid robot places fewer of them than pt50.