Why Complex, Multi-Step Tasks Are the Hardest Problem in Humanoid Robotics

_{Guest contributor: Nikita Rudin is Flexion's co-founder and CEO. Formerly at NVIDIA and ETH Zurich, he now leads Flexion in building AI systems for humanoid robots.}

Last month, we gave a humanoid robot a single instruction in plain language:

A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area.

Then we stepped back.

Without teleoperation, a scripted sequence, or anyone correcting it off camera, the robot navigated a real multi-floor building, opened doors, called the elevator, unpacked the box, and stocked the shelf. When things didn't go exactly as expected, it adapted and continued. Four minutes later, the task was complete.

On the surface, it's an ordinary office chore, but I think it illustrates a significant technological leap in humanoid robotics: long-horizon autonomy – or complex, multi-step tasks. 

Robots today can already perform many individual skills that seemed out of reach only a few years ago – from following natural-language instructions to navigating buildings and manipulating objects. The harder challenge is completing an extended task that requires all of those abilities to work together without human intervention. That's the difference between a robot that can perform an impressive demonstration and one that can do useful work in the real world. For example, a robot on a factory floor that needs someone to redirect it every thirty seconds isn't actually saving anyone time, because someone still has to stand there and run it.

A real-world job involves a sequence of decisions. The robot has to interpret instructions, navigate unfamiliar spaces, manipulate objects, recover when conditions change, and continue toward its objective without someone stepping in. Every successful action creates new opportunities for failure, making reliability over time fundamentally different from mastering isolated skills.

Robots have climbed stairs and opened doors before, so those capabilities alone aren't what made our demo notable. What made it different was asking a robot to combine language understanding, navigation, manipulation, planning, and error recovery into a single, continuous task in a building designed for people rather than robots. It's that integration of capabilities in a real-world environment that represents something new.

It’s also much closer to the work robots will eventually need to perform in factories, warehouses, hospitals, hotels, and commercial buildings. In these environments, the challenge is rarely executing a single movement correctly, but completing a sequence of decisions in an environment that changes over time.

The system behind our demo, which we call Reflect, combines language understanding, perception, motion planning, whole-body control, and reinforcement learning into a single autonomous stack. Rather than optimizing each capability independently, we optimize for successful completion of the entire task.

We saw the importance of that approach in a 16-step internal evaluation. Off-the-shelf VLMs (Vision-Language Model), even when prompted carefully and given in-context examples, failed almost immediately, completing the task 0 percent of the time. A mission controller trained only with supervised fine-tuning completed the task successfully 38 percent of the time; adding reinforcement learning increased completion to 90 percent. That improvement wasn't about making any one capability dramatically better, but making the entire sequence more reliable.

A huge challenge in building long-horizon autonomy comes down to how you train the robot. Much of today's industry relies on teleoperation to collect demonstrations. Those demonstrations have been valuable for teaching robots new behaviors, but scaling them across every possible task, environment, and hardware platform requires significant human effort.

Reinforcement learning in simulation offers another path. Instead of collecting every example manually, robots can accumulate millions of experiences in virtual environments before entering the physical world. Simulation doesn't replace real-world data, but it dramatically expands the range of situations robots can experience and helps them learn to recover from mistakes that would be impractical to demonstrate by hand.

There is still a long road ahead. Our system operates within a defined set of tasks rather than arbitrary real-world situations, some objects remain difficult to manipulate reliably, and recovery behaviors will continue to improve. But those limitations reinforce the point that long-horizon autonomy is still an unsolved engineering problem, and every failure reveals where the next advances need to come from.

The benchmark for humanoid robotics will increasingly become simple to describe: can you give a robot an objective, walk away, and trust that it will finish?

Hardware will also continue to improve across the industry – as it should – but the systems that consistently complete long, complex tasks without human intervention will define the next generation of useful humanoid robots.