How to Evaluate General-Purpose Robot Policies for Real-World Deployment

Robotics foundation models have made remarkable progress. Today’s best systems can follow natural language instructions to pick, place, sort, and manipulate a wide variety of objects. But as these models grow more capable, evaluating them rigorously has become one of the field’s hardest unsolved problems. In this blog post, we introduce the key problems and our method for addressing them.