Why doesn’t the game rely on traditional enemy patterns or scripted encounters?
Martin Singh-Blom: We can’t really do that because of the physics.
We’ve tried building more traditional behavior systems, but as soon as something unexpected happens, like an enemy getting pushed, it breaks those patterns.
It all comes back to physics. The physics adds a layer of unpredictability that seeps into the design’s fundamentals. So we had to move away from that approach entirely.
How does machine learning fit into the overall AI system?
Martin Singh-Blom: The machine learning part is actually more limited than people think. It’s strictly for locomotion, such as how the robot places its feet and moves. That’s a very hard problem, and traditional methods don’t work well for legged robots, so we had to go to the research frontier and use reinforcement learning.
For drones, we don’t need that since we can use traditional control systems, as you’d see in real-world drones. But for legged robots, we haven’t found any other way that works.
Once you move into higher-level decisions, like where to go or what to do, that’s handled by more traditional systems like behavior trees.
There’s a boundary between the two. For example, if there’s a box in front of the robot, the behavior tree might decide that it wants to move forward, but the locomotion system decides how to get there, whether to go over the box or around it.
As the models improve, we can push more decision-making into the machine learning side. That’s where it gets interesting, because the robot can start making its own decisions, like deciding to squeeze through a space or jump over something, and that creates more surprising situations for players.”
