Reinforcement Learning — From Simulation to Deployment on the Robot Platform Spot

Title: Reinforcement Learning — From Simulation to Deployment on the Robot Platform Spot
Author: Markus Lejon and Fredrik Sundt
Date & Time: June 15th, 9:00-10:00
Location: Seminar Room M 3170-73 in the M-building, LTH
Supervisor: Björn Olofsson, Volker Krueger (Dept. Computer Science)
Examiner:  Karl-Erik Årzén


Abstract:
Recent advancements in GPU-accelerated simulation have made it possible to train reinforcement learning policies for robotic control using large-scale parallel data collection. This thesis explores the pipeline from task description to deployment of control policies on the robot platform Spot. The focus is on two important skills needed for future loco-manipulation tasks: robust standing and command-following locomotion. The thesis presents the design choices made when creating an Isaac Lab environment, including the observation space, action space, reward functions, and deployment setup. The resulting policies were tested both in simulation and on real Spot hardware using the Spot RL Researcher Kit. The experiments showed that the trained policies could be deployed on hardware and that successful locomotion was possible. At the same time, a sim-to-real gap was observed, especially in the joint tracking and load-bearing behaviour of the robot. The thesis therefore shows both the potential of the pipeline and the practical challenges involved in transferring reinforcement learning policies from simulation to real hardware deployment.