RSL-RL

RSL-RL is a GPU-accelerated, lightweight learning library for robotics research. Its compact design allows researchers to prototype and test new ideas without the overhead of modifying large, complex libraries. RSL-RL can also be used out-of-the-box by installing it via PyPI, supports multi-GPU training, and features common algorithms for robot learning.

Key Features

• Minimal, readable codebase with clear extension points for rapid prototyping.
• Robotics-first methods including PPO and Student-Teacher Distillation.
• High-throughput training with native Multi-GPU support.
• Proven performance in numerous research publications.

Extensions (this fork)

This repository extends upstream RSL-RL for mjlab / humanoid RL research. Highlights:

• AMP (PPOAMP, AMPRunner): adversarial motion prior with task + style reward mixing; discriminator supports BCE / LSGAN / WGAN-GP and optional adaptive style scale.
• AMP replay: DiscReplayBuffer and replay helpers for terminal / demo disc observations.
• ASE (PPOASE, ASERunner): AMP + encoder / MI-style reward for transfer and style learning.
• SMP (PPOSMP, SMPRunner): task reward + frozen diffusion prior (SDS-style style reward), no online discriminator.
• Decoupled PPO (PPODecoupled): separate lower-body / upper-body policies for locomotion + manipulation.
• FPO + AMP (FPOAMP): flow-policy variant with AMP-style discriminator training.
• Models / utils: MoEMLPModel, ASEDiscriminatorEncoder, AMP / ASE / SMP loggers, diff normalizer.

Used together with unitree_rl_mjlab tasks such as velocity tracking, AMP-v2, ASE, and SMP.

Learning Environments

RSL-RL is currently used by the following robot learning libraries:

• Isaac Lab (built on top of NVIDIA Isaac Sim)
• Legged Gym (built on top of NVIDIA Isaac Gym)
• mjlab (built on top of MuJoCo Warp)
• MuJoCo Playground (built on top of MuJoCo MJX and Warp)

Installation

Before installing RSL-RL, ensure that Python 3.9+ is available. It is recommended to install the library in a virtual environment (e.g. using venv or conda), which is often already created by the used environment library (e.g. Isaac Lab). If so, make sure to activate it before installing RSL-RL.

Installing RSL-RL as a dependency

pip install rsl-rl-lib

Installing RSL-RL for development

git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .

Citation

If you use RSL-RL in your research, please cite the paper:

@article{schwarke2025rslrl,
  title={RSL-RL: A Learning Library for Robotics Research},
  author={Schwarke, Clemens and Mittal, Mayank and Rudin, Nikita and Hoeller, David and Hutter, Marco},
  journal={arXiv preprint arXiv:2509.10771},
  year={2025}
}