Skip to content

RSL-RL

Bases: Wrapper, VecEnv

A wrapper that makes your genesis forge environment compatible with the rsl_rl training framework.

Warning

This should be the last wrapper, as the change in the step and get_observations methods might break other wrappers.

What it does
  • Combines the terminated and truncated tensors into a single tensor (i.e. terminated | truncated).
  • Add the truncated tensor to the extras dictionary as "time_outs".
  • Returns observations and extras from the get_observations method.

Parameters:

Name Type Description Default
env GenesisEnv | Wrapper

The environment to wrap.

 required
cfg dict | object | None

The logger configuration that RSL-RL will pass to the external logging service (neptune, wandb, etc)

 None
Source code in genesis_forge/wrappers/rsl_rl.py 
34
35
36
37
38
39
40
41
42
43
44
def __init__(self, env: GenesisEnv | Wrapper, cfg: dict | object | None = None):
    super().__init__(env)

    self.rsl3 = False
    self.cfg = cfg if cfg is not None else {}
    try:
        major_version = int(metadata.version("rsl-rl-lib").split(".")[0])
        if major_version >= 3:
            self.rsl3 = True
    except:  # noqa
        pass

action_space property

The action space of the environment.

can_be_wrapped = False class-attribute instance-attribute

cfg = cfg if cfg is not None else {} instance-attribute

device property

dt property

The time step of the environment.

env = env class-attribute instance-attribute

episode_length_buf property

extras property

The extras/infos dictionary that should be returned by the step and reset functions. This dictionary will be cleared at the start of every step.

max_episode_length property

num_actions property

The number of actions for each environment.

num_envs property

The number of parallel environments.

num_observations property

The number of observations for each environment.

observation_space property

The observation space of the environment.

robot property

Get the environment robot.

rsl3 = False instance-attribute

scene property

Get the environment scene.

unwrapped property

Returns the base environment of the wrapper.

This will be the bare :class:ManagedEnvironment or :class:GenesisEnv environment, underneath all layers of wrappers.

build()

Builds the scene and other supporting components necessary for the training environment. This assumes that the scene has already been constructed and assigned to the .scene attribute.

Source code in genesis_forge/wrappers/wrapper.py 
 97
 98
 99
100
101
102
def build(self) -> None:
    """
    Builds the scene and other supporting components necessary for the training environment.
    This assumes that the scene has already been constructed and assigned to the <env>.scene attribute.
    """
    self.env.build()

close()

Closes the wrapper and :attr:env.

Source code in genesis_forge/wrappers/wrapper.py 
121
122
123
def close(self):
    """Closes the wrapper and :attr:`env`."""
    return self.env.close()

get_observations()

Returns observations as well as an extras dictionary with the observations added to the extras["observations"]["critic"] key.

Source code in genesis_forge/wrappers/rsl_rl.py 
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
def get_observations(self):
    """
    Returns observations as well as an extras dictionary with the observations added to the `extras["observations"]["critic"]` key.
    """
    obs = self.env.get_observations()

    # rsl_rl 3.0+ just returns the observations
    if self.rsl3:
        obs = self._format_obs_group(obs, self.env.extras)
        return obs

    # Earlier versions of rsl_rl adds critic observations to the extras dictionary
    extras = self._add_observations_to_extras(obs, self.env.extras)
    return obs, extras

reset()

Converts observations into a TensorDict for rsl_rl 3.0+

Source code in genesis_forge/wrappers/rsl_rl.py 
84
85
86
87
88
89
90
def reset(self):
    """
    Converts observations into a TensorDict for rsl_rl 3.0+
    """
    (obs, extras) = self.env.reset()
    obs = self._format_obs_group(obs, extras)
    return obs, extras

step(actions)

Returns a single "dones" tensor, instead of the terminated and truncated tensors (via terminated | truncated). Add the truncated tensor to the extras dictionary as "time_outs".

Source code in genesis_forge/wrappers/rsl_rl.py 
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def step(
    self, actions: torch.Tensor
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, dict[str, Any]]:
    """
    Returns a single "dones" tensor, instead of the terminated and truncated tensors (via `terminated | truncated`).
    Add the truncated tensor to the extras dictionary as "time_outs".
    """
    (
        obs,
        rewards,
        terminated,
        truncated,
        extras,
    ) = super().step(actions)

    # Combine terminated and truncated
    dones = terminated | truncated

    # Add observations and timeouts to extras
    if extras is None:
        extras = {}
    extras = self._add_observations_to_extras(obs, extras)

    obs = self._format_obs_group(obs, extras)
    return obs, rewards, dones, extras