Command Manager#

The Command Manager generates high-level commands for goal-conditioned RL tasks. It handles command generation, resampling, visualization, and gamepad control for interactive testing.

Generates random commands from specified ranges
Resamples commands at regular intervals
Provides debug visualization
Supports gamepad control for testing

You can see a full example using the command manager in examples/command_direction.

Velocity Command Manager#

The most common command manager is VelocityCommandManager for locomotion tasks:

from genesis_forge.managers.command import VelocityCommandManager

class MyEnv(ManagedEnvironment):
    def config(self):
        self.velocity_command = VelocityCommandManager(
            self,
            range={
                "lin_vel_x": [-1.0, 1.0],  # Forward/backward
                "lin_vel_y": [-0.5, 0.5],  # Left/right
                "ang_vel_z": [-1.0, 1.0],  # Rotation
            },
            resample_time_sec=10,  # Resample new commands every 10 seconds
            debug_visualizer=True, # Show command arrows
            debug_visualizer_cfg={
                "envs_idx": [0],  # only debug env 0
            }
        )

In this example, the VelocityCommandManager will generate random new X/Y/Z commands from the dict of ranges every 10 seconds.

Debug visualization#

VelocityCommandManager(
    self,
    range={...},
    debug_visualizer=True,
    debug_visualizer_cfg={
        "envs_idx": [0],  # Only add this to environment zero
        "arrow_offset": 0.5,  # Height above robot
    },
)

When debug_visualizer is True, arrows will be displayed above your robot showing which direction is commanded v.s. which direction your robot is actually moving.

Green Arrow: Commanded velocity (robot-relative, shown in world frame)
Blue Arrow: Actual robot velocity (world frame)

Caution

The debug arrows can slow down the simulation since they need to be calculated and rendered for each environment on every step.

It’s recommended to only enable them for a small number of environments at a time with the envs_idx configuration setting.

Standing Probability#

Include periods where the robot should stand still:

VelocityCommandManager(
    self,
    range={...},
    standing_probability=0.2,  # 20% chance of zero command
)

Using Velocity Commands in Rewards#

Track how well the robot follows commands:

from genesis_forge.mdp import rewards

RewardManager(
    self,
    cfg={
        "track_lin_vel": {
            "fn": rewards.command_tracking_lin_vel,
            "params": {"vel_cmd_manager": self.velocity_command},
            "weight": 2.0,
        },
        "track_ang_vel": {
            "fn": rewards.command_tracking_ang_vel,
            "params": {"vel_cmd_manager": self.velocity_command},
            "weight": 1.0,
        },
    },
)

Using Commands in Observations#

Include commands in the observation space:

ObservationManager(
    self,
    cfg={
        "velocity_command": {
            "fn": self.velocity_command.observation,
        },
    },
)

Gamepad Control#

After your policy is trained, you can control the commanded values with a physical game controller:

train.py#

from genesis_forge.gamepads import Gamepad

#...

# Setup your environment
env = MyEnv(num_envs=1, headless=False)
env.build()

# Add your gamepad to the velocity command manager
gamepad = Gamepad()
env.velocity_command.use_gamepad(gamepad)

# Run policy...

Custom Command Manager#

You can also create arbitrary commands with the basic CommandManager.

# Create a random target height between 0.1 and 0.2
self.height_command = CommandManager(self, range=(0.1, 0.2))

# Arbitrary number of ranges to support your command
self.target_command = CommandManager(self, range={
  "target_x": range=(-1.0, 1.0),
  "target_y": range=(-1.0, 1.0),
  "gait": range=(0.0, 5.0),
})