Rewards

Reward functions for the Genesis Forge environment. Each of these should return a float tensor with the reward value for each environment, in the shape (num_envs,).

class genesis_forge.mdp.rewards.action_acceleration_l2(env: GenesisEnv, action_manager: PositionActionManager = None)

Bases: MdpFnClass

Targets jittery oscillations (rather than smooth consistent movement), by penalize the second-order finite difference of actions (discrete acceleration) using the L2 squared kernel.

This encourages a smooth consistent movement, where a smooth ramp has zero acceleration even at high velocity.

A smooth action ramp looks like this: 0.5 → 0.6 → 0.7 → 0.8

• Velocities: 0.1, 0.1, 0.1

• Accelerations: 0.0, 0.0 (zero – perfectly smooth)

• Penalty: zero

A jittery action ramp looks like this: 0.5 → 0.8 → 0.5 → 0.8

• Velocities: 0.3, -0.3, 0.3

• Accelerations: -0.6, 0.6 (large – direction keeps reversing)

• Penalty: very large

The acceleration is computed as:

\[\text{acc}_t = a_t - 2 \cdot a_{t-1} + a_{t-2}\]

and the penalty is \(\sum \text{acc}_t^2\) across all action dimensions.

Parameters:

• env – The Genesis environment containing the robot

• action_manager – Optional action manager to source actions from. If not provided, actions are read from env.actions.

build()

Called during the environment build phase and when MDP params are changed.

reset(envs_idx)

Clear the action history for the specified environments.

class genesis_forge.mdp.rewards.body_acceleration_exp(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None, sensitivity: float = 0.1)

Bases: MdpFnClass

Penalize jerky body acceleration to encourage smooth locomotion.

Parameters:

• env – The Genesis environment containing the robot

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

• sensitivity – The sensitivity of the exponential decay. A lower value means the reward is more sensitive to the error.

build()

Called during the environment build phase and when MDP params are changed.

reset(envs_idx)

Called when environments are reset. Override to clear per-env state.

genesis_forge.mdp.rewards.action_rate_l2(env: GenesisEnv) → torch.Tensor

Penalize the rate of change of the actions using L2 squared kernel.

Parameters:

env – The Genesis environment containing the robot

Returns:

Penalty for changes in actions

Return type:

torch.Tensor

genesis_forge.mdp.rewards.ang_vel_xy_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None)

Penalize xy-axis base angular velocity using L2 squared kernel.

Parameters:

• env – The Genesis environment containing the entity

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

torch.Tensor

genesis_forge.mdp.rewards.base_height(env: GenesisEnv, target_height: float | torch.Tensor = None, height_command: CommandManager = None, terrain_manager: TerrainManager = None, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor

Penalize base height away from target, using the L2 squared kernel.

Parameters:

• env – The Genesis environment containing the robot

• target_height – The target height to penalize the base height away from

• height_command – Get the target height from a height command manager. This expects the command to have a single range value.

• terrain_manager – The terrain manager will adjust the height based on the terrain height.

• entity_attr – The attribute name of the entity in the environment.

• entity_manager – The entity manager for the entity.

Returns:

Penalty for base height away from target

Return type:

torch.Tensor

genesis_forge.mdp.rewards.command_tracking_ang_vel(env: GenesisEnv, commanded_ang_vel: torch.Tensor = None, vel_cmd_manager: VelocityCommandManager = None, sensitivity: float = 0.25, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor

Reward for tracking commanded angular velocity (yaw)

Parameters:

• env – The Genesis Forge environment

• commanded_ang_vel – The commanded angular velocity in the shape (num_envs, 1)

• vel_cmd_manager – The velocity command manager

• sensitivity – A lower value means the reward is more sensitive to the error

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Reward for tracking of angular velocity commands (yaw)

Return type:

torch.Tensor

genesis_forge.mdp.rewards.command_tracking_lin_vel(env: GenesisEnv, command: torch.Tensor = None, vel_cmd_manager: VelocityCommandManager = None, sensitivity: float = 0.25, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor

Reward for tracking commanded linear velocity (xy axes)

Parameters:

• env – The Genesis environment containing the robot

• command – The commanded XY linear velocity in the shape (num_envs, 2)

• vel_cmd_manager – The velocity command manager

• sensitivity – A lower value means the reward is more sensitive to the error

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Reward for tracking of linear velocity commands (xy axes)

Return type:

torch.Tensor

genesis_forge.mdp.rewards.contact_force(_env: GenesisEnv, contact_manager: ContactManager, threshold: float = 1.0) → torch.Tensor

Reward for the total contact force acting on all the target links in the contact manager over the threshold.

Parameters:

• env – The Genesis Forge environment

• contact_manager – The contact manager to check for contact

• threshold – The force threshold for contact detection (default: 1.0 N)

Returns:

The total force for the contact manager for each environment

genesis_forge.mdp.rewards.dof_similar_to_default(env: GenesisEnv, actuator_manager: ActuatorManager | list[ActuatorManager] | None = None, action_manager: PositionActionManager | None = None) → torch.Tensor

Penalize joint poses far away from default pose(s).

Pass actuator_manager as one manager or a non-empty list/tuple (e.g. per-limb stacks); penalties are summed per environment across all included DOFs.

Parameters:

• env – The Genesis environment containing the robot (unused today; accepted for MDP signature consistency).

• actuator_manager – One or more actuator managers.

• action_manager – (deprecated) One or more position-action managers. Use actuator_manager instead.

Returns:

Penalty summed over included DOFs, shape (num_envs,).

Return type:

torch.Tensor

genesis_forge.mdp.rewards.dof_torque_l2(env: GenesisEnv, actuator_manager: ActuatorManager) → torch.Tensor

Penalize joint torque effort using the L2 squared kernel.

Discourages the policy from applying unnecessary force, particularly when the robot is near equilibrium. This helps reduce actuator oscillation when the robot is stationary or moving slowly.

Parameters:

• env – The Genesis environment containing the robot

• actuator_manager – The actuator manager to retrieve DOF forces from.

Returns:

Penalty for joint torque effort, shape (num_envs,)

Return type:

torch.Tensor

genesis_forge.mdp.rewards.dof_velocity_l2(env: GenesisEnv, action_manager: PositionActionManager) → torch.Tensor

Penalize joint angular velocities to encourage slow, deliberate motion.

genesis_forge.mdp.rewards.feet_air_time(env: GenesisEnv, contact_manager: ContactManager, time_threshold: float, time_threshold_max: float | None = None, vel_cmd_manager: VelocityCommandManager | None = None) → torch.Tensor

Reward long steps taken by the feet using L2-kernel.

This function rewards the agent for taking steps that are longer than a threshold. This helps ensure that the robot lifts its feet off the ground and takes steps. The reward is computed as the sum of the time for which the feet are in the air.

If the velocity commands are small (i.e. the agent is not supposed to take a step), then the reward is zero.

Parameters:

• env – The Genesis Forge environment

• contact_manager – The contact manager to check for contact

• time_threshold – The minimum time (in seconds) the feet should be in the air

• time_threshold_max – (optional) The maximum time (in seconds) the feet should be in the air. The reward will be capped at this value.

• vel_cmd_manager – The velocity command manager

Returns:

The reward for the feet air time

genesis_forge.mdp.rewards.feet_ground_time(env: GenesisEnv, contact_manager: ContactManager, time_threshold: float) → torch.Tensor

Penalize brief ground contacts (foot tapping) using a linear kernel.

Fires at the moment a foot lifts off. The penalty is proportional to how much the stance duration fell below time_threshold. A proper stance phase (contact_time >= time_threshold) produces zero penalty.

Intended to be paired with feet_air_time (positive reward) to fully shape gait timing: feet_air_time rewards long swings while this penalizes taps. Use a negative weight in the RewardManager.

Parameters:

• env – The Genesis Forge environment

• contact_manager – The contact manager to check for contact

• time_threshold – Contacts shorter than this (in seconds) are penalized. Set independently from the feet_air_time threshold based on the expected stance duration of your target gait.

Returns:

The penalty for brief ground contacts, shape (num_envs,)

genesis_forge.mdp.rewards.feet_slide(env, contact_manager: ContactManager, entity_attr: str = 'robot') → torch.Tensor

Penalize feet sliding.

This function penalizes the agent for sliding its feet on the ground. The reward is computed as the norm of the linear velocity of the feet multiplied by a binary contact sensor. This ensures that the agent is penalized only when the feet are in contact with the ground.

This penalty is less effective at longer foot-contact links (for example, long legs without dedicated foot links), because they might have some velocity while they’re being used to move the robot. However, dedicated foot links will be stationary on the ground and not moving while pushing the robot forward.

Parameters:

• env – The Genesis Forge environment

• contact_manager – The contact manager for the feet

• entity_attr – The attribute name of the robot entity that the feet are attached to.

Returns:

The penalty for the feet slide

genesis_forge.mdp.rewards.flat_orientation_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor

Penalize non-flat base orientation using L2 squared kernel. This is computed by penalizing the xy-components of the projected gravity vector.

Parameters:

• env – The Genesis environment containing the robot

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Penalty for non-flat base orientation

Return type:

torch.Tensor

genesis_forge.mdp.rewards.has_contact(_env: GenesisEnv, contact_manager: ContactManager, threshold=1.0, min_contacts=1) → torch.Tensor

One or more links in the contact manager are in contact with something.

Parameters:

• env – The Genesis Forge environment

• contact_manager – The contact manager to check for contact

• threshold – The force threshold for contact detection (default: 1.0)

• min_contacts – The minimum number of contacts required. (default: 1)

Returns:

1 for each contact meeting the threshold

genesis_forge.mdp.rewards.is_alive(env: GenesisEnv) → torch.Tensor

Reward for being alive and not terminating this step. This assumes that env.extras[“terminations”] is a boolean tensor with the termination signals for the environments.

genesis_forge.mdp.rewards.lin_vel_xy_l2(env: GenesisEnv, entity_manager) → torch.Tensor

Penalize horizontal base linear velocity.

genesis_forge.mdp.rewards.lin_vel_z_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor

Penalize z axis base linear velocity

Parameters:

• env – The Genesis environment containing the entity

• entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

• entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Penalty for z axis base linear velocity

Return type:

torch.Tensor

genesis_forge.mdp.rewards.stand_still_joint_deviation_l1(env, vel_cmd_manager: VelocityCommandManager, actuator_manager: ActuatorManager = None, command_threshold: float = 0.06, action_manager: PositionActionManager = None) → torch.Tensor

Penalize offsets from the default joint positions when the command is very small.

Parameters:

• env – The Genesis Forge environment

• command_threshold – The threshold for the command to be considered small

• vel_cmd_manager – The velocity command manager

• actuator_manager – The actuator manager to get the joint positions and recent actions from.

• action_manager – The action manager to get the joint positions and recent actions from.

Returns:

Penalty for offsets from the default joint positions when the command is very small

Return type:

torch.Tensor

genesis_forge.mdp.rewards.terminated(env: GenesisEnv) → torch.Tensor

Penalize terminated episodes that terminated. This assumes that env.extras[“terminations”] is a boolean tensor with the termination signals for the environments.