Rewards#

Reward functions for the Genesis Forge environment. Each of these should return a float tensor with the reward value for each environment, in the shape (num_envs,).

class genesis_forge.mdp.rewards.body_acceleration_exp(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None, sensitivity: float = 0.1)[source]#

Bases: MdpFnClass

Penalize jerky body acceleration to encourage smooth locomotion.

Parameters:

  • env – The Genesis environment containing the robot

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

  • sensitivity – The sensitivity of the exponential decay. A lower value means the reward is more sensitive to the error.

build()#

Called during the environment build phase and when MDP params are changed.

genesis_forge.mdp.rewards.action_rate_l2(env: GenesisEnv) → torch.Tensor[source]#

Penalize the rate of change of the actions using L2 squared kernel.

Parameters:

env – The Genesis environment containing the robot

Returns:

Penalty for changes in actions

Return type:

torch.Tensor

genesis_forge.mdp.rewards.ang_vel_xy_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None)[source]#

Penalize xy-axis base angular velocity using L2 squared kernel.

Parameters:

  • env – The Genesis environment containing the entity

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

torch.Tensor

genesis_forge.mdp.rewards.base_height(env: GenesisEnv, target_height: float | torch.Tensor = None, height_command: CommandManager = None, terrain_manager: TerrainManager = None, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor[source]#

Penalize base height away from target, using the L2 squared kernel.

Parameters:

  • env – The Genesis environment containing the robot

  • target_height – The target height to penalize the base height away from

  • height_command – Get the target height from a height command manager. This expects the command to have a single range value.

  • terrain_manager – The terrain manager will adjust the height based on the terrain height.

  • entity_attr – The attribute name of the entity in the environment.

  • entity_manager – The entity manager for the entity.

Returns:

Penalty for base height away from target

Return type:

torch.Tensor

genesis_forge.mdp.rewards.command_tracking_ang_vel(env: GenesisEnv, commanded_ang_vel: torch.Tensor = None, vel_cmd_manager: VelocityCommandManager = None, sensitivity: float = 0.25, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor[source]#

Reward for tracking commanded angular velocity (yaw)

Parameters:

  • env – The Genesis Forge environment

  • commanded_ang_vel – The commanded angular velocity in the shape (num_envs, 1)

  • vel_cmd_manager – The velocity command manager

  • sensitivity – A lower value means the reward is more sensitive to the error

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Reward for tracking of angular velocity commands (yaw)

Return type:

torch.Tensor

genesis_forge.mdp.rewards.command_tracking_lin_vel(env: GenesisEnv, command: torch.Tensor = None, vel_cmd_manager: VelocityCommandManager = None, sensitivity: float = 0.25, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor[source]#

Reward for tracking commanded linear velocity (xy axes)

Parameters:

  • env – The Genesis environment containing the robot

  • command – The commanded XY linear velocity in the shape (num_envs, 2)

  • vel_cmd_manager – The velocity command manager

  • sensitivity – A lower value means the reward is more sensitive to the error

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Reward for tracking of linear velocity commands (xy axes)

Return type:

torch.Tensor

genesis_forge.mdp.rewards.contact_force(_env: GenesisEnv, contact_manager: ContactManager, threshold: float = 1.0) → torch.Tensor[source]#

Reward for the total contact force acting on all the target links in the contact manager over the threshold.

Parameters:

  • env – The Genesis Forge environment

  • contact_manager – The contact manager to check for contact

  • threshold – The force threshold for contact detection (default: 1.0 N)

Returns:

The total force for the contact manager for each environment

genesis_forge.mdp.rewards.dof_similar_to_default(env: GenesisEnv, actuator_manager: ActuatorManager = None, action_manager: PositionActionManager = None)[source]#

Penalize joint poses far away from default pose

Parameters:

  • env – The Genesis environment containing the robot

  • actuator_manager – The actuator manager for the robot/entity.

  • action_manager – (deprecated) The DOF action manager

Returns:

Penalty for joint poses far away from default pose

Return type:

torch.Tensor

genesis_forge.mdp.rewards.feet_air_time(env: GenesisEnv, contact_manager: ContactManager, time_threshold: float, time_threshold_max: float | None = None, vel_cmd_manager: VelocityCommandManager | None = None) → torch.Tensor[source]#

Reward long steps taken by the feet using L2-kernel.

This function rewards the agent for taking steps that are longer than a threshold. This helps ensure that the robot lifts its feet off the ground and takes steps. The reward is computed as the sum of the time for which the feet are in the air.

If the velocity commands are small (i.e. the agent is not supposed to take a step), then the reward is zero.

Parameters:

  • env – The Genesis Forge environment

  • contact_manager – The contact manager to check for contact

  • time_threshold – The minimum time (in seconds) the feet should be in the air

  • time_threshold_max – (optional) The maximum time (in seconds) the feet should be in the air. The reward will be capped at this value.

  • vel_cmd_manager – The velocity command manager

Returns:

The reward for the feet air time

genesis_forge.mdp.rewards.feet_slide(env, contact_manager: ContactManager, entity_attr: str = 'robot') → torch.Tensor[source]#

Penalize feet sliding.

This function penalizes the agent for sliding its feet on the ground. The reward is computed as the norm of the linear velocity of the feet multiplied by a binary contact sensor. This ensures that the agent is penalized only when the feet are in contact with the ground.

This penalty is less effective at longer foot-contact links (for example, long legs without dedicated foot links), because they might have some velocity while they’re being used to move the robot. However, dedicated foot links will be stationary on the ground and not moving while pushing the robot forward.

Parameters:

  • env – The Genesis Forge environment

  • contact_manager – The contact manager for the feet

  • entity_attr – The attribute name of the robot entity that the feet are attached to.

Returns:

The penalty for the feet slide

genesis_forge.mdp.rewards.flat_orientation_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor[source]#

Penalize non-flat base orientation using L2 squared kernel. This is computed by penalizing the xy-components of the projected gravity vector.

Parameters:

  • env – The Genesis environment containing the robot

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Penalty for non-flat base orientation

Return type:

torch.Tensor

genesis_forge.mdp.rewards.has_contact(_env: GenesisEnv, contact_manager: ContactManager, threshold=1.0, min_contacts=1) → torch.Tensor[source]#

One or more links in the contact manager are in contact with something.

Parameters:

  • env – The Genesis Forge environment

  • contact_manager – The contact manager to check for contact

  • threshold – The force threshold for contact detection (default: 1.0)

  • min_contacts – The minimum number of contacts required. (default: 1)

Returns:

1 for each contact meeting the threshold

genesis_forge.mdp.rewards.is_alive(env: GenesisEnv) → torch.Tensor[source]#

Reward for being alive and not terminating this step. This assumes that env.extras[“terminations”] is a boolean tensor with the termination signals for the environments.

genesis_forge.mdp.rewards.lin_vel_z_l2(env: GenesisEnv, entity_attr: str = 'robot', entity_manager: EntityManager = None) → torch.Tensor[source]#

Penalize z axis base linear velocity

Parameters:

  • env – The Genesis environment containing the entity

  • entity_manager – The entity manager for the robot/entity the reward is being computed for. This is slightly more performant than using the entity_attr parameter.

  • entity_attr – The attribute name of the entity in the environment. This isn’t necessary if entity_manager is provided.

Returns:

Penalty for z axis base linear velocity

Return type:

torch.Tensor

genesis_forge.mdp.rewards.stand_still_joint_deviation_l1(env, vel_cmd_manager: VelocityCommandManager, actuator_manager: ActuatorManager = None, command_threshold: float = 0.06, action_manager: PositionActionManager = None) → torch.Tensor[source]#

Penalize offsets from the default joint positions when the command is very small.

Parameters:

  • env – The Genesis Forge environment

  • command_threshold – The threshold for the command to be considered small

  • vel_cmd_manager – The velocity command manager

  • actuator_manager – The actuator manager to get the joint positions and recent actions from.

  • action_manager – The action manager to get the joint positions and recent actions from.

Returns:

Penalty for offsets from the default joint positions when the command is very small

Return type:

torch.Tensor

genesis_forge.mdp.rewards.terminated(env: GenesisEnv) → torch.Tensor[source]#

Penalize terminated episodes that terminated. This assumes that env.extras[“terminations”] is a boolean tensor with the termination signals for the environments.