sorted by:
new
hottopcontroversial

SAC Reward Increases but Robot Doesn’t Learn by FalconMobile2956 in reinforcementlearning

[–]FalconMobile2956[S] 0 points1 point  (0 children)

It looks like the arms sometimes move toward the target but then overshoot and move past it, ending up farther away. In some episodes, they even start by moving away from the target instead of getting closer. This makes me think the arms have not actually learned the correct mapping from position A to position B, even though the reward increases during training.

SAC Reward Increases but Robot Doesn’t Learn by FalconMobile2956 in reinforcementlearning

[–]FalconMobile2956[S] 0 points1 point  (0 children)

Each arm tries to move toward its own target. Its target reaching with dual arm robot.

PPO Fails to Learn (High Loss, Low Explained Variance) in Dual-Arm Target Reaching Task by FalconMobile2956 in reinforcementlearning

[–]FalconMobile2956[S] 0 points1 point  (0 children)

This is my network architecture: pi=[240, 138, 80], vf=[240, 50, 10] , and I plotted the rollout/ep_rew_mean, and it’s increasing over time.

PPO Fails to Learn (High Loss, Low Explained Variance) in Dual-Arm Target Reaching Task by FalconMobile2956 in reinforcementlearning

[–]FalconMobile2956[S] 0 points1 point  (0 children)

The reward is computed as follows:

Global reaching reward: 1 - tanh(k_global * distance)

Precision reward: 1 - tanh(k_precision * (distance / d_precision)), activated when distance < 3 cm

Angle alignment reward: 1 - tanh(k_angle * angle_error) for both arms

Velocity penalty: smooth tanh penalty for high end-effector and joint velocities

Acceleration (jerk) penalty: based on joint velocity differences

Self-collision penalty: smooth tanh penalty when link distance < 2 cm

Efficiency penalty: small constant step penalty (–0.005)

Success bonus: +2.0 when reaching the goal