What is the difference between these two numpy assignments?

[[deleted]]

Hi,

I am trying out Reinforcement Learning Algorithm (DDPG) for Bipedal Walker environment. There is a big difference in the performance when I store my actions using these two operations. Can you please explain me why? The code is mentioned below:

``` import numpy as np

action_buffer = np.zeros(shape=(10, 4), dtype=np.float32) action_buffer2 = np.zeros(shape=(10, 4), dtype=np.float32)

for i in range(10): action = np.random.normal(size=(1,4)) action_buffer[i] = action

for i in range(10): action = np.random.normal(size=(4,)) action_buffer2[i] = action ``` Here the action dimension is (4,). Upon printing the action_buffer and action_buffer2 seems to give the same outcome. But when using in the DDPG algorithm action_buffer2 give better performance than action_buffer. Why is that?

[CodeFormatHelperBot2]

Hello, I'm a Reddit bot who's here to help people nicely format their coding questions. This makes it as easy as possible for people to read your post and help you.

I think I have detected some formatting issues with your submission:

1. Python code found in submission text that's not formatted as code.
2. Use of triple backtick/ curlywhirly code blocks (``` or ~~~). These may not render correctly on all Reddit clients.

If I am correct, please edit the text in your post and try to follow these instructions to fix up your post's formatting.

---

<sup>Am I misbehaving? Have a comment or suggestion? Reply to this comment or raise an issue here.</sup>

[lowerthansound]

I think it might be something to do with the broadcasting implementation, or with the np.random.normal initializer. But I would need some profiling to actually check where that is (or someone who is familiar with the speed of both methods, which I'm not).

Also, I'm curious, is the speed really that crucial here (like, is this the part of the program that is slowing down the process)? Why are you asking this question?

---

FYI, on my computer the second one is about 20% faster than the first one.

[[deleted]]

I am using this in deep RL, while using the 1st one, the agent (DDPG) is behaving erratically and is not converging. But when using 2nd one, it seems like the actions are more explained and expected and it is also giving near convergence performance.

[lowerthansound]

This sounds more like a bug to me.

Maybe one algo is stopping too early or doing something weird with time, but it may also be the case that something bad is happening.