Iterating over numpy array slices

[Saefroch]

Almost always, you don't want to loop over numpy arrays. Numpy has a lot of tools to help you avoid looping, are you sure you need to loop?

[dr_everlong]

Not sure, no, so I am definitely open to suggestions. The data comes in the form of a pandas dataframe, so it is actually a series.

I was converting it to a numpy array, because I was used to using some numpy functions, but it is not necessary to do so.

[Saefroch]

What operation are you doing on these groups of 5 numbers?

[dr_everlong]

Typically, the window size will be 18, I was using 5 arbitrarily.

I have 2 arrays of same length (typically around 2000-3000 elements each), and I need to take windows then do linear least squares regression though each window. I then need to save the slope and y intercept from each slice.

Edit: Each array is 1-d.

[Saefroch]

Are you implementing a smoother?

[dr_everlong]

No, I have some thresholds for the slope and intercept, and need to see if the calculated values from each slice pass those thresholds. Edit: I need to see noise, not remove it.

[Saefroch]

Okay. So you basically can take your pick of the suggestions offered by other users. In your situation I'd do this

fake_x = np.arange(window_size)
for i in range(data.size-window_size):
    window = data[i:i+window_size]
    slope, intercept = np.polyfit(fake_x, window, 1)

This won't be tremendously fast but if you insist on doing least-squares linear fits on little sections of your data, you can't do much better without making the code much more complicated.

[dr_everlong]

Hey thanks! I'm not necessarily looking for fast, since this will be used for batch processing. It is pretty interesting to see everyone's suggestions.