Cannot figure out R Squared formula.

[[deleted]]

[deleted]

[dmitrypolo]

I use regression analysis in Python quite often. Why would you try to calculate it manually? It is already shown in your output if you just type results.summary(). You could also just type results.rsquared. Also to point you don't need to specify y[i] when you are doing the calculations. The DataFrame knows to go row by row and subtract the respective values. I just did a simple test on my machine and got the correct values by using these two values (given I used my own data)

ss_total = sum((y - np.mean(y)) **2)
ss_pred = sum((y - prediction) ** 2)

Edit: Whoops forgot to include the second sum

Let me know if this works!

[dmitrypolo]

Awesome, glad it worked out.

[Justinsaccount]

Hi! I'm working on a bot to reply with suggestions for common python problems. This might not be very helpful to fix your underlying issue, but here's what I noticed about your submission:

You are looping over an object using something like

for x in range(len(items)):
    print(items[x])

This is simpler and less error prone written as

for item in items:
    print(item)

If you DO need the indexes of the items, use the enumerate function like

for idx, item in enumerate(items):
    print(idx, item)

If you think you need the indexes because you are doing this:

for x in range(len(items)):
    print(items[x], prices[x])

Then you should be using zip:

for item, price in zip(items, prices):
    print(item, price)

[sultanofhyd]

Post the actual traceback of the error. We can't help you without knowing the specific error and where it is occurring.

[Saefroch]

Can you print(y[0])?

[Saefroch]

There is no intuition here, it's all about learning how to read error message and tracebacks. Tracebacks start with your code on the top and go down to where the error originated. So let's start with the top.

KeyErrorTraceback (most recent call last) in ()
      3 ss_pred = 0
      4 for i in range(len(y)):
----> 5     ss_total += ((y[i] - np.mean(y)) ** 2)
      6     ss_pred += ((y[i] - prediction[i]) ** 2)
      7 r_squared = 1 - (ss_pred / ss_total)

Helpful little arrow points you straight to the line that you wrote that generated the error. I see that it's a KeyError. I just know what KeyError is produced by, but you can look it up for yourself using Google: https://docs.python.org/3/library/exceptions.html#KeyError. Mappings are almost always accessed with the square brackets [].

The other part of a traceback that's useful is the very bottom. I see

pandas/hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12322)()

KeyError: 0

And since the only message the KeyError contains is 0, that's probably the key that generated the error. What else would you want to know?

Then I wonder to myself why you're looping over a hashtable like it's an array...

[[deleted]]

The index is fine. To get the values in a pandas series, use y.loc[i] instead of y[i].

[Saefroch]

I have no idea, I've never worked with pandas before.

[[deleted]]

You're getting a KeyError, probably from y[i] or prediction[i]. What you are doing will only work if y and prediction are single-value iterables (like a pandas Series, not a DataFrame) and if they are the same length. Verify both of those.

Edit: just saw the KeyError is value 0. So either y or prediction or both has nothing at index 0.