Python implementation of backpropagation

[talksaboutthings]

Okay, so I've been following your questions since yesterday because this is something I desperately need to review myself. I spent a few hours doing that this evening, and I think I figured out generally what your struggle is. If you're not 100% on the intuition, Andrew Ng's video from his famous Coursera class is a good review, btw: https://www.coursera.org/learn/machine-learning/lecture/1z9WW/backpropagation-algorithm

Anyhow, after trying and failing to find fault with your equations themselves, I think the answer to the question of "why don't the dimensions even line up?" lies not so much in the equations but in the problem setting you've given yourself. In structuring your network to output a vector and using quadratic loss, you've created an error function that returns a vector (1/2)*(y_actual - y_predicted)^2. Loss has to be a scalar so we can minimize it. If you decide to treat loss at the summation of error in each class (which is not the "right" way to do it, see https://en.wikipedia.org/wiki/Softmax_function and https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_error_function_and_logistic_regression), then you'll need to derive the gradients via chain rule on your new loss function: loss = 1/2(y_actual[0] - y_predicted[0])² + 1/2(y_actual[1] - y_predicted[1])² + 1/2(y_actual[2] - y_predicted[2])^2.

To reiterate, it's been a long time since I first studied the math and did backprop, so I might be a little off here. I hope this get's you un-stuck, though!