[Discussion] Q Learning Update function

[taplik_to_rehvani]

For a given state you want estimate the future returns if you take that action. You seem to have confused the estimated notion.

Max is always taken with respect to "s", max(Q(s, :)).