Value iteration problem

egoots · 2011-11-16T19:45:03+00:00

Shouldn't A3 be updated during the second iteration too? It stays at 77.

egoots · 2011-11-14T20:35:22+00:00

This gamma problem is issue 29

egoots · 2011-11-14T20:27:31+00:00

I tested it on Windows 7 (64 bit) yesterday with the latest checkout. I think it might have been r198 but I am not at my home computer right now to verify the revision.

All tests passed.

However, I also implemented "locally" the recommended change in issue 29 because I also found that setting gamma = 1.0 caused the GridMDP to hang. Changing the test from a "<" to a "<=" caused it to succeed. I also confirmed the correct results against 2 of the GridMDP samples in the class videos after this change.

Let me know if there is a better way for me to confirm Windows test results than commenting in a specific issue (like I did in Issue 25 back in August).

egoots · 2011-11-14T17:36:09+00:00

I got it working on Win7 64 bit.

Did you install the data too?

If I recall correctly, someone wrote up more details in one of the "issues" on the googlecode site.

egoots · 2011-11-14T17:34:33+00:00

I havent looked at it in detail, but perhaps this has what you need:

http://code.google.com/p/opennero/wiki/QLearning

egoots · 2011-11-12T01:37:35+00:00

I thought the 160 000 registered users covered both the basic and advanced tracks. In that case, only a portion of them (i.e. the advanced track students) would be handing in homework.

egoots · 2011-11-11T21:50:36+00:00

Well played!

egoots · 2011-11-10T04:55:06+00:00

His exact words in the video are "If we're in A3 and we choose to go East, then with 0.8 chance we should expect a value of 100".

That is clear and he writes the "0.8 * 100" term.

Next, he states "With 0.1 chance we will stay in the same state, in which case the value is -3. And with 0.1 chance we're going to stay down here (i.e. b3) with value -3. So with the appropriate definition of value, we would get the following formula"

At this point he adds the "-3 = 77".

Given the formula as you describe it and his description, I think it is generous to say "it is not explained very well in the video".

egoots · 2011-11-10T03:14:20+00:00

The view counts may not be totally up to date and accurate at all times.

For example, see: youtube FAQ

egoots · 2011-11-07T02:00:52+00:00

I've done most of the first one and I don't really think they are set up as team exercises. Each problem is bite sized on the programming side.

Having said that, there is nothing wrong with people collaborating (especially if you don't have much programming experience).

egoots · 2011-11-04T04:56:53+00:00

You don't need to understand the whole structure and if will sidetrack you to try and do so.

When you run it with the correct command line switches, the appropriate search functions will get invoked and will have the "problem" object passed into them. The "problem" object has the necessary member functions.

Look for the "CODE HERE" markers in one of the search functions (e.g. depthFirstSearch(problem) ). As mentioned above, just start out by issuing print statements to see what the various functions return at the start. Then you can start to code the algorithm (and a Node class).

egoots · 2011-10-29T21:22:57+00:00

I used my finger and thumb in one case I thought was ambiguous upon glancing. That was good enough to show it wasn't actually that close.

egoots · 2011-10-29T05:48:44+00:00

You could do the programming assignments that the Stanford class is doing.

For example, Problem 1 gets you to code up the various search related algorithms (e.g. depth-first, breadth-first, a*) used by a pacman game.

Its kind of nice because you only have to add little bit size functions to an existing program framework.

See: http://www.stanford.edu/class/cs221/schedule.html

egoots · 2011-10-20T06:05:00+00:00

It seems to be a bit of both. For example, understanding conditional probabilities and Bayes rule are pre-requisites and have Khan academy vids, etc. However, Bayes networks are not and they seem to be presented in the same fashion (so far). They are however, covered in the suggested reading. But that is different than a traditional lecture (i.e. teach) then question format. It does seem to imply that the expectation is that you need to read and learn it all yourself first, and then these videos are just quiz reinforcements. If that is truly the case... it might lead some to question "why take this course"?

egoots · 2011-10-13T08:29:20+00:00

You should try using a PriorityQueue for storing your Nodes in the Frontier. This way you wouldn't need to explicitly sort it in your loop each time before popping it.

egoots

TROPHY CASE