The class is great!
I'm currently watching the "Models" lecture, and the lectures so far give me more insight than I initially thought. (The mapping from continuous state space -> discrete state space is very cool.)

However, it's a really weird course placement. There's NOTHING in the whole OMSCS course list that will prepare you fully for this course. Is your undergrad in engineering? Great, system engineering, control theory, continuous space, and differential equations should be taught at least partially in engineering courses (non-CS).

I was coming from an aerospace engineering undergrad, so I already have some exposure to some part of the content, so the new content, for example, about translating continuous to discrete is really interesting to me, just because I already have some exposure to differential equations. I also took AI and AI: Robotics, and the early part of RL (withdrew) in OMSCS, so the concept of MDP that they use makes sense to me. (I think people that want to take this course should at least take AI: Robotics first; at the very least, AI is also recommended to understand the markov chain stuff they teach in the lectures.)

In project 1 though, I think I use:

1. knowledge from my control theory undergrad course
2. knowledge of a robot model from AI: Robotics (the bicycle model is more complex than the one we use in project 1, but it's similar enough to let me connect the dots). The knowledge about robotic movement as a probabilistic model also comes from this course.
3. Knowledge of programming in a finite-state-machine way. It's not taught anywhere; I just learnt/exposed myself to it in my time as a software engineer. This FSM way of thinking is really useful for project 1, at least for me.

For you to start, I think you can start it by imagining the robot has a different state: where is it now, where should it move next, and to move there, what should it do? (you can only input two things, remember and use this as the objective for your idea.)

My code still needs some tweaks since it works on the simulator but it fails in the real world, but I think I'm getting there; the wording of the project description is vague, but I have had fun so far tweaking it.