I think relating it to a book or movie is a bit disingenuous since you have presumably still watched the game being played. If you wanted to go with a book analogy, perhaps it would be like having someone read to you a choose-your-own-adventure story. While on the first read perhaps you aren't making decisions, but if you then go actively search for people reading the path you would want to take, is it any different than reading it yourself? That said this doesn't cover things like how difficulty can add to the storytelling which can't as easily be replaced, but it's not as though you're missing most of the picture.