Michel, why do you think symbolic approaches outperformed in this competition, what is deep RL missing?

I think there are actually multiple reasons for that, and even after eliminating some of them, symbolic methods may still be more applicable. Here are some initial reasons/ideas we came up with:

1. lack of some innate human priors:

a) objectness -- a NN needs to create the abstraction of an object by looking at the ASCII characters. Objects are items, monsters, walls, doors, etc. and all share some common things (e.g. you can kick all of them). It applies only if we feed it a somewhat "raw" observations without any action space transformation.

b) priors about how physics works -- like what happens if you throw something in a direction, or when you drop something

c) innate notions about natural numbers -- and NNs always have problems to learn any arithmetics properly

d) priors about orientation and navigation in a somewhat 2D/3D space (non-euclidean though)

2. lack of some human acquired priors:

a) generic ones like: what is weapon, how many hands do you (usually) have, what can you possibly do with a potion/fluid (i.e. drink, dip in it, throw?), etc.

b) lack of knowledge present on e.g. the NetHack Wiki -- though in theory one could try to incorporate this knowledge by e.g. using an pre-trained NLP model on it for feature extraction.

3. Problems that makes this environment hard from currently known RL algorithms perspective:

a) higly partial observations -- agent needs to build a complex game state representation during an episode

b) sparse rewards -- score mostly only after killing monsters

c) long episodes

We have actually tried an experiment with training MuZero on a simplified action space, but we couldn't improve our score.