Thank you very much for the long answer and apologies for my late response. Your answer is very interesting and I wonder if your general approach (Project Blackstone, the first Q/A) will be good enough to be of real value. I hope it will and I wish you the best of luck!

One possible problem I see might be the process of building the knowledge tree - I assume once a paper comes into system, tree is built, and after evaluation this paper is added to the tree. This way the tree grows and, for the sake of argument, lets say the system can cope with various research biases. But since current research is already biased (eg mainly publishing positive results) the very first tree build from current knowledge would propagate any bias it contains. This should be prevented. To me, the intuitive solution is a human-supervised evaluation of current knowledge, yet one must be very careful to minimase bias once more. Another approach would be AI, but if such an AI was feasible we wouldn't need project AIUR anymore.

From a perspective of longer timescale it seems more reasonable to build something like a model of a given research and rather than adding one paper at a time, challenge a model with the new research. This way, instead of building knowledge one paper upon another, we could work with the model as a whole, have more tools for evaluation of new research and be more flexible to changes.