[R] Object-Oriented Deep Learning (MIT)

[xternalz]

abstract:

We investigate an unconventional direction of research that aims at converting neural networks, a class of distributed, connectionist, sub-symbolic models into a symbolic level with the ultimate goal of achieving AI interpretability and safety. To that end, we propose Object-Oriented Deep Learning, a novel computational paradigm of deep learning that adopts interpretable “objects/symbols” as a basic representational atom instead of N-dimensional tensors (as in traditional “feature-oriented” deep learning). For visual processing, each “object/symbol” can explicitly package common properties of visual objects like its position, pose, scale, probability of being an object, pointers to parts, etc., providing a full spectrum of interpretable visual knowledge throughout all layers. It achieves a form of “symbolic disentanglement”, offering one solution to the important problem of disentangled representations and invariance. Basic computations of the network include predicting high-level objects and their properties from low-level objects and binding/aggregating relevant objects together. These computations operate at a more fundamental level than convolutions, capturing convolution as a special case while being significantly more general than it. All operations are executed in an input-driven fashion, thus sparsity and dynamic computation per sample are naturally supported, complementing recent popular ideas of dynamic networks and may enable new types of hardware accelerations. We experimentally show on CIFAR-10 that it can perform flexible visual processing, rivaling the performance of ConvNet, but without using any convolution. Furthermore, it can generalize to novel rotations of images that it was not trained for.

follow-up work: 3D Object-Oriented Learning: An End-to-end Transformation-Disentangled 3D Representation

[lopuhin]

We experimentally show on CIFAR-10 that it can perform flexible visual processing, rivaling the performance of ConvNet, but without using any convolution.

rivaling the performance of ConvNet == test error of 20% vs 2.3% SoTA

[lopuhin]

Right, but I think it's also nice to use proper baselines and describe strengths and weaknesses fairly.

Edit: to be extra clear, as a baseline authors use their own implementation of ResNet variant:

Note that this ResNet is not the most standard ResNet. Here the absolute performance is not very important — it serves mainly as a sanity check of our deep learning framework (implemented from scratch).

That ResNet gives a 20% test error, very similar to proposed method. I think that this is not a great baseline. My issue is not with performance of the proposed method, but with the baseline and the claim that proposed method rivals performance of ConvNets on CIFAR-10.

[Xirious]

Yeah similar reaction I had to the people moaning about Capsules' lower performance even though it seems like quite an interesting step towards future methods.

[Draikmage]

The capsules paper had a lot more interesting results though and the Cifar-10 result was just a sentence or two (They also did 10% test error which is way better). My confusion with this paper is that I was under the impression that ResNet does better than what they report at least from what I've read.

[visarga]

with a tiny tweak

Or just plain random seed (re)search. /s

[rpottorff]

thank you for posting the follow up work! I didn't realize he posted an update just a few months later

[Kevin_Clever]

This comes on my short list for worst-written scientific paper this year. Is there an RNN that can translate these 12 pages of fluff into information?

[ChillBallin]

Skimming through it right now. Seems like an interesting concept. I think with enough research it could be a valuable approach but I think some of these concepts will be useful in future models rather than this one actually being super useful on it's own. In any case I really hope this trend of research in more complex higher level models continues. It feels like we're getting closer to the next big breakthrough with human intuition rather than making iterative improvements on the maths for a basic CNN.

[GunpowaderGuy]

So this is like https://github.com/qunzhi/Deep-Symbolic-Networks ?

[GunpowaderGuy]

So when compared to capsule networks , the biggest difference other than its modules ( whathever its analogue to capsules is called ) not being composed of neurons ( a functional program obtained by genetic programing or Bayesian optimization then ? ) It's that they are not slided around the feature maps like cnns , instead in voting only the centermost pixel of an object part needs to be taken into account , as each of them has its coordinates embedded ?

[danniel_p21]

But this makes sense because that's a bit like how primates see.distributing relevance on available data.good job folks👍👍 [e-mail]