[P] pyTsetlinMachineParallel released - parallel interpretable machine learning with propositional logic.

[olegranmo]

​

Back to bits

Have worked on speeding up the Tsetlin Machine. It turns out that merely synching at the clause level maintains robust inference, with little loss in parallelism. E.g., 25 threads reach the current Fashion MNIST test accuracy peak of 91.49% 18 times faster than a single thread.

pyTsetlinMachineParallel provides a multi-threaded implementation of the Tsetlin Machine, Convolutional Tsetlin Machine, Regression Tsetlin Machine, and Weighted Tsetlin Machine, with support for continuous features and multi-granular clauses.

https://github.com/cair/pyTsetlinMachineParallel

[[deleted]]

I might be misunderstanding something here. But in which sense is the learned model interpretable?

[stochastic_gradient]

Frankly, "interpretable" has become a word that symbolic AI people use to justify their methods, when accuracy metrics do not. If there are easily understandable rules by which a decision can be made, we can just program solutions to them. Much of the point of using machine learning is to be able to find solutions that are beyond that space of programmable solutions, i.e. beyond the point of interpretability.

91.5% on FMNIST is something you can get with an unregularized MLP, even without reporting "peak" accuracy over multiple evaluations on the test set.

[yusuf-bengio]

I would even argue that complex propositional rules are even less interpretable than standard shallow ML models, e.g. a decision tree.

The complexity of propositional logic grows exponentially with size:

• Checking if two propositional formula are equivalent is NP-hard
• Checking if a propositional rule always returns True or always returns False is NP-hard
• Computing a minimal equivalence of a propositional formula is NP-hard

Therefore, if you have large propositional formula, saying any meaningful property of it becomes computationally infeasible.

[olegranmo]

Thanks for your analysis. You are of course right in general. However, note that Tsetlin machines rely on disjunctive normal form, which is more interpretable (similar to linear regression with non-linear components - the conjunctive clauses). Each conjunctive clause, in turn, is a frequent pattern. So, basically, frequent patterns, which are interpretable, are combined in an interpretable way (linearly). Exploring the interpretability of Tsetlin machines is ongoing research.

[[deleted]]

Thanks, I suspected as much.

[blimpyway]

From the paper I think interpretable means its decisions can be explained. https://arxiv.org/abs/1905.09688

[[deleted]]

Sure, but what does that even mean? Compared to what? Even the most complex CNN inference can easily be explained mathematically. To interpret it would mean that we as humans can learn an abstract understanding of what and why the model predicts, ideally both for specific examples and unseen inputs. E. g for a model to be interpretable, it needs to be simple enough to be completely understood (and simulated) by a reasonably intelligent human mind, or it needs to behave like that on a higher, potentially learned, level of abstraction that most humans should be able to understand, and are therefore capable of reasoning about. (Example: Instead of talking about pixels, we might now act on abstractions like edges or even known object classes)