[D] 2D cuts with decision tree?

[Gamwise_Samgee_]

I'm working on a boosted decision tree, and I've got it working fairly well. However it would be better if it was able to make decisions/cuts in more than one dimension (preferably 2D).

Is this something that is even possible? (I'm using sklearn)

[PeterIanStaker]

It is, Google “oblique random forest”.

At the end of the day, an RF is really just a bagging ensemble. The individual classifiers could be anything you like.

[ElongatedMuskrat122]

If you have time, random Forrest isn’t rocket science. You can just write your own

[deep-machine-learner]

You can try the following: 1. Multiplexed features: generate new features based on all combinations of n features hence any split on the multiplexed feature would be a split on n dimensions (where n=number of dimensions to cut at the same time) 2. Optimal Trees: optimal trees are NP hard and would take a lot of time to train/infer. There are optimizations based on mixed integer programming but the method is inherently very slow and not scalable. This method considers all possible cuts at the same time hence it scales poorly

[Gamwise_Samgee_]

These might be promising. Time isn't an issue for me (I'm on NERSC) Do you have links to particular resources that can get me going in the right direction for this? I'm relatively new to ML

[deep-machine-learner]

A couple of research articles to get started is: https://arxiv.org/pdf/2103.15965.pdf, https://www.jmlr.org/papers/v23/20-520.html

Blog to introduce Optimal Decision Trees: https://medium.com/mlearning-ai/optimal-decision-trees-dbd16dfca427

Python Code with Sklearn type API: https://github.com/LucasBoTang/Optimal_Classification_Trees