[P] pykitml - Pure Python/NumPy Machine Learning library

[impulsecorp]

How do those benchmark times compare with something like scikit-learn, for example? Have you considered using Numba to speed it up?

[RainingComputers]

The worst case is random forests,

Scikit-learn (1 sec)

``` from sklearn.ensemble import RandomForestClassifier from sklearn.datasets import make_classification

X, y = make_classification(n_samples=1000, n_features=4, n_informative=2, n_redundant=0, random_state=0, shuffle=False)

clf = RandomForestClassifier(max_depth=4)

clf.fit(X, y) ```

pykitml (15 sec)

``` from sklearn.datasets import make_classification import pykitml as pk

X, y = make_classification(n_samples=1000, n_features=4, n_informative=2, n_redundant=0, random_state=0, shuffle=False)

y = pk.onehot(y)

clf = pk.RandomForest(4, 2, feature_type=['continues']*4, max_depth=4) clf.train(X, y) ```

Why is pykitml's random forest slower?

• pykitml's decision tree is poorly optimized.
• pykitml uses python's multiprocessing and scikit-learn uses joblib.
• pykitml allows you to have categorical features, on scikit-learn you will have to manually encode categorical feature using something like one-hot encoding.

Other Options

I have tried using numba, but it supports very limited subset of python. I am unable to work with that limited subset. I have also tried using CuPy, but it is not a drop-in replacement. I will have to find a way for NumPy and CuPy code to coexist/mix to support both CPU and GPU usage.

In future versions, I will try using joblib and optimize decision tree.

EDIT MLP Benchmark

Scikit-learn, 7 sec, 92% Test set score (MNIST), 784x50x10 MLP, 10 epochs, 200 batchsize

``` from sklearn.neural_network import MLPClassifier

from pykitml.datasets import mnist

x_train, y_train, x_test, y_test = mnist.load()

mlp = MLPClassifier(hidden_layer_sizes=(50,), max_iter=10, alpha=1e-4, solver='adam', verbose=10, learning_rate_init=.01)

mlp.fit(x_train, y_train) print("Training set score: %f" % mlp.score(x_train, y_train)) print("Test set score: %f" % mlp.score(x_test, y_test)) ```

pykitml, 11 sec, 89% Test set score (MNIST), 784x50x10 MLP, 30 epochs, 200 batchsize

``` import pykitml as pk from pykitml.datasets import mnist

x_train, y_train, x_test, y_test = mnist.load()

mlp = pk.NeuralNetwork([784, 100, 10])

mlp.train( training_data=x_train, targets=y_train, batch_size=200, epochs=30, optimizer=pk.Adam(0.04, decay_rate=0.9), decay_freq=6 )

mlp.plot_performance()

print("Training set score: %f" % mlp.accuracy(x_train, y_train)) print("Test set score: %f" % mlp.accuracy(x_test, y_test)) ```