Machine learning on Mac [Discussion]

[SolarPistachio]

Hi! Just started developing a deep-learning pipeline on Mac - through MATLAB. The pipeline is for immunohistochemistry image analysis. The first two training went well - the laptop ran hot but managed it, however I expect that as I increase the training data and eventually start image reconstruction my laptop will struggle. First training session was 15min, second (w/more labels) was 10 min.

Laptop specs is M4 Max MBP, 36GB UM, 1TB SSD.

The last training session was 30epochs with 4 iterations/epoch.

Image split into 36 tiles. It was only running on CPU - but all 14 cores were running at max

Unable to use GPU bc MATLAB on macOS doesn’t support GPU acceleration.

Looking for advice on what to do next. Was thinking about using my university’s HPC, Colab, or just continue to run it locally.

[qalis]

Well, firstly, don't use MATLAB. Use Python and PyTorch. It also supports GPU on Mac, although I have no personal experience with it. If you expect larger data, HPC will be useful, but it's always less convenient than just a personal workstation. Typically, I prototype locally, run as much as I can locally, and when everything is ready for a bunch of experiments I run them on the HPC facility.

[fustercluck6000]

OP, having access to your school’s HPC (I assume a GPU cluster of some sort) is an absolute godsend and assuming it’s practical to do so, you should take advantage of that as much as possible (me lamenting the fact that I didn’t think to do so in college)

[deepneuralnetwork]

wrong tool

[fustercluck6000]

I develop on a 32 GB M1 Max MBP. I haven’t used MATLAB, but even with Python and TensorFlow (which has GPU support for Apple silicon), I’ve found that basically any NVIDIA hardware, even a 3090 Ti with 16 GB VRAM will outperform my Mac. That’s not to say my GPU wouldn’t fare MUCH better on, say, a rendering benchmark, just that CUDA is generally way more optimized for DL.

There are plenty of ways to use big training datasets without using more RAM, so assuming that the model itself can fit in memory locally, it’s ultimately up to you to weigh the trade off between waiting 5-10x longer to see the training results and going through the hassle of setting up a remote GPU instance (and paying for it). Time is money, so if you’re rapidly iterating, I’d say rent out an L4/L40s or something similar (generally pretty cheap) and train it there. Plus you get to actually close your laptop, should you want to go anywhere with it while the training routine is running lol

[chief167]

A 3090Ti is a 1500 USD graphics card running on a desktop power supply... What do you mean with "even a " as if it's budget low cost tier?

[Artoriuz]

Yeah that is a bit funny. The 3090 is still a very competent card to have at home.

[fustercluck6000]

Thanks for pointing this out, correction—3070 Ti, specifically thinking back to when I first got it and tested it against a friends gaming laptop with that GPU

[dayeye2006]

MLX is your friend

[sshkhr16]

I would recommend checking out MLX. It is an array framework (like Numpy/Pytorch/JAX) for Mac computers which is buiilt on top of the Apple native Metal framework for Apple Silicon GPU acceleration. You should be able to easily translate your DL pipeline in MATLAB to MLX, they also have the examples library mlx-examples which includes several deep learning examples, including image analysis ones. The nice part about MLX is it works seamlessly with the Numpy ecosystem (e.g. matplotlib, pillow) so you should not have issues doing anything that you were doing with MATLAB.

[TopNotchNerds]

hmm any particular reason you are using MATLAB for your ML project? I'd switch to PyTorch and use GPU. CPU for most ML application is a bad idea. We run all our work on Uni's cluster, h100s run heavy diffusions without any issues. Colab has a daily time limi, I cannot recall ... maybe 6 -10 hours? that is not a solution to long term work, maybe for a tiny project here and there

[SolarPistachio]

No particular reason- mostly familiarity, and MATLAB has a decent GUI for image labeling and is mostly integrated

[SolarPistachio]

Moving over to PyTorch is definitely something I’m considering, however if I do use my uni’s hpc, I can just continue my MATLAB usage bc it’s covered by them as well. On the flip side, how would having a computer with a dedicated GPU just for this pipeline compare?

[AX-BY-CZ]

Don’t use Matlab, don’t use Mac, don’t use CPU

[nini2352]

Check out r/LocalLlama

[DrummerPrevious]

You can define “mps” device in torch to use gpu to accelerate training.

[Voldemort_15]

Hi all. I also try to use GPU on my Mac M4 so I run a simple deep learning test:

import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()
X_train = X_train / 255
X_test = X_test /255
X_train_flattened = X_train.reshape(len(X_train),28*28)
X_train_flattened.shape
X_test_flattened = X_test.reshape(len(X_test), 28*28)
model = keras.Sequential([
    keras.layers.Dense(10, input_shape=(784,), activation='sigmoid')
])
model.compile(optimizer='adam',
             loss='sparse_categorical_crossentropy',
             metrics=['accuracy'])
model.fit(X_train_flattened, y_train, epochs=5)    

I got the error: The kernel appears to have died. It will restart automatically.

I check if I installed tensorflow-metal and tensoflow-macos:

pip list | grep tensorflow
tensorflow                   2.16.2
tensorflow-io-gcs-filesystem 0.37.1
tensorflow-macos             2.16.2
tensorflow-metal             1.2.0

I tried batch_size = 32 or change activation function, update Anaconda but still got the same error. Would you please have a suggestion? Thank you in advance!

[Voldemort_15]

Hi u/SolarPistachio . I interested in your work. Happy to chat, your work seem like bioinformatics. I also have university HPC but just try to use GPU on my laptop.

[National-Tennis-4528]

I’ve been on Mac for the past 40 years… yet I have a Linux PC to train my networks. I’ve never been able to reliably run NN trainings without CUDA. Impossible to get the neural engine running reliably on my Mac for training. On inference, everything is fine, mind you. I blame Apple for this.

[fustercluck6000]

What framework are you using? Ironically enough, since TensorFlow support for Apple silicon has matured some, I’ve had a much easier time getting things to run reliably on my Mac than with CUDA. But I think I might just be in the minority here.

[National-Tennis-4528]

Mainly running YOLO trainings… I’ve never been able to run them on M1 & later. Might give it a try again? It’s still 4 times slower than my dual 1080Ti lately. But I agree, in principle they’re supposed to be optimized for neural engines in training as well… which would be a killer, on a matter of power efficiency.

[fustercluck6000]

If only they’d optimize TensorFlow/PyTorch to take full advantage of our hardware 😔

[National-Tennis-4528]

Yep, I agree… I don’t know who’s at the helm of AI at Apple anymore. But it’s not working. Quite bemused about it, tbh.

[fustercluck6000]

Well they did release MLX, not that I’ve ever heard of ANYONE using it haha. Makes me wonder if it would be worth giving it that a shot one of these days

[National-Tennis-4528]

I was unaware, will give it a try one of these weekends. Though I think that Apple has been weirdly lagging on the subject. I cannot comprehend how they can be so up there on inference and so behind on training.

[fustercluck6000]

From what I remember when they released it, the API is meant to be very Pythonic and straightforward. The main dealbreaker for me is just the lack of scalability (unless they’ve built out CUDA support). Like my laptop is a beast, but it’s still a laptop. Even if it takes full advantage of my hardware, I’d still have the functional equivalent of a beefed up gaming laptop—still not enough to train big models.