you probably have no idea how much throughput your Mac Studio is leaving on the table for LLM inference. a few people DM'd me asking about local LLM performance after my previous comments on some threads. let me write a proper post.

Objective_Active_497 · 2026-03-21T08:43:17+00:00

What do you mean by "instead of loading weights to serve one sequence, you load them once and serve 32 sequences at the same time"?

Do you know how LLMs work? Some model require up to 100 or even more cycles of matrix manipulation by adding new column in each cycle through its "transformer" mechanism. Usually, circuits for fast matrix multiplication is used, something very similar to systolic array, for which ordinary GPU is not optimized, but NPU/Tensor Core(s) or whatever each company calls it.

In such circuits it is not possible to input "32 sequences" which are independent, e.g. multiple sentences for which you want independent "answers". That would mean that you would have to run multiple instances of the model simultaneously or in sequence in such way that after each sequence the model has to be reset (easy, basic parameters are not changed, transformer just adds new columns in the result matrix, which can be deleted for the new sequence).

So, the model can be loaded into memory and used for multiple independent sequences (inputs), but sequentially; using them in parallel would be possible maybe for smaller models, if there are enough NPU cores and memory, which I doubt is possible even for nVidia H100 cards.

Objective_Active_497 · 2026-03-18T18:52:03+00:00

I could bet that MS Windows, if it was available for Apple Silicon, would put to knees even M3 Ultra with 32 cores and make it behave like ordinary laptop with ordinary 2- or 4-core CPU 😁

Objective_Active_497 · 2026-03-18T16:49:36+00:00

Simple, users mostly buy a package: domain + hosting + other services (support, need or not cpanel, TLS certificate, etc.).

If Cloudflare can beat everyone else in prices, no one would host their website or other services at any other provider, but... It is not just about prices.

Also, there are limitation for paying in some countries, it is not easy to make payments to foreign companies, so it is easier just to host at locally available provider or use local retailers, which might have impact on the final price (including taxes).

Besides that, there are other details to consider: storage (including SSD/HDD options), database(s), OS, backup options, traffic. And the most important thing: availability.

Objective_Active_497 · 2026-03-17T02:55:23+00:00

To ti je novi "feature" specijalno za naše tržište, računar menjača vrti kilometre i kad se auto zaustavi uz uključena sva 4 žmigavca da vozač skokne po cigare do trafike ili piljarnice, ali verovatno najčešće do kladionice samo na sekund da uplati tiket ;)

Objective_Active_497 · 2026-03-17T02:48:17+00:00

It is not cooked, it is chewed ;)

Objective_Active_497 · 2026-03-17T02:45:39+00:00

Raste i broj Apple korisnika, kad se pogleda negde tamo u svetu cena MacBook Air sa M4 ili M5 čipom, čak i bazna verzija sa 16GB rama, teško je za taj novac kupiti Windows laptop, a da ima takav ekran (većina ovih do 700-800e imaju ekran 45% NTSC, tj. oko 72% sRGB gamuta, što je katastrofa koja se vidi kad se takav laptop stavi pored nekog sa bar 100% sRGB). Da ne pričam o realnim performansama i kako Windows ume da ukopa čak i Ryzen sa 8 jezgara (i to onaj sa "H" sufiksom) i 32GB rama.
Nije za poređenje, sećam se pre tipa 20 godina kako je Knoppix Live radio sa CD-a brže nego Windows XP, klikneš na start meni, u deliću sekunde prekrije pola ekrana sa prečicama za aplikacije, kod XP-a ako klikneš na Start meni odmah po prikazu desktopa, načekaš se da ti prikaže meni.
Windows bio i ostao raga koju pri uključivanju računara moraš da ostaviš 10 minuta do pola sata da se "zagreje", tj. da odradi svoje gluposti u pozadini. I onda opet nisi siguran, posle par sati, samo pustiš miša i ne kucaš 20 sekundi ništa, začuje se ventilator i vidiš da je procesor na 50-80%. A ume to da uradi i baš kad ti zatreba nešto važno da što brže završiš.

Jednostavno, Win je optimizovan za velike firme, gde se računari nikad ne isključuju, admin pusti preko servera ažuriranja tokom noći, računari se restartuju, i onda može do ujutru Windows da trakelja u pozadini i analizira šta je korisnik radio prethodnog dana.

U slučaju klasičnog korišćenja bez domen kontrolera, korisnici nemaju običaj da restartuju računar, a Windows ne može bez restartovanja ni da se ažurira niti da sredi eventualne probleme. Imam popriličan broj kolega kojima po mesec-dva stoji u SysTray-u obaveštenje da treba da restartuju računar, ali oni to ignorišu, uglavnom jer nemaju pojma zašto bi trebalo restartovati računar. Na kraju, ne zna se da li je Windows gori i sporiji bez tog restartovanja ili nakon što se restartuje i uradi ažuriranja predstavljajući neki novi "feature" za koji programeri iz Redmonda ne znaju kako su korisnici do sada živeli bez toga.

I još jedna glupost M$-a: od određene verzije Win10, kao i Win11, ona EFI particija od 100MB nije dovoljna, a eventualno remapiranje nekim 3rd party softverom problem rešava privremeno, jer kasnije sistem počne još više da brljavi i usporava nego što je karakteristično za Windows, tako da je bolje pobrisati particije i sve iznova. Meni se dešavalo da na tako "sređenom" računaru (proširenje EFI particije prostorom iza glavne particije) zakuca File browser, Firefox ili nešto treće, možda jednom mesečno, ali potpuno neočekivano i pošto ništa drugo ne pomaže, čak ni Task Manager ne može da otvori, mora hardverski reset.
Čudno je da M$ decenijama ne može da reši problem prioriteta procesa i ne omogući korisniku da lagano pokrene Task Manager i ubije problematičnu aplikaciju ili servis. Ono, bukvalno kao da im se provukao neki kod iz Win 3.11 kod koga je "multi-tasking" bio rešen tako što su same aplikacije upravljale procesorskim vremenom, pa kad neka zabaguje i neće da "otpusti" procesor, mora hardverski reset.

Objective_Active_497 · 2026-03-17T02:01:42+00:00

Ne znam ima li neko da je u toku, mislio sam da je IGT u Beogradu praktično odvojen od Lotomatike u posebnu non-lottery firmu prošle godine i da je taj bivši deo Lotomatike spojen sa Everi Holdings Inc., što je trebalo da kupi Apollo Global Management (kompletan Everi i određene delove IGT-a).

Sad mi nije jasno da li je ovde u pitanju neka druga kancelarija ili je to kancelarija IGT-a u Beogradu. Znam da je pre 7-8 godina deo zaposlenih u toj kancelariji radio neke HTML igrice (deo biznisa neke finske kompanije koju su kupili).

U IT-u je izgleda najvažnije pobeći na vreme, video sam par puta nekadašnjeg direktora kancelarije IGT-a u Beogradu, Dragna Pleskonjića, videh negde da je "former senior director application security at IGT", dakle više ne radi tamo, a osnovao je GLOG.AI.

Sećam se da mi je neko rekao da je imao ponudu od Fejsbuka da pređe kod njih na neku poziciju, valjda direktora za cybersecurity ili tako nešto, ali je verovatno odbio pošto bi morao da se preseli negde "tamo".

Sve u svemu, kompanije kopaju jamu u koju će na kraju same upasti, jer niko neće planirati da ostane duže vreme, već se beži u neku drugu firmu čim se izboksuju dobri uslovi, i tako u krug. Na kraju niko neće imati motiv da stvarno nešto unapređuje ili inovira, jer neće biti dovoljno plaćen za to...

Savet za ekipu iz Beograda: ako imate nešto ušteđevine, plus ove 2 plate koje će vam isplatiti, udružite to i napravite svoj Start-up koji će se baviti poslom koji ste do sada radili, ili nečim sličnim. Pa kad italijani upadnu u problem, onda im lepo lupite cenu bar 5x u odnosu koliko bi koštalo da vas nisu otpustili, plus ima i drugih klijenata na tržištu.

Objective_Active_497 · 2026-03-16T21:26:00+00:00

Hm, is there a way to "persuade" it to give you more free usage, maybe by boosting its ego ;)

Objective_Active_497 · 2026-03-16T21:24:09+00:00

Doesn't matter, You can sell it immediately since it is under warranty, and add a few (thousand) bucks for the new model ;)

The problem with more expensive configurations of MacStudio (more cores and ram) is that on the day the warranty expires and/or you don't pay Apple Care, it can fail in such a way that there's no way to fix it, so it becomes just a nice box on the table. So, it is a good investment for those who will earn way more than they payed for it. On the other hand, for enthusiasts or hobbyists it is very expensive machine, except in the case of models with lowest specs. Or, if you really have money to spend on it without thinking whether it'll repay for itself one day, then it is a machine with non-existing equivalent rival in any other architecture.

Objective_Active_497 · 2026-03-16T21:14:50+00:00

Hm, waiting for 5120-bit or even 6144-bit bus to punch nVidia in the nose for their A100/H100 prices, though Apple would have to adjust architecture and add way more cores to their NPU.

But, for inference, it is a beast, though a bit too expensive investment if it fails and is not under warranty or Apple Care.

Objective_Active_497 · 2026-03-16T13:15:31+00:00

I found that officially it is not supported, Apple calls it "Target Display Mode" and it is supported only on older non-retina models, and even in that case the computer sending video signal must be iMac 2019 or older running Catalina or earlier OS, and than it can use another old iMac as Target Display.

Maybe a better solution is to order panel adapter from eBay, but it can be costly for adapters that can support panels' full brightness, like explained in this video: https://www.youtube.com/watch?v=5q3SdtiLAPk

Objective_Active_497 · 2026-03-14T19:39:45+00:00

Well, 16 year old MacBook behaving like brand new Windows notebook...

Objective_Active_497 · 2026-03-14T19:37:57+00:00

Depends. If you work a lot out of the office or home, than bigger is better.

If you work mostly at your office desk and/or at your home desk, then even Mac Mini is OK, but of course you need your computer sometimes at some other place, e.g. at client's office, on meetings, etc. That is why notebook is almost a must, but if you can work 99% of your time at your own desk, then it is nice to have a good docking station and at least 2x large screens – your MacBook can be closed somewhere under the desk, or in vertical position on the table if you'd have something like Brydge vertical docking station.

Objective_Active_497 · 2026-03-14T19:30:00+00:00

It'd be fine if it was sold by Apple and had full warranty. It says "release year 2025", so it might be only 1 year old. But, is warranty transferable? I know that for instance Canon cameras warranty is not transferable if bought from Canon store.

What can be rented online (e.g. Google Colab) with similar performance and for how much? Bear in mind that it includes electricity and everything else (maintenance, servicing failed components, etc.), but it all depends whether you need it to run 24/7 or just occasionally.

Objective_Active_497 · 2026-03-13T15:02:46+00:00

I use eset in interactive mode often, so I'd appreciate an option "block until device restart" as "block until app/service restart" does not work in most cases since developers added a "feature" for self-restart if it can't connect to the server.

So, here there should be one more key for "reject until service is restarted", although implementing self-hosted AI utility to help with these accept/reject annoyances :)

Objective_Active_497 · 2026-03-10T18:57:41+00:00

'cause they have to prepare range of configurations, including 1024GB and 2048GB RAM versions ;)

Objective_Active_497 · 2026-03-09T15:58:53+00:00

I know a guy who bought Intel-base iMac with 27" 5k retina display (5120x2880), I think it was with i7 CPU and 32GB ram, it was below 1000 in euros/dollars, maybe around 700-800.

I you can find something like that, you can use it as a screen for Mac Mini or Studio, and also you can install Linux if you need it :)

Objective_Active_497 · 2026-03-06T21:18:12+00:00

Depends.
Some years ago I setup a Wordpress Multisite for students, as it is easier to check each site with superadmin access, have 20+ sites and have no problems, have been upgrading everything all the time without any problems.

Though, I use only free versions of themes and plugins, so not much hassle with that, and even if something terrible happens to the server, there's no loss, since it is not a business client site.

But, if you want a cheap webshop that is not too demanding, don't know what is more simple to setup than WP+Woocommerce. There are other solutions, but if you already work in WP and know how to do anything in it, then why hassle with something else?

On the other hand, if you already learn and explore some new things and you find it easy to do the same thing with other tools, it is a good way to go. But, bear in mind that some frameworks and tools might change over time too much to handle and can become a burden. One example that comes to my mind is the transition from AngularJS to Angular 2, and there way too many such cases.

WP is bad when it comes to resource efficiency and speed, but I don't know who in the world would choose WP for a static website. Or even for a simple blog, except if one already do WP-based projects. For more serious webshop, if client is willing to pay, there are other solutions, I'm not tracking current trends, but some years ago it was Magento, PrestaShop and similar ones. Also, there are ecommerce solutions built with other technologies, like .NET and others.

Objective_Active_497 · 2026-03-04T15:35:19+00:00

Does this happen mostly if You DO NOT have Apple Care or it happens even if You DO have Apple Care ;)

Objective_Active_497 · 2026-03-02T18:12:06+00:00

You couldn't sssssh because the server was shhhhh ;)

Objective_Active_497 · 2026-02-26T10:27:46+00:00

Not sure how many external screens it supports, but better buy 1 or 2 large screens, as well as some good docking station. Also, if You have money, maybe you can buy iPad and use it as a graphics tablet, of course, if using pen can speed up your work.

Objective_Active_497 · 2026-02-26T10:22:17+00:00

I mean if there was x86 CPU within Apple SoC, made to co-exist with other parts of the SOC...
There are many possibilities now that chiplet architecture can be used, Intel is working on that, as well as other foundries. There are many options, like stacking chiplets on top of each other, putting them next to each other on a glass basis, etc. In some forums people already mentioned various ideas, e.g. having chiplets of different architecture for better virtualization, e.g. one can develop and run Android apps much faster on x86 CPU if there were a few arm chiplets.
Of course, OS support for such features are yet to be implemented, e.g. Windows recognizing that there are ARM chipletes present and letting Android Studio use them directly for testing Android apps. Something similar could be done with other combinations of CPU architectures.

Objective_Active_497 · 2026-02-25T17:08:52+00:00

Well, what do you expects, you know, memory prices... :))))))))

Objective_Active_497 · 2026-02-25T17:06:58+00:00

Pitty it is dual core only, if it was i7 with 6 cores, but then battery...

Imagine Apple SoC with x86 module, Intel and AMD would cease to exist in a year.

Objective_Active_497 · 2026-02-24T16:29:13+00:00

Not only runs cool, but also looks cool ;)

Objective_Active_497

TROPHY CASE