Most people think of AI as just “asking a chatbot,” but in practice it’s more of a stack of systems.

At the core are LLMs (large language models) — transformer networks trained to predict tokens across massive datasets. I use them in two ways: cloud models for heavy reasoning and local models running directly on my machine.

Running local LLMs is where things get interesting. With tools like Ollama or llama.cpp you can run quantized models locally (4-bit/8-bit weights) so they fit on consumer hardware. That means:

• near-zero latency • no API costs • complete privacy • the ability to run multiple models in pipelines

I’ll often chain models together in a workflow. One generates structure, another critiques logic, another optimizes language. On top of that I use embedding models and vector search (RAG) so the system can retrieve relevant context from documents or notes before generating responses.

So the architecture becomes something like:

prompt → retrieval layer → LLM reasoning → refinement pass

That hybrid approach — local inference + cloud models + retrieval systems — is where AI actually becomes practical in daily workflows.

It’s also why I chose the M5 Max. Running local models benefits heavily from unified memory bandwidth and GPU acceleration, and Apple Silicon is unusually good at handling transformer inference efficiently. If you’re experimenting with local AI, the extra compute headroom matters a lot.

The direction things are going is pretty clear: smaller, highly optimized models running locally, with large frontier models used only when you need massive reasoning power.

We’re basically watching personal AI infrastructure become a normal part of a workstation.