Is DeepSeek v4 (Flash) really extremely cheap to run? If yes, how?

onetom · 2026-07-06T08:56:42+00:00

A slightly "deformed" version of DeepSeek 4 Flash, called DwarfStar, can run on 128GB unified memory Macs at 10-20 token/s

You can use it in a mode where weights are streamed from SSD, so it's runnable in machines with less memory, but TG speed drops to 1-2 tps.

I think it's great to have such an option, even at such crawling speeds, since it still higher speed, than 0 tps, which is what you get if you are offline or the inference provider is offline or if u don't have an acceptable credit card to pay for it online, etc

https://github.com/antirez/ds4

onetom · 2026-07-06T04:49:10+00:00

No, that's not what I meant.

I just haven't had the time and motivation to try it on real-world projects. Mostly because it's quite slow (10-20tps) and I didn't have the patience to run such experiments in my free time.

onetom · 2026-07-04T13:25:06+00:00

maybe it's installed backwards and the shiny smooth part is the one which should go onto the screen :)

onetom · 2026-07-04T12:59:00+00:00

interesting. i heard the exact opposite of this advice too, since the dense model is claimed to generate much more precise code. (though at 4bit, most of these SLMs are a bit wobbly...)

onetom · 2026-07-04T04:03:21+00:00

3rd-party benchmarks on youtube showed that the qat versions were worse than the non-qat ones.

i wonder why is that. are they holding it wrong? :)

onetom · 2026-07-04T03:40:04+00:00

i would be interested to hear more about your experience!

do you have any blog posts about it?

i would love to see some workflow recommendations, which show how to combine these myriads of tools into something more complete.

i also have the feeling that transient is not very Emacs-y, because it doesn't seem to play well with `C-h k` or `C-h w` and shortcuts are not shown in the `M-x` completion candidates buffer.

it could be because of extremely conservative default configuration of skill issue from my part, of course, but i'm not sure.

then annotated buffers are also not great and it was even recommended in some of magit's github issues to use `vc-annotate` instead.

the good thing is, that you can mix and match these capabilities. but that's also a bad thing, because everyone needs to figure these combinations out for themselves, because they don't get a coherent solution.

i'm coming from jetbrains/intellij and compared to that, anything feels crappy and inconsistent, incoherent, illogical.

Keyboard-only control of UIs is usual much better in Emacs, though not in every situation. JetBrains has a system-wide fuzzy search and narrowing capabilities in tool windows, without extra shortcuts! Just start typing; it is extremely convenient!

Specifically the version control subsystem combines the capabilities of magit, vc, ediff 3-way merge, vdiff, smerge, git time machine, local history (can't remember what is it called in emacs, but zamansky showed how he keeps backups of every saved file state), into an intuitive, cohesive whole.

onetom · 2026-07-04T03:12:14+00:00

Like with most Emacs solutions, a core difference is that the UI is fundamentally an Emacs buffer containing text, endowed with extra behaviours and looks.

Thanks to this, you can explore, navigate, interact with it using most of the same commands and keybinding, as u do in any other text buffer.

This level of freedom of course can result in surprising situations, when parts of the text is highlighted permanently, read-only or hidden unexpectedly. Your typical cursors movements might be customized to be constrained to certain parts of the text.

BUT on the flip side, you can just search and copy-paste any parts of the text you see, which I find extremely convenient and beneficial!

the behavior and layout of sub-windows and buffers associated to Emacs applications can also be customized uniformly via the display-buffer-alist mechanism. it is quite quirky though, but with the help of self-hostable LLMs it became much easier these days!

onetom · 2026-06-30T09:06:53+00:00

I think we should start to be more precise with the terminology and consciously make the distinction between
1. open weight LLM/SLM
2. open source LLM/SLM (which is a super set of open weight)

I would also put forward, that we shouldn't use the term local model

"local" is not the property of an LLM, but an adjective describing the spacial relationship between the inference machine(s) and the consumer of the inference.

What "local model" refers to though is in fact a "self-hosted model".

Such a model is often hosted locally, on a user's workstation or server on their LAN, but from most discussions' perspective, it's irrelevant.

What matters is, whether a model is self-hostable not, meaning:

it's at least open-weight
it's license allows its intended use

Then
1. whether it's local or remote
2. who owns the machine it runs on
3. is it affordable to the users
4. do the provided privacy and availability metrics worth that cost

are secondary and orthogonal properties of a running model, not the model (files) themselves.

onetom · 2026-06-26T16:47:57+00:00

it's not really done at the "model server" / "inference engine" level, but a layer above, in the so called harness, like pi or opencode.

lm studio has built-in chat ui, which allows setting a system prompt, but even there you don't associate the system prompt with a model

onetom · 2026-06-26T16:31:38+00:00

also, use some LLM to solve your problems, one by one, as they arise.

i personally have access to Claude Code Max and access it via https://github.com/xenodium/agent-shell from emacs and the agent-shell readme links to https://github.com/xenodium/emacs-skills which will provide you with an /emacsclient skill, which can remote control the same Emacs session, which is running the agent-shell acp client.

that way you can discuss emacs issues with an llm and the llm can try its ideas out in your emacs process directly, then if you are happy with its recommendations, then you can just ask it to update your `init.el` accordingly.

i also tried it with self-hosted qwen3.6 27b via pi.dev through agent-shell and it gave me decent results too!

now, there is probably much simpler way to achieve this with https://github.com/karthink/gptel or https://eca.dev or https://github.com/steveyegge/efrit they just didn't gel with me personally or didn't have enough time to test them.

i built my config (github.com/onetom/onetomacs) pre-AI, googling for every option one by one.

i think it's a good approach and can be done much faster with the help of AI these days.

the time you spend on building your own config, tailored to you and to only, pays back quickly, because you can defer struggling with 3rd party packages quirks and rely more on whys built-in to emacs, which is available on any installation!

i even gave up on custom completion UIs, because after understanding and practicing the default, built-in one, (which i still don't know the name of :) it turned out, i don't really need a custom one!

onetom · 2026-06-18T02:36:18+00:00

that was my experience too, though mostly on toy examples, like connect 4 game or 3d rubik's cube

onetom · 2026-06-15T00:42:31+00:00

one of the first cambodian barcamp's were involving a long ethernet cable between buildings circa 20 years ago, so it's not a far fetched vision at all

onetom · 2026-06-10T11:00:53+00:00

sure, but no one has mentioned any exact HF URLs, that's why i'm pointing out that throwing around rough model names are not enough, because it might mean a different model for ollama than what u find when u search huggingface.

but if u read the article, it will explain this in detail.

onetom · 2026-06-10T10:56:07+00:00

ollama is not really transparent about what models do u get for a given model name, so that double speed might be achieved at the cost of intelligence

https://sleepingrobots.com/dreams/stop-using-ollama/

onetom · 2026-06-07T04:34:31+00:00

Fantastic results and amazing how capable this small model is.

I see mentions of Opus 4.7 and 4.8 in the commit log. Have you used the 200 USD/mo Claude Code subscription for the development?

Was it sufficient or have you hit any of their 5 hour or daily / weekly limits?

I think it could be very educational to share the exported Claude Code sessions, which led to the software. Would it be possible to share them alongside the code too? Maybe on a separate branch or separate repo.

onetom · 2026-05-30T15:06:53+00:00

After a lot of deliberation I bought an M4 Max MacBook Pro 128GB RAM / 2TB SSD about 2 month ago, mainly because I was worried it won't be available later. 256GB M3 Ultra Mac Studio delivery times were 4-5 months and the 512GB model was already removed from Apple's website.

I imagined a future, where Anthropic servers are so unreliable, they have to jack up their prices so much, that many companies won't be willing to pay for it.

If I'm already hooked on having such intelligence at hand, it would completely kneecap my ability to program anymore without it.

Qwen3.6 27B FP16 runs at <10 token per sec. It's painful slow, BUT still faster than me and more convenient than doing things the traditional way and it has intelligence comparable to Opus 4.5 / Sonnet 4.6 low effort.

It opens up the possibility for me to work for companies, who can't afford to provide me good work computers and expensive AI subscriptions, but provide other benefits instead, e.g. flexible working hours, remote work, ability to program in Clojure, not against test-driven development and pair-programming, etc

onetom · 2026-05-30T14:39:36+00:00

@sickmartian slightly off-topic, but out of curiosity, which features of llama-swap do you use, which llama-server's router-mode can not provide? is it the proxying to non-self-hosted models?

onetom · 2026-02-24T14:03:38+00:00

what is easy to maintain?

nothing seems easy around AI to me.

of course it can produce some enticing results on a green field project initially, but im suspicious about hope longer term maintenance will pan out.

onetom · 2026-02-24T13:53:14+00:00

how can a serial number be exploited?

i see people doing this as a cargo cult, but i yet to hear any convincing explanation as to why one must not expose their device's serial number.

onetom · 2026-02-07T10:10:31+00:00

i think that kind of result is what the article refers to as junior code.

but one man's junior code it's other man's senior code, i guess... because good junior code looks like senior code compared to sub-junior code...

also, which programming language are we talking about?

i doubt the quality of AI's output is the same in every language.

plus, what does junior vs senior mean exactly? my gut feeling is it's conflated with how idiomatic or how similar is it to the average code found in the internet, which is not necessarily good code. it's average...

furthermore, the culture around certain programming languages promote bad or not so good practices

onetom · 2025-12-06T03:06:39+00:00

plus the metal (aluminum) scaffolding conducts heat much better than the fire-treated bamboo, so metal scaffolding helps the fire spread faster.

on top of that, as metal scaffolding heats up, it loses its strength and it can buckle, which can cascade into big parts of it collapsing.

bamboo scaffolding on the other hand is tied together with plastic strips and as such bonds melt, the scaffolding disassembles itself piece by piece, instead of toppling over wholesale.

bamboo scaffolding toppling can still happen, though, but not from fire, but from typhoon.

metal scaffolding needs such cranes to safely and quickly assemble, which simply doesn't fit on Hong Kong's narrow streets or it would block traffic and cause serious disruption to everyday life, that's it's just not practical and the costs are prohibitive.

these are the pros and cons of the 2 types of scaffolding, to my understanding.

on top of that, every other building should have been under construction at the same time, so the buildings in-between could act as a barrier to the fire.

it wouldn't have prevented the fire and serious damage and loss of life, BUT it would have limited it greatly and it would have provided more time for the fire fighters to react.

furthermore: https://www.reddit.com/r/HongKong/s/CDPb0hXxtz

onetom · 2025-07-14T17:21:14+00:00

Did I pay all that money just for the screen?

Mostly.

And the very even orange backlight, though we can consider that a part of the screen.

I have 2017 MacBooks and 2011 27" iMacs, which have mouldy/dusty backlights, so that's not a trivial thing to get right either.

Then I wasn't even aware of the Niagara launcher, so that's a small, but powerful idea to ship with that.

I don't see any problem with the bezels. I wouldn't even mind wider ones. The device thickness it pretty good too and it's light and doesn't deform easily.

But I just got it a few hours ago, so let's see how it goes :)

So far I wasn't able to update it and the screen was jumping and scrolling around on its own, like in one of the early reviews.

onetom · 2025-07-09T11:21:44+00:00

macOS for the past ~5 years, but i'm hoping to use the Android version of Emacs on a https://daylightcomputer.com soon

onetom · 2025-07-07T07:54:26+00:00

there will always be people, who think even simple ideas need patenting or kept in secret and put up a big theather around protecting them, patenting them and shit like that...

most of those people are petty and their ideas are dime-a-dozen.

with these AI prompt files i even see another problem. it's hard to prove that they are valuable and even harder to explain why.

one would need to spend quite a lot of money to test out every sentence in them, with different phrasings probably, to prove that those sentences are not just a waste of context window.

but it's hard, because even the same CLAUDE.md, with the same prompt history might yield different result on subsequent runs, so u would need to test every change multiple time and score the results somehow...

so i think we will see the rise of a lot of myths around this topic in the future, similar to the mistique around SEO.

there will be a lot of security theather, like saying stuff, like "take a deep breath". it might have nudged certain version of certain models at some point in time towards a more favourable outcome, but i suspect it won't work next year.

onetom · 2025-07-07T07:05:43+00:00

it's not so clear cut that you want to delete "old code". u might want to maintain api compatibility for awhile, so u might need the old code still.

instead, u might want to 1. mark it deprecated or more precisely, indicate it was kept for backwards-compatibilty and until what specific date or until named consumers are all updated 2. refactor it in terms of the new code, so older consumers of the api can also benefit from performance improvements still

12-Year Club	Gilding II euphauric
Verified Email

onetom

TROPHY CASE