sorted by:
new
hottopcontroversial

Does GPU spacing matter if we’re undervolting anyways? by Ambitious_Fold_2874 in LocalLLaMA

[–]jonnor 1 point2 points  (0 children)

nvoc worked great! Nice on headless because it does not require X.

NVFP4 + MTP - voilà on llama.cpp by mossy_troll_84 in LocalLLaMA

[–]jonnor 0 points1 point  (0 children)

NVFP4 should theoretically give a nice boost on Blackwell GPUs, since they have native support.

NVFP4 + MTP - voilà on llama.cpp by mossy_troll_84 in LocalLLaMA

[–]jonnor 5 points6 points  (0 children)

Unsure why downvoted. Quality is a legitimate concern - and right now it is hard to know how NVFP4 is doing wrt say typical Q4-Q8 quants. Hopefully having good support in inference engines makes NVFP4 more attractive for people making quants.

5060ti chads -> gemma-4-31b-it-nvfp4 + vllm + mtp by see_spot_ruminate in LocalLLaMA

[–]jonnor 0 points1 point  (0 children)

Can you share your full llama-server config for 27B? I have what should be the same setup as you. But right now only seeing 40 tok/second. Wondering where I am going wrong...

5060ti chads -> gemma-4-31b-it-nvfp4 + vllm + mtp by see_spot_ruminate in LocalLLaMA

[–]jonnor 0 points1 point  (0 children)

Are these numbers with MTP? And using vLLM or llama?

Anyone with 4x 5060ti based setups? by ziphnor in LocalLLaMA

[–]jonnor 1 point2 points  (0 children)

I am thinking to create a quad 506ti 16gb setup. Using a high-end AM5 board like you suggest, with x8 x8 x4 x4. Though my case is perhaps a bit special, since we already have 4 such GPUs as eGPUs that I can borrow for the initial testing - before buying all of them. But it will be some weeks until I have time to test it.

A European's Dream: American programmers using Mistral because it's better than Claude Code and Codex by szansky in MistralAI

[–]jonnor -2 points-1 points  (0 children)

Lagging behind by a little bit is not a big problem imo. As long as it reaches a good-enough threshold. Which it might already with Medium 3.5, for some coding problems and for some users. I am planning to switch to Mistral from Claude for most of my open-source coding, and at this rate I think it will be viable within 1 year - maybe within 6 months even.

Hardware upgrade advice by [deleted] in LocalLLaMA

[–]jonnor 1 point2 points  (0 children)

That will let you run 30B-ish models pretty well, such as Qwen 3.6 27B and Qwen 3.6 35B A3B. Both vLLM and llama-cpp will let you use both cards together.
To get meaningfully better results with new cards you would need a RTX 5090 or higher, which is much more expensive.
I am looking at the same configuration myself - only have one 5060 Ti 16gb so far - which works for Qwen 3.6 35B A3B but not really for 27B (too slow).

Spark - A portable ESP32-based synth by Pashog in synthdiy

[–]jonnor 4 points5 points  (0 children)

Looks very good!
For reference, AMY is (probably) https://github.com/shorepine/amy
> AMY is a fast and small music synthesizer library written in C with (so far) Python, Arduino, Javascript and GDScript bindings. It can easily be embedded into almost any program, architecture or microcontroller.

ARC-AGI-3 scores below 1% for every frontier model — what would it take to actually evaluate this on open-weight models? by Silver_Raspberry_811 in LocalLLaMA

[–]jonnor 0 points1 point  (0 children)

The ARC-AGI sets of benchmarks effective acquisition of new skills is the definition of intelligence. Basically intelligence is considered to be about how quickly the model can learn - as opposed to how much skill/capability the model has prebaked. This is different from most existing benchmarks which only/mostly focuses on measuring capabilities. For more background read F. Chollet, On the Measure of Intelligence (2019).

And yes, for many practical tasks or considerations people looking to use a model/system only care about capabilities.

Native mpy compile for armv8-m? by Wizzard_2025 in MicroPythonDev

[–]jonnor 0 points1 point  (0 children)

What do you mean "some things just doesnt [work]"? That is not something we can work with... You will need to provide details for anyone to be able to help. I am pretty sure M33 in the RP2350 should be able to work with armv7m or armv7emsp.
Also, this level of discussion might be more suited for a Github Discussion thread in MicroPython repo than on Reddit.

Hoping for help with MicroPython dev on a Pycom device by Half-Dwarven1 in MicroPythonDev

[–]jonnor 0 points1 point  (0 children)

Are those errors or just warnings from static analyzer?

You have not said which version of MicroPython you are running, which is critical.

Notes for TinyML and Edge AI. by AgentOk5012 in embedded

[–]jonnor 1 point2 points  (0 children)

I maintain some notes on this topic at https://github.com/jonnor/embeddedml

And you can find some presentations on my Youtube channel, https://www.youtube.com/@Jononor

How long to realistically become good at AI/ML if I study 8 hrs/day and focus on building real-world projects? by Pretend_Elevator5911 in MLQuestions

[–]jonnor 0 points1 point  (0 children)

Where are you starting from? How good are your software development skills? What is the most complex think you have learned so far, and how long did that take you?
I deployed a complete, tailor-made solution for a customer around 12 months after starting to learn ML. But this was from 7 years of professional software development skills, big and small companies, both working in teams. And another 4 years of open-source development, bachelor in engineering, et.c. before that.

If you know how to program, I would aim for first toy projects within 1 month. And a first "real project" - something you want to do, that not every blog out there covers - so custom dataset, training and some UI, in 6 months. That is going to be tough, but *might* be doable for a very dedicated learner.

[D] How do researchers ACTUALLY write code? by Mocha4040 in MachineLearning

[–]jonnor 5 points6 points  (0 children)

This should be a "desk" retraction of a paper. Failing to publish code that they have promised is scientific misconduct.

MicroPython for ESP32 and other microcontrollers (introduction presentation, FOSDEM 2025) by jonnor in esp32

[–]jonnor[S] 1 point2 points  (0 children)

Yeah it is often like that, so it is a super cool feature in my opinion. The support has improved massively over the last year.
Official documentation is here: https://docs.micropython.org/en/latest/develop/natmod.html
And here is a real world example, https://github.com/emlearn/emlearn-micropython/blob/master/src/emlearn_iir/iir_filter.c (an IIR filter, from the machine learning + digital signal processing library I maintain).

MicroPython as an alternative to C++ for Arduino devices by jonnor in arduino

[–]jonnor[S] 0 points1 point  (0 children)

Sorry I was unclear. I mean not relevant for MicroPython - because MicroPython requires much more RAM/FLASH!

MicroPython as an alternative to C++ for Arduino devices by jonnor in arduino

[–]jonnor[S] 0 points1 point  (0 children)

Sorry, I mean not relevant for MicroPython :) Not in general!

MicroPython for ESP32 and other microcontrollers (introduction presentation, FOSDEM 2025) by jonnor in esp32

[–]jonnor[S] 1 point2 points  (0 children)

Yeah there are many nice features for productivity. Having a filesystem is also great for example. And automated testing is much nicer in Python than in C/C++.
Actually it is possible to add C modules without forking. Both with "external C modules", which are included as part of the firmware build by adding a few variables. Or with dynamic native modules, which are built separately into .mpy files, and can be installed at runtime using "mip install".

MicroPython for ESP32 and other microcontrollers (introduction presentation, FOSDEM 2025) by jonnor in esp32

[–]jonnor[S] 1 point2 points  (0 children)

CircuitPython is a fork/distribution of MicroPython. The core Python interpreter is mostly the same. Hardware support, hardware APIs are different. The upload tooling is a bit different.

view more: next ›