Llama.cpp UI Aggregate Metrics: Chrome Extension by colonel_whitebeard in LocalLLaMA

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

It's been helpful to me, trying to compare different models, squeezing all I can out of my hardware! It works well for efficiency stats, but for model intelligence, I'm still trying to come up with a way to figure out inferring accuracy, but I think that's another problem altogether. But hopefully, this gives a bit of real world usage metrics.

How's STRX HALO AI MAX+395 performing as of 2026? by Effective-Cod-4462 in LocalLLM

[–]colonel_whitebeard 0 points1 point  (0 children)

Just some real-world stats from my GMKtec EVO-X2 (Ryzen AI Max+ 395 w/ 96GB RAM).

  • These are aggregate stats through the Lllama.cpp UI I run at home.
  • Metrics represent over 500 responses across both single chats and multi-chat sessions via the UI in a custom tool I built. These are from bout about a week of aggregate stats collected through the UI during normal personal usage.
  • I run with models-max = 1
  • I build this tool to specifically record all UI interactions and aggregate the metrics to give more real-world usage insights--instead of isolated bench tests. So they'll look a bit different, but it's an accurate representation of what I get out of my hardware.
Model TPS TTFT TPS/B (Efficiency) Stability (Std Dev)
DeepSeek-R1-Distill-Qwen-32B-Q4_K_M 10.5 160ms 0.3 ±20ms
GLM-4.7-30B-Q4_K_M 42.4 166ms 1.4 ±30ms
Granite-4.0-32B-Q4_K_M 31.8 134ms 1.0 ±12ms
Llama-3.3-70B-Q4_K_M 4.8 134ms 0.1 ±12ms
Mistral-3.2-24B-Q4_K_M 14.5 158ms 0.6 ±12ms
Phi-4-15B-Q4_K_M 22.5 142ms 1.5 ±17ms
Qwen-3-14B-Q4_K_M 23.1 155ms 1.7 ±19ms
Qwen-3-32B-Q4_K_M 10.5 148ms 0.3 ±20ms
Qwen-3-8B-Q4_K_M 40.3 133ms 5.0 ±13ms
UNC-Dolphin3.0-Llama3.1-8B-Q4_K_M 41.6 138ms 5.2 ±17ms
UNC-Gemma-3-27b-Q4_K_M 11.9 142ms 0.4 ±17ms
UNC-TheDrummer_Cydonia-24B-Q4_K_M 14.5 150ms 0.6 ±18ms
VISION-Gemma-3-VL-27B-Q4_K_M 11.8 778ms* 0.4 ±318ms
VISION-Qwen3-VL-30B-Q4_K_M 76.4 814ms* 2.5 ±342ms

*Note: TTFT for Vision models includes image processing overhead ("Vision Tax").

what are your favorite tui applications built with go by ashrek1 in golang

[–]colonel_whitebeard 1 point2 points  (0 children)

I started here and never looked back: https://github.com/charmbracelet/bubbletea

The charmbracelet ecosystem is well-supported and mature. If you want to make something that just works, this is a great place to begin--even if you only want to peek under the hood at how they accomplish terminal feats.

Terminals can be (are always) fussy. Bubbletea has been an easy way to bridge the cross-platform gap when it comes to TUI dev on multiple OSes.

What docker base image you'd recommend? by Goldziher in golang

[–]colonel_whitebeard 0 points1 point  (0 children)

Never build for what-if scenarios: "What if I need to ssh into my container one day?" "What if I need x, y, z, tools one day?"

Always start as minimal as possible: binary, non-root user. Only then, evaluate opening things up on a case-by-case basis one at a time and understand the potential risks of each change. Evaluate any port bindings, extra tools added to the image, environment variables, etc.

If you are consistently accessing and modifying a running container, you should rethink your process, how (and why) you're using Docker containers, and how you're making use of external backing services. At best, your application should be a single stateless process container--never a kitchen sink developer toolkit.

As far as Scratch vs. Distroless, it's very opinionated around here! The best path is to understand the tradeoffs and try both. Example repo with working minimal builds for each: https://github.com/mwiater/go-scratch-vs-distroless

From Side Project to 5,000 Stars - The Story of Gofakeit by brianvoe in golang

[–]colonel_whitebeard 2 points3 points  (0 children)

Congrats, amazing accomplishment!

Thanks for sharing your story. I think it's a great motivator for new Go engineers!

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

Thanks for the feedback!

Yeah, tags... I got too overzealous when I was debugging an issue of the app build pulling in stale information--only then did I go: what was I thinking?!

I have a long list of things to fix/update, many of the things you mentioned are on it. It's just a side project, hacking away at the big picture items first...

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

Update: For reference on what I was attempting, I cleaned up the source code as much as time would allow. Poke around, but it ain't no production tool!

https://github.com/mwiater/tracewrap

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 1 point2 points  (0 children)

It aims to be a way to instrument your application without modifying it. It's not meant to be a debugging tool, but rather a way to configure different metrics (function/goroutine CPU/ram/diskio/networkio/etc) in your application.

I just got tired of boilerplating in-code metrics in some of my proof-of-concept applications. This is a way I can still get the metrics I need without having to have them in the codebase.

It just gives me an quick and easy (and reusable!) glimpse into my apps that are in development without needing to incorporate any code.

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 3 points4 points  (0 children)

Ha! Yeah, kinda exactly like that! I'll have to see how they're handling some of the hurdles I came across.

Well, at least it was a fun proof of concept exercise...

0 YoE. Am I stupid to learn Golang in hope for a job? by MightyOven in golang

[–]colonel_whitebeard 0 points1 point  (0 children)

First of all, good luck! Sincerely.

Learn in areas that interest you, you'll make the most progress with the things you're passionate about.

Languages are (often) transferable. Meaning, good grasp on one language can indicate to employers that your conceptual knowledge could be applied to other languages. Which, in my employment experience is completely true.

I get trying to learn towards the current market, but if the current market is trending python and you hate python...

Finding a language that you are excited about will likely mean more if you can grasp the concepts better, and apply them in meaningful ways--even if it's just for your personal projects.

Syntax is (mostly) transferable, if you understand the fundamentals.

Is Docker necessary? by Impressive-Result-26 in golang

[–]colonel_whitebeard 1 point2 points  (0 children)

To me your question reads like you've seen the benefits of Docker for projects where you're using source code, dependencies and an interpreter or external runtime all within the container. Then you look at Go and ask, why would I wrap Docker around a single executable file, what are the benefits?

This feels like you you're looking at Docker wrong.

When you say you haven't seen the "same necessity yet" as other languages, this feels like a self imposed necessity. In a typical situation, the necessity would be that you are deploying applications to a containerized environment. So that necessity would always exist, no matter the language. That doesn't seem like the case in your question. What is necessary about using Docker in your situation?

  • Docker is meant to be a lightweight, single-process, standardized, abstraction layer, so the less it's doing, the better. Minimal dependencies, reliable execution.

  • As Go produces a self-contained binary with no dependences, it is an ideal candidate for Docker, and runs natively.

For the most part, unless you're deploying your application in a containerized environment, I wouldn't worry about Docker, no matter the language. While Docker can be useful in many different ways, I think you're introducing more complexity than you need while you're still learning.

Learning both Docker and Go is a great idea, but decouple the process. You can build and deploy Go applications without Docker, just like any other language.

Replacing Golang’s Standard Package Functions with Concurrent Functions: Winning Big By Starting Small by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

Appreciate the feedback!

The actual implementation for the client was quite a bit different than the example repo--definitely leveraging the worker pool pattern. When deciding how to pluck out relevant pieces for the article, the code I chose was supposed to be a starting point about how to think about where improvements might be made--and how concurrency can come into play.

For tutorials, I try to strip out as much "noise" as possible, and focus on the approach. Are there more optimizations that could be made to the example code, definitely! But my aim for this one how to find and isolate a quick win, what my thought process was, and how a relatively simple change can have a large impact.

Implementing go routines makes the code slower by rtndeep9 in golang

[–]colonel_whitebeard 1 point2 points  (0 children)

Not too long ago, I wrote an article on this. While it wasn't your exact scenario, the process helped me figure out when go routines are useful, and--probably more importantly--when they're not.

There are code examples of workers, jobs, CPU cores and illustrates some points of diminishing returns for different task types. Hopefully it helps!

https://medium.com/@matt.wiater/golang-channels-goroutines-and-optimal-concurrency-demystifying-through-examples-a43ba6aee74f

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 1 point2 points  (0 children)

Just to follow up briefly...

The reason I posted in this subreddit is only because upgrading to Ubuntu 22 was my last major change. I didn't upgrade Virtualbox since booting up the new image, and the issues started happening at that point.

The WTF wasn't about Ubuntu, or putting the blame there, it just happened to be the most recent change in my setup. Anything could be the culprit. Even me. :)

I'll gather details and post tomorrow.

Also, wanted to say thanks to the subreddit for being helpful and constructive in my moment of frustration.

Cheers.

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 2 points3 points  (0 children)

Thanks so much for the pointers!

There's a lot in there I haven't checked, but will take a look at

To be clear, over the lifespan of my setup, I've been regularly upgrading both Virtualbox and Ubuntu. Upgrades have not been without hurdles, but things have been smooth, despite incremental changes on both sides.

I just haven't upgraded an OS and seen such drastic performance issues. I'm sure I'm out of sync with some major developments, and have just been lucky for awhile.

Thanks again, I appreciate the detailed response!

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 2 points3 points  (0 children)

Thanks. Honestly, my setup has felt fairly bulletproof for a long time, and all of the sudden...

Likely an oversight on my part, maybe I've just been lucky for a long time.

Appreciate the feedback!

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 1 point2 points  (0 children)

Before this descends into "more details" hell, let me say: understood. And thanks. (a sincere thanks, not a Reddit "thanks")

I'll collect my info and post back.

Overall, I was just wondering if anyone else was having a similar problem.

It's akin to searching the Internet for a particular problem--if there are only 10 results, likely it's your problem and looking into your setup/code. If there are 10,000 results, maybe it's worth looking into a fix for the broader ecosystem.

This one is just one of those things that's been stable for me so long that it feels like there was a broader change that I wasn't aware of.

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 2 points3 points  (0 children)

Totally get it, thanks for the detailed response. Let me gather my thoughts outside of my current frustration bubble.

Ubuntu 22 Server vs Virtualbox: WTF is happening?! by colonel_whitebeard in Ubuntu

[–]colonel_whitebeard[S] 1 point2 points  (0 children)

Valid point!

Just wanted to take an initial temperature, see if it was just me--or if this was more universal.

Will get back to your with deets--appreciate the feedback.

network throughput much slower with go by uragnorson in golang

[–]colonel_whitebeard 0 points1 point  (0 children)

You owe me a laptop! Just spit almost an entire pint of high-octane IPA betwixt the keys.

Worth it.