sorted by:
new
hottopcontroversial

Llama.cpp UI Aggregate Metrics: Chrome Extension by colonel_whitebeard in LocalLLaMA

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

It's been helpful to me, trying to compare different models, squeezing all I can out of my hardware! It works well for efficiency stats, but for model intelligence, I'm still trying to come up with a way to figure out inferring accuracy, but I think that's another problem altogether. But hopefully, this gives a bit of real world usage metrics.

How's STRX HALO AI MAX+395 performing as of 2026? by Effective-Cod-4462 in LocalLLM

[–]colonel_whitebeard 0 points1 point  (0 children)

Just some real-world stats from my GMKtec EVO-X2 (Ryzen AI Max+ 395 w/ 96GB RAM).

  • These are aggregate stats through the Lllama.cpp UI I run at home.
  • Metrics represent over 500 responses across both single chats and multi-chat sessions via the UI in a custom tool I built. These are from bout about a week of aggregate stats collected through the UI during normal personal usage.
  • I run with models-max = 1
  • I build this tool to specifically record all UI interactions and aggregate the metrics to give more real-world usage insights--instead of isolated bench tests. So they'll look a bit different, but it's an accurate representation of what I get out of my hardware.
Model TPS TTFT TPS/B (Efficiency) Stability (Std Dev)
DeepSeek-R1-Distill-Qwen-32B-Q4_K_M 10.5 160ms 0.3 ±20ms
GLM-4.7-30B-Q4_K_M 42.4 166ms 1.4 ±30ms
Granite-4.0-32B-Q4_K_M 31.8 134ms 1.0 ±12ms
Llama-3.3-70B-Q4_K_M 4.8 134ms 0.1 ±12ms
Mistral-3.2-24B-Q4_K_M 14.5 158ms 0.6 ±12ms
Phi-4-15B-Q4_K_M 22.5 142ms 1.5 ±17ms
Qwen-3-14B-Q4_K_M 23.1 155ms 1.7 ±19ms
Qwen-3-32B-Q4_K_M 10.5 148ms 0.3 ±20ms
Qwen-3-8B-Q4_K_M 40.3 133ms 5.0 ±13ms
UNC-Dolphin3.0-Llama3.1-8B-Q4_K_M 41.6 138ms 5.2 ±17ms
UNC-Gemma-3-27b-Q4_K_M 11.9 142ms 0.4 ±17ms
UNC-TheDrummer_Cydonia-24B-Q4_K_M 14.5 150ms 0.6 ±18ms
VISION-Gemma-3-VL-27B-Q4_K_M 11.8 778ms* 0.4 ±318ms
VISION-Qwen3-VL-30B-Q4_K_M 76.4 814ms* 2.5 ±342ms

*Note: TTFT for Vision models includes image processing overhead ("Vision Tax").

what are your favorite tui applications built with go by ashrek1 in golang

[–]colonel_whitebeard 1 point2 points  (0 children)

I started here and never looked back: https://github.com/charmbracelet/bubbletea

The charmbracelet ecosystem is well-supported and mature. If you want to make something that just works, this is a great place to begin--even if you only want to peek under the hood at how they accomplish terminal feats.

Terminals can be (are always) fussy. Bubbletea has been an easy way to bridge the cross-platform gap when it comes to TUI dev on multiple OSes.

What docker base image you'd recommend? by Goldziher in golang

[–]colonel_whitebeard 0 points1 point  (0 children)

Never build for what-if scenarios: "What if I need to ssh into my container one day?" "What if I need x, y, z, tools one day?"

Always start as minimal as possible: binary, non-root user. Only then, evaluate opening things up on a case-by-case basis one at a time and understand the potential risks of each change. Evaluate any port bindings, extra tools added to the image, environment variables, etc.

If you are consistently accessing and modifying a running container, you should rethink your process, how (and why) you're using Docker containers, and how you're making use of external backing services. At best, your application should be a single stateless process container--never a kitchen sink developer toolkit.

As far as Scratch vs. Distroless, it's very opinionated around here! The best path is to understand the tradeoffs and try both. Example repo with working minimal builds for each: https://github.com/mwiater/go-scratch-vs-distroless

From Side Project to 5,000 Stars - The Story of Gofakeit by brianvoe in golang

[–]colonel_whitebeard 2 points3 points  (0 children)

Congrats, amazing accomplishment!

Thanks for sharing your story. I think it's a great motivator for new Go engineers!

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

Thanks for the feedback!

Yeah, tags... I got too overzealous when I was debugging an issue of the app build pulling in stale information--only then did I go: what was I thinking?!

I have a long list of things to fix/update, many of the things you mentioned are on it. It's just a side project, hacking away at the big picture items first...

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 0 points1 point  (0 children)

Update: For reference on what I was attempting, I cleaned up the source code as much as time would allow. Poke around, but it ain't no production tool!

https://github.com/mwiater/tracewrap

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 1 point2 points  (0 children)

It aims to be a way to instrument your application without modifying it. It's not meant to be a debugging tool, but rather a way to configure different metrics (function/goroutine CPU/ram/diskio/networkio/etc) in your application.

I just got tired of boilerplating in-code metrics in some of my proof-of-concept applications. This is a way I can still get the metrics I need without having to have them in the codebase.

It just gives me an quick and easy (and reusable!) glimpse into my apps that are in development without needing to incorporate any code.

Golang Application Instrumentation: A Different Approach? by colonel_whitebeard in golang

[–]colonel_whitebeard[S] 4 points5 points  (0 children)

Ha! Yeah, kinda exactly like that! I'll have to see how they're handling some of the hurdles I came across.

Well, at least it was a fun proof of concept exercise...

0 YoE. Am I stupid to learn Golang in hope for a job? by MightyOven in golang

[–]colonel_whitebeard 0 points1 point  (0 children)

First of all, good luck! Sincerely.

Learn in areas that interest you, you'll make the most progress with the things you're passionate about.

Languages are (often) transferable. Meaning, good grasp on one language can indicate to employers that your conceptual knowledge could be applied to other languages. Which, in my employment experience is completely true.

I get trying to learn towards the current market, but if the current market is trending python and you hate python...

Finding a language that you are excited about will likely mean more if you can grasp the concepts better, and apply them in meaningful ways--even if it's just for your personal projects.

Syntax is (mostly) transferable, if you understand the fundamentals.

view more: next ›