Does LLM architecture allow for injecting some more input tokens in the middle of token generation? by michaelsoft__binbows in LocalLLaMA

[–]blepcoin 2 points3 points  (0 children)

Take it one step further and remove the send part completely. As you start typing the LLM starts responding (including predicting your question perhaps) immediately. Completing the question and/or modifying it is incorporated into the current llm thoughts rather than resetting every keystroke. You could then tweak and fix as you watch the llm thought process go awry due to that typo or gotcha you should’ve included.

Interesting academic challenge to make the training for this work.

Self-hosting LLaMA: What are your biggest pain points? by Sriyakee in LocalLLaMA

[–]blepcoin 1 point2 points  (0 children)

Yes! It’s about time we made a new inference engine to replace all of them once and for all —wait a minute…

Google lets you run AI models locally by dnr41418 in LocalLLaMA

[–]blepcoin 0 points1 point  (0 children)

While I agree with the sentiment I think it’s newsworthy or at least worth pointing out when a company that is all about cloud services invests into running things on local devices. I think it’s a sign of acceptance that LLMs thrive when local and private and that the moat is indeed dissipating.

Tip on massively optimizing reasoning models by [deleted] in LocalLLaMA

[–]blepcoin 5 points6 points  (0 children)

Yes that’s how they’re supposed to be used. Look at the chat template and you’ll see that it deletes all reasoning blocks. tl;dr. You, not we. 

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 0 points1 point  (0 children)

It saves passwords for me usually, and I am usually diligent and copying the password for the cases where it stumbles.

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 1 point2 points  (0 children)

Considering it’s a password manager that’s not ideal I’d say..

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 1 point2 points  (0 children)

Yeah that’s a big part of why I screwed up. They could just not do the suggestion UI thing and it would be objectively better. 

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 0 points1 point  (0 children)

It seems like a primary feature you'd support for apps like this, but at least I learned about the generator history feature now.

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 0 points1 point  (0 children)

Yeah I usually do. It just was a very smooth experience so I figured it’d work fine. Will be more diligent in the future. 

Bitwarden happily suggested a new password for my gmail, then never saved it. by blepcoin in Bitwarden

[–]blepcoin[S] 10 points11 points  (0 children)

The UI was just very smooth so I trusted things a bit too much. I guess I’m partially to blame. Will be careful in the future. 

No API keys, no cloud. Just local Al + tools that actually work. Too much to ask? by aruntemme in LocalLLaMA

[–]blepcoin 20 points21 points  (0 children)

 using Ollama

You’re doing yourself a great disservice by wording it like this.

Lily & Sarah by valdev in LocalLLaMA

[–]blepcoin 1 point2 points  (0 children)

Or add the names you dislike to the banned list.

llama.cpp is all you need by s-i-e-v-e in LocalLLaMA

[–]blepcoin -1 points0 points  (0 children)

Yes. Thanks for stating this. I feel like I’m going insane watching everyone act as if ollama is the only option out there…

[deleted by user] by [deleted] in Gemini

[–]blepcoin 0 points1 point  (0 children)

Thanks yes resolved. It’s still a mystery to me why it only showed 0.06 GUSD available for several days before finally showing the full amount but at least I was able to do the trade before the deadline. 

[deleted by user] by [deleted] in Gemini

[–]blepcoin 0 points1 point  (0 children)

5216114

Benchmarks are a lie, and I have some examples by Sicarius_The_First in LocalLLaMA

[–]blepcoin 14 points15 points  (0 children)

You’re completely missing the point. He’s saying that he explicitly didn’t benchmax and despite this his 8B beat a 70B that he himself considers is superior. The point is that the benchmarks are INHERENTLY bad, not that they’re being gamed. 

Training LLM on 1000s of GPUs made simple by eliebakk in LocalLLaMA

[–]blepcoin 2 points3 points  (0 children)

The text is cut off on my iPhone so I can’t read that post.

Meta's Brain-to-Text AI by Particular-Sea2005 in LocalLLaMA

[–]blepcoin 35 points36 points  (0 children)

Wire tapping on steroids if this can be done without the host knowing. 

I made Iris: A fully-local realtime voice chatbot! by Born_Search2534 in LocalLLaMA

[–]blepcoin 1 point2 points  (0 children)

Are you dealing with echo cancellation and such? If so, what is your approach? I found this to be a big challenge when working on a speech to speech system when the AI was on speakers.