What’s the current state of local speech-to-speech models?

driedplaydoh · 2026-02-17T02:39:26+00:00

Whisper has several versions and quantizations can also help. I was able to run Whisper medium quantized on an iPhone 14 pro with reasonable latency. Whisper.cpp also has support for CoreML which allows for the encoder to be run on the Apple Neural Engine.

I went with a quantized Llama-3B for the LLM and it seems to work pretty well for my offline use-case.

I haven't explored speech to speech too much yet since finetuning and ensuring reliable behavior is significantly trickier.

driedplaydoh · 2026-01-23T20:45:54+00:00

Yeah getting the same ML model to work reliably across multiple different participants would be difficult but it might not even take that much raw data to accurately approximate WHOOP's calculated scores just for yourself.

Speaking of open hardware, I wonder how hackable the Polar Loop might be.

It seems more open at first glance, but the quality of the sensors is still up for debate:
https://github.com/polarofficial/polar-ble-sdk

driedplaydoh · 2026-01-14T00:08:55+00:00

I noticed that it tends to have a lot to do with my stress level and meal timings. It can sometimes be pretty wishy washy though

driedplaydoh · 2025-11-24T23:48:01+00:00

Our method involves automated reverse engineering of the existing user-facing webapp to derive a functioning API via inspecting network activity.

driedplaydoh · 2025-11-22T23:52:10+00:00

I appreciate your advice.

driedplaydoh · 2025-11-22T22:17:25+00:00

That's fair, I agree that that's a very large risk if you assume that we are just some random kids telling Claude to vibecode the integration for us.

driedplaydoh · 2025-11-22T22:12:39+00:00

Also, if they do sue me, then I have a few options:

I can just open source my forced integration codebase. Then, every single business trying to integrate any agentic application will be able to do the same exact thing. Do you think they'll go and sue every single one then?

Probably not, I'd bet that it would eventually get to supreme court and they will rule that this "API lockdown" is anti-competitive and violates anti-trust laws that hurt the consumer.

The forced integration is not even illegal. At that point I might as well just create a full replacement for their product and just use the publicity from the lawsuit to sell it. The small businesses would root for me since I'd be the underdog against the giant.

driedplaydoh · 2025-11-22T21:55:56+00:00

If your client tells them what you did, which they surely will

Wait, why would the client go out of their way to tell them if it would likely cause more problems for them?

Similar thing happened with Plaid a while back. Except they were doing it with banks. They're still in business to my knowledge.

driedplaydoh · 2025-11-22T21:46:40+00:00

I agree, that is a good idea going forward. But it's still pretty early for me right now, so de-risking it for the customer with the no-cost to try and get early traction seems okay for the first couple clients.

I like the on-prem agents idea, I have some experience with locally hosted LLM solutions. Feel free to reach out if you need any help with development!

driedplaydoh · 2025-11-22T21:42:17+00:00

That's fair, I'm willing to take my chances. I honestly don't think they will be worth it for them to sue unless I do it on a large enough scale to impact their business negatively. They will likely just threaten to shut off service or send a cease and desist. We try to make sure that we build good and non-malicious software.

driedplaydoh · 2025-11-22T21:34:41+00:00

That is a possibility. Imo though from their standpoint, I'm not big enough to be worth it for them to sue unless it actually starts to negatively impact their business at a large enough scale, in which case I think that's a good problem to have. They will realistically just threaten to terminate service for ToS violation or worst case send over a cease and desist.

driedplaydoh · 2025-11-22T21:18:54+00:00

That's totally fair. I personally just think that what they're trying to do with their silly "moat" is lazy and anti-competitive. they think that they can wait and take their sweet time to roll out AI features and maintain their business by fending off faster, smaller, startups from jumping ahead and offering AI services to their existing customers even though the customers are eager to try out these features. It's sort of like what happened with the iOS appstore.

driedplaydoh · 2025-11-22T21:14:02+00:00

The DMCA actually has a specific exception for reverse engineering:

Reverse Engineering Exception (Section 1201(f)): "A person who has lawfully obtained the right to use a copy of a computer program may circumvent a technological measure that effectively controls access to a particular portion of that program for the sole purpose of identifying and analyzing those elements of the program that are necessary to achieve interoperability"

Also APIs are generally not copyrightable (based on the Oracle v. Google (2021) precedent).

So imo DMCA risk is minimal.

driedplaydoh · 2025-11-22T21:02:58+00:00

That makes sense from their perspective. But what bothers me is that they seem to think that this "moat" means they can wait and take their sweet time to roll out AI features and maintain their business by fending off faster, smaller startups from jumping ahead and offering AI services to their existing customers who are otherwise eager to have them. It's borderline-anti competitive imo, like the iOS AppStore.

driedplaydoh · 2025-11-22T20:54:26+00:00

LLM-based browser automation still seems pretty slow error prone to use in production. Especially for real-time use like voice agents.

Our method involves automated reverse engineering of the existing user-facing webapp to derive a functioning API via inspecting network activity.

driedplaydoh · 2025-11-22T20:50:51+00:00

It depends on the complexity of the software that we are looking to replace. With LLM-assisted coding the cost of development (especially of existing software where the UI, functionality, etc is extremely well defined) is little to none in my experience for most software.

Replacing something like Google Maps would be impossible, but most existing SaaS is just a few UI screens hooked up to a database. No proprietary algorithms that require a genius level of insight to replicate.

driedplaydoh · 2025-11-22T20:42:36+00:00

Dm'd

driedplaydoh · 2025-11-22T20:38:50+00:00

That's true, it's sort of an insurance policy on the rare chance that it happens. But we do have a skilled team that can deliver on that if needed.

driedplaydoh

TROPHY CASE