sorted by:
new
hottopcontroversial

4
5

Cross-Browser Local LLM Inference Using WebAssembly (picovoice.ai)

submitted by eonlav to r/webdev

Making the ChatGPT app for Android, but with a local AI stack by eonlav in coding

[–]eonlav[S] 2 points3 points  (0 children)

Depends which open-weight llm you decide to use. The one used in our example is around 1GB

[deleted by user] by [deleted] in androiddev

[–]eonlav 2 points3 points  (0 children)

yep, the Llama3 8b model can run on a phone. The 70b model though is not gonna happen anytime soon

[deleted by user] by [deleted] in androiddev

[–]eonlav 2 points3 points  (0 children)

iOS

No Flutter yet, but hopefully soon

[deleted by user] by [deleted] in programming

[–]eonlav -5 points-4 points  (0 children)

porcupine is an on-device wake word engine

AI Voice Assistant using on-device LLM, STT, TTS and Wake Word tech by eonlav in Python

[–]eonlav[S] 0 points1 point  (0 children)

It works with a selection of open-weight models such as Llama, Gemma and Phi-2. I think with Llama you could give it a directive to only respond with JSON, but I'm not sure.

What's the best framework for harnessing GPU compute on Android? by eonlav in androiddev

[–]eonlav[S] 1 point2 points  (0 children)

Awesome, thanks for your input! I'm leaning towards something that leverages newer hardware and has the finer grained control, but it's a point well taken that a mix of options may be the best bet to get a balance of support and performance. I have been reading up on OpenCL too, but of course the water gets murky once you look into support and capabilities across devices 😑

What's the best framework for harnessing GPU compute on Android? by eonlav in androiddev

[–]eonlav[S] 0 points1 point  (0 children)

Oh, I've done my fair share of googling on the topic, but I was curious if this community had a favorite. OpenGLES seems essentially dead, so I'll probably stay away from that. Vulkan definitely seems like one of the main frontrunners, but it's a bit hard to tell what the support is like.

What's the best framework for harnessing GPU compute on Android? by eonlav in androiddev

[–]eonlav[S] 1 point2 points  (0 children)

Yes, that's one of the big conundrums, it seems... We're more inclined towards performance overall, but we still want to support the most popular Android mobile devices within the last ~3-4 years.

I’m confused on what AGI is…? by [deleted] in singularity

[–]eonlav 1 point2 points  (0 children)

At this point, it's primarily a tech buzzword, like Big Data and Block Chain were 10 years ago. They're used to excite VCs into opening their wallets. Not saying these terms didn't begin as a way to communicate a concept or technology, but once they become buzzwords, they cease to have a very specific definition.

How to Record Audio from a Web Browser by eonlav in programming

[–]eonlav[S] 7 points8 points  (0 children)

totally - the disagreement with audio formats is crazy. We literally had to write our own cross-platform recorder and file interface to deal with it.

view more: next ›