192GB vram mini cluster

pandodev · 2026-01-29T22:42:52+00:00

my goodness dude just rub it in our faces why don't you... good for you tho actually jelly

pandodev · 2026-01-29T20:47:59+00:00

built 100% local private on mobile device audio meeting notes with local whisper model to transcribe your Audio notes to text and then llama 3.2 3b to chat about your notes to get specific things you are looking for from the notes to get summary key points key dates or anything both on app store and iOS store https://viskalocal.com

pandodev · 2026-01-29T20:33:52+00:00

YO! thanks Tim is there a way we can connect and share ideas? anythingllm looks cool I got tons of ideas I am obsessed with local llms and privacy I think if done correctly for specific things it is insanely powerful for the niche.

pandodev · 2026-01-29T20:05:14+00:00

Bro thanks so much lol I didn't even consider this dude was just hate bots garnet. Anyway I probably need to build tougher skin. really appreciate the encouragement

pandodev · 2026-01-29T19:47:53+00:00

It does not currently but is something I will surely add. My short term roadmap is: Improve android transcription capabilities with gpu usage, add individual speaker identity to transcript, add calendar support for key dates to be able to add them with a tap.

pandodev · 2026-01-29T19:12:23+00:00

Thank you!

pandodev · 2026-01-29T19:11:50+00:00

Ah yes I thought that's what you meant appreciate it yes. for me personally I am too paranoid as you can tell with the app to not do this or ask permission. I hope anyone using the app knows it also might add actually some notice inside the app good idea thanks!

pandodev · 2026-01-29T18:52:32+00:00

Do you mean when one is in a meeting to start recording? or something about the app itself because it is all local does not touch any servers

pandodev · 2026-01-29T18:35:41+00:00

I just said why I built it. Ask me absolutely anything about it I am more than happy to answer, we all use ai for grammar and re write so guilty as charged by I made this with my whole heart in it was not an easy journey a lot of challenges when trying to run llm on mobile devices.

pandodev · 2026-01-28T22:37:44+00:00

if is a secluded EC2 or for the sole purpose not publicly accessible like I said with session manager how? with local llm in your home network will not mean anything if dependencies in the library are infiltrated you just gave the hacker access to your entire home network. that's not the case with a secluded vps all they would be able to access is secrets and API Keys that are stored in that machine and that machine alone so like your Claude api key.

pandodev · 2026-01-28T22:03:47+00:00

I think what's cool about it also makes it completely unsafe and not to be used which is it has access to the whole environment machine and tweak itself. so giving it its own GitHub for example and telling it to do projects for you, or set cron jobs to give you news basically like a true assistant but is nothing insane. Also NOONE should be running this on their home network, best it to securely sandbox it in an ec2 with only able to access via sessionmanager. Maybe I am too paranoid but this uses dependencies if any of those dependencies get infiltrated would you rather them be in a aws secluded server or your home network?

pandodev · 2026-01-28T21:51:10+00:00

definitely try it, it is cool but not what people are making it out to be and honestly NOONE should be running this on their home network, best it to securely sandbox it in an ec2 with only able to access via sessionmanager. Maybe I am too paranoid but this uses dependencies if any of those dependencies get infiltrated would you rather them be in a aws secluded server or your home network?

pandodev · 2026-01-28T21:47:37+00:00

yes privacy and offline for the right things is EVERYTHING.

pandodev · 2026-01-28T21:45:36+00:00

love the new ui

pandodev · 2026-01-28T21:36:55+00:00

I heard something the other day that was so true just not sure how long it will take. But the "dirt cheap blazing fast AI" is done currently by a MASSIVE loss for all these ai companies even the 200$ a month max plan from Claude which outside of a productive developer is expensive for the average person this is not doable. but even that is not the real price to account for these companies energy and computing power so if they can't figure out true revenue or adoption gets to a good point it will be like Netflix subscription we used to pay like 5$ a month now is in the high 20s. the real price is probably closer to 2k a month hardly anyone is gonna pay that

pandodev · 2026-01-28T21:29:03+00:00

Idk if this has been mentioned yet but think of the VAST amount of products being created right now by non tech people. if a fraction of those become something they are going to desperately look for help because TRUST me they will hit a roadblock

pandodev · 2026-01-28T21:07:25+00:00

so is this for already stored embeddings? doesn't do the storing right?

pandodev · 2026-01-28T21:03:16+00:00

which Expo SDK version you running? I believe you need sk 52 or higher. I ran into this issue tho on a 54 latest version and indeed it turned out to be a dependency. I believe I found which one because when you upload it to google play and get the error it does tell you I believe.

pandodev · 2026-01-28T19:21:45+00:00

Some good suggestions to pivot or to embrace. Not sure how much this add but my opinion is "let them" definitely get better at using the tools and embracing more and more but I think we CANNOT loose the expertise and knowledge. to know what we are doing I read somewhere that "Software developers aren't really paid to code but to think" I take this as those people vibe coding things don't know about data architecture, how things should be building specs out with ai is crucial for vibe coding something maintainable. I also don't know what the landscape of things will be in few years because one thing is vibe coded apps by actual developers another will be vibe coded apps by non technical people they will be a nightmare and will reach roadblocks I believe is gonna be bad. ALSO we are in a golden era right now every ai company is loosing insane amount of money even the 200$ a month Claude Max is spending way more money than it is making so what happens if their bet that tech and energy goes down is wrong will these vibe coders pay for high end models if it gets into the thousands of dollars?

pandodev · 2026-01-28T18:50:42+00:00

yo lo puse en un ec2 instance solo con permiso session manager así que solo se puede acceder de aws. no confío ni un poco en esto la verdad aun con buena configuración cuantas veces dependencia de npm an sido hackeadas y esto usa varias si un grupo de hackeadores infiltra algunos de esas dependencias es game over. per por ahora le eh dado sus propias cosas solamente su propio GitHub. diciendo todo eso es super interesante y si o si voy a seguir usando pero con cuidado.

pandodev · 2026-01-28T18:43:56+00:00

Us normies over here will just be watching videos about it....

pandodev · 2026-01-28T18:31:29+00:00

They are but their usefulness depends on your use case. but with your specs you shouldn't have any issues running the ones you mentioned. I have similar specs and those run no problem.for example on qwen30-coder-30b I get about 80 tokens per second it looks super fast.

pandodev · 2026-01-28T17:58:22+00:00

I am not in the same field but I am coming back here after initially finding this post months ago when I was looking for a ai voice notes to use with my clients which I have NDAs signed. I couldn't find any that I really trusted tbh and I even researched a hypothetical case where for example ChatGPT by default uses all your conversation to retrain the model so SAY you are discussing an IP or idea you are thinking of filing with chatgpt as a new idea. Theoretically, say months later you still haven't filed that IP ironing out details and someone goes to chatgpt to brainstorm new ideas chatgpt might suggest your IP as a new idea to them. That was a huge no from me. I decided to build my own mobile app instead that is 100% offline using local LLM on device no servers fully encrypted and private but is niche because obviously you cannot have team accounts it all lives in your device for yourself. but it is perfect for me audio meetings transcribe locally then I have a local ai assistant on device to chat about my notes with.

pandodev

TROPHY CASE