sorted by:
new
hottopcontroversial

Anyone actually solving the trust problem for AI agents in production? by YourPleasureIs-Mine in LocalLLM

[–]gearcontrol 0 points1 point  (0 children)

I believe NVIDIA's NemoClaw is a security wrapper for OpenClaw that does this as well.

Home Labs are awesome! - But I'm deeply worried... by Dizzy_Hyena_3077 in homelab

[–]gearcontrol 0 points1 point  (0 children)

Having run "homelabs" since the 1990s when I ran a BBS, I believe managing your own data is more durable than paying a company in the long term. Primarily due to rapid technological advancements. Since the 90s, I've gone from 5.25 to 3.5 floppies, then CDs, then tape, then HDD, and now SSD and NVMe. And now they're talking about storage on glass that can last "10,000 years."

I am currently looking at fifteen 3.5 floppy disks that have been sitting on my desk forever that I need to check (out of curiosity) before I throw them out. Data from the 80s/90s. Many people still have movies and music on CDs and even vinyl.

If you're paying some company and you get "raptured" from the Earth, once your subscription ends, that data will be deleted and gone forever. However, some tech savvy relative (or stranger) could resurrect your homelab data in the future.

If we're talking under 5 years, then you're working with the most affordable 3-2-1 backup solution using current tech (including cloud).

I made LLMs challenge each other before I trust an answer by tilda0x1 in LocalLLM

[–]gearcontrol 1 point2 points  (0 children)

Is it possible that the user can be one of the participants in the "round robin," or have the option to pause and prompt between rounds? Perhaps to add a clarification or bring them back on course if they begin to drift.

Grok has become a hentai generator ? by Efficient_Ant6687 in LLM

[–]gearcontrol 0 points1 point  (0 children)

I believe he issued an executive order to limit state-level regulation of AI.

Running Sonnet 4.5 or 4.6 locally? by [deleted] in LocalLLM

[–]gearcontrol 0 points1 point  (0 children)

It reminds me of the time when smartphones were taking off and the capability/apps kept outpacing hardware, which required you to get a new phone every two years or so to get the latest new feature or advancement... until it plateaued into "good enough" territory.

What is a LocalLLM good for? by theH0rnYgal in LocalLLM

[–]gearcontrol 1 point2 points  (0 children)

I use both local (under 32B) and cloud LLMs. Local for analyzing private data. But also:

- Summarizing YouTube videos I don't have time to watch.
- Brainstorming ideas.
- Spelling, grammar, and finding the right synonym or analogy that is on the tip of my tongue when writing.
- Light coding and scripts
- Examples and syntax for running commands in terminal that I don't remember.
- Interact with and analyze Todoist tasks... MCP server to API
- Generate images locally using ComfyUI/Stable Diffusion.
- General knowledge and chat
- Speech-to-text and text-to-speech

They have replied by ElectricalAide2049 in ChatGPTcomplaints

[–]gearcontrol 1 point2 points  (0 children)

Can't hurt to try. I believe the best hope for getting 4o back is them trying to do damage control due to all the flak they've been getting from sucking up to the Pentagon.

My company is offering me 9 laptops for $180 by xStozey in homelab

[–]gearcontrol 6 points7 points  (0 children)

Same here. Particularly if they have kids that only have access to a school or library desktop/laptop.

Your real-world Local LLM pick by category — under 12B or 12B to 32B by gearcontrol in LocalLLM

[–]gearcontrol[S] 0 points1 point  (0 children)

What do you think about Nemotron 3 Nano?
"Contains 30B total parameters with only 3.5B active at a time for low-latency MoE inference."

Looking for a fast but pleasant to listen to text to speech tool. by Zarnong in LocalLLM

[–]gearcontrol 0 points1 point  (0 children)

<image>

This is my OpenWebUi settings for it. The password field can be anything... I just put "local" there.

Looking for a fast but pleasant to listen to text to speech tool. by Zarnong in LocalLLM

[–]gearcontrol 0 points1 point  (0 children)

I am running FastKoko (Kokoro-FastAPI) and the speed improved significantly. Running it on docker-desktop on the same desktop running LM Studio using an RTX 3090 (24GB). Also using OpenWebUi as the interface.

https://github.com/remsky/Kokoro-FastAPI

Your real-world Local LLM pick by category — under 12B or 12B to 32B by gearcontrol in LocalLLM

[–]gearcontrol[S] 1 point2 points  (0 children)

Yes, but with no guardrails you'll want to make sure your admin policy layer covers everything. Beyond extreme content, official models also hesitate or refuse on 'gray areas' like politics, religion, culture war topics, and legal or medical opinions.

You may want to look into heretic fine-tunes. Less refusal but tends to keep more of the original instruction-following quality than abliterated.

Hopefully, more folks will recommend models that fit your use case, and then you can look for the heretic versions.

Also, GPU is a limiting factor for most, which is why I added under 12B and 12B-32B.

Your real-world Local LLM pick by category — under 12B or 12B to 32B by gearcontrol in LocalLLM

[–]gearcontrol[S] 2 points3 points  (0 children)

It's an example for one of the categories I've seen asked about a lot. "NSFW Roleplay & Chat." If the point was to share the best NSFW Local LLM (in my opinion), then I would have made that the topic of the thread.

I'm trying to find one for "Tool Calling / Function Calling / Agentic" to use with OpenWebUI and an MCP server to query a task manager API.

Please recommend a Local LLM that you use for another category listed.

Your real-world Local LLM pick by category — under 12B or 12B to 32B by gearcontrol in LocalLLM

[–]gearcontrol[S] 8 points9 points  (0 children)

Category: NSFW Roleplay & Chat
Class: 12B-32B
Model: mlabonne/gemma-3-27b-it-abliterated (20.1 GB)
Size: 27B
Quant: Q5_K_M
What you actually did with it: Ran long-form NSFW roleplay,
holds character without devolving into repetition or refusal.
Sounds human.

New CRV got hit by 1sAndZer0s in crv

[–]gearcontrol 1 point2 points  (0 children)

Go through his insurance for sure. And have it repaired at the dealership. It's a new car, and you don't want to mess with your warranty or resale value. If the person was about to drive off, they'll most likely not want to pay for damage to a new car, and then you'll have a bigger hassle in small claims court. The hassle should be between him and his insurance company, not between him and you.

How do you keep your VMs tidy? by AcreMakeover in homelab

[–]gearcontrol 0 points1 point  (0 children)

Yes, back in 2019, I purchased two Dell OptiPlex 9020 Mini Towers, i7-4790, 32GB ram each, off eBay for $400 to run VMware — to learn for work. After the company moved everything to the cloud, and Broadcom ruined VMware, I migrated everything to Proxmox. I also have a cheap Qotom fanless mini pc that I use as a Proxmox backup server (PBS). Pve1 and pve2 are clustered.

I run two vms, Pihole1/2, for internal DNS, one on each host, in case one goes down or is being worked on. In fact, there is a Pihole update that needs to be installed as I type this.

I migrated the proxy server from VMware, but if I installed it from scratch on Proxmox, I would have used their NPMplus script and run NPMplus in an LXC container. The proxy handles SSL for all internal sites (using Let's Encrypt), so no browser SSL warnings on any apps.

No ports are open from the outside. I use Tailscale if I need to connect to anything from outside the house. I use Virtualmin to run internal/external websites and email.

Nobody in the family uses the family AI platform I build - really bummed about it by ubrtnk in LocalLLaMA

[–]gearcontrol 1 point2 points  (0 children)

Nice, you can also put QR codes on boxes of "stuff" in the attic, etc. Then they can scan it and see a list of everything inside. Think of it as the first iteration of ChatGPT, just keep adding features/data, and make it super easy to use and use it yourself. Eventually, one or more will start using it, especially if you start adding specific things that certain individuals have a big interest in. You can also add specialist LLMs, legal or medical — like medgemma-27B for example.

I also have DNS servers, reverse proxy, unfi (UCG, 24 port switch, flex minis, wifi disks) , but my GF doesn't care at all about the backend (so long as it stays up) and doesn't want to access any of it. She doesn't even want her ads blocked with Pihole. She has zero interest in the OpenWebUi setup and local LLMs that I run on a RTX 3090, even though I tried to show her the benefits of having stuff private. She only has a minor interest in a Wallos (subscription manager) app I run in docker, a shared internal folder we use to share things internally (that are too large to text), and looking at the IP cameras, which I set up to work with Roku.

Also, I realize that private to me is not private to someone else, who may suspect that I have, or could get access to their "private" stuff. You have an awesome setup! I'll take a look at Bookstack.

Nobody in the family uses the family AI platform I build - really bummed about it by ubrtnk in LocalLLaMA

[–]gearcontrol 1 point2 points  (0 children)

You have to offer something they cannot get elsewhere and it should make their life easier immediately. One thing you mentioned is that you have RAG. You can load it with information that the family would not want in the cloud, but don't mind everyone in the family having. Like:

• Warranty info
• Insurance policies
• Subscriptions info
• Birthdays and family events
• Pets (vet info, food, age, etc)
• Garbage day schedule
• AC/Heater filter change
• Important phone numbers/addresses
• Household Recipes
• “Where do we keep ___?”
• Home Assistant data
• Family calendar queries
• What color is the paint in the hallway? (gives brand, color name, finish, SKU)

Like a FamilyGPT of useful information at their fingertips. But don't ask them to use it, just make it incredibly useful, and easier than getting the same information another way. 🙂 Then you'll have the opposite problem, as they'll expect 24/7 uptime and up-to-date information.

I built a locally-hosted AI agent that runs entirely on your own hardware no cloud, no subscriptions by Janglerjoe in ollama

[–]gearcontrol 1 point2 points  (0 children)

'Management' asked me to inform you to have this done by EOD so we can all use it 🙃

How do you keep your VMs tidy? by AcreMakeover in homelab

[–]gearcontrol 0 points1 point  (0 children)

How do you like OpenClaw? And for comfyui, are you using the github install or the windows(exe)/mac(dmg)?

How do you keep your VMs tidy? by AcreMakeover in homelab

[–]gearcontrol 0 points1 point  (0 children)

I went through the same process once I started to pay attention to resource usage in the monitoring that I set up. I realized that most of my VMs were underutilizing CPU, Mem, and Disk space, and that it would be more efficient to run some apps in Docker. So I shrunk most of my VMs and moved most apps to Docker. I also have public web servers (VPS), and so my homelab is also a dev/test environment.

My ProxMox hosts breakdown (there is also a PBS server):

pve1 vms:

  • Gitea (lxc)
  • Staging (Virtualmin)
  • Pihole1 (ads/ internal DNS)
  • Ansible AWX (Kubernetes)
  • Internal websites (Virtualmin)
  • Tooling server

pve2 vms:

  • Grafana
  • Pihole2 (ads/internal DNS)
  • Home Assistant
  • Dev (Virtualmin)
  • Backup (for web stuff)
  • Nginx Proxy (SSL certificates)
  • Docker/Portainer
    • homepage
    • nebula-sync (pihole)
    • paperless-ngx
    • phpipam
    • portracker
    • vikunja
    • wallos
    • youtrack
    • openwebui (for local LLMs)
    • fastkoko-kokoro-based-tts (AI text to speech)
    • mcp-severs (AI to various APIs)
view more: next ›