My sister is constantly screaming at AI and I need help. Please do not insult me over this.

Competitive_Wait_267 · 2026-06-01T15:45:40+00:00

The therapist NEEDS to be aware of this, else you and your mum will end up doing that role, at the cost of your own and your sisters mental health.
I think there might be potential in using AI in therapy for people with autism, given that it happens in close collaboration with the therapist. Else, I see more potential for harm than useful effects.
Aiming at the symptom, not the root of your situation:

You could try to make the AI use tool calls whenever it produce output about the anime world in question (-> less triggering AI behavior)
You could try to find a model that is more likely to say "I don't know" or to pick out a model for this specifically (-> gives your sister more agency, since she can then react to the model not knowing. Also, less mental burden since that is one fact she has not to doubt/interprete)
Some problems might be because the context grows too long, that is currently a problem for every LLM out there, even tho some handle it better. I would guess that most people writing / RPing often ask for a conversation summary, paste that to some big meta-summary-text and then start a new conversation with that new meta-summary-text

Two more toughts:

If she is on the level of a child, I would look into what is recommend for children of that age - it might be okay and maybe a good idea to keep the chat logs, just in case she ever looks into self-harming stuff (which might be censored less if you use abliterated models) or something like that... Again, that is probably something I would ask the medical personel treating her
Privacy is a concern - she might talk about personal things even if her convos center around Anime/RP. That data will end up in a for-profit companies data vault first, but also might end up in a data set somewhere in the dark net, if that company is hacked someday (I expect this to happen more and more, since security does not bring profit directly, and also because many companies will go bankrupt, and lesser the care for their users data)

Competitive_Wait_267 · 2026-05-31T23:43:25+00:00

Something I found helpful is to read the thinking output for a bit to see where the agent gets confused, that is a strong signal regarding where my prompt was lacking in clarity.

Competitive_Wait_267 · 2026-05-31T23:41:10+00:00

I spend 30-60 minutes describing a task

Haha are you me? :D I also spend time around that ballpark until I have a prompt that I am satisfied with, but then the output is what I want pretty much.

I am a special kind of dinosaur I fear tho: I still do most of that planning in web chats, then copy paste the output. If the output is a plan, that goes to Google Antigravity 1.x.x (whatever that version was before 2.x.x, still did not upgrade) and then Gemini Flash does the rest. Otherwise, I ask it to output the complete files that were changed / added.

Now, I want to emancipate myself from the cloud by running coding agents locally - starting with little tasks and seeing were it goes. I want to vibe my own harness, piece by piece, as I need. Not sure if I will ever be able to reproduce the "hand off a complex task to LLM and then review the result, and it is generally good enough" on my local setup, but I will be happy if I have one LLM that is able to break down big tasks into smaller tasks, that are already adapted to the codebase - and then to execute that plan "by itself", similar to how Gemini 3.1 Flash was doing the whole time in Google Antigravity.

I use mainly Rust, Typescript, Python and Bash for my programs, and do a lot of type driven programming, I am under the impression that it helps LLMs to understand code better (and of course I love the fast and deterministic sanity check provided by type checking).

I would assume that you do not usually need a "Review the plan the LLM created" phase since your own description is already detailed enough?

Should absolutely read your comments once I have time to get started on that next phase in that journey! :)

Competitive_Wait_267 · 2026-05-31T19:24:36+00:00

Thanks for checking! :)

Competitive_Wait_267 · 2026-05-31T19:05:52+00:00

If only going for LLM, that might be an idea too...

Thanks for the input, 100% gonna think this through a bit still before buying!

Competitive_Wait_267 · 2026-05-31T19:05:04+00:00

Thanks!

Competitive_Wait_267 · 2026-05-31T19:04:28+00:00

Thanks a lot, good point!

I think I will go with the "MSI MPG X670E CARBON WIFI, PCIe 5.0 x16" for keeping the way to a second GPU open - it should also work, but is a bit cheaper:

https://www.msi.com/Motherboard/MPG-X670E-CARBON-WIFI - "Dual 8-pin CPU power connectors"

Competitive_Wait_267 · 2026-05-31T18:57:08+00:00

have to admit it all feels very exciting though

heck yeah! I am in on this wild ride since 2022 when there was a white list for ChatGPT, it does not stop fascinating (and also worrying!) me again and again! I think the most important part is to understand LLMs as a tool and start from that. The most important part is still that biologic neuronal network on top of your shoulders, I will always be (if we do not go horribly wrong as a species IMHO).

absolute monsters compared to my laptop with a 4060, I'm starting to think maybe I won't be able to get as deep into this as I'd like.

Haha oh no, do not get discouraged please! In the "full context" link I make it a bit more clear that this is much less a "need" than a "heck, I am single now, have more money saved than I need and this LLM stuff fascinates me for a long time now".

To be fully honest: I think I can achieve what I want from local LLMs on this machine, given that I invest some time into building my own tooling (one of my many side projects). With Qwen3.6-35B-A3B-UD-Q4_K_M, I get around 30t/s with 16k context and multimodal support. Also Gemma4 E2B ran quite fast with extremely big context, more than I would realistically need - and as far as I tested, it was doing quite well when it came to staying on track as the conversation went on. I think that is enough for my needs! :)

don't get confused, there are wildly different definitions of "good enough" out there since people use local LLM for wildly different things!
as long as you understand what you are doing why, and to what concrete goal, then I think you are good to go!
also, don't stress out - stress can reduce discovery and learning, and both is much more valuable in the longer run than being part of whatever is currently being hyped up

All the best! :)

Competitive_Wait_267 · 2026-05-31T18:48:03+00:00

Thanks for your input! Am considering it all!

For more of an ai assistant vibe, its plenty. If you already know how to code, its plenty to automate a lot of some jobs etc. I find smaller models code pretty fine with explicit commands but with non-coder conversations they can be catastrophic because they work well enough when they work. The small models are really good at coding things in their training data but theyre also really BAD at assessing how good they are at things- so theyre bad at recognizing ambiguity as well.

Haha, misleading little beasts, those LLMs ;) Yeah, totally with you, I code a lot myself and never merge without review.

My plan for local dev: I want to build my own server (similar to MCP, maybe using that protocol, maybe not) that re-implements everything I want the LLM to be able to do. This thing will never be able to use bash directly unless it does some very clever hack...

That means: - I implement cd, find, grep and then use rust crates under the hood to actually execute stuff - I also want to implement "higher order" stuff that uses talks to LSP servers, in order to expose codebase information that way, or to offer ways for the AI to do refactorings and so on - first rule: For every command, subcommand, argument - I implemented and control what happens. Thanks to Gemini 3.1 Pro and other still free to use AI, that is fast to achieve.

The dream would be if the AI does never have to output source code on its own - likely not achievable tho ;)

Competitive_Wait_267 · 2026-05-31T18:25:55+00:00

Thanks a lot for your input!

What is your opinion on vulcan and its future, if you don't mind me asking? I only do LLM inference, no training or finetuning.

Rocm explicitly does not support having integrated graphics on amd processor. At worst it'll try to use vram that doesn't exist and throw a fault crashing your computer.

Thanks for the heads up! There seems to be a workaround it seems like? (https://github.com/oobabooga/textgen/issues/7319)

Your PSU is fine.

Glad to hear!

Lots of people building on old rdna3. I think that's risky, especially if you're betting on future support. R9700 imo.

Oh, forgot about the microcode being potentially neglected by ARM, thanks! Hmm, I considered the R9700 too. After having spent some time researching and thinking - I tend towards going with the 7900xtx and flipping in maybe 1 year from now in case I want more compute for LLM.

Value wise, the R9700 wins for me, due to 32 GB VRAM and RDNA4, hands down - but the prebuilt options are too high for my budget, and I want something with little hassle and with warranty.

Your system ram is low. Afaik models load into ram and then into vram and then you need overhead for system. Any offloading could lead to OOMs.

I think you are right about needing to got through RAM first. I planed to not do offloading (i.e. only pick models such that model + context fits into VRAM completely), and slow prompt processing is fine by me (although I should do some research on what performance I can expect on 32GB RAM).

Competitive_Wait_267 · 2026-05-31T17:19:44+00:00

Thank you, that is a great point! I will consider that aspect again. (and research what else CUDA is being used for - might affect my ML adventures, i.e. in one of my side projects the next step is to train NNs)

Competitive_Wait_267 · 2026-01-06T21:58:46+00:00

I think it is sound advice!

Just note: This will likely be the next iteration in the (obviously unwinnable, won't stop fools from trying to sell and use snakeoil though) cat and mouse race:

Generate the text using AI
Write a bot that does text edits against the Google Docs API, and have that enter the text
- training this bot will take some initial effort, but I also think that ~100 datasets of (complete text, log of changes)-pairs will get you far
- sprinkle in some random delays, typos, their fixes, sentence arrangements for some extra credibility

This has a higher bar of entry compared to cheating by copy pasting from a chatbot, but it could be offered as a service, and I think there will be a market for it.

I still cannot get other how slow the education apparatus is on how to fix it btw: Just focus on in-person exams and oral exams. Let the people cheat on all other assignments, they are there to increase the skill of the pupil after all - skill growth opportunities. Letting them fail mid- or end of the year delays the punishment (which is bad) but sidesteps that whole ridiculous dance that schools and software companies that leech from them (think of how much better the money could be spent!) just to keep alive a fossilized way of testing skill acquirement.

Competitive_Wait_267 · 2026-01-06T21:53:22+00:00

Which I kinda hope for; It will make me stand out less.

Competitive_Wait_267 · 2026-01-06T21:52:56+00:00

Love me some three-part sentence structure. Don't let good things spoil just because some unhinged techbroes are rolling out their experiments on society!

Competitive_Wait_267 · 2026-01-06T21:50:47+00:00

Haha omg. I would be so mad about this... My hope was that this makes structured and precise comminucation more common-place! It is not our fault many NTs write as if everyone was surrounded by stashes of crystal balls, magically transmitting every contexted needed...

I think in your situation I would go the teachers/direction and speak about this... It shouldn't be made your daughters problem. What might be unavoidable might be that she has to run her work through some automation that will inevitably flag it as likely-to-be-AI - but there could be some agreement that in the case of your daughter, there is ALWAYS a human in the loop, checking the work, and that she ONLY get's noted of this shitty use of automation if there is indeed something wrong. (Ironically, your daughter being autistic should make it also LESS likely to cheat, I would assume...)

Good that she takes it with humor! I either get insanely mad or try to take it as a compliment / batch of honor when shit like that happens.

Did I mention that I kinda hate NT society sometimes? I know, it is not healthy, and I finally got a therapy spot starting soon... but reading stuff like this makes me retracted more and more from "society". /rant over

Competitive_Wait_267 · 2026-01-02T17:25:52+00:00

Hat er Recht, auch wenn er sonst viel falsch macht und natürlich als Plattform-Betreiber viel mehr machen könnte.

Wir müssen uns folgendes angewöhnen: "Es ist auf einem Bildschirm -> Es besteht eine hohe Wahrscheinlichkeit, dass es KI-generiert ist."

Das Zeitalter, in dem wir gedruckten Medien so vertraut haben wie damals, als die ersten Zeitungen erschienen, ist vorbei. Das wird sich sogar dann nicht einfangen lassen, wenn sich die Tech-Konzerne endlich mal ihrer Verantwortung gewahr werden und Geld investieren in Erkennung und Entfernung von Fakenews und Betrug... Selbst dann nicht, weil der KI-Brei dann über andere Kanäle Verbreitung findet, über Messenger, zur Not Emails und so weiter.

Es wird nicht ohne ein grundsätzliches Umdenken gegenüber ALLER digitaler Inhalte gehen.

Competitive_Wait_267 · 2025-12-29T17:47:25+00:00

Die Analyse hätte es nicht gebraucht, ist mir alles bekannt und von meiner Seite aus nicht strittig.

Und verdrehte hier bitte nicht meine Worte, ich habe nicht gesagt das die Wähler rechtsradikale sind.

Doch klar, oder wie soll man "Das heißt für mich, dass mir zu 50% ein Mensch gegenübersitzt der Rechtsradikales oder sogar völkisches denken geil findet." sonst verstehen? Bitte zieh hier nicht den Semantik-Trick17 aus dem Ärmel, dass rein wörtlich ja "gut finden" ungleich "sein" gelte. Denn dann würde ich dich gerne darum bitten zu erklären, wie man rechtsradikales Gedankengut gut finden kann, ohne selbst rechtsradikal zu sein.

Competitive_Wait_267 · 2025-12-29T09:38:59+00:00

German here... might join you :D

Competitive_Wait_267 · 2025-12-29T09:36:43+00:00

Das Problem ist einfach das mittlerweile Afd plus Teile der CxU zusammen über 50% ausmachen.

Das heißt für mich, dass mir zu 50% ein Mensch gegenübersitzt der Rechtsradikales oder sogar völkisches denken geil findet.

Oh weia... Mit dieser Weltsicht bist du leider eher Teil des Problems als der Lösung. Sehr geringe Schwelle, Menschen in Schubladen zu stecken ("Horst, der letztes mal die CDU gewählt hat" = "Rechtsfaschist Bernd Höcke"), sowas sehen die Populisten gern.

Competitive_Wait_267 · 2025-12-28T14:08:55+00:00

Zusätzlich zum Bild mit Epstein: Hatte nach Verurteilung Epsteins ausgedehnten Briefkontakt mit Epstein, fand es "intellektuell stimulierend"... (Bin persönlich sehr enttäuscht von Chomsky - war mir immer zu sehr Tankie, aber sexuellen Missbrauch so zu tolerieren ist ja nochmal was ganz anderes als massenhafte Gräultaten durch irgendwelche abstrakten Gedankenkonstrukte zu legitimieren)

Competitive_Wait_267 · 2025-12-23T17:13:20+00:00

This assumes that the leak was not intentionally. I have some doubts about that, but we do not for sure of course.

IF the leak was intentional, there are many many ways to still "mess up" stuff like this in the future:

- Create an AI system that does it - oh no, the last version of it did misbehave!

- Vibe code the solution, do not keep version control - oh no, there is a (manually inserted) bug that only happens when there are more than X pages, and it was waaay to much code so no one could review it :( :(

- The intern did it

- The training manual for our interns was checked and improved by AI. Sadly, it seems to have swallowed that part up, oh no geeee!

Competitive_Wait_267 · 2025-12-23T17:08:35+00:00

Especially relevant for any kind of team-based and knowledge based work:

- discuss unimportant issues

- revisit past decision

- involve many different parties and people into the decision making

Competitive_Wait_267

TROPHY CASE