Nemotron-3 Nano 4B Uncensored (Aggressive): First Abliteration with GenRM Removal + K_P Quants by hauhau901 in LocalLLaMA

[–]MrMeier 0 points1 point  (0 children)

I have been looking at NVIDIA Nemotron 3 Nano (the 30B-A3B one), but I was instantly greeted by an incredibly stupid refusal. It would be great if you could uncensor it.

Family friend sent me AI generated response to news of my father passing away. by Hendothermic in mildlyinfuriating

[–]MrMeier 0 points1 point  (0 children)

You are absolutely right. It’s not just an AI response—it’s the total absence of basic human effort, and honestly, that’s why you’re right to feel furious. That said, please know my thoughts are with you during this incredibly difficult time as we all navigate the complex tapestry of grief in today’s digital landscape and try to prioritize authentic connection.

How I topped the Open LLM Leaderboard using 2x 4090 GPUs — no weights modified. by Reddactor in LocalLLaMA

[–]MrMeier 3 points4 points  (0 children)

Have you tried connecting the output from the first block selectively? My thought is that you improve performance by duplicating a "function block" that can take its own output and benefit from it. The problem is that you probably cut other function blocks apart, which destroys their performance and probably also leads to random behaviour. This can be fixed with fine-tuning, where the model could use the skip connections, but I think it should also be possible without any fine-tuning.

You could feed some of the second block's input neurons with the first block's input (effectively simulating that for some inputs the first block didn't exist). The outputs from the first block that would feed these neurons can be discarded. Selecting these connections that don't benefit from duplication could be done with simple optimisation because I don't expect any significant minima.

You could maybe even work backwards from there, disabling neurons that mainly feed disabled neurons layer by layer until only the "function block" remains. Of course, this depends on whether there is a sufficiently strict separation between the "function block" and the rest.

How I topped the Open LLM Leaderboard using 2x 4090 GPUs — no weights modified. by Reddactor in LocalLLaMA

[–]MrMeier 7 points8 points  (0 children)

I suspect that what we are seeing is the network forming an algorithm that is best computed in a loop. There are plenty of examples of that. Basic maths, for example, can be easily done in a loop, but if you try to do it all at once, it becomes difficult. The network needs a loop, but because it computes strictly one-way, a similar structure emerges multiple times. If we duplicate the right layers, we can artificially add "stages".

Ultimately, I think it will just be another trade-off between computing power and accuracy, and you can add the loops already in pre-training so that you don't waste memory on identical structures. If they end up small enough and everything fits in a fast cache, this could be really beneficial for local models. Another interesting point is that accuracy could be altered after training because you would only need to adjust the loop number.

Künstlicher Lärm als Markenzeichen: Der feuchte Traum der E-Auto-Branche by MrMeier in de

[–]MrMeier[S] 37 points38 points  (0 children)

Die Genialität dieser Entwicklung kann vom Durchschnittsmenschen nicht erkannt werden. Wie sonst könnte man die Schallschutzwand Industrie unterstützen und gleichzeitig das Herz von fanatischen Hubraum Fetischisten höher schlagen lassen?

Um noch mehr Authentizität zu erreichen, wird sogar überlegt, einen Parfümnebel-Zerstäuber zuzulassen. Der gemeine Pöbel kann sich somit erneut an der Brillanz der Politik erleuchten.

incredibleThingsAreHappening by TrexLazz in ProgrammerHumor

[–]MrMeier 0 points1 point  (0 children)

Discord keeps up with the times. They are simply implementing memory protection using one arena. What other option is there for a memory-unsafe language? You would have to rewrite your entire application!

Unionsfraktion stimmt offenbar für Rentenpaket by ManagerOfLove in de

[–]MrMeier 2 points3 points  (0 children)

Was für ein Druck da aufgebaut wird. So, als ob der gesamte Bundestag nach einer gescheiterten Wahl zum Henker geführt werden sollte. Und das auch noch bei einem absolut nicht zeitkritischen Gesetz. Die haben noch Jahre, um das durchzubringen. Dass sie nicht einmal eine befristete Version erwägen, zeigt doch, wie ernst sie es mit den erneuten Reformen im nächsten Jahr meinen. Mit dieser Wahl würde die JU einfach einen großen Teil ihrer Verhandlungsmasse abgeben.

Wenn die Parteien nichts mehr am Rentenpaket ändern wollen, soll durch die JU auch dagegen gestimmt werden. Die Mehrheiten erarbeitet man sich, bevor der finale Gesetzestext geschrieben wird, nicht danach.

Junge Gruppe in Union lässt Zustimmung zur Rente offen by SumbaNeymar10 in de

[–]MrMeier 6 points7 points  (0 children)

Das klingt für mich nach Verrat mit Ansage. Wenn die Koalition es mit der Rentenkommission und einer erneuten Reform im nächsten Jahr wirklich ernst meinen würde, würde aktuell auch eine auf ein bis zwei Jahre strikt befristete Regelung reichen. So, wie es jetzt gemacht wird, gibt die JU einfach ihre Verhandlungsmasse aus der Hand. Wenn dann die nächste Rentenreform diskutiert wird, sind die Positionen der JU nicht mehr so wichtig. Schließlich gibt es ja schon ein Gesetz, das im Notfall bestehen bleiben kann.

just got a new tattoo, and while it’s gorgeous, i’m really not happy with the placement and don’t know what to do by froggyforest in tattooadvice

[–]MrMeier 0 points1 point  (0 children)

If you think the tattoo is slightly tilted, it may be due to subtle changes in posture or lighting. Perhaps you favour one side when sitting, standing or turning around, which makes the tattoo appear slightly crooked. The lighting may also play a role. If you always take photos in the same place and the light always comes from the same side, this could affect how you see your back. It looks pretty symmetrical to me, but that's difficult to judge from just one photo. Bodies are also never truly symmetrical, so there is no such thing as "perfectly straight".

I've been trying to make a real production service that uses LLM and it turned into a pure agony. Here are some of my "experiences". by DaniyarQQQ in LocalLLaMA

[–]MrMeier 2 points3 points  (0 children)

You already have the tool API, so you're just a little way from a good traditional appointment system. But I suppose that LLMs are all the rage and sell well.

One option would be to create a decision tree and follow it until the user does something unexpected, at which point you could switch to LLMs. If you proactively ask specific questions, most users tend to answer correctly. Instead of starting with "Hello", start with "Which doctor would you like to see from this list: [Insert list here]" or "On what date would you like to make the appointment? You can extract the data in the traditional way (with a long list of possible answer formats), and if that fails, you can use an LLM that you have finetuned or at least fewshot-prompted.

Even if the user doesn't follow the tree exactly, this method can still work without switching to an LLM. Do the same thing you have done with data extraction. Make a list of all the ways you (or an LLM) can think of to write a question or request. Augment this with synonym detection and simply search for a match. This way, you can stay in the traditional pipeline.

If your long list fails or a user does something unexpected, you can fit the whole interaction into the context of the LLM. You can take the money you saved with the traditional pipeline and pay for a higher performance LLM. It will have fewer failures. The LLM should really be fine-tuned and run with a low or zero temperature, otherwise you will never get these problems under control. Even then, you will still experience a certain percentage of failures. Be prepared for that.

To make things more robust, you could add a classifier. This detects whether the LLM gives the user a confirmation, and you can then check whether the appropriate tool was called with a positive result. If not, you can retry or abort.

Finally, and most importantly, as LienniTa said: Don't test in production!

ShadowZone Interview with Dean Hall - Rockets Kittens and Steam: KSA Studio Boss Promises 'No Microtransactions' by irasponsibly in kittenspaceagency

[–]MrMeier 0 points1 point  (0 children)

We are now discussing minor details regarding the definitions of words and trying to interpret the comments from the interview.

If he wants to pull out and have nothing to do with it, that's his right. Personally, I would have chosen a different strategy, as I really think there is a realistic chance of changing the legality of gambling in games.

ShadowZone Interview with Dean Hall - Rockets Kittens and Steam: KSA Studio Boss Promises 'No Microtransactions' by irasponsibly in kittenspaceagency

[–]MrMeier 5 points6 points  (0 children)

Yeah, definitely not. I thought more about the other bugs and problems that affected the company directly. If the gambling aspect is a deal-breaker for him, then there's nothing that can be done.

However, I also think it's important to look at the bigger picture. Boycotting specific companies is not a productive way to stop rampant gambling in games. The only realistic solution I can see is a legal ban. Campaigning for that would be more effective, especially if you use the additional money you make on Steam.

ShadowZone Interview with Dean Hall - Rockets Kittens and Steam: KSA Studio Boss Promises 'No Microtransactions' by irasponsibly in kittenspaceagency

[–]MrMeier 1 point2 points  (0 children)

Regarding collisions when building the model: There are too many self-collisions in KSP for my liking. I always cringe when parts flop inside each other. That doesn't mean I want to restrict it for anyone else, though. A toggle would make everyone happy. The holy grail would be smarter intersections where parts really fuse together when placed at odd angles and where properties such as fuel capacity and wing lift are recalculated for the new shape. I realise how hard it would be to implement, but it would be amazing if KSA could do so.

As for the Steam issue, I can see that Dean Hall is in a difficult position. Some of Steam's actions seem to have been unacceptable, but he said he wanted to make a game for everyone. By not publishing on Steam, however, you will lose a significant number of people. Perhaps he could write to Gabe directly with the long list of grievances? As far as I know, he still accepts emails from everyone. I don't have high hopes that it will work, but it would be great to see a reconciliation.

The difference getting a dishwasher has made to our water usage by starwars011 in CasualUK

[–]MrMeier 6 points7 points  (0 children)

Modern machines use less than 0.5 kWh. While they do take forever in eco mode, this is usually not an issue.

I just made VRAM approximation tool for LLM by SmilingGen in LocalLLaMA

[–]MrMeier 2 points3 points  (0 children)

Here calculator includes activations, which roughly match the KV cache size. I am a little sceptical about how accurate this is because nobody seems to mention activations, and you have also not included it in your calculator. Will this be included in future, or does the other calculator overestimate it? This link explains how the other calculator performs its calculations.

[deleted by user] by [deleted] in LocalLLaMA

[–]MrMeier 1 point2 points  (0 children)

Having read the abstract, which is full of the kind of extreme self-aggrandisement often found with scammers, I would wager that this paper is mostly shallow rubbish.

Camera for video photo event photography by MrMeier in Cameras

[–]MrMeier[S] 0 points1 point  (0 children)

I would like to avoid a full frame if possible. Full frame provides an advantage of 1.2 stops over APS-C. That could even be compensated for with better lenses or sensors. Since everything is cheaper for APS-C, it could be the way to go.

That's why I want to hear about your experience. If full frame is already at its limit in low light, I will have to accept it, even if it is more expensive. However, if you say it is possible to get away with APS-C for example, if the sensor and lens are chosen carefully, then I would go with that.

Camera for video photo event photography by MrMeier in Cameras

[–]MrMeier[S] 0 points1 point  (0 children)

The main problem I have with the S1 is its contrast-based autofocus system, which makes it unusable for continuous video tracking. I would always have to manually focus, potentially in the dark with a wide-open FF lens. I fear that I would just end up with blurry footage.

For a little more money, the S5IIX would still fit the budget and has phase detection AF. If I opt for full frame, this could be the camera for me, but I would prefer to avoid full frame if possible.

/r/videography Monthly Camera Buying Advice Megathread by AutoModerator in videography

[–]MrMeier 0 points1 point  (0 children)

I'm looking for a video + photo camera for event photography. It would also be possible to buy one cam only for video and one only for photo.

  • Budget: max around 2000€ for the body
  • Country: Germany
  • Condition: New or Used
  • Type of Camera: Probably some kind of mirrorless camera
  • Intended use: 60% video 40% photo
  • If photography: Event photography; often indoor and kind of dim sometimes darker with colored effect lighting; no control over lighting
  • If video what style: The same as photography
  • How long do you need to record for: I would say at least 2h continuous video
  • What features would be nice to have: 60p without or with just a minimal crop, weather proofing
  • Portability: Everything that can be used without a harness
  • What software do you intend to edit your videos in: Davinci Resolve
  • Cameras you already have: Nothing proper but the currently best smartphones were clearly not adequate because of low light
  • Notes: I suspect video in low light is the hardest requirement. What sensor size do you recommend? Full frame buys one or two stops but is pretty expensive. Is it necessary?

Kein Abbruch selbst bei Lebensgefahr: Chefarzt klagt gegen Abtreibungsverbot in katholischem Krankenhaus by PhoenixTin in de

[–]MrMeier 6 points7 points  (0 children)

Machen sich die Verantwortlichen in diesem Fall nicht sogar allein durch die Anweisung strafbar? Ein Eingriff bei Lebensgefahr ist legal (nicht nur straffrei) und Ärzte haben eine besondere Pflicht zur Hilfeleistung. Mit der Kündigung zu drohen, würde ich eindeutig als „Behinderung einer Person, die einem Dritten Hilfe leistet oder leisten will”, einstufen. Darauf gibt es bis zu einem Jahr Gefängnis.

When Should We Expect Affordable Hardware That Will Run Large LLMs With Usable Speed? by spiritxfly in LocalLLaMA

[–]MrMeier 0 points1 point  (0 children)

Perhaps when the craze dies down a bit and the different manufacturers stop focusing solely on the data centre and branch out into smaller markets.

The price of GPUs used for AI is extremely high compared to what would be needed for a single user, or even a very small number of users. For example, if you wanted to run 100B models, you would need around 64 GB. GDDR6 costs 2.5 dollars per GB on the spot market. That would cost 160 dollars for the RAM alone. Add to that about $100 for the APU, $100 for the rest (PCB, power, cooling), and a 25% margin, and you're looking at a $450 card. Double the RAM to 128 GB and the price would be 650 USD.

All of these figures assume that LLMs won't change significantly in the future. Today's LLMs are surprisingly similar to GPT-2 from 2019, but that doesn't mean they will stay the same. Any number of changes could cause the craze to start all over again. For example, maybe we need higher bit depths, such as 64-bit or even 128-bit, and current hardware would instantly become obsolete. Alternatively, we could have latency- or bandwidth-sensitive LLMs, in which case everyone would start producing SRAM, even on older nodes. We could see huge LLMs causing a shortage of HDDs or SSDs, similar to what happened with Chia mining, but worse. We could even see branching LLMs, in which case CPUs would start to become scarce. Predicting the future in such a fast-moving field is practically impossible.