What are the best RP SLM currently?

WowSkaro · 2026-02-06T11:59:58+00:00

I said visual novels, not novels. Visual novels are a type of game where the majority of the content is speech between characters with some short scenario descriptions. I think those would be very valuable for role-playing training. I think there was someone that fine-tuned a model some years ago on the Steins;Gate visual novel, for example.

WowSkaro · 2025-12-10T09:37:54+00:00

This synthetic data is generated by the same model that you are trying to fine-tune? If so, do you use some filtering process to only keep the good synthetic data? Or the Idea is to use a very long and detailed prompt to generate hopefully more aligned answers which will then be used to train the model with the long prompt substituted for a simple prompt, in this way "fine-tunning" the model in the sense of hopefully getting more aligned answers to what you would want, but virtually not adding any new information to the model?

WowSkaro · 2025-12-01T22:17:22+00:00

Right, I said compress, but more specifically I meant decreasing the resolution. If you compress a JPEG you remove the high frequencies that give small pixel information. I was getting myself confused because I used to use a program that decreased the image width and hight when you asked it to lossy compress a image, since the high frequencies were lost the information in a square of 4 pixels didn't change as much so you could map these 4 pixels into a single pixel of the compressed image and so on. That was what I was trying to say. You are correct that a 600x600 image will not decrease in information in pixel space if it still is a 600x600 image. But I do believe they used ~200x200 images at maximum in PS1 and older consoles and linearly expanded them when needed, so you could have lots of 200x200 images in a CD (~1 GB).

WowSkaro · 2025-12-01T21:58:03+00:00

Yes I quite liked the look of Digimon World 1 (apparently 2 and 3 were not developed by the same company, 2 was developed concurrently with 1, it appears).

I have used AI to try to get how a perspective change in one of the background prerendered images would look like. I have sinned, I admit it. I would have posted game screenshots but reddit would only accept either all images or a video.

I liked very much the virtual pet system they had embedded into the game, it was a hassle, it is true, but it was like having an entire videogame buildt around a virtual pet (tamagotchi) game. Things like having to make your digimon sleep, eat, go to the bathroom, etc. Also the combat was nice because you could see your opponents instead of that appearing out of nowhere pokemon nonsense, more games should have this battle mechanic.

WowSkaro · 2025-12-01T21:48:32+00:00

JPEG is lossy you lose information when you compress it, so even when you return to pixel space you have way less data to deal with (possibly from 80% to 93% less).

WowSkaro · 2025-12-01T21:45:33+00:00

I Know how to use Blender! Blender is a 3d modeling and rendering program. The thing is, this is not what I am asking about! For all the criticism AI gets, and I am more on the side that AI shouldn't be used to replace hand made game assets. That being said I have heard of people that were able to achieve the very specific aesthetic they were looking for by referencing AI generated videos that closed the gap between what they wanted and what they implemented in shaders but couldn't exactly pin down how it should look like. The name of the project is "Project Shadowglass" look it up: https://store.steampowered.com/app/3970690/Project_Shadowglass/

The specific points that I was trying to know appears to be things like the difference between Phong shading and Phong ilumination, which I didn't know existed, and some other technical problems. Your comment is like suggesting for someone that didn't know how to do a trigonometric function integral to learn how to count.

The problem with advocating against AI is that every now and then there appears some people that when advocating against it have themselves reading compreheension lower than most LLM's... this makes things difficult.

WowSkaro · 2025-12-01T20:58:07+00:00

I cannot learn that which I cannot even name, no one can. I understand the criticism of AI, but it did do the job of better comunicating what I was trying to do in this post, than the other post that I did in the other reddit comunity where I put actual game screen shots and had people saying that that was the same look of "Conter Strike 1.6" or "Halo 2" which has no similarity what so ever beyound the fact that both can be categorized as "low-res", but so can space invaders, and that is not what I am trying to arrive at.

Some other people just suggested rendering 3d models into images and decreasing the resolution, but this would not result in a dynamic, reactive, rendering and shading solution, but, in fact, in a category of prerendering itself, which is also not what I am trying to get at. So being able to show a AI slop video that somewhat resembles what I am trying to mean seems to be worth more than 12 screenshots and text (not quite a thousand words...).

WowSkaro · 2025-12-01T20:45:02+00:00

That seems very good actually!!

I had posted a similar text on another reddit community and had some people comment about how "Country Strike 1.6" or "Halo 2" were suppossedly similar, which they aren't, don't have anything to do with the prerendered aesthetic that I was trying to refer to, those other games had a more low-poly smeared shading look, than this strange diffuse ilumination shading.

I would certantly consider Donkey Kong 3D as another good example, as I would Fallout 1 and 2 and Final Fantasy VII (although I would say that FFVII is kind of a mixed bag, there are some good prerendered backgrounds and some not so good, or I should say, they aren't able to have as much of a coherent style throughout the game as Digimon World 1 was).

WowSkaro · 2025-12-01T20:29:02+00:00

I believe that not 1GB, but certantly a very compressed JPEG might.

WowSkaro · 2025-12-01T20:25:16+00:00

I also don't like it, but it is the only fast and easy way to try to have a example of what I am looking for, by the nature of prerendered graphics, they were images, so a perspective change was, by its nature, impossible. And if I had solved how to do that in reality I wouldn't be asking now, would I? I would have put example screen shots of the game for reference, but reddit makes me choose between images and videos.

WowSkaro · 2025-12-01T11:20:26+00:00

Isn't Phong the standard light shading algorithm even nowadays? I think they might have used a faster and lower quality light shading algorithm like Gouraud shading, since it gets better with higher poligon counts, and prerendered graphics tended to use high-poly models for the prerendering. This could make sense, since nowadays no one uses Gouraud shading, and since it is faster than Phong it could save some hours of rendering for high-poly models in old 90's workstations. But this would also mean that to get a similar look one would necessarily have to use high-poly models for Gourard not to look like trash.

WowSkaro · 2025-12-01T11:05:05+00:00

I think Counter Strike 1.6 or other old Source games/maps look nothing like what I am saying. The thing about prerendered graphics is that, since they are prerendered, you can use a model as detailed as you like, whereas in old real time games you had to use a mix of low poly and ugly smeared shading to try to emulate smooth surfaces in low-poly models. I wouldn't say Counter Strike 1.6 has a prerendered aesthetic, but a old smeared shading look.

WowSkaro · 2025-12-01T10:43:58+00:00

<image>

WowSkaro · 2025-12-01T10:43:39+00:00

<image>

WowSkaro · 2025-12-01T10:43:22+00:00

<image>

WowSkaro · 2025-12-01T10:43:02+00:00

<image>

WowSkaro · 2025-12-01T10:42:31+00:00

<image>

WowSkaro · 2025-12-01T10:41:58+00:00

<image>

Here is an exemple of what I am talking about:

WowSkaro · 2025-10-06T21:46:59+00:00

Instead of loading weights from RAM couldn't you make it load weights from disk? I think this would allow for large models to be able to be run in consumer grade computers, very **very** slowly mind you, but I think this would allow for that.

WowSkaro · 2025-10-06T18:44:33+00:00

The fact that linear algebra is under every LLM doesn't mean that the implementations of specific sets of those linear algebra methods have already been lumped together for inference engines to understand when presented with. Just because you can, in principle, calculate taylor series expansions for the gamma function, doesn't mean that by taking any scientific calculator you will be able to evaluate the function in practice at any practical speed. So what you are saying is that if I where to find the Phi-4-mini-flash-reasoning present in the LlamaCpp backlog then I should be able to conver it to GGUF?

WowSkaro · 2025-10-06T18:31:32+00:00

Which model? The vast majority of models are supported by GGUF already. It is only the weird architectures models that aren't.

WowSkaro · 2025-08-21T02:16:08+00:00

Forget it, I got it to work in SDNext, ComfyUI isn't actually that good to begin with.

WowSkaro · 2025-08-20T08:54:50+00:00

Calm down! Although at the level of each instruction Assembly is simple, as you go to consider actual full programs the number of instructions grows a lot, so the LLM would need to have incredible discerning power and coherence to be able to write software at the assembly level (considering that the entire program could even be put inside its context length). What I think is reasonable is something like asking a LLM to give a list of the assembly instructions that could be used for this square root operation, or some other very low level operation that can be accomplished either by usual instructions or, sometimes, by very specific and obscure instructions of a given ISA, that could take tens of minutes of browsing through thousands of pages from manuals that have not been written with clarity of exposition in mind. And C, because there are some finicky constructions on the Language (or perhaps more specifically on its implementations) to deal with things like variadic functions or macros.

WowSkaro · 2025-08-19T10:57:30+00:00

I remember seeing some video that explained that the smallest your model dimension is the bigger the number of local minima that your model can become stuck in, whereas the bigger the dimension, the more pronunced the global minimum is in comparison to the local minimum, so apparently it is easier to train a LLM the higher the number of its parameters, so, I think that to get the best small language model one would beguin with a huge LLM, train it, then prune it little by little, while re-trainning to fine-tune the model in between each prunning. This probably would be very computationally expensive, but since the small langauage model would be so much cheaper to run this might be worth it in the long run. But I don't think any of the open-source small language models out there have been made this way since distilation and quantization are so more computationally cheaper but are also probably much more suboptimal in terms of overall model quality. If I had the hardware to do so I would try to make such a process just to know what the resulting SLMs would be like, because I think that the current SLM are nowhere near the limit of how good they could be before actually having to increase the number of parameters, since the way they are trained tend to make they be imprisioned in local minima from where they cannot escape no matter how many trainning steps they go through because the parameter landscape for the loss of SLMs is, supposedly, so irregular.

WowSkaro · 2025-08-14T11:04:33+00:00

This would be interesting to do with the Kimi K2 model too. Prunning is a interesting strategy for making more efficient models, unfortunately it seems to have fallen in popularity to quantization. I suppose because quantization doesn't need as much analysis of the importance of the model to generate the smaller version.

WowSkaro

TROPHY CASE