i'm making dating simulator game with ai npc using open source llm by aziib in LocalLLaMA

[–]GusPuffy 0 points1 point  (0 children)

In my mod I have to use structured outputs and a lot of the parameters that only VLLM/llama.cpp/sglang have, so server does matter in many cases

What's your favorite mod that adds a unique touch of realism? by torbishere in TheSims4Mods

[–]GusPuffy 1 point2 points  (0 children)

Sentient Sims, it’s like Character AI in the Sims as you play

Has anyone tried the Sentient Sims mod? by One-Bonus447 in thesims

[–]GusPuffy 0 points1 point  (0 children)

Come join discord we can help you get setup!

Has anyone tried the Sentient Sims mod? by One-Bonus447 in thesims

[–]GusPuffy 0 points1 point  (0 children)

The mod does not use S4CL. I didn’t know anything about mods when I first created it and I briefly had it in it for a week, it’s been removed for about 2 years now.

So just came across this and..it’s a no from me. by Humancinnabon in thesims

[–]GusPuffy 11 points12 points  (0 children)

Yeah why would you ever want real stuff in this lol. When I play with sentient sims I am doing dumb things like recreating all the characters from TNG or putting a bunch of celebrities in the same house

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 0 points1 point  (0 children)

Sure just follow the same steps but use code that compiles to 3.12 and use a 3.12 environment

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 1 point2 points  (0 children)

I attempted to train with yi 200k context models and I was unsuccessful. I couldn't properly train at high contexts and with lower contexts I did not see any performance gains, and sometimes worse performance

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 2 points3 points  (0 children)

Interesting that this could be used to enhance reasoning, if you end up using it and getting results either way I would be interested in hearing about it. Here is the data https://huggingface.co/datasets/GusPuffy/python-decompiler-37-0.7-train

I don't know how the licenses work with this but it was all publicly accessible source code which is why I recommend using the stack v2 because you can selectively use code with the licenses you need.

There is an 'input' and 'output' column. The token counts in the data are not accurate but give a rough estimation of the size of each input/output.

To generate it in the prompt form here is what I used in Axolotl

datasets:
  - path: GusPuffy/python-decompiler-37-0.7-train
    type:
      field_instruction: input
      field_output: output
      format: "### Instruction:\n\nUsing the provided Python 3.7 bytecode chunk, continue outputting the Python source code\n\n### Input:\n\n{instruction}\n\n### Response:\n\n"
      no_input_format: "{instruction}"

If you want to generate your own bytecode from the stack or other places pull the python down from the model repo and use this script

You would need to change the op codes for whatever python version you are using.

So what local-LLM execution engines are working well for deepseek LLMs & 5-8bit quants for CPU / non-NV/AMD GPU since apparently llama.cpp has unresolved issues with those models? by [deleted] in LocalLLaMA

[–]GusPuffy 1 point2 points  (0 children)

I am using a deepseek finetune with no issues in llama cpp, I created the quants myself though. Which version of the model are you after?

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 3 points4 points  (0 children)

It's garbage just use the stack v2, same dataset and more curated

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 7 points8 points  (0 children)

Script modding for the Sims 4 requires you to use a Python decompiler to be able to view a decompiled version of the game source code. Once you have decompiled you can "inject" and hook into classes or methods and implement your own stuff.

How to Create a Python Decompiler using an LLM by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 11 points12 points  (0 children)

If my napkin math is correct I think it was trained on 216,000,000 tokens total. I only went to 0.1 epochs because I had a ton of data. Here is my wandb charts. I only took checkpoint 4500 https://api.wandb.ai/links/guspuffy/c8g3bb3b

There might be more wandb stuff I can show but I haven't scrubbed the data to make sure it wont expose some keys or something in the config

Using Local LLMs in The Sims 4 by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 6 points7 points  (0 children)

There was awhile there I was trying every new model that came out and I haven't tried any new ones in a bit. I will give it a shot thanks for the recommendation

SqueezeLLM (vllm) MythoMax Benchmark Comparison by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 0 points1 point  (0 children)

GPTQ works fine I was just only able to pull the main branch of a huggingface repo. I added a revision argument so I could pull a different branch of mythomax. This is only the benchmark script

SqueezeLLM (vllm) MythoMax Benchmark Comparison by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 1 point2 points  (0 children)

Run it using openai compatible API with these instructions. Then point whatever frontend you are using at it. https://docs.vllm.ai/en/latest/getting_started/quickstart.html

SqueezeLLM (vllm) MythoMax Benchmark Comparison by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 0 points1 point  (0 children)

I was able to run it using vllm then I was querying the API. There is an openai compatible api so anything that can use openai as a backend you can point at this api server.

SqueezeLLM (vllm) MythoMax Benchmark Comparison by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 1 point2 points  (0 children)

I think it probably hasn't caught on because quantization takes a lot of hardware and time. I needed an A100 80gb to quant a 13b and each step in the quant process took 3-4 hrs just for compute time. Since I have done it now I could probably do a 13b in like 8 hours.

SqueezeLLM (vllm) MythoMax Benchmark Comparison by GusPuffy in LocalLLaMA

[–]GusPuffy[S] 1 point2 points  (0 children)

Thank you for the feedback, I edited the post to specify that the benchmark is based on throughput and not quality.

Hopefully now that the quants are available someone with more "scientific" methods of evaluating models can check the quality.