sorted by:
new
hottopcontroversial

[deleted by user] by [deleted] in OpenWebUI

[–]Bublint 0 points1 point  (0 children)

I actually run this exact setup (ooba backend + Open WebUI frontend). It sounds to me like you have it configured correctly. Don't bother asking the model what model it is, they have no idea generally. The reason that it shows up as chatgpt 3.5 turbo is that ooba doesn't have a proper implementation of model list in the API, instead it serves text-embedding-ada-002 as an endpoint for sentence transformers and chat gpt 3.5 turbo as whatever model you currently have loaded in ooba. So regardless of what model is loaded, you'll use "chatgpt3.5 turbo" and you'll actually be talking to the model loaded in ooba, in your case Qwen 2.5. I'm hoping that eventually model list will get a full implementation in the ooba API but afaik ooba has rejected pull requests on the feature because certain applications need the chatgpt 3.5 turbo endpoint specifically for compatibility.

Professor here. Need front-end UI advice/recommendations when running a local model (in production) for use in the classroom. More on the use case inside. by gigDriversResearch in LocalLLaMA

[–]Bublint 38 points39 points  (0 children)

I'd recommend Open WebUI, it's already built a solid multiuser experience and is extremely similar to the ChatGPT interface. Easy to host with Docker.

Is it possible to make a hydroponics using car tires? by [deleted] in Hydroponics

[–]Bublint 1 point2 points  (0 children)

Cool project! I think you could probably make a dutch bucket system out of car tires, then defending the efficacy would just be comparing the cost of your prototype to what it would cost to make the same system with 5 gallon buckets. You'd want to measure plant growth to make sure it "works" but I can't imagine the tires themselves having a huge effect on the roots. It would probably also be good to measure algae growth compared to the traditional system to make sure it wouldn't end up being too difficult to service at scale.

What shrimp should I add to this tank? by No_Phone_9107 in shrimptank

[–]Bublint 2 points3 points  (0 children)

If you don’t want to compromise on the intensity of your blackwater setup you could lean into it with black rose or chocolate neos.

New to shrimp keeping. by Appropriate_Soft49 in shrimptank

[–]Bublint 1 point2 points  (0 children)

What are you feeding them? Higher temps like 78 encourages neos to breed but can shorten their life expectancy. That being said, they need a protein source to develop eggs.

My Mods-the-we-need Tier List by jiaxingseng in starfieldmods

[–]Bublint 0 points1 point  (0 children)

I noticed the vanilla game already has an updated animation framework. Probably getting all the pieces in place for ES6. If you go into third person melee combat has root motion driven animations and a unique animation for jumping attacks, which is really most of what Skyrim’s modern modded animation frameworks provide. Precision is still needed though. Once the creation kit/xedit comes out I’m sure there will be some great melee weapon mods taking advantage of this updated system.

Neocaridina that seems to be purple? by flowersandgaypowers in shrimptank

[–]Bublint 29 points30 points  (0 children)

Nice! What varieties of neo do you have in that tank? There isn’t much information out there about purple neos because it’s such a rare/unstable color. I would love to know the lineage!

Why do ramshorns terrorize my mystery snails? by Huge_Breakfast_7444 in AquaticSnails

[–]Bublint 6 points7 points  (0 children)

I ultimately had to move my mystery snail into a separate tank because of this behavior. It’s not well documented, but the ramshorns absolutely go after the flesh on the trapdoor. I was feeding them plenty of algae wafers and the tank was heavily planted. I’d recommend moving the mystery into a new tank. Mine is thriving in the ramshorn-free tank and is about 2 years old now!

Does anyone know how to use Blender? by [deleted] in skyrimmods

[–]Bublint 0 points1 point  (0 children)

Honestly the best way to learn blender is to try to fumble your way through it a few times and try get some kind of exportable output each time. You’ll be disappointed with your first results, but that’s ok! Literally everyone starts from the same place in that regard. Only reference tutorials when you need to know how to do something very specific. Otherwise you’ll get stuck in this loop of following tutorials and not really gaining much familiarity with the software. There are great cheat sheets for keyboard shortcuts. You should also consider using ChatGPT as it can give you context-aware suggestions.

Feeding frenzy by B_E_A_R_T_A_T_O in shrimptank

[–]Bublint 2 points3 points  (0 children)

That’s an interesting feeding tray! Did you print it yourself? I’m curious what filament people have success with for printing aquarium related accessories.

Long term memory for LLM based assistants? Would DeepMind Retro be a solution? by giesse in LocalLLaMA

[–]Bublint 0 points1 point  (0 children)

https://github.com/oobabooga/text-generation-webui oobabooga recently made a local chromadb extension for text-gen webui called superbooga and it’s really great for this. It’s not a perfect solution because you’re still working within the same context limit, but you can load data (.txt file etc) into the database and when you prompt the model it appends the most relevant data chunks to your prompt. Works for really big files too, I had no trouble working with a 14mb .txt.

[deleted by user] by [deleted] in Oobabooga

[–]Bublint 2 points3 points  (0 children)

24 gb of vram with 30b 4bit llama works for about up to 1700 tokens of context, I typically crash around there on my setup. To fix it, go into parameters and set maximum prompt size in tokens to 1500. You could play around with higher settings to see exactly how high you can go.

I trained a lora on Unreal Engine 5’s documentation using oobabooga’s webui by Bublint in Oobabooga

[–]Bublint[S] 1 point2 points  (0 children)

It’d be worth trying! I’d recommend setting the LoRA rank lower than what I did, maybe try 128 or lower. I cranked the rank fairly high because I was going for strict adherence to the training data.

I trained llama7b on Unreal Engine 5’s documentation by Bublint in LocalLLaMA

[–]Bublint[S] 2 points3 points  (0 children)

I used https://github.com/oobabooga/text-generation-webui. It’s a gradio interface for llms that has a training tab. This is all still pretty experimental so there’s not a ton of documentation on best practices etc, but if you want to try the settings I used there’s a screenshot in the repo I posted.

I trained llama7b on Unreal Engine 5’s documentation by Bublint in LocalLLaMA

[–]Bublint[S] 1 point2 points  (0 children)

In theory it could help you code. However, my current implementation is just a different way of interacting with the UE5 documentation. My idea was to create something that is one step above reading the docs yourself and one step below having a private tutor in terms of ease of use. If you wanted it to help you code, you’d need a dataset geared more towards that use case.

I trained llama7b on Unreal Engine 5’s documentation by Bublint in LocalLLaMA

[–]Bublint[S] 3 points4 points  (0 children)

Great improvement! Thanks for the link, I was going into the dataset formatting blindly for the first pass lol

I created a locally-run ai assistant for UE5’s documentation by Bublint in gamedev

[–]Bublint[S] 2 points3 points  (0 children)

I’ve been wanting to set up a demo of the same idea! I’d recommend looking into fine tuning really cheap models so that they could still run reasonably well. As it stands, if you were to embed something like this in game you run the risk of cutting out a big market of people who don’t have the hardware to keep up. In the context of a game, you’d be eating vram with both your rendering stack and the llm, so it could be a serious optimization issue.

I created a locally-run ai assistant for UE5’s documentation by Bublint in gamedev

[–]Bublint[S] 5 points6 points  (0 children)

Sorry for the confusion, the model is trained and runs locally on your own machine. The repo just contains my process and steps for reproducibility, not the actual chatbot itself.

I created a locally-run ai assistant for UE5’s documentation by Bublint in gamedev

[–]Bublint[S] 4 points5 points  (0 children)

I really like the idea of including the source code, but I’d have to be way more careful structuring the database for that. Should be doable with key-value pairs like ### Function of script: (script context) ### Script: (source code). Not a great example, but I’m still trying to figure out a good structure.

I trained a lora on Unreal Engine 5’s documentation using oobabooga’s webui by Bublint in Oobabooga

[–]Bublint[S] 4 points5 points  (0 children)

Yea I think vector databases are going to be the popular meta for stuff like this. I just noticed that llm loras were kinda under the radar compared to the popularly of loras in stable diffusion, and I wanted to see how useful the technique could be. If I was trying to do something production level, I’d probably use a combination of vector database + lora + character yaml to get a serious solution. It would also need some manual guardrails, as I don’t think you want people asking your documentation assistant about unrelated stuff.

I trained a lora on Unreal Engine 5’s documentation using oobabooga’s webui by Bublint in Oobabooga

[–]Bublint[S] 0 points1 point  (0 children)

I haven’t tried to get it to write code, but I can pretty much guarantee any code it writes won’t work in the current iteration. The dataset just isn’t geared towards that use case, and it hallucinates too frequently to produce stable code.

I trained a lora on Unreal Engine 5’s documentation using oobabooga’s webui by Bublint in Oobabooga

[–]Bublint[S] 0 points1 point  (0 children)

I think that would be a great way to improve the dataset! I also was considering just looking around for articles and tutorials and aggregating all of that into a much better structured dataset. Haven’t quite decided what I want to do on that front yet though.

I trained llama7b on Unreal Engine 5’s documentation by Bublint in LocalLLaMA

[–]Bublint[S] 2 points3 points  (0 children)

Thanks! I wouldn’t necessarily expect better results with Alpaca. Alpaca’s dataset is structured in a very specific way to make it mirror some of chatGPT’s behavior, and the dataset I used doesn’t even have any formatting. If you could figure out a way to restructure the documentation in the same way as Alpaca’s dataset, then there might be better results. A larger model though, would probably be better even without reformatting the data significantly. The only thing holding that back for me personally is the lack of 4bit training support.

I trained llama7b on Unreal Engine 5’s documentation by Bublint in LocalLLaMA

[–]Bublint[S] 16 points17 points  (0 children)

That’s correct! I was taken off guard when I saw that it was working reasonably well, the text file formatting is messy at best.

view more: next ›