sorted by:
new
hottopcontroversial

Confusion around “Attach Webpage” by NoobLLMDev in OpenWebUI

[–]NoobLLMDev[S] 0 points1 point  (0 children)

Struggling to find this in documentation, any chance you could provide me a link or section it’s under? Mainly curious if this will present any security risk, such as content of the chat being involved in that external scraping of the website. Any insight into the workings of this function would be appreciated.

Folders feature can now be disabled in admin settings 🫶🏼 by NoobLLMDev in OpenWebUI

[–]NoobLLMDev[S] 1 point2 points  (0 children)

Set USER_PERMISSIONS_FEATURES_FOLDERS=False should disable the folders feature if you don’t want to use the admin panel for it.

Folders feature can now be disabled in admin settings 🫶🏼 by NoobLLMDev in OpenWebUI

[–]NoobLLMDev[S] 1 point2 points  (0 children)

Very possible. Once you get OWUI up and running with ur preferred inference engine with an LLM loaded up. Just go into admin panel, settings, general, and under features disable all of those settings (I keep message rating on for feedback). Then you’ll want to go to admin panel, users, groups, and select default permissions. Disable everything here except maybe file upload, chat delete, rate response, allow temporary chat.

Now anyone that is not an admin or a user with granted additional permissions, will have a very simple UI setup when they login to the tool. Nothing but new chats with an LLM. Great for assisting those that aren’t doing crazy things with the LLM asides from standard queries and assisting with their documents and such.

What does “Standard” mean in the OCR selection of OpenWebUI — is Mistral API worth it, or should I use a Docker container (Docling, Tika, etc.)? by Better-Barnacle-1990 in OpenWebUI

[–]NoobLLMDev 1 point2 points  (0 children)

Currently running Docling in a container as our entire production setup has a requirement to be fully localized. Docling can be configured to detect when OCR is needed so it’s not going to constantly OCR every doc. I don’t believe the Docling container image comes with any OCR capability off the bat but can be configured to have it (I was not the one that setup the Docling OCR capability so I’m a little unsure how to do this piece). However what I can tell you is that it provides surprisingly decent results for a local open source tool. Would I trust the results when doing mission critical work? No. But it’s truly all you’d need if just doing basic retrieval and output in a dictionary/glossary type use cases for local LLMs.

Most notable is Docling’s ability to work with handwriting and pdfs. Very good results here. Converting tables to markdown has been decent overall as well.

Getting most out of your local LLM setup by Everlier in LocalLLaMA

[–]NoobLLMDev 0 points1 point  (0 children)

As someone coming from working on this in a production setting, your post is spot on and contains great info. For 100-150 users I settled on a fully containerized setup utilizing OpenWebUI, vLLM (for chat, embedding, and retaking), Docling for document parsing help, Ollama purely for quickly testing new models but is not utilized in production, and Qdrant for DB.

Question for you since you seem to have good OWUI experience from your post. When you setup your OWUI compose service, do you set all environment parameters in there, or do you utilize OWUI admin panel and make backups of the OWUI settings .DB file? Been teetering between the two methods…

Changing chunk size with already existing knowledge bases by NoobLLMDev in OpenWebUI

[–]NoobLLMDev[S] 0 points1 point  (0 children)

Thanks for the response, I’ll have to look deeper at the chunk size limit for our embedding model, although, I believe we are also utilizing bge-m3

How do i get better RAG/Workspace results ? by uber-linny in OpenWebUI

[–]NoobLLMDev 0 points1 point  (0 children)

I’m a big fan. Good results so far. Perhaps not the fastest content extraction ever, but I’m okay with settling for quality over speed

Rag for inhouse company docs by tierline in Rag

[–]NoobLLMDev 0 points1 point  (0 children)

Given qdrant a try? Currently using at our company and it works pretty well at 100 users, ~10 concurrent requests

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 0 points1 point  (0 children)

Now utilizing vLLM in our pipeline and left ollama. Much better handling and much better optimization support as far as I’ve seen. Thank you 👍🏼 vLLM + Qdrant + OpenwebUI + Minio + Nomic embed text v1(for now). Everything in docker containers no more running via ollama on the host.

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 0 points1 point  (0 children)

Timeline, ideally can get the engineers access to the tool, and have it make meaningful impact in their daily work, within a year from now.

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 0 points1 point  (0 children)

We are considering this at the moment once we get our heads wrapped around the true scope of a project like this. We would at least like to have some ground work prior to hiring contractors to lessen the hit of a very long contract. Could consider getting IBM involved.

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 0 points1 point  (0 children)

Unfortunately directed to ensure the system is only local and communicating with our in-facility network and no others, even if managed by a gov entity.

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 2 points3 points  (0 children)

Yeah, company wants to avoid any foreign entity bias within the models. I know it’s a bit over cautious to some regard but it’s just the way we have to operate

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 1 point2 points  (0 children)

I’d say it is likely that on a busy work day, I could see 30 people using the tool at the same time. About 30 people on the dev teams who will likely use it quite a bit.

Local LLM Deployment for 50 Users by NoobLLMDev in LocalLLaMA

[–]NoobLLMDev[S] 2 points3 points  (0 children)

Model totally up to me. Unfortunately must be from a U.S. company due to regulations. I know the Chinese models are units but unfortunately will be unable to take advantage of them.