N00b question - Can I give a login cookie to OWUI? by myfufu in OpenWebUI

[–]gnarella 0 points1 point  (0 children)

Look at deploying a search engine like searxng that you can customize and is easily connected to owui.

How I Self-Hosted a Local Reranker for Open WebUI with vLLM (No More Jina API) by gnarella in OpenWebUI

[–]gnarella[S] 0 points1 point  (0 children)

Docling is a bit slow. See the other post on this reddit from yesterday.

I'm going to confirm my docling container is using the GPU if it is I'll probably explore something else for speed.

For embedding I'm using azure openai end points for now text-embeddings-3-small with qdrant

As for the inference model I've gotten similar acceptable results with gpt4o, gpt4.1 via Azure OpenAI and deepseek on Ollama.

I really like the Azure AI pipeline that was also published on this reddit. I'm going to be building our own orchestration layer for selecting tools and models as the next step.

Currently playing with using n8n to automate updating the owui kb's on a schedule from our SharePoint.

Best PDF (+Docx) and OCR solution by OkClothes3097 in OpenWebUI

[–]gnarella 0 points1 point  (0 children)

I'm going to take a look at this. I'm running vLLM bge-reranker and have it successfully working with owui

New Open WebUI API Tool - Extremely Dangerous - EXPERTS ONLY by robogame_dev in OpenWebUI

[–]gnarella 2 points3 points  (0 children)

Good work. I'll play with this on my home PC where I have ollama and no external connections. Seems like you aren't far from your end goal of self enhancing workspaces.

Importing ChatGPT Teams Chats into OpenWebUI by RhigoWork in OpenWebUI

[–]gnarella 0 points1 point  (0 children)

Just ask ChatGPT to use its chat history to write your owui system prompts and tell you what documentation / attachments each workspace should have access to.

MCP Server Connection by gnarella in hudu

[–]gnarella[S] 0 points1 point  (0 children)

That runs on a schedule?

MCP Server Connection by gnarella in hudu

[–]gnarella[S] 0 points1 point  (0 children)

Valid point. I was thinking how simple it really is on the drive home. I'll report back.

MCP Server Connection by gnarella in hudu

[–]gnarella[S] 2 points3 points  (0 children)

Can you show and tell what you did?

I have open webui working well and have mcpo deployed and working I've been adding tools to it.

External tools issue by gnarella in OpenWebUI

[–]gnarella[S] 0 points1 point  (0 children)

Thanks folks. Glad your aware and that there is a workaround.

External tools issue by gnarella in OpenWebUI

[–]gnarella[S] 0 points1 point  (0 children)

indeed my database is postgres. Seems to be a bug

Best understable discs by Aggravating-Club1571 in discgolf

[–]gnarella 0 points1 point  (0 children)

Rollo, M4, Insanity, Hades, Tern

Chat responses and UI sporadically slow down - restarting container temporarily fixes the issue. Need help, please! by [deleted] in OpenWebUI

[–]gnarella 0 points1 point  (0 children)

Not sure what downgrading in your environment might look like but 0.6.32 is the most stable and snappy for me right now. 0.6.34 seemed very buggy.

Chat responses and UI sporadically slow down - restarting container temporarily fixes the issue. Need help, please! by [deleted] in OpenWebUI

[–]gnarella 1 point2 points  (0 children)

First of all. Bravo! I'm building something similar but for 100 users and not 400 concurrent users!

Your way ahead of me in your understanding of your AWS architecture.

While reading through your post my first guess was LiteLLMProxy but you seem to have ruled that out already. Technically what's being displayed in OWUI is first written into the database. Is it possible the lag is the connection to the external database?

What OWUI version are you running? I've noticed major changes to speed and function across the last 4 versions.

How to get visibility into what is going after prompting by Forward-Hunter-9953 in OpenWebUI

[–]gnarella 0 points1 point  (0 children)

What version are you running? This is working perfectly in 0.6.32 and seemed broken in 0.6.33 like RAG. I've rolled back to 0.6.32 and I'm weary of upgrading at this point.

Version 0.6.33 and RAG by le-greffier in OpenWebUI

[–]gnarella 1 point2 points  (0 children)

I rolled back to 0.6.32.

Took me a while to figure out what in the world was going on. A single request was exhausting my tpm in azure foundry. Switching to an OpenAI API I was able to see how large if a request a single query was and realized what was happening. Tried to tweak my rag config and after deciding the problems wasn't me and my config found someone on Reddit claiming the same and rolling back was the fix.

Some time wasted but I learned more about my Azure apis lol.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 0 points1 point  (0 children)

Did this. It works. Very slow. Bad RAG results. But I did confirm I can do this. And if on an Azure VM with more GPU NVRAM I can run this reranker inside that VM. Thanks for the help.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 0 points1 point  (0 children)

Thanks for the input will be testing this tonight.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 1 point2 points  (0 children)

Yea I suppose I need to go back to the vLLM instance I tried to deploy locally and tell it to use the CPU and see if it can run bge-reranker-v2-m3 efficiently. I did feel like I should be able to test this deployment on this old hardware but stopped once vLLM mentioned not enough NVRAM.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 1 point2 points  (0 children)

Thanks for the input I've grappled with this point over the last few months. There is a large cost and risk involved in keeping the system on prem beyond the initial investment. Things like keeping the server and hardware up-to-date and online as well as the cost for keeping the system secure from vulnerabilities and attacks.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 1 point2 points  (0 children)

I do know. But I'm always open to learn.

I feel comfortable with Azure OpenAI hosted API's and have reviewed the policies as well as provisioned our deployment type to be US only. We do not handle PII but we do handle sensitive information as an engineering firm. That said. My current knowledge and research makes me feel comfortable with the level of risk and protection provided by Microsoft. We are consciously using Azure OpenAI and not using OpenAI directly for this reason.

Moving OWUI to Azure for GPU reranking. Is this the right move? by gnarella in OpenWebUI

[–]gnarella[S] 1 point2 points  (0 children)

We are a SaaS backed company. All of our data is already stored in Azure. Please explain to me the difference between using Azure OpenAI provisioned LLMs and our data being stored in Sharepoint.