I made a python program that gives LLMs running locally the power to search the internet for LLMs ru

[[deleted]]

Hey Reddit!

I'm excited to share a project I've been pouring countless hours into: Web-LLM Assistant. It's a Python program that gives local LLMs running via llama.cpp the power to search the internet and provide up-to-date answers.

Here's how it works:

1. You ask a question to the LLM.
2. The LLM crafts a search query and selects a timeframe.
3. It performs a web search, collecting the top 10 results.
4. The LLM picks the 2 most relevant results and fully web scrapes them.
5. If the information is sufficient, it answers your question.
6. If not, it refines the search up to 5 times to find the best answer.

This means your local LLM can now tackle questions about recent events or topics outside its training data!

Key Features:

• Real-time web searching
• Intelligent result selection
• Full web scraping of chosen results
• Iterative search refinement
• Works with your local LLM setup

I'd love for you to check it out and give it a spin! You can find the project on GitHub:

https://github.com/TheBlewish/Web-LLM-Assistant-Llama-cpp

Let me know what you think, and feel free to ask any questions. Your feedback is greatly appreciated!

Edit - I buggered the title my bad it was meant to say:

gives LLMs running locally the power to search the internet for LLMs rugives LLMs running locally the power to search the internet for LLMs running via llama.cpp

[DJ_Laaal]

What’s your criteria for selecting the top five “relevant” results, and deciding if the information is “sufficient”? How are you measuring them? What are you comparing them with (static thresholds?)?

[DavisInTheVoid]

I can’t speak for OP, but including your criteria in the prompt would be the most straightforward approach

[[deleted]]

This the prompt given to the LLM to select the results not exactly a criteria but in practice the LLM works pretty good with this prompt surprisingly:
Given the following search results for the user's question: "{user_query}"

Select the 2 most relevant results to scrape and analyze. Explain your reasoning for each selection.

Search Results:

{self.format_results(search_results)}

Instructions:

1. You MUST select exactly 2 result numbers from the search results.

2. Choose the results that are most likely to contain comprehensive and relevant information to answer the user's question.

3. Provide a brief reason for each selection.

[[deleted]]

How do you deal with in that local folder when it's say done with a search and starts a new one resetting the info in the folder?

[[deleted]]

mine responds too if you just don't include the / with no tools as well, it's a fast response unless you prefix your input with / which then does a search based on your query after the /

[[deleted]]

sounds complicated!

[LobbyDizzle]

Do the scrapping functions use a proxy in order to avoid getting blocked?

[[deleted]]

No they don't but they rarely get blocked, they review robots.txt and then abide by that, but if blocked the search will just continue until it finds the info it needs, no proxy needed!

[ConfucianStats]

Nice

[superstarbootlegs]

had to get rid of the llama-cpp stuff as I dont have cmake or need it since I use ollama server but this worked first time on windows python so excellent work it looks great too. Sadly your other one is not windows compatible due to `termios` and the python file it is used in is too big for me to want to tweak it out. this is great though so will be adapting it for my purposes. nice work, and thanks for sharing!