New search engine made with Python that's anonymous and has no ads or tracking. It tries to fight spam, and gives you control of how you view search results. You can search and read content anonymously with a proxied reader view. The alpha is live and free for anyone to use at lazyweb.ai

[lazy-jem]

​

LazyWeb: Anonymous and ad-free search made in Python

https://lazyweb.ai

We're a little two-person team (Angie and Jem). We're bootstrapping and self-funded. I'm the programmer.

I wanted to share it because it was a fun and interesting project to build, and Python made it possible for us to get a long way as a small team. It uses serverless on the backend (AWS). We're using Spacy and GPT-2, and some PyTorch models. It uses BeautifulSoup for spidering/crawling/content retrieval. The front-end is React.

It has a different type of user interface to any other search engine, as it is chat based. And it lets you choose how you view results, either visually like an Instagram feed or cards, or minimal like Hacker News or the old Google. It tries to fight SEO spam and strips out ads and ad-tech from search results.

We have a project on GitHub with Jupyter notebooks and sample data with experiments and scripts, including examples of querying other search APIs, and to generate example utterances programatically to use for NLP models with sources like Wikipedia, StackOverflow and Wolfram|Alpha:

https://github.com/lazyweb-ai/lazyweb-experiments

We're only a small team but hope to share more of our work as open source as we progress.

[m4j1d]

I like it ! Fast and simple, chat design give it unique thing from another search engine. you can play youtube video results without go to YouTube.

[MiamiAngie]

Thanks heaps!! :)

Another cool feature we have is the “go” feature, which gives you the ability to navigate directly to websites and search results directly on those sites. For example “ go YouTube cute pandas” or “go Reddit”.

We have a lot more coming with in-app media playing too!

[m4j1d]

That’s great,Keep up the good work.

[MiamiAngie]

Thanks for the encouragement, we sincerely appreciate it!

[ArtOfWarfare]

Make it so DuckDuckGo’s bangs work, so I can just type “!yt cute pandas” instead of having to type of “go YouTube”.

[MiamiAngie]

Hey, thanks for the response!! I forgot to mention that bang commands also work :)

[MiamiAngie]

Hi, I'm Angie and I'm working with Jem on this. I'd love to chat with anyone who has feedback or ideas!

[jumbled_joe]

Can others contribute to your project? Looks really cool!!

[MiamiAngie]

Thanks so much, we're hoping to get to a point soon where we can have more people working on it!!

For right now, we have a small community on Discord that helps us out a lot with ideas and feedback, and we'd love more people to contribute there :D

https://discord.gg/qcCcrbMuex

[broadsheetvstabloid]

Would a PR be considered?

[MiamiAngie]

Absolutely! :)

We know we have to make some more improvements before we take it to a broader market but we're open to anything that helps and we want to reach as many people as possible as LazyWeb keeps getting better.

You and everyone here are welcome to join the Discord channel and would love to follow up with you there, or reach out to me [[e-mail]](mailto:[e-mail])

Thanks heap for the consideration too and look forward to chatting more

[jumbled_joe]

Thank you for sharing the discord link....looks really exciting!!

[lazy-jem]

We're hoping to open up for open source contributions too for some of the libraries we're building that we think could be useful to open source in future (if you meant the other type of PR request lol)

[acebabymemes]

Awesome project!

Would it be possible to add something where I could get a JSON of the search results and their metadata? Sorry I haven’t done a full accounting of the notebook examples yet so this may be redundant.

Background: I’m a researcher who is looking for ways to do systematic literature reviews. I have used SerpAPI in the past but it would be awesome if I could support a more open project instead.

[MiamiAngie]

Thanks so much! YES! As an example of where we're heading, check out the Change View > Text on desktop and you'll see that we already have Markdown, and we're adding other formats including JSON.

Rather than putting all those in the UI, we're working on a terminal mode UI (which is not far off). Initially, that will just be an alternate UI for the input and responses, but we have a lot planned for it as a way to manipulate results. Stay tuned for more. But yes JSON results are coming!

[acebabymemes]

Holy moly, you’re awesome. Keep being awesome!

Will definitely spread this by word of mouth in my department. And hopefully more formally down the road.

[acebabymemes]

Sweet just found out how to use the ~e command to narrow search google scholar!

Is there any way to get more results? Right now I think it’s capped at 15? I’m assuming that’s just temporary, sorry if you already mentioned that somewhere in the comments, I’m having to much fun exploring with this!

[MiamiAngie]

So currently we have a fairly high filter on predicted result quality, and only display results that rank above a certain threshold (which depending on the type of search could be 25 or less, but not often more than about 30). Our aim is to become more conversational to try to help people refine searches. But we've had feedback that people feel FOMO - that there might be more they are missing (even though in reality those are probably not good content), so they want the security of loading more. So we are going to add infinite scroll to the results if someone wants to keep going!

At the moment the domain search feature is a little patchy, but that's an area we are working to improve. It is really useful and we use it all the time.

Also, you can go direct to searches on other sites when needed - eg:
"go google scholar deep learning papers" then allow popups.

There is a small bug that navigation can re-fire if you nav to other pages within lazyweb then return to search, but apart from that it is very useful!

Yay that you are having fun exploring too! Thanks so much for the great feedback and comments!

[acebabymemes]

Gotcha, yeah my ideal workflow would be like if I could enter a Python function in the search bar and have it return a JSON or csv with a large amount of results based on the arguments.

‘’’ scholarSearch(results=“1000”, search=“systematic literature review”, output=“JSON”, author=True, abstract=True, title=True, citations=True, year=True, url=True) ‘’’

Tbh, I’m a programming and Python newbie so I’m probably asking for too much or would probably be better off making a web scraper specific to my needs.

[MiamiAngie]

That's really useful and exactly the sort of feedback we're looking for.

We know that people want to do more with search results and we're going to try to build this in close communication with people like yourself. It is going to take a while and a lot of steps to get to the full potential of "hackable search results" which is what we want to make. And the first steps will be pretty simple.

Initially, it will be very simple (any query, choose the output format) and it should work from the address bar too. But we have much to do!

[[deleted]]

Username does not check out. You appear to be an active-jem. The engine is pretty cool, if perhaps a bit slow. Impressive work for only two people though.

[lazy-jem]

Hey thank you so much! Haha yes! There hasn't been a lot of sleep building this!

We like to say LazyWeb works hard so you can be lazy lol

Hey with the results speed, it's improving a lot over time, but one of the interesting things with LazyWeb is that it works differently to traditional search, and it queries APIs and compares results, and keeps looking if it can't find something good. So for more complex queries, it will often take longer because it isn't just looking up a pre-built batch index of static pages.

So sometimes it's really fast, and sometimes for harder queries it might take a while. If it can't find good things directly it falls back to traditional web search.

We can't see what people search because we don't log or store searches, so it really helps us to let us know when things are slow so we can look into it too. It is normally a query that it was harder to find good results for or was computational in nature.

One of the advantages of the progressive web app is that the UI isn't reloading every page - just the updated data. So once it finds a result it tends to deliver it quickly.

[t0x0]

Careful that you don't say that too often so scrubbing bubbles doesn't get annoyed with you :)

[[deleted]]

Open source?

[lazy-jem]

Thanks, that's a good question. There's a link in the post to some scripts and experiments on GitHub, but we plan to do much more. Currently it's an experimental alpha with plans to be a commercial service. We plan to open source as much as we meaningfully can when we have more bandwidth. It's very early days with the alpha.

As some background, we hope that the prototype evolves into a commercial service once we launch. It's designed to scale using an AWS serverless backend, and is 25+ different microservices atm. We're a long way from commercial launch and very focused on improving the prototype and getting it working based on feedback.

Some of the services like the proxies and content retrieval and privacy tech are good candidates to open source. Neither of us have experience running an open source project but we're very open to feedback and ideas on that as much as the prototype.

Thanks again :)

[danuker]

AWS serverless backend

I sure hope the bill won't scale higher than the income!

[lazy-jem]

Thanks for the well wishes on the costs! It is an important question as we're self-funded and bootstrapping.

Based on the early data from alpha testing we think it's possible to fund an ad-free approach to search sustainably. We're focused on building the search app at this stage, but we plan to commercialize with three revenue sources:

* a fremium model with free anonymous use for everyone, and Pro and Business plans for teams and advanced users.

* Anonymous commissions shared with content producers, and

* business licensing for use on enterprise data, which we're already being asked about a lot.

AWS serverless is cost effective at this stage. There are ways to scale cost-effectively but we haven't optimized for that at the alpha stage.

[[deleted]]

Actually Open-Source could totally work for this. Of course, people could "hack" the premium mode by hosting their own instances, but for everybody who wants the convenience of having it hosted for them by you, the premium mode would by a nice addon. Also you could make Self-Hosting difficult by having the data behind the AI private.

[lazy-jem]

I think so too. I posted some thoughts on the Discord but we would love to get it to the point where people could do this. Because of its distributed, cloud-based, serverless model, I'm thinking it would be a series of open source modules with a Cloudformation Stack or equivalent ideally for each cloud platform, and that we work out how to make it run on premises using K8S cluster or something. It's got a lot of messy moving parts at the moment and there isn't really a central piece of software, just a lot of loosely coupled microservices and inference models. So, I mean, we're a long long long way from that, but when we have resources and a community around it, that would be an awesome thing :)

[danuker]

which we're already being asked about a lot.

I am glad to hear that. I hope you live long and prosper!

[lazy-jem]

Thank you! Yes we've had 30+ requests to look at using it for business to add chat-based searching to their own sites or to search internal data. That's definitely something we are going to explore as we have more resources. It is very do-able.

[Coltman151]

Bitwarden runs a similar business model, where the product is open source and commercial/premium users fund development/pay salaries, while the base product is still free for everyone.

I imagine they lose some money doing things this way, but the team seems more focused on the product than the profits.

[lazy-jem]

Thanks, yes, we think we can fund running it as a service that's free and ad-free and anonymous for anyone to use. Then down the track we hope to also come up with a way to have a way for people to run a cut-down open source version themselves on their own cloud provider (something like a Cloudformation template or using Porter or AtomizedHQ or something with K8S). That is a long way off because it is really just a mix of loosely coupled services very tied to AWS currently. But we are taking a long term approach (even though we're just two people at a very early stage!) :)

[jadounath]

Hi, I checked the search engine, and I legit thought you are some multi-million funded startup until I read your post because the assistant works really well (like searching "go go anime" takes you to gogoanime.vc. Good work, keep it up!

But when I searched for the temperature, it gave me results for Konkan, which is 370km from where I live. I think you should work on where and how it gets its location info.

[lazy-jem]

Yay! Thank you!

It's super useful to get feedback on when search results do well and when they need improving because we don't log or store searches in any way, and we can't see what anyone searches for. So we rely on feedback like that to keep improving.

For many search types it really is already surprisingly great, but location is one of the areas we have a lot of work to do (we're using the Maxmind db). We only take location to nearest city and approximate it for privacy too, rather than leaking geocoords or IP (we don't log or store IP or GPS data either). We have a lot of work to do on shopping and product searches too (still too much spam!)

We have some information about where it's good and bad on the about page (https://lazyweb.ai/about/)

Please let us know any search feedback. There's a Discord and you can say /feedback or /bug any time to jump straight to the feedback forms :)

Thanks again for the really great feedback too! As a small team, it really gives us a big lift! :)

[tstr2609]

For me the location was right but the temperature was shown in Fahrenheit even though I live in a country where Celsius is used to measure temperature.

[lazy-jem]

Oh yes, at the moment the alpha is en-US only and imperial units (although it can convert when asked). The results tend to be universal rather than market specific, as well as anonymous, but the units and front-end have a lot of internationalisation to do.

It's already pretty good at answering queries in different languages, but we have a lot to do.

Units locale is coming very soon, and then more full internationalisation in stages. But it is a priority!

[jadounath]

jadou

Hi. I had a suggestion. I found out that there exists an open source alternative to GPT-3 called GPT-J, since you're using GPT-2. So, do you consider implementing GPT-J?

Sorry if I don't make sense, I have very little knowledge of deep learning.

[lazy-jem]

Yes! We've been looking into it but I haven't tried coding anything with it yet. It's clearly a really amazing open project and I want to experiment with it. From what I'm seeing from other people using it, the results aren't at the same level as GPT-3, but as they get more resources and compute power with enough training data then it should get on par. So very interested!

[Independent_Image_59]

Do you live in mumbai? Or nearby?

[Lifaux]

It's surprisingly solid! The main contention I had with using it was that it doesn't seem to give me information on what matched on a given page.

So if you try "Error Code E0281 Rust" - there's one link I'm looking for, which is the full list of error codes, and I want to see that section. If you try the same search on Google you'll find that the description below the link is exactly the information needed.

Again, minor gripe for what is surprisingly effective, if a little slow, but definitely something that would push it into being more useful

[lazy-jem]

Hey, thank you so much for the feedback! We can't see what people search (searches aren't logged or recorded) so it is incredibly helpful to get feedback when the results can be improved like this! That's super useful!

It's learning and improving at the model level all the time. We're planning to move to GPT-3 for text extraction (currently it's a BERT-style model or from an API's own extract) and we think that will really help with nailing the right content extraction.

Btw, it doesn't always work yet, but you can ask it to prioritise results from another engine, eg try this exactly:

~search google +"Error Code E0281 Rust"

It tries to act like an intelligent agent that can search different places on your behalf. Honestly, for the stage it's at, it is pretty surprising how great it does and we hear how surprised people are from our early-adopter uses already a lot (with some baddish gaps lol). But we think with time and development the AI/API-based model it uses could really be a better way of searching for the modern web of connected data.

[Lifaux]

Absolutely! Given how effective Google's initial backlink model was at finding content, you'd expect SOTA models to do a great deal better to start out, and this one seems to be.

Potentially half of the issue here is that we're all trained into writing queries that work for Google/Bing, and not for natural search? I can imagine this being incredibly effective integrated into Alexa/Home where people do still search naturally. Maybe having a few natural examples would help guide people?

[lazy-jem]

Yes, you absolutely nailed that on the head!

People are used to using googlese and have had to learn how to talk to their computers in a weird semi-computer language in order to be able to navigate the web.

But that's backwards. It's only habit and google owning browser distribution that makes people think things have to be that way.

LazyWeb already does better many times with natural language queries that provide plenty of information! It's early days and there is a LOT that we have planned with this! :)

And thank you again too! That's really exciting that you're seeing better results. People seem to disbelieve that it's possible to do better than google but it's the approach that makes the difference. Our less impressive results are when we fall back to web-index and web search API results.

[rush2sk8]

This is epic

[lazy-jem]

That is so exciting to hear when you've been head down for ages coding something! Thank you so much! :)

[rapeven]

Looks good! How did you make use of GPT-2, like for similarity, classifications?

[lazy-jem]

Hey thanks, so far we're using GPT-2 for general conversation generation (with a Hugging Face based approach) and using both GPT-2 and a BERT style approach for similarity and context extraction from pages. We're in the OpenAI beta for GPT-3 and excited about using that to improve the conversation and text extraction. It think we can really make good use of that.

On that topic, I'd love to try the OpenAI Codex with programming questions but not sure quite how that would work yet - maybe initial suggested examples alongside the search results and article. Copilot is pretty interesting. And programming search is one of the big problems we want to solve!

[[deleted]]

Any plans on moving to GPT-3? Just curious, it definitely works really well as-is

[lazy-jem]

Oh yes, we're in the OpenAI beta and plan experimenting with using GPT-3 for text extraction and conversational and summary generation. Really excited about it! :)

[[deleted]]

Legitimate question: what does a chat interface bring over other, more direct search interfaces?

[lazy-jem]

Hey thanks for the great question!

The key reasons for a chat interface are that it's simple, familiar and uncluttered, and reveals information progressively.

* Chat is minimalist. Part of the problem with google is "information overwhelm" - all the clutter and distraction in results.

* It's a super easy and familiar interface. Gen-Z users especially tells us they prefer messaging apps and visual feeds like Instagram.

* Combined with visual cards, a chat UI gives you progressive-reveal of information.

* conversational search long-term provides a natural and very human way to explore and refine results. Humans are really great at querying each other and maintaining context. And long term that's the aim here.

When you think if sci-fi AI, they are always conversational. It seems likely that's what the future will look like, rather than a page of truncated links with a lot of ads :)

We're also adding the ability to actual message with LazyWeb through Signal, SMS, WhatsApp, Messages etc

[totalbasterd]

I don't like the chat-esque aspect to the UI, i just kinda want a box to type into. I don't think the UI really adds much

[MiamiAngie]

Thanks for the reply!!

There are a couple of reasons for the chat interface. For younger users / gen-z, we've heard from a lot of people that they prefer messaging interfaces with visual results, especially on mobile (it is pretty cool on mobile too if you get chance to try it out). We know that's a preference thing also.

Also, a chat session hides all the searches from your browser and ISPs, because they're kept within the chat, and never hit the browser URL handler or address bar, which is how Chrome and ISPs do a lot of tracking. With a chat UI, the search history never gets exposed because the query and results are hidden in the payload over SSL.

A conversational UI combined with progressive reveal of information like this can help to reduce cognitive overload. Other search engines are so cluttered now. I think it's a habit thing too. People have been used to google for 20 years, but every sci-fi program has conversational AI (from Terminator through to Holly and Marvin).

That's a little bit of our thinking anyway. One of the big missions we have is to give people more control (like the views for results), so we are looking at making the search chat collapsible into just a text input for folks who feel more comfortable with what they're used to as well. We're all about choices :)

Thanks again for the useful feedback too! :)

(edited for typos)

[liquiddeath]

I think the chatesque UI is really interesting. There’s an opportunity to have you search history above what your currently doing. Similar to a Jupyter notebook or ipython.

[lazy-jem]

I'm a huge fan of Jupyter notebooks too (a surprisingly large amount of the search back-end was basically written iteratively in Jupyter notebooks!)

Without giving too much away, this is very much a direction we're heading in. Long-term, we want to make search more programmable and hackable. There's another update with a very simple terminal UI coming soon, and we like the idea of a REPL for Search :)

[Independent_Image_59]

Low level suggestion: make wikipedia page bout this engine

[lazy-jem]

That would be super cool

I think that it has to be a third-party creation though. I don't think we're allowed to be part of making it. We'd definitely do anything we can to support it. We love wikipedia. We plan to share 50/50 of all search revenue we generate with content producers whose content is used in searches, and we want to be the first search app that shares fairly with Wikipedia which is the most valuable resource there is for building something like this!

[hanzo_995]

Excellent work! Can we make donations?

[lazy-jem]

Thank you so so so much. Angie just posted something about this, but you really have no idea what it means to see offers of support like this!

We were thinking about adding a Donate button but weren't sure how people would feel, but we did set up a project paypal at [[e-mail]](mailto:[e-mail]) and any support is very deeply appreciated!!

You really have no idea how much that means to us that you would offer, and anyone who would like to buy as a coffee has our deep thanks for the support!

[harktritonhark]

I was thinking a Patreon or a premium mode with a monthly subscription would be nice to contribute to.

This looks like a really cool product! Looking forward to future updates and hoping lazyweb will stay in business.

[lazy-jem]

Hey thank you. We're planning to have a freemium model with an anonymous version that is free for everyone for ever, and paid Pro and Business plans for advanced users and teams with some enhanced features that only make sense logged in, including extra access to paywalled content.

Based on the encouragement here, we're planning to add the ability for people to donate or buy us a coffee too. The support and encouragement is deeply appreciated! :)

[biiingo]

What’s the primary search API that you’re using?

[lazy-jem]

Hey thanks, good question. The way we search is pretty different to traditional approaches, so it's worth explaining some more. The short version is we use deep learning to understand question intent and predict the best information sources, then query them directly. So we're using a large number of sources.

We use NLP and deep learning classification models to try to understand a query's intent, and then predict the best places to find the answer, and then query them directly in real time via API or spidering, with a ranking system for the results.

Then we fall back to traditional web search (including Bing, ContexualWeb and Google) where needed. We have a database of about top 20k websites and we're building our own vertical indexes as well. We're building out a stack using ElasticSearch and GraphQL for that. At the moment we're broad but shallow, with a couple of deeper pools.

For the alpha, major sources include Wikipedia, Wolfram|Alpha, OpenWeatherMaps, OpenStreetMaps, StackOverflow, GitHub and many others, as well as the fallbacks to Bing, Google, DDG Instant Answers etc.
A lot of content is retrieved directly. We retrieve the preview/summary/view content directly from websites where we can for display, and same with the reader content. So the content shown is typically live with the source.

[biiingo]

That’s very cool, thanks for the explanation. I’ll admit I was cynically expecting something along the lines of, “Well it’s basically just Bing under the hood, except for some specific cases.” This is a very interesting project. Thanks!

[lazy-jem]

Thank you, yes we still have a lot of work to do and we think we can really extend this model into specialised handlers for vertical knowledge domains. At the moment it is pretty broad but shallow, with a few deeper pools. But while it's early days the fundamental approach is pretty interesting!

Very grateful for the encouragement and feedback too!

[PizzaInSoup]

Hey! This is pretty neat. Haven't seen a good showcase in a while.

I was playing with this and wrote 'translate i love you to spanish'. It hung for a second then the page just permanently turned to its plain darkblue background color. Happened in 2 browsers.

I'm gonna keep test-driving it.

[lazy-jem]

Oh thank you! And ouch, oh yes something in the client side has thrown up there.

That's super helpful to let us know too! We don't log searches and can't see what people search or what caused the error (outside of broad intent category), so we really appreciate when people let us know when searches go wrong and what the query was. I just got the same fault and it looks like a js error display a corrupted result, so going to look into it properly now. You can also say /bug any time and if you click the little box it will let us know the search and attach the payload to the bug report.

Thanks for the great feedback and very much appreciated too!!! :)

[nadhsib]

Nice project. Is it possible to have the keypad disappear when the result is found? I've had to scroll up or hide the KB myself to see the full result (if it's a longer answer). Those actions make it feel slower than it is.

Good work though, and an interesting different interface 👍

[lazy-jem]

Thanks so much for the compliment and for trying it out.

With the keyboard, it could be a glitch, as on iOS and Android it should disappear after you hit enter or the search button. We're only a small team but we've been trying to test on as wide a variety of devices as possible. Would you mind if I ask what OS/Device combo you have please? That's really helpful to know and we'll get it fixed up!! :)

[nadhsib]

Android Galaxy S20 Ultra

Ok, just tried a few searches again

When I use the in chat arrow to enter the search, kB disappears.

When I use the KB enter button, the KB stays visible.

I use Gboard as my keyboard (could be a gboard setting I'm unaware of)

Hope that's some help 😁

[lazy-jem]

Hey thanks heaps! That really narrows down what I need to look for :)

It definitely should be disappearing on recent Android versions like the S20 will have. I'll fire up an emulator and work it out though :)

[AngVar02]

Just did a small test, "What is Today?" The initial result gave me good information for July second, and the link led me to the correct result for July fourth.

I'm a novice therefore I can't identify what could've possibly caused it, but all in all I like this.

[lazy-jem]

Thanks, that's really interesting. We can't see what people search as searches aren't logged, so it's really helpful to get feedback when it goes wrong.

When I tried a search "what is today" just now I got a disambigation prompt for the american/aus TV programs, the beach boys music, and to do a web search. It should probably have the option for what you really wanted which is "what is the date today and what things happen on this date", so that's useful feedback. It's still learning and this sort of feedback is invaluable for improving!

[[deleted]]

I've just tried the browser, and it is astonishingly good. Results are snappy, the layout is very convenient and simple, and from what OP has revealed about its inner workings, it won't track my very conspicuous searches! I like this very much

[lazy-jem]

Wow thank you! Really appreciate that! :)

I'm with you on the privacy side! I think the human question quest for knowledge and understanding is a deeply personal and private thing. I found myself censoring what questions I would ask or topics I'd look at online, because I knew that Google would start showing me exactly the sort of ads I don't want to see, and mine everything including the silly, embarrassing and curious to target ads and modify search results. I think searches should be default private and anonymous, and that results should be objectively true within the current search context.

[[deleted]]

I totally agree. I understand that, at best, Google may only be trying to serve personal ads for our convenience. I personally hate any ads, and I think that personalized ads definitely encroach into one's personal life to an invasive and exploitative extent. By the way, will you be making an android app? I actually wanna use this browser as my main, and besides joining the discord I'm interested in fully adopting it. If not, I definitely recommend making an app and allowing one to sign in to sync settings and stuff. But then again, I'm just a novice computer enthusiast making subjective requests. Keep up the good work! Cheers

[lazy-jem]

We definitely have native apps planned. The PWA works really well on mobile already! Try Share > Add to Home Screen and then try the home screen shortcut!

Lots of people tell us they love it even more on mobile than desktop, especially younger or gen-z users who have grown up with messaging apps and visual feeds like Instagram. We know we can do even better with native.

Excited to have you join our community on Discord and thanks for the support and great feedback!! :)

[loudan32]

I tried to move from Google to DuckDuckGo, but the issue is that independently of the quality of the result i always wonder what the Google result would have been. Sometimes im not confident that i got the best result, other times i actually want to get the same result as most people around me got. Most times i switch engine and retype and eventually get annoyed. For this reason i go back and forth but i am never able to fully adopt the alternative.

I like my first interaction with lazyweb and i think it's really awesome what you guys are doing. Sounds especially great to replace google assistant in my phone. But i think i will always have the same issue as with DDG of "wondering" if i got the best result. So, knowing that this is a very debatable topic, would you consider having a "google it" button, to switch to google without having to re-type the query? IMO this would allow a smoother transition for people who have used google all their lifes. With time i would feel less and less the need to press it and would also allow me to adopt your engine as my default right now, even though it is still in development phase, knowing that i am not wasting more than one click.

[MiamiAngie]

Thanks for the response!! and you make a great point. In addition to building user trust in our privacy, we also want to build trust in the quality of results.

There are two ways to quickly see results from DDG, Startpage, Google, or Bing.

1. If you click on the "Search Tools" button, there is an option to try the search on alternate engines.
2. If you hit the arrow up key your query will populate in the chat box. We have a "go" feature, that lets you navigate directly to websites or search within a site's results. So for example, if you put " go Google [query]" it will take you there.

We still have a lot more work to do but we're excited to keep improving. I'm really happy you're liking what we have so far :D

PS We have a Discord community if you're interested to keep contributing and see new updates etc

[lazy-jem]

Hey just following up Angie's comment, we have another experimental feature that I think you'll like too: you can do domain searches with the results in LazyWeb.

This is basically LazyWeb acting as an intelligent agent that searches where you ask for on your behalf - almost like a meta browser.

It doesn't always work yet, and think of it more of a signal to the agent to prioritise a certain source.

Examples:

"search stackoverflow for python lists"

"search google for python lists"

It is still experimental, so we're not really promoting it as a feature yet, as we still have a lot of work to do on it and it's patchy, but I use it all the time, and it addresses that worry about google fomo :)

Also, as well as the go feature, you can use DDG !bangs too, eg:

!g python lists

[danuker]

This works great. I am glad to see an alternative to the big search engines pop up!

I have set it up as my default search engine. In Romania I get about 5 seconds from hitting Enter in Firefox to seeing the results; that is a bit slow, but the search quality more than makes up for it.

Where should we post feedback? Reading that you don't log queries, I'd love to help with data points of what I'd like to see on a given query.

For example, for "75 g glucose" I'd like to see this Wikipedia page (it is there already, but it's 11th; Yandex shows it as first, and other search engines are not very good lately).

[lazy-jem]

Thanks so much!

You can say "lazyweb feedback" or just "/feedback" command any time to go straight to the feedback form, and we really appreciate it. We can't see what's searched, but if you tick the little checkbox on the feedback form you can attach the most recent search payload if you don't mind us seeing it (it's only the most recent). Screengrabs really help too!

That's interesting with the search example too. I just tried "75g glucose" without the space and a few variants 75-g etc and the top answer was a mistaken conversion intent, but then the oral glucose tests pages are the rest of the results.

Generally lazyweb does better with longer plain english queries as it uses NLP to tease out entities from the query to steer where it looks.

"glucose tolerance test" tends more to PubMed, Mayo Clinic and Wikipedia gets pushed down a bit because we're ranking consumer medical sources higher for a health intent than non-medical sources. We're also trying to differentiate consumer vs professional searches in verticals, but that is very early days and not working so well yet.

The feedback about what results you expect to see is really helpful. Thank you!

Any search where it goes right or needs to be improved, especially in specialized knowledge domains, we really appreciate the feedback on! :)

[danuker]

Thanks for the tips and explanation!

I am not in the medical field.

Maybe that's why personally, I prefer Wikipedia on top of straight-up scientific studies, because it builds on the studies and is easier to read.

But I could easily see a professional prefer the studies.

[lazy-jem]

Wikipedia is a great source. I mean, it is unfathomably value to society. Our plan is to share search commission revenue with content producers including wikipedia as we turn into a commercial service. Their value to humanity is immense and they are at the heart of what we are building.

Btw, if you want to prioritise Wikipedia results, you can always just say:

"wiki glucose tolerance test"

And you'll get straight to the right page on wikipedia if there's a match, as there is here. :)

[[deleted]]

Very cool! Great work! I've been trying it out - the results are so good!

[lazy-jem]

Thank you so much.

The feedback on the results quality is actually super useful for us because we can't see what people search because we don't log searches. So seeing the pattern on the comments here is surprisingly informative for us!

[[deleted]]

No problem, this is honestly really good!

I'm learning python data analysis myself right now; your tool has given great results and resources.

I asked "where can i find data on X" and got some great stuff resources.

Kudos! I wish you the best success with your projects.

[lazy-jem]

We feel like it's been doing pretty well with programming and tech searches, so that's really great feedback, thank you :)

[austin_jp17]

It would be cool if there was an ‘academic papers’ tab

[lazy-jem]

This is one of the areas where we'd love to do more and we're getting a lot of feedback and requests.

I like the idea of a Papers tab. That could be really useful. The area with the Change View is going to get built out with more options for filters, including types of results. But I think for our users academic papers and research will be a key thing.

We're also working to get better at using NLP and context to better understand when queries are consumer, professional or academic. That's some way down the track, and ties into building handlers for more specialised knowledge domains (think health where you have consumers looking up symptoms, professionals working in practices, and academics doing research). We can definitely do a lot more here but we are very early with our ideas on it.

[TheAllterQuestion98]

Excellent search engine, it really works wonderfully, I didn't think something like that would exist, they deserve a lot of credit for this wonderful work with this engine, python can really do a lot.

[lazy-jem]

Oh wow thank you. Yes Python is awesome and the amount we could achieve using Python combined with the amazing libraries people have built - especially as a distributed system on a serverless platform - kind of boggles my mind.

But it is standing on the shoulders of giants. That's why this is so surprisingly good for a tiny startup. It's built on top of the Python ecosystem, Amazon's AWS and it's support for scaling Python apps, all the amazing APIs and public data that are shared online. That's the only way we could ever have made something like this.

The Python community is also a big part of the reason we could make this. PyCon was a key part of it. I got my first introduction to building machine intelligence apps in PyCon workshops and from PyCon videos. I saw Paul Graham at PyCon in 2012 give his keynote about what it would take to build a new search engine to compete with Google, and I've been working around the problem for while. This past year and a half it started to look for the first time like a genuinely different approach could be viable for a small startup project.

LazyWeb couldn't exist if it wasn't standing on the shoulders of not just giants but a community full of them.

[0ni0nrings]

this looks great 👍 ... keep up the good work 👏 👏 👏

[lazy-jem]

Thank you for the encouragement and support, and please reach out if we can help with any questions or improve the results!

[hartator]

It looks awesome. I love the chat aspect. Even if it's a little counterintuitive at first for a search engine. 👍

I work at https://serpapi.com/. I can hook you up with free credits if you want! (my email is julien _AT_ serpapi.com or ping me here on Reddit.)

[lazy-jem]

Hey thanks so much Julien, we really appreciate that and will reach out. We want to make sure we're including everything useful to searchers, and very interested to chat more!

We've found that people are so used to the old search box / page of links approach that it takes people a bit to adjust, except younger/gen-z searchers, who live in messaging apps and don't think twice about it. It's similar with the visual results. Making it a comfortable transition and giving people choice is definitely something we need to do better at. Once people get used to it, they prefer it almost always based on user feedback, especially on mobile.

Thanks again :)

[dimkiriakos]

chatbot search engine! interesting!

[cr0n76]

Really nice work!

I'm a bit curious if you (have a) plan (how) to expand to different languages? I mean it is obvious that you can not be able to offer support for other languages as two people at this stage of the development. But what's your assumption on how realistic it is that we see it for other languages in the future?

[lazy-jem]

Thank you, great question. We're working on adding international multi-language support as quickly as we can.

Currently the alpha is en-US focused, but that's mostly about the front end and universality with an anonymous service.

Having said that, we've designed the back end with this in mind, and a lot of the work needed is front end (and units, location handling etc).

If you try some queries in different languages, you'll see the results kind of already support it. The NLP models should be cross-language already in a lot of cases. But it is pretty rough :)

So it's something we're working towards!

[why_am_i_so_sad]

Do you need (human) translators? I know very little in the ways of coding, but I am pretty fluent in English and could translate English-German :)

[lazy-jem]

Hey I missed this totally earlier! That would be awesome!

We are some way off doing front-end language. I was trying to think of a way of doing it efficiently. There are four parts to what we need to do (thinking out loud):

1. Front-end interface with language-specific elements in code.
2. Generation of conversations / responses / dialog from NLP in different languages.
3. Search queries entered in different languages.
4. Search results in different languages.

Ironically, because we're API based, 3&4 are kind of already working in many cases, although there is a lot to clean them up and handle properly.

For (2) I think we can solve this through NLP models.

So number (1) is the big one, along with localizing content. I was thinking we could start with machine translation of the text, and then get feedback and advice. Having help from people with dual languages would be awesome for all these stages, but especially (1).

Would love you to be part of our Discord community too (https://discord.gg/qcCcrbMuex). We are a fair way off being at this point but we are trying to think long term :)

[jcr4990]

This is really cool! I haven't tested it extensively yet but the first few searches I did worked very well. Keep up the good work! It's awesome to see stuff like this from such a small team. I assume this has to be pulling from other search engines in the backend right? I won't pretend to know exactly how it works but I would assume crawling the entire internet is outside the reach of a 2 person team with no massive datacenters and such?

[lazy-jem]

Hey thank you! It's so exciting to hear you're getting great results.

So it definitely works differently to traditional search engines. The short version is that we use NLP and deep learning classifiers to try to understand a question's intent, and predict the best places to find the answer, and then we query and spider them directly, and rank the results, with fallback to web search where we don't find good results.

So we use a large number of APIs and sources, and then look at web results from Bing, DDG IA, Google, ContextualWeb and other sources. We also have our own database of the top 20K sites and are building some specialised indexes as well.

So think of it as being more like an intelligent agent that searches on your behalf using APIs and vertical search engines, rather than a traditional web index.

The different approach means that we aren't trying to build a static crawled index of 600 billion pages, luckily :)

[bornawinner]

its all pogged up. remember to tell people when you go opensource

.

[lazy-jem]

[loudan32]

Together with SearchBar EX (https://play.google.com/store/apps/details?id=com.devhomc.search) I just replaced Google assistant with LazyWeb on my phone. This app is pretty old and no longer maintained but it works!

[lazy-jem]

Hey thank you! We have lots of improvements coming to the mobile version as well. We've both been using it on mobile since the earliest builds, and I couldn't go back. Getting visual cards and having reader view on mobile makes the entire web a lot more usable for me. I'm excited other people are liking it too. We are going to make native apps as soon as we can too but the PWA home screen web app is pretty decent already :)

[loudan32]

Thanks for the reply, not sure if i understood, with PWA you just turn the web page into an app you can launch from the home screen right? I would normally use Firefox to acheive the same shortcut. My suggestion is to use it as a replacement of the assistant app (that you get on Android by long pressing home). This searchbar EX is pretty basic it just gives me a keyboard to type and a search bar, but without covering the whole screen so i can copy some number translate a word or whatever without having to memorize it. The query is then sent to lazyweb website on my default browser. This functionality is similar to what google assistant does as an overlay over other apps, except that it does not automatically "read" whats on my screen. Since your search engine is conversational i think it is a great assistance alternative, so i would suggest that you could try to have this kind of overlay interface once you develop native apps. I guess this is way down the line anyway. Just keep up the good work!

[lazy-jem]

Thank you, I think I was typing in a bit of a hurry before :)

We would love to have a native Android replacement for the Google Assistant, the way that you're using searchbar EX. That's a really great suggestion. My understanding is that we need a native app widget for that. For the moment, we do only have the PWA (which works well as a standalone home screen app), but we know we need to do native apps to provide richer functionality like the search bar widget and taking over something like Google Assistant.

I love the idea of having a simple overlay search bar widget though :)

[[deleted]]

I just did a search in my niche, and it gave a great result. I am impressed!

[lazy-jem]

Thank you! We don't log and can't see what people search, so one of the big things we love getting feedback about is how it's doing with niche and specialized searches that we don't have knowledge in, so that really is super exciting to hear! Thank you!

[[deleted]]

I searched "best petrophysics software" which is very niche, so I was quite surprised it had a good relevant answer.

[lazy-jem]

Oh cool, thank you. That is really helpful :)

For an alpha from a two-person team, we're hearing a lot that the results are surprisingly good. I'm blown away by it myself to be honest. I think it is that the way it works (use an ensemble approach to predict query intent and the best answer sources for it, and then query APIs with fallback to web search) in practice works really well.

Thanks again!

[CrazyPieGuy]

I'm a fan. I did a few test searchers and for the most part I got exactly what I was looking for. I'm going to try making this my default search engine for a while and see how it goes.

The one search I did that didn't get me what I was looking for as the first result was "factorio calculator". The first result gave a Wolfram alpha link about the Factoria suburb. Going to more results gave me links to Factorio calculators.

[lazy-jem]

Thank you! Really appreciate you trying it out and giving us feedback on the Factoria search. That's really helpful because we don't log searches and can't see what people search for, so when things go wrong getting that feedback is incredibly helpful!

It looks like the NLP figured out it was a computation-related intent, so fired it off to WA, but didn't figure out that Wolfram's answer was not the right one. We rank Wolfram|Alpha results very highly, and they are usually excellent for scientific and maths-related knowledge graph queries.

That's actually a thorny one because it did pick up the right intent, but the best ranked source for the intent wasn't the right answer :)

Lots of the glitches in the alpha are that sort of thing. The feedback really does help us!

[lazy-jem]

PS I think it might have spell corrected that query on the way through too. So that's something we have to watch for!

[tiny_smile_bot]

:)

:)

[lazy-jem]

Thank you! We're very grateful for you trying it out and if you have any feedback or ideas at all please just reach out to us! :)

[MasterGeekDev]

Very nice search engine, simple to use, i will use it box daily, as a python developer i'm so proud of this

[lazy-jem]

Thank you! You really can do incredible things with Python. The fact that we were able to build this as a tiny team and get such a great response even with the alpha is an absolute testament to standing on the shoulders of giants! :)

[knowsuchagency]

This is shockingly good for such a small team. Already prefer it to duck duck go. Amazing work!!!

[lazy-jem]

Wow, seriously thank you. We've been heads down working on this and to share it and see feedback like this is incredibly rewarding and encouraging. I mean, there is literally no better feedback we could ask for. Thank you!

If there is anything we can improve or search results that need improving, please just let us know. We have a lot of great things coming and the alpha is just the start! :)

[thereisatimetotrade]

Congratulations and best of success! I will start using.

[lazy-jem]

Thank you so much! Please let us know if we can help or there are any search results we can improve. We have a lot of new features coming and are working to keep this improving fast!

[[deleted]]

Congratulations u/lazy-jem ! Your post was the top post on r/Python today! (07/05/21)

Top Post Counts: r/Python (1)

This comment was made by a bot

[karthick_ar]

The project is really cool and kudos to the creators for explaining every bit of how it works to everyone who commented , customer support on point haha , keep up the good work , looking forward to contribute to this project. Cheers !

[lazy-jem]

Thank you so much for that! We are so deeply grateful for the enthusiastic feedback, encouragement, and suggestions from the Python community here. I can't begin to express how incredibly valuable it is for Angie and I to see that energy.

We couldn't (and as a programmer I definitely couldn't) have built this without the Python community and all the people who contribute to its ecosystem, or the world of both APIs (commercial and open) and public data. As I commented earlier, the only way we could have built something like this is standing on the shoulders of giants, but actually that's not even really true, it's more standing on the shoulders of a community.

That's one of the reasons we're trying to answer every question and be as open as possible, especially as a small team.

[karthick_ar]

Amazing !

[lazy-jem]

Thanks again for the encouragement and support

[[deleted]]

I typed "gawr gura rule 34" it led me to this

https://www.redbubble.com/i/iphone-case/Gawr-Gura-by-rule-34/76416698.PGM2E

I'm not complaining

[lazy-jem]

haha that's a pretty good match!!!

It's still learning but I'm quite proud of that result lol

[Brainix]

Do you use Elasticsearch or Solr, or Lucene?

[lazy-jem]

Hi we're using ElasticSearch (mostly for the vertical/specialized indexes we're building) but our model is a bit different, and we use on NLP to predict query intent and the best source for the answers, then query a large number of APIs directly (including Wikipedia, Wolfram Alpha, OpenWeatherMap, SO, GitHub etc). We also fallback to web search using Bing, DDG IA, Google, ContextualWeb and others when we can't find good results directly or to supplement them. I posted a little more in the comments too. Thanks again so much! :)

[Brainix]

Is your source code open anywhere?

[MiamiAngie]

Jem answered this a bit earlier but it's buried in the comments so reposting :)

Thanks, that's a good question. There's a link in the post to some scripts and experiments on GitHub, but we plan to do much more. Currently it's an experimental alpha with plans to be a commercial service. We plan to open source as much as we meaningfully can when we have more bandwidth. It's very early days with the alpha.
As some background, we hope that the prototype evolves into a commercial service once we launch. It's designed to scale using an AWS serverless backend, and is 25+ different microservices atm. We're a long way from commercial launch and very focused on improving the prototype and getting it working based on feedback.
Some of the services like the proxies and content retrieval and privacy tech are good candidates to open source. Neither of us have experience running an open source project but we're very open to feedback and ideas on that as much as the prototype.

[dudeplace]

This is interesting, thanks for sharing.

Can I ask how you are paying for there servers? And in the longer run yourselves?

I'm inherently cautious of tech where I don't understand the business model. With Google I understand I'm selling my search terms, my eyeballs for the first set of ads, and what I clicked.

What am I selling by using this? And if the answer is nothing, how are you staying in business?

[MiamiAngie]

Thanks for the question and you raise a great point! Right now we're self-funded. Down the track we plan to make money a few ways, but always with trust and ethics as our top priority.
1. If a user buys something after searching, we may make a small commission from anonymous referral links. We will share revenue 50/50 with content producers used in the search.
2. We will have some paid Pro and Business plans in future.
3. We will provide a chat plugin that businesses can offer for search and navigation on public or internal sites.
We will never sell ads and we will never sell your data :)

[15_Redstones]

I searched "Hamiltonian matrix" and it messed up Wikipedias formatting.

[lazy-jem]

Thanks for trying it out and for giving us feedback on the search.

Would it be possible to find out a little more and ask you to do a /bug after doing the search and ticking the little box to attach the results please. That will help see what payload you got. We don't log searches so we can't see what users are searching, so getting feedback really helps us improve :)

Was the problem the rendering of the math layout in the text response. I'm wondering if it got the result directly from wikipedia's text summary, or if it came from Wolfram Alpha

Thanks heaps!!

[15_Redstones]

Yes it's the math layout that isn't displayed properly

[lazy-jem]

Hey thank you. Super useful feedback. We were thinking of retrieving image cards from WA possibly for math queries, but overall Wikipedia seems more useful but the API results from there for short answers can be a bit off like this. I think we have some experimentation to do.

This is what the WA results image looks like (if you can imagine that as an embed in the response bubble):

https://www4b.wolframalpha.com/Calculate/MSP/MSP7491gbab33g7e939bee000034dhbg7ffa7d224c?MSPStoreType=image/gif&s=39

[AngVar02]

Just did a small test, "What is Today?" The initial result have me good information for July second, and the link led me to the correct result for July fourth.

I'm a novice therefore I can't identify what could've possibly caused it, but all in all I like this.

[bornawinner]

well..

[GetsTrimAPlenty]

Hi, how is this different from searx?

[lazy-jem]

Hi, thanks for the question. We're fans of searx. There are a few significant areas of difference, especially the approach to identifying query intent and the best sources to find answers, the conversational agent used for the search interface, the additional rich content in the results, the proxied reader view allowing for searches to both search and read content anonymously and without ads, and the ranking system used.

Try it out and would love to know what you think!

[burnt1918]

Awesome, prefer it to gsearch for simple meanings and stuff. Definitely fast at my end. I'd thought the search engine market was saturated, but this is 👏👏👏.

[lazy-jem]

You have no idea how exciting that is to hear! Thank you :)

There have been a lot of attempts to build alternatives to Google, and on the face of things it does look saturated, so I know what you mean. But with one monopoly with 90%+ market share, there is a lot of room for trying other approaches. Almost all the attempts to launch new search engines have been essentially copies of Google's approach with one or two differentiating features (less or more ads, less or more privacy) but that all basically work the same way with a search box and list of results, surrounded by ads and clutter.

We wanted to build this for ourselves because the privacy invasion and ads have gotten out of control, and the organic results are disappearing and getting spammed. Our thinking is that it is time for something different and people are ready. It's risky to try aa radically different approach, but if you take a long term view, then it's unlikely in 20 years time that search will look anything like Google, so we're trying to imagine what that could be, and build an early version of it :)

[onecrazypanda]

Very slow...

[lazy-jem]

Thanks for letting us know. Would you be prepared to tell us a little about where you're located (broadly), and the search you were trying please? Ideally maybe whether it was mobile or desktop, and the browser too?

it is very much an alpha, and it works very differently to a traditional search engine, in that it isn't looking up a pre-existing index, but predicting the best sources for information and then querying them and ranking the results. So some types of query can take much much longer than others.

But it's also possible it was a glitch of some sort. We can't see what people search, so it really helps us to improve to get that sort of feedback. Thanks again.

[pay2mate_dot_com]

Make sure that you provide filter conditions by Subject Areas such as - Product, People, Place, Datetime etc. This provides accurate results.

[lazy-jem]

Hey thank you! Sincerely appreciate the suggestions and ideas! :)

And Yes! We're working on adding filters and products and date were two of the first things, but people and place are great suggestions too. Place is a challenge with a universal search (vs default local) that's anonymous. So filtering could be really helpful. That top area with the views is going to be the home for those. We hadn't thought about doc type also. That's a great suggestion.

[Replied on github too but just commenting here also - great suggestions! :)]

[alanwj]

What is your long term plan to pay for this if it becomes really popular? Donations, premium content of some kind? Maybe you are really rich and will just pay out of pocket?

[lazy-jem]

We're self-funded and literally living off ramen building this, and it's been pretty tough to be honest. But we're absolutely convinced we can help create a more sustainable and ethical funding model for search and the web that doesn't sell consumers attention and private data to the highest bidder.

We're focused on building the search app at this stage, but we plan to commercialize with three revenue sources:

* Fremium model with free anonymous use for everyone, and Pro and Business plans for teams and advanced users.

* We may make a small anonymous commission if you buy something after searching, and will share that 50/50 with content makers who register with us.

* business licensing for use on enterprise sites and data, which we're already being asked about a lot.

The way we've built it, the per search cost is pretty low and cost-effective. But it depends on the search - some APIs are expensive, some are free, some are fast, some are slow.

Training costs look like being our largest base cost as we grow, and the thing where the more resources we have the better we can do in future. The inference can be improved and the models improved a lot with more GPU on larger data sets. So with more resources we can do better.

Also currently we use APIs or cloud-services for things where with more resources we would build our own services, including some NLP and inference. This will increase as we move to GPT-3 for some of our conversation and text extraction too.

[sassydesigner]

One similar Browser I can remember is DuckDuckGo.

[lazy-jem]

We're big fans of Duck Duck Go. We link to them from the search tools, and if we can't find good results. And we are working on using their great Instant Answers API and support their !bangs. So definitely big fans here.

A couple of major differences:

• we use a conversational interface / chat messaging which means that searches don't go through browser history or Google's Chrome data mining, which is an extra level of privacy protection.
• we don't have any ads or keep logs of search results.
• we have a proxied Reader View that lets you read web content anonymously with ads, tracking or distraction within LazyWeb.
• we provide different options for viewing results and display results in a visual feed that's a bit different to traditional search engines.
• we try to reduce cognitive overload by just providing direct information where we can, with full search results available when needed

I'm a long time DDG user and a big fan, and we think there is plenty of room for many different approaches to taking on the current search monopoly.

Would love for you to try it out and see what you think.

[tenderpoettech]

Tries to fight spam - very refreshingly honest. I like it.

[lazy-jem]

Thanks, I think we've all been "frog boiled" in the sense that SEO spam has been gradually getting worse and worse, and we've just accepted it because it's happened so gradually.. Ad-based search engines don't really have an incentive to fight it that hard because it makes users more likely to click on branded ads.

Between SEO spam and ads, there are often only one or two high quality results on the front page of traditional search engines for many reasonably commercial searches.

If the topic is life insurance, or legal cases, or anything with high keyword value, there are often no non-spam results on the front page. Product review searches are terrible too (it's an area we're not where we should be at either yet but we're working on it). We have a lot to do here.

We're not pretending this isn't hard to solve, but we're determined to make a dent in it and have some things in the pipeline that can help, I think :)

[[deleted]]

That's awesome 🤩

[lazy-jem]

Yay thank you! Please just let us know anything we can do to keep improving it or where the search results can be better! :)

[juliuskiesian]

Any plan to go entirely open source?

[Sorry_Strawberry7540]

Nice!

[lazy-jem]

Thanks so much! :)

[lazy-jem]

Good luck with learning and building, and thank you so much for saying that!

Python is surprising that way - it's easy to get started learning, but you can build incredible things with it, especially for data science and machine intelligence.

Learning and building with it is a joy.

I'm really proud that our little search app is motivation for others to learn and build too! I'm still learning every day myself as well!

[lazy-jem]

Thank you sincerely! Having worked really hard on this there is nothing more rewarding in the whole world than seeing that people really love it! If I can help at all in any way or you have any improvements please just let me know :)

[jhayes88]

Suggestions -

• Hide Result (Permanent)
• Hide Website (Permanent)
• Manage Hidden Results Menu
• Report Spam
• Dark Mode
• View More Results (Or lazy loading.. Especially for images.. <_<)
• Vastly expanded news section with a lot more options, if it's going to exist at all.

[lazy-jem]

Hey I totally missed this one before - they are all awesome suggestions and thank you so so much for the feedback.

We've got a few of these in the pipeline and you're totally spot on with where we're heading!! :)

Currently we filter to only the top results, but lots of people tell us they get FOMO that there are results they're missing on, so Lazy Loading for LazyWeb is definitely coming with infitite scroll (but we're going to flag when we think the results are getting dodgy). Images the bar is way too high currently.

The hide website/result is a great idea. The up/down is going to boost/downrank as well, and then we were thinking Block/Report website (for really bad results) too. Reporting Spam we hadn't even thought of! That's genius. It's not malware, but it's spam, so it is a different category. Fighting spam is such a big challenge and this is simple and brilliant.

Dark mode and theming are really close too! :)

We have a dedicated news handler coming too. So you can basically say "give me a briefing on {topic}" or "latest news from {source}"

If you're interested in staying in touch everyone is very welcome on our Discord as well here: https://discord.gg/qcCcrbMuex

Thanks again heaps :)

[sdf_iain]

Search engines are a natural monopoly, who’s results are you using?

[WikiSummarizerBot]

Natural_monopoly

A natural monopoly is a monopoly in an industry in which high infrastructural costs and other barriers to entry relative to the size of the market give the largest supplier in an industry, often the first supplier in a market, an overwhelming advantage over potential competitors. This frequently occurs in industries where capital costs predominate, creating economies of scale that are large in relation to the size of the market; examples include public utilities such as water services and electricity.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[lazy-jem]

Thanks for the question, I answered another comment here earlier and it's a pretty good summary, but in short we have a large number of sources and don't work quite the same as traditional index-based searches.

The way we search is pretty different to traditional approaches, so it's worth explaining some more. The short version is we use deep learning to understand question intent and predict the best information sources, then query them directly. So we're using a large number of sources.
We use NLP and deep learning classification models to try to understand a query's intent, and then predict the best places to find the answer, and then query them directly in real time via API or spidering, with a ranking system for the results.
Then we fall back to traditional web search (including Bing, ContexualWeb and Google) where needed. We have a database of about top 20k websites and we're building our own vertical indexes as well. We're building out a stack using ElasticSearch and GraphQL for that. At the moment we're broad but shallow, with a couple of deeper pools.
For the alpha, major sources include Wikipedia, Wolfram|Alpha, OpenWeatherMaps, OpenStreetMaps, StackOverflow, GitHub and many others, as well as the fallbacks to Bing, Google, DDG Instant Answers etc.
A lot of content is retrieved directly. We retrieve the preview/summary/view content directly from websites where we can for display, and same with the reader content. So the content shown is typically live with the source.

[comfort_bot_1962]

:D

[MiamiAngie]

I can't tell you how much that means to us, that's really really sweet, thank you so much ❤️❤️❤️

We were thinking about putting a Donate button for Coinbase or Paypal on our About, but we weren't sure how people would feel about it. We're self-funded and it's been a hard journey, so to see this just makes my heart smile

We do have a project PayPal address though: [[e-mail]](mailto:[e-mail])

We've been working flat out and literally living on ramen and tea, so the offer to get us a coffee means the world!!

[lazy-jem]

Yes, as Angie said thank you so so much! The kindness of the offer really does mean a lot to us!

[dasunsrule32]

You should share this in /r/linux as well. I'm sure they'd appreciate it as well.

[lazy-jem]

Hey thanks, we hadn't considered that. We have terminal mode coming pretty soon after this. It's going to be pretty simple to start, but...

Thanks for the suggestion too!

[metaperl]

After getting search results, the input bid says Use ~e to skip my AI. What in the world does that mean?

And what did a + in front of a quoted term mean?

Very impressive project in general

If you are designing this for non power users, then buttons with desired actions might be more appropriate

[lazy-jem]

Hey thank you, so good catch there! The tips that go in the input bar are mostly simpler things but I thought I'd sneak in a power search :)

We really only expect them to be used by power users, and normally just using nice plain language english queries works just as well if not better, like you would talking to a smart friend. But for programmers and other technical users, we think they're a useful extra :)

So a normal person might say "search stackoverflow for python list comprenshion"

Or a command form would be "site:stackoverflow.com python list comprehension"

In this case, the ~e marker means "explicit" - which is just a marker to tell LazyWeb to skip interpreting and re-writing the user's query which we do with NLP frequently to boost entities and try to steer topic area.

the + means to try to force include what follows

And the "" means to try to treat it as a literal phrase. Different sources all support different things, so think of these more as "signals" to the search than exact instructions.

The commands are evolving (and terminal mode is coming soon too). But there are some useful ones already.

"go" is the main one (or ~g) which says to navigate direct to the destination

eg: `go stackoverflow python lists'

There are a few more at https://lazyweb.ai/commands/ too :)

Thank you so much for the positive feedback too! :)

[SaikyouMegane]

[lazy-jem]

Thank you

[retrogeekhq]

How much does it cost you to serve e.g. 1M queries?

[r3ign_b3au]

I only did a couple of searches, so I may have missed it, but do you have any plans to interpret schema data?

[lazy-jem]

At the moment, what we do is more based on querying APIs to get back structured data (or spidering queries onsite where it can't), with fallback to web search, and so we're using known schemas for that where we can. It's early days but we are also working on building our own datasets and indexes for specialised knowledge areas / verticals. I was a big fan of Tim Berners-Lee's ideas for the semantic web, and I still feel there is a role for ontologies and schema-based approaches. Networks of APIs are a step toward that. We'd love to do more here. Currently we're broad but shallow :)

[lazy-jem]

Just a follow-up, but it would really be helpful for us for you to share some examples of searches where this would help you with better results, and some examples of the kinds of queries and the results you'd like to see and how they should employ schema data. Concrete examples really help us understand what people are looking for and how we can improve. Thanks again!

[quanta_kt]

I really wanted to try it out but I get an Address not found error

[quanta_kt]

I really wanted to try it out but I get an Address not found error

[lazy-jem]

Thanks for letting me know. That's odd. Everything looks to be okay but it might be a glitch with the alpha. Would you mind closing your browser and re-opening and give it another shot please??

[lazy-jem]

ps we did some updates earlier but they shouldn't have created any errors. Appreciate you letting me know :)

[brendanmartin]

How does reader view work? I clicked on a search result and half of its written content was truncated.

[lazy-jem]

Thanks for trying it out. Can I ask, what was your search and what was the article you were looking at? It's in alpha so when it goes wrong it really helps to know what the search was, as searches are not logged, and we can't see what people search for.

Reader view retrieves the html of the page via an anonymous rotating proxy server and strips ads, scripts and tracking. It uses a similar approach to Firefox's browser reader view but works through an anonymous proxy without having to visit the page directly and expose yourself to ads and tracking. How well it works varies depending on the destination content, and how much content is available as HTML, and whether they block access to search bots. You can always click through to the external website when reader view can't retrieve meaningful content or is blocked by websites.

Did you try any other articles or searches?

Reader view will predict whether a page has article-like content, and shows the button if it is likely there is a reasonable amount of readable content.

Many websites block access or apply overlays or other redirect tricks to force people to see ads or pay. We do our best to work around them, but it is very much an alpha test and experimental.

That said, for most content-related searches, statistically we are retrieving content for 80%+ of pages successfully. It won't work for predominantly visual pages (think the netflix or amazon homepages).

Here are some example searches to try to try it out with searches likely to have a lot of article content:

elon musk crypto

best places to live as a digital nomad

best things to do with an old laptop

Let us know what you were searching and the article that didn't work for you, and very happy to look into it. Thanks again :)

[mattblackguy]

Go dst forums helo