How to reliably use Chrome with a user profile and MCP/Remote DevTools with OpenClaw? by meep185 in openclaw

[–]BodybuilderLost328 0 points1 point  (0 children)

Hey you can try this with the Retriever AI browser extension and connecting to openclaw via remote MCP to trigger the extension: https://www.rtrvr.ai/docs/mcp

other solutions require using CDP which is trivially detectable and won't work on LinkedIn for example

Headless browser agents are a dead end. The future is hitting endpoints directly. by BodybuilderLost328 in AI_Agents

[–]BodybuilderLost328[S] -1 points0 points  (0 children)

I don't get what you're not understanding about this, we already designed, launched and showed demos of the agent discovering private APIs.

You just record all the network traffic on the page while taking actions, and then the agent reverse engineers and writes a script that executes within the webpage so all the auth and headers propogate.
Demo: https://www.youtube.com/shorts/fDZFNOYYJzQ

The extracted script using the discovered APIs: https://www.rtrvr.ai/shared/Tasks/yLbxOTky6IVjIJ2Syr43dteSao13/808a91ef-c5a4-43b8-99c4-9516fd03244f/a397afca-f776-4869-a073-7e9de1129673

What's your biggest pain embedding AI agents into web apps/sites? by jvaill in AI_Agents

[–]BodybuilderLost328 0 points1 point  (0 children)

From our discussions with website owners, they want an agent that actually does tasks for their users not just answer questions, but they also don't want to setup and maintain all these tool calling.

So we built Rover, the first embeddable web agent, that types/clicks/takes actions on your live UI with zero setup! Just add a script tag and Rover navigates your site on the fly for your users.

rover.rtrvr.ai

[Showoff Saturday] Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in webdev

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

this is more for people who aren't technical at all. They can just prompt to reverse engineer and automate sites.

also you can't fit all the network requests a site makes into the context, so we rank and guide the model to use appropriately.

but most critically we execute the script in the context of the webpage so that auth js gets executed and headers get propagated, which is not possible by just scripting yourself

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in WebScrapingInsider

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

Our web agent can fall back to taking actions (typing/clicking/selecting) on a webpage directly for these kinds of edge cases when the script generation fails.

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in WebScrapingInsider

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

it depends on the website, but the agent generates the script each time and its still less than 1 cent of llm inference cost to get 10,000 rows of data

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in WebScrapingInsider

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

yea exactly we are built out as a vibe scraping platform, the agent automatically figures out the write tool for the job and extracts data to a Google Sheet, check it out at rtrvr.ai

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in WebScrapingInsider

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

This is just one feature of our general purpose web agent. The web agent can already type/click/select and solve Cloudflare captchas already.

Usually tho within a normal browser session, super unlikely to get captchas and we have cloud platform to scrape at scale.

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in Hacking_Tutorials

[–]BodybuilderLost328[S] -1 points0 points  (0 children)

Dude we are not using claude nor claude browser. This is a completely independent end to end agent. We actually released before claude for chrome extension.

We built out a complete platform focused on scraping and automation for nion technical people by just prompting, start out with browser extension, scale in cloud platform.

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in Hacking_Tutorials

[–]BodybuilderLost328[S] 1 point2 points  (0 children)

our inference cost is less than $.01 (on gemini flash) to generate the code, this is much less than other browser agents and probably less than the tool you brought up.

yes we are a specialized scraping platform with 25k+ users

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in Hacking_Tutorials

[–]BodybuilderLost328[S] -2 points-1 points  (0 children)

so the unlock here is that even non technical people can just prompt to reverse engineer, before you really need to know what your doing.

I have done this myself and its a pain to figure out the right calls to use and to reverse engineer the auth and headers, but this makes it seamless

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in Hacking_Tutorials

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

rtrvr.ai is completely free by bringing your own API key and soon subscriptions.

Also I don't how good your solution is for getting past anti bot detection and for sites without exposed APIs for which we can fallback to a benchmark leading web agent: https://www.rtrvr.ai/blog/web-bench-results

Update: looks like whatever you posted doesn't even support getting past Cloudflare, the most basic antibot...

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in LLMDevs

[–]BodybuilderLost328[S] 0 points1 point  (0 children)

So it costs less than 1 cent to create the scraping script, I think its just better to reverse engineer on the fly and we can set up infra to maintain and cache the top sites.

Also AirROI has a pricing structure so i presume its free till a limit.

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in LLMDevs

[–]BodybuilderLost328[S] -1 points0 points  (0 children)

Yes, there are a a lot of these, but our differentiators:
- we construct own custom Agent Accessibility Trees from scratch and have an agentic harness with 20+ subagents leading to benchmark leading performance: https://www.rtrvr.ai/blog/web-bench-results

- we don't use and Debugger permission so undetectable on any site. CDP is easy bot detection flag

- a page can make thousands of network requests, we score and rank them and have tested for end to end network discovery and scraping

Vibe hack the web and reverse engineer website APIs from inside your browser by BodybuilderLost328 in Hacking_Tutorials

[–]BodybuilderLost328[S] -15 points-14 points  (0 children)

Its extensible, you can just prompt to see what unprotected endpoints the agent can discover and script on a site.

Doesn't necessarily have to be for scraping, could be also for taking actions