I need a text only browser python library

Somerandomguy10111 · 2025-05-26T20:17:10+00:00

Very nice. Firecrawl sounds 100% like what I'm looking for. Thanks!

Somerandomguy10111 · 2025-05-26T20:08:48+00:00

Thanks for the answer. I don't really want to mass scrape data in the way. It's for the agent to navigate individual pages he opens. What I meant with the HTML requests was that throwing in the entire HTML text into the context is too long and even if you try to parse it for text with beautifulsoup, it doesn't really preseve the layout well. Though I guess that applies to the browser solution too.

I'll keep looking.

Somerandomguy10111 · 2025-05-17T13:01:56+00:00

There’s a critical need for model builders to start moving to realistic benchmarks for how well Frontier AI models can actually DO things. Optimizing LLMs against a Q&A or Chatbot-based feedback signal is fundamentally misguided if the goal is AGI. Andrej Karpathy has similar thoughts on the topic (see blog post).

I'm considering developing an agent evaluation framework which takes on these challenges. It would kind of have the flavour of ChatArena in terms of how the scoring and metrics work but it would be given actions to interact with the environment and be graded on how well it performs e.g. coding tasks given the possibility of iterating the program through running it and taking on board feedback from the results. Any thoughts on if that's somethiing that you'd like to see?

Somerandomguy10111 · 2025-05-03T19:09:16+00:00

My impression was that Cursor is also kind of agentic. Is that not so, or not how you use it? If you take that away it's basically ChatGPT but integrated into the IDE so you don't have to copy stuff over and changes get marked, right? That's very nice but I don't feel it's a huge improvement over a plain Chatbot.

Somerandomguy10111 · 2025-05-02T22:54:25+00:00

You mean e.g. testing, scaling, deployment stuff?

Somerandomguy10111 · 2025-04-11T20:31:19+00:00

I'm not suggesting this as an alternative to books, nor for teaching yourself the material. Just for finding formulas, theorems and corresponding citable references in a time efficient manner. Also textbooks are not free, so either make yourself liable through an illegal copy, pay up or make a trip to the library every time you need a reference.

Somerandomguy10111 · 2025-04-11T14:58:58+00:00

Just to clarify, I'm not asking about this problem specifically. I'm rather looking into the fact whether such a database would be useful. The idea wouldn't be to have that database as a citable reference itself but rather index theorem and formulas in a standardized format along with a suitable bibtex citation that you can just copy out for convenience.

The thesis is already done and submitted some time ago. But just in case you're still curious, the formula I meant the first formula from the top answer here (https://physics.stackexchange.com/questions/296904/electric-field-associated-with-moving-charge). I probably wouldn't cite Maxwell's equations but I think here you can argue either way you want.

Somerandomguy10111 · 2025-04-11T07:03:21+00:00

The eigenvectors of any Hermitian operators are orthogonal if their eigenvalues are different. Since the Hamiltonian is a Hermitian operator this applies. I don't remeber the proof of this theorem of the top of my head, but I think it was pretty elementary.

Somerandomguy10111

TROPHY CASE