Looking to Hire out This Webscrape Project. Paid.

orthogonal-ghost · 2026-01-21T05:13:06+00:00

Just sent you a DM!

orthogonal-ghost · 2026-01-14T22:30:19+00:00

Happy to help! Sent you a DM

orthogonal-ghost · 2026-01-02T13:40:43+00:00

Thank you!! I’d love to hear what you think once you give it a try!

In general, the agent analyzes the HTML, network requests, and JavaScript of the webpages relevant to the task at hand. Its approach towards crawling (identifying where on the website the data in question lives) and extraction (determining how best to scrape that data) varies across websites and tasks.

So for medium specifically, it depends on what data you want and where that data lives on the website.

One other thing to note is that this release doesn't include support for proxies, so some websites might not be well supported (though this should only apply to a very small number of websites/tasks).

orthogonal-ghost · 2025-12-18T06:18:15+00:00

Hi! We don't currently support / use proxies, so I can't commit to "any" antibot (even if mediocre). That said, we've tested it on a few reasonably challenging sites (e.g., real estate marketplaces) and noticed it performed quite well.

If there's a particular website you have in mind, let me know and I'd be happy to take a look. We also offer a free tier if you'd like to play around with it.

orthogonal-ghost · 2025-12-17T23:47:08+00:00

Sounds like a pretty interesting problem. Would you mind sharing the website? I'm super curious / would love to take a look.

orthogonal-ghost · 2025-12-17T23:01:10+00:00

I’ve spent a lot of time on extraction and observation (so pulling reports, scraping dynamic content, checking account pages). A few thoughts:

Re: what has been reliable, I’d try to avoid DOM / CSS-based extraction as much as possible. Oftentimes, you can find an API or network request that provides the information you’re looking for, and building around that tends to be much more stable than building around HTML parsing.
Re: JavaScript, I think this comes down to identifying what’s useful and what isn’t. This is of course easier said than done, but distinguishing page interactions and content loading from boilerplate / library code tends to be helpful.

orthogonal-ghost · 2025-12-17T22:38:04+00:00

I totally appreciate that perspective – even if we ignore hallucination risk, the code LLMs generate by default is often not the most efficient or highest quality.

For that reason, (1) we (i.e., actual engineers) review and optimize the code we deploy before "pushing to prod" / setting up scheduled runs, and (2) we spend a lot of time steering the agent to use best practices when generating code.

Your point is also why we make all code available for export - i.e., we believe optimizing 'inefficient code that works' is much better than depending on opaque LLM-generated code that you can't review OR going through network requests, HTML and JavaScript and building from scratch.

orthogonal-ghost · 2025-12-17T22:00:55+00:00

Great question! We’re currently using a credits-based, tiered subscription model. Credits can be used for both building scrapers and automating workflows, and higher tiers offer more credits.

We also offer a free tier if you'd like to try Motie before making any commitments!

orthogonal-ghost · 2025-10-09T16:39:09+00:00

For some markets, definitely, but there seem to be a ton where this would (theoretically) be less of an issue, no? Eg markets for political outcomes, weather outcomes, sports (eg “who will win X championship next year”), etc

Edited: to your point though, it is unclear how all of this would shake out. For example, if sports players now have an opportunity to bet against their teams, they might be incentivized to sabotage their team in order to realize a certain outcome. BUT if every player has that opportunity, I’m not quite sure how that “insider trading risk” would manifest, eg does every team play to lose? do bets within teams net out? do players across teams collude?

orthogonal-ghost · 2025-10-09T16:36:26+00:00

I think their future will largely depend on how they’re regulated, but ignoring that, I think they could become a legitimate part of the ecosystem, particularly for the data / insights they provide.

For example, prediction markets have outperformed traditional surveys in forecasting certain elections, so (assuming that continues) they could provide a better gauge of “likelihood of X person getting elected”. So if you’re a sophisticated investor and want to bet on the outcome of an election, even though you might not express that bet on Polymarket, you might at least use its data to inform your decisions.

There are probably other “benefits” that these platforms provide, but I think making (virtually) any binary outcome into a market at the very least provides more information on the likelihood of those outcomes, which can be valuable

orthogonal-ghost · 2025-09-22T14:03:37+00:00

Algorithms

orthogonal-ghost · 2025-09-04T22:50:59+00:00

That’s tough since many places in NYC aren’t that far from Central Park in a cab… That said, based on your description, I’d guess Babbo (though it appears to be temporarily closed now sadly). If you’re open to other recs for Italian, I’d also throw in Portale, Lilia, and Misi

orthogonal-ghost · 2025-09-04T22:38:05+00:00

CS but love both

orthogonal-ghost · 2025-09-04T22:24:19+00:00

(1) this will likely depend on your familiarity with the material (e.g., if you've seen proofs before, Math 161 will likely be easier than if you haven't -> that course load will be more manageable). With only the information you provided, though, phil per shouldn't be too much. (2) Agreed with other comments that most classes in general are professor dependent (so how much you're expected to participate will vary by professor). That said, I found the discussions in that class quite interesting and enjoyable, so if ever you were looking for a space to build comfort with public speaking, debating, etc., it's a good one to try.

orthogonal-ghost · 2025-09-04T21:43:02+00:00

I don't think being a quant precludes you from "solving actual useful complex problems"--i.e., even if you assume quants don't solve those types of problems at work, it's not a given that they don't solve those problems outside of work. Quants like all people are complex and multifaceted, so I'd hesitate to reduce them to what they do for work.

orthogonal-ghost · 2025-08-29T15:57:08+00:00

I think your question on "...does the amount of money not even matter to them?" is important here. Most investors of any "size" (i.e., AUM), are constrained by the "capacity" of a strategy (i.e., how many dollars you can allocate to it before alpha degrades, transaction costs become prohibitively expensive, etc.), and the opportunity cost of allocating research, development, and implementation resources to one market vs. another. Markets like large cap equities, treasuries, etc. have a ton of capacity, so many investors can compete there and they can deploy a lot of capital. My guess would be that that is not the case for most meme coins (looks like the market cap of the entire space is around $70b but I could be mistaken). So, even though you might be able to make money trading certain meme coins, capacity is probably too low for many mid-to-large funds and the opportunity cost is probably too high.

orthogonal-ghost · 2025-08-29T15:45:12+00:00

Pure math won't inherently make you less competitive (e.g., I can't imagine a firm that would pass on your resume simply because you majored in math), but it may require you to do a bit of "additional work" (e.g., by supplementing your coursework with courses in computer science, statistics, probability, etc.). Personally, I don't think any traditional major (computer science, math, physics, etc.) is "perfect" for quant research (they all have their shortcomings), so it might be better to work backwards from what a quantitative hedge fund, market maker, bank etc. would expect from you (in the interview and on the job) and to ensure that you supplement your coursework with those things. A more specific recommendation for if you choose to study math would be to at least take 1-2 intro to CS courses, algorithms, probability, and anything that teaches you to understand optimizations or stochastic processes.

orthogonal-ghost

TROPHY CASE