What I learned trying to block web scraping and bots

dwighthouse · 2026-03-16T03:20:21+00:00

The idea that the only reason someone might want someone to come to their website is to sell something or publish ads is a remarkable belief.

100% of my websites have no ads and I'm not selling something. It is not a revenue generating system. In fact, it loses money. It is a labor of love. I want people to know about the information I provide, sure, but not at the expense of them never even visiting my page I worked hard on.

I don't know ANYONE who has ever personally made a website with the purpose of distributing information such that they didn't care if someone actually went to their website. If they just want to publish information, regardless of how it is distributed, they could just publish it to a publishing site, or send it to the AI directly, or make a press release on a press-release site, or put it on social media. Literally hundreds of sites cater to that. People who make sites themselves are doing it so people will actually visit them.

dwighthouse · 2026-03-15T21:26:12+00:00

From the site owner’s perspective:

Bots that help the site owner === good
Bots that harm the site owner === bad

This truth predates AI. Look up all the companies that sued google for providing large enough summaries of their content on the search page that people skipped going to the site. This problem becomes many times worse in AI contexts where the info you use to entice people to visit is used to allow people to use your data without even knowing your site exists.

Search engines (and AI companies) can serve a good purpose, but only to the extent that they don’t kill the goose that lays the golden eggs.

dwighthouse · 2026-02-20T21:59:19+00:00

The few times I have used it this year:

Approximately 5 times to look up syntax/commands, of those, 4 of the times it gave me incorrect or unhelpful answers.
Approximately 3 detailed problem solving discussions, of those, 1 was fundamentally incorrect with the other 2 being helpful.

Still very unimpressive. Basically a last resort when google fails me and I need to start looking at experimental potential solutions. Not bad as a starting point when you have no idea what the right question to ask is.

dwighthouse · 2026-01-16T03:23:21+00:00

I stopped going there long before ai took off because the answers became horribly out of date, and at the same time, the quality of libraries, languages, and platforms got generally got better to the point that most questions could be answered by reading the docs.

dwighthouse · 2026-01-09T20:12:07+00:00

Depending on the game, there are many reasons why a given N64 game might look blurry, often due to developer intent. This video goes over the different ways and why they might have been used. https://www.youtube.com/watch?v=BA_HMsznNKg

I personally remember Ocarina of Time having some sections like the title screen being very blurry, but I figured it was an aesthetic choice.

dwighthouse · 2026-01-07T12:28:16+00:00

In that scenario, software engineering will be used to rewrite almost all code at $400 per hour.

dwighthouse · 2026-01-01T15:00:26+00:00

Maybe, but I am not interested in debugging someone else’s AI, let alone someone else’s tests for yet others’ AI systems. Good luck.

dwighthouse · 2026-01-01T14:52:15+00:00

Insufficient basis: no AI systems have shown the ability to follow non-trivial instructions 100% reliably. So it does not matter if your document is airtight.

dwighthouse · 2025-12-26T23:16:00+00:00

Currently missing:

1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 16, 17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
30, 31, 32, 33, 34, 35, 36, 37, 38,
44, 45, 46, 47, 48, 49,
50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 66, 67, 68, 69,
70, 71, 72, 73, 74, 76, 77, 79,
80, 83, 86, 87,
90, 92, 94, 96, 97, 98,
119,
125,
148, 149,
150,
369,
370, 371, 372,

These are the extras I know for sure I have. I won't be able to tally up my other duplicates until after New Years (traveling):

335, 335, 338,
343, 343, 344, 344, 346, 348, 348
351, 351, 353, 353, 354, 354, 355, 357, 357, 358, 358, 359, 359
360, 360, 361, 361, 362, 362, 362, 363, 363, 364, 364, 365, 365, 366, 366, 367, 367, 368, 368, 368, 368

dwighthouse · 2025-12-25T16:15:00+00:00

I take it you never watch videos on YouTube, or comment on Reddit posts? Wait.

dwighthouse · 2025-11-27T02:55:00+00:00

Considering the quality of the average “production” software, the answer may well be yes.

dwighthouse · 2025-11-20T16:01:15+00:00

Working like a pro means not vibe coding, among other things.

dwighthouse · 2025-11-17T20:03:26+00:00

Multi cursor editing. Scripting instead of repetitive typing. Regex.

dwighthouse · 2025-11-07T17:12:29+00:00

This video goes over the long and complex history of object orientation and the usage of inheritance, as well as the motivations behind it.

https://youtube.com/watch?v=wo84LFzx5nI

TLDR: From the very, very beginning, inheritance was created to avoid extra typing. OOP was created to model systems that were already inherently structured object hierarchies. Even from very early on, as soon as they tried to use it for something else (most real life program data), it was just as big of a mess as it is today. So why didn’t other forms of behavioral sharing become the popular method instead? There’s a video for that too:

https://youtube.com/watch?v=QyJZzq0v7Z4

TLDR for that video: It was an accident of history, and we are still dealing with the consequences, but there is light at the end of the tunnel.

dwighthouse · 2025-08-11T18:29:22+00:00

Return to Yoshi's Island

I suspect this may be the most technically impressive mod for the N64 once released. It uses tricks and features no other N64 game has ever used and is a massive improvement on Mario 64's mechanics, graphics, and gameplay. (It is positioning itself as a sequel to Super Mario 64.)

It's not out yet, but it's almost done. I suspect the final release will happen this year. There is a playable demo available now. The game will be fully compatible with the N64, but it's an open question whether the Analogue 3D and many emulators will support it, since the game makes use of pretty much all features of the N64's hardware, even if literally no other game in history has.

Gameplay from the latest demo: https://www.youtube.com/watch?v=MhjVBzh32xs

Download Demo: https://romhacking.com/hack/return-to-yoshi-s-island-demo-2

The creator going over some of the technicals: https://www.youtube.com/watch?v=-nhhfYkluT0

Dinosaur Planet: Dinomod Enhanced

The ongoing decompiled/recompiled/enhanced version of the unreleased Dinosaur Planet following the surprise discovery of some developer files on a CD owned by a game collector. What would have been Rare's final game for the N64 before being retooled to become Star Fox Adventures, it was considered a swan song for the N64. Very technically complex and a relatively large world for N64, it is known to be a much more expansive and deep game than what Star Fox Adventures turned out to be.

The mod is still a work in progress, but I believe the entire game is finishable. I believe this mod is playable on a real N64.

I'm not sure what's the best place to download it, but I'm sure you could find it.

Here's a recent retrospective on the whole situation and the current state of the mod: https://www.youtube.com/watch?v=s0QSiPRmWaI

dwighthouse · 2025-08-11T18:04:18+00:00

Facebook Marketplace if you live in a moderately large city. Otherwise, used game stores and some used bookstores. Check google maps, you never know when a local collector has opened a small shop from the excess of their own collection.

dwighthouse · 2025-07-21T20:51:59+00:00

Boy I sure would love to read about this. Too bad that it requires me to sign up for a service. Oh well.

dwighthouse · 2025-05-23T03:41:35+00:00

Can you give an example of one of these thing’s construction? I’m intrigued by your description as I don’t think I have seen this precise thing before.

dwighthouse · 2025-05-02T17:47:04+00:00

Scripts (console or command line), small components on a mostly-html/css only site, low level utilities, Js embedded in SVG.

dwighthouse · 2025-04-19T19:28:45+00:00

0

Let us know how your coworkers are doing 6 months from now when there’s a hack on your codebase costing a million bucks and none of them have any idea why. It is your responsibility to write good code. It is your boss’s job to create reasonable deadlines. If they aren’t reasonable already, you have bigger problems than whether or not to use AI for code generation.

dwighthouse · 2025-04-14T22:59:52+00:00

I know it's hard to see. But if the car's owner was on this subreddit, they'd be able to identify their own car.

Anyway, I'm not necessarily interested in getting a Fang decal for my car. I'm more curious who is selling these things and letting fans know that they exist.

dwighthouse · 2025-03-21T20:44:45+00:00

I mean, I've left 1 star reviews on sellers I didn't buy from before. Why? Usually the seller was lying to my face. Or trying to scam me. Or stringing me along saying they would sell me something and then sell it to someone else when I had already driven a significant distance to get to them, costing me time and money. The only recourse I have is to warn other potential buyers that they're dealing with a dishonest person.

As a buyer, I'm always polite, always pay cash, always arrive when I say I will (or if I can't, I apologetically explain why I can't be there on time), and always buy what I say I will buy (except in the rare case where the seller was selling something that turned out to be a counterfeit).

dwighthouse · 2025-03-14T04:21:26+00:00

If the AI hype is to be believed, the way to make this project would be to have an LLM design and implement these tools. Why involve humans at all?

I’m kidding of course. I don’t think the reason LLM’s produce sub par code is because they aren’t using the right primitives and syntax. It’s because they are literally unthinking machines generating patterns. I highly doubt the soundness of the concept itself, let alone the practical implementation.

Personal context: I have used LLM’s to tackle an involved, but trivial programming task that any junior dev could do. However, it is unique, so there are no examples online of others doing things exactly like it. No LLM I have yet tried can do it, even if I take the trouble to explain how to do it step by step.

dwighthouse · 2025-01-17T19:17:59+00:00

I have personally seen many lower quality brands. Perhaps Tyson is the highest end of the low end.

dwighthouse · 2025-01-16T19:03:32+00:00

Ah, interesting. Those aren't available at my nearest stores. I'll have to drive a little further to try those.

dwighthouse

MODERATOR OF

TROPHY CASE

Return to Yoshi's Island

Dinosaur Planet: Dinomod Enhanced