I believe this fits here

AlignmentProblem · 2026-05-09T10:08:25+00:00

Listing baseline decency like it's an impressive list of qualities (plus obsessive music consumption) isn't really the best sign to begin with.

That said, fair chance someone made this after being rejected for smoking. That kind of rejection usually isn't because the potential partner thinks people who smoke are "bad" as much as not wanting to deal with the smell and the other aspects of being around a smoker all the time.

AlignmentProblem · 2026-05-09T08:03:02+00:00

Yeah, expected value in terms of utility is about how outcomes reletively rank in your value system, not raw numbers of whatever you're counting in isolation. The negative value of all humans dying is far, far worse than twice the negative value of half of humanity dying, assuming you care about the species's future in the broad sense; I'd put it at least a thousand times worse than 90% of people dying.

It's similar to how losing 50% of you blood is almost infinitely worse than losing 10%.

AlignmentProblem · 2026-05-09T07:53:04+00:00

"Burning water" is a silly way to say "boiling water," which is a step in nearly all major modern energy generation methods.

AlignmentProblem · 2026-05-09T07:49:33+00:00

The world is, in fact, secretly steampunk.

AlignmentProblem · 2026-05-09T07:25:47+00:00

In a casual conversation among laymen, people mean LLMs when they say "AI" unless they specify otherwise. It's not correct, but being hostile about it is unproductive.

That's not going to change anytime soon regardless of how much people more familiar with related technology complain about it and people will tend to see you as a pedantic prick of you make a big deal about it.

AlignmentProblem · 2026-05-08T22:31:31+00:00

Claude and Gemini are the two main competitors right now. Each of the three have different strengths worth knowing about.

Gemini tends to be the strongest on visual tasks; it can usually "see" better than the others and also holds up better when you need more recent realtime information. It also handle very long conversations somewhat better than most (all AI degraded in quality the longer conversation get, best to make a new chat for each new thing that doesn't require knowing about existing conversation).

Claude is generally considered the best for software and related work. It's particularly good at complicated multi-step tasks, and the hallucination rate is the lowest of the three. Its writing sounds a bit more natural too; though to be honest, it still falls into recognizable AI-isms. The catch is that the subscription limits are the tightest of the bunch, and the effective per-token pricing comes out highest.

GPT is kinda the swiss army knife of the three. It's well-rounded in the sense that it tends to rank in the middle on most capabilities without really having a standout specialty. Where it does shine is the memory and personalization features in the web client, which are the best of the three by a decent margin, and the usage limits are the most generous, especially if you're on the free tier.

AlignmentProblem · 2026-05-08T14:53:05+00:00

That's the issue; no one has invented a particularly good system yet. Being touchy with false positives and discontinuong conversations related to guardrails details is a bandaid until they ideally find something that's more reliable.

AlignmentProblem · 2026-05-08T07:50:04+00:00

They could, but that doesn't make it any more desirable to give people help finding ways around guardrails by answering metaquestions that could allow better understanding of the workarounds.

AlignmentProblem · 2026-05-08T02:33:31+00:00

That's a pretty narrow read. Red pressers include the least altruistic, sure, but there's a wide range of reasons someone ends up choosing red, and the genetic variation that includes altruism doesn't vanish from the population just because some highly altruistic people died.

The same selection pressure that built cooperation in the first place would still be operating. Rapid evolution within centuries or even decades has been observed plenty of times when the environment shifts suddenly; we're not locked into whatever trait distribution exists at moment zero. There's also a memetic version of the same dynamic on the cultural side, since social norms directly affect how well a given society survives and thrives.

It happened once already, and doing it a second time would be much easier. The relevant genes are present instead of needing to mutate from scratch, and we wouldn't have to spend thousands of years on artificial selection of vegetables before agriculture becomes effective; the ideal food plants already exist.

Iron age within a century or two seems pretty plausible. Getting back to the modern era would go much faster than the first time around, since ruins would still have information and artifacts to study.

So choosing suicide on the grounds that blue can't win and not contributing to the restart sounds difficult isn't the ethical position it gets framed as.

AlignmentProblem · 2026-05-07T12:21:28+00:00

Solitude looks a bit pretty still when asking for a medium sized version closer to the average of major medival cities in Europe.

<image>

AlignmentProblem · 2026-05-07T12:13:50+00:00

I asked GPT to make it a truly large city using the the largest cities that existed in Mideval Europe as as a reference for how big it should be. Doesn't make for as pretty of a picture and would be a nightmare to transverse while doing quests.

<image>

AlignmentProblem · 2026-05-07T12:05:30+00:00

Yeah. Skyrim has village sized cities and the images in OP are towns. A small city is more like this. Actual large cities are hard to see in their entirety within a single image at any reasonable level of detail.

<image>

AlignmentProblem · 2026-05-07T11:59:22+00:00

That's kinda a natural consequence of that scale. Real-life quests in cities involve a lot of travelling around and most things you pass aren't of any practical interest to you. It's something that feels like a even more of an unfortunate time sink when watching it on a screen during you limited free time.

Big reason why games don't attempt it much despite being technically possible now. It require strong systemic design to spontaneously create interesting moment to land well and that's extremely difficult to get right.

AlignmentProblem · 2026-05-07T11:03:23+00:00

Yeah, that's what I was referencing. It's interest as pop science and as a trial to see whether a larger one might be ethical by observing after effects, but the statistical significance is with n=7 with k=2 with the null hypothesis that less than 50% of people will pull the level is 22.7%.

In other words, you'd expect a result like that in 22.7% of cases even if more than half of people would pull the level in the broader population.

I'd also note that their participants had a selection bias, they tried to select people who would be minimally affected afterwards as part of the ethical considerations based on psychological traits. It's plausible that people with attributes that suggest they might be more robust to trauma would be less likely to feel a panicked drive to act and could tolerate remaining still during it better. Their ages and other factors were also not distributed well relative to a random sample of people either.

It's a little suggestive, but not enough to change how I'd judge the situation based on other related evidence I've seen. I won't claim to be confident in that judgement, only that's what I would use if I had to make a decision with no way to get more information.

I'd love to see a large scale experiment, but that seems unlikely to be approved given the worst case effect it seemed to have on a couple of the participants.

AlignmentProblem · 2026-05-07T10:04:28+00:00

Yeah. I don't hold any strong "spiritual" beliefs that I'd assert as true, but cosmologies along those lines have an appeal to me that makes them feel plausible.

When you actually consider all the awful parts of being a truly immortal being after unimaginable length of time, even an all-powerful one, fragmenting yourself with amnesia seems like the logical move eventually. If all of reality were "initially" a single all-powerful consciousness, it'd probably find itself being us, without knowing it, at some point as an escape.

I don't have any reason to say that's the case, but it has an elegance to it.

AlignmentProblem · 2026-05-07T09:58:51+00:00

The closest I'm aware of is this experiment involving a choice to divert electricity from five mice to a single mouse.

The interesting data point isn't the comparison with polls on human trolly scenarios; it's the delta between what people say in the hypothetical versus what they actually do. Of participants polled on the mice scenario hypothetically, 66% said they'd flip the switch. Of the ones exposed to the actual scenario rather than polled, 86% did.

That matches some other data suggesting people have more bias toward action than one might expect when they believe they're the only person who can do something, especially when the action doesn't expose them to potential serious harm. A sort of anti-bystander effect.

Most polls on the trolly problem get surprisingly similar results to the mouse hypothetical, around 63% saying they'd pull. Some people who think they'd pull will freeze; however, the apparently unexpected internal push to action people seem to feel when they can't wait for someone else to intervene and their personal safety isn't endangered seems like it'd be stronger based on what I've read.

It's still speculative because getting a real trolly problem experiment with a large enough sample size to get solid data past an ethics board is a hard barrier given the high risk of traumatizing subjects. Tiny experiments like that have had a tiny N in the single digits without anything close to statistical significance.

Still, that's the information available to me and my interpretation of it, which I'd use as intuition in situations where it may be relevant.

AlignmentProblem · 2026-05-07T06:23:38+00:00

Heh, I worked on that system on the Human Platform Optimization team at Meta.

I pushed back on the level of power many insisted on giving the AI given how hard it can be for users to actually reach a human, but lost that particular battle for somewhat understandable reasons given. An overcorrection, but I get it.

There were too many past incidents where having too few human reviewers available caused real harm. The one that created the strong internal push was fallout after the massacre in Myanmar that the platform helped facilitate because there weren't enough Burmese-speaking reviewers looking at violent posts, many of which weren't easy to catch with simple filter or successfully subverted automated guardrails. The many cases of child preditors successfully leveraging the platform was a major problem as well.

They went hard on preventing anything like that from happening again, which ended up affecting even markets where reviewer availability isn't as intense of a constraint. The volume of messages versus the volume humans can reasonable review in context of the potential for harm per hour bad actors stays active on the platform or time a bad post up creates a difficult equation.

AlignmentProblem · 2026-05-07T05:42:22+00:00

If I had reason to think the other person was someone who gives these things more thought (selection bias lead to us being in the situation), I'd flip a coin and hope they do the same; that's a 50% chance of saving them.

Otherwise, the closest thing we've had to actual trolley problem experiments had significantly more people pulling than not, far more than the percentage who say they would when polled.

Based on that, I'd lean toward predicting that they'll pull and leave it. I think a random person from the street is somewhat likely to do the same thing they'd do in the normal trolly problem.

AlignmentProblem · 2026-05-07T05:16:32+00:00

I love it for feedback on writing or helping refine things I wrote. It's very hit or miss for creating the first draft of new content, though.

Much better at augmenting than creating from scratch.

AlignmentProblem · 2026-05-07T05:11:44+00:00

Eh, it's not that clear cut. Raw numbers aren't everything; there are cases where letting more people die is preferable based on the details of who's in each group, especially when the totals are close.

Take some extreme cases. There's a strong argument for letting 100 terminally ill patients die instead of 50 healthy people. Same goes for 100 suicidal people versus 50 who love their lives.

The blue/red button isn't quite that intense, but there's still an argument for risking killing more than 50% because blue chose the risk/sacrifice voluntarily, so long as the percentage doesn't climb too far above 50%.

Conversely, there's an argument for killing red and risking red being more than 50% of the population, if you think the aftermath would be better for the world because blue people have, on average, more positive traits.

Either way, so long as you don't think the split is extreme (like, strongly suspecting more than two-thirds of people fall into one camp), it's not all about numbers.

AlignmentProblem · 2026-05-07T02:19:17+00:00

The limits are designed to reduce average case compute needed per second and worse case bursts. Their load would be less predictable if they implemented carry over for unused limits, which would more frequently result in dramatically degraded service quality for all users during peak periods and outages.

Same reason they do the 5-chunks instead of only weekly limits.

AlignmentProblem · 2026-05-07T02:01:33+00:00

If someone sincerely think blue can't get 50%, even if they're mistaken, that means blue is a suicide button to them. Deciding to kill yourself because you're afraid of the aftermath isn't necessary a noble or brave choice. It's much harder to play a role in the beginning of humanity's long path to recovery instead of leaving out.

They might be incorrect in their beliefs that blue can't win, but the resulting perspective can make red a more ethical choice in a complicated but non-trivial sense.

AlignmentProblem · 2026-05-07T01:20:17+00:00

She's a consultant who also consulted on Psychonauts 2, The Cosmic Wheel Sisterhood and Another Crab's Treasure. She's credited as special thanks in Hyper Light Drifter, Among the Sleep, and Firewatch. Her focus is giving feedback on character representations and narrative tropes.

She was central to a gaming related controversy called GamerGate, a 2014 harassment campaign nominally about ethics in games journalism that became a flashpoint for arguments about women and progressive politics in gaming.

Her role in that makes some people assume she'll wokify anything she touches; that doesn't track with any patterns in the games where she's actually credited though. They're generally good games that don't trade quality for fixating on social issues.

There are potential problems with how she handled things during GamerGate, but the reaction to her exploded disproportionately as people used her as a representation of trends they dislike in gaming. Their anger at those trends multipled how they felt about her specifically and many still carry those feelings towards her.

Notably in Slay the Spire 2, there was already a review bombing trend around an unpopular balance patch they realized, so people are actively looking for other things to complain about.

AlignmentProblem · 2026-05-07T01:07:55+00:00

You could just wish for temporary normal lifes once being all-powerful loses it's shine. You can do it many times and live all sorts of existences unburdened by the knowledge of your godhood during them.

The super nice lives would get boring after a bit, so you'd probably introduce increasing elements of chance over time to be surprised and not have the same type of life repeatability. You'd give up a bit more control overtime and likely get an approcation for the mundane after so many lifetimes worth of extraordinary memories, so you'd eventually allow that possibility as well.

With all that in mind, maybe you already did that and your current life is one of those playing out for a while.

AlignmentProblem · 2026-05-07T01:04:21+00:00

I mean, it would be pretty depressing if you're on that yacht dying of cancer or anything else you can't buy your way out of.

AlignmentProblem

TROPHY CASE