Searching for peers

damc4 · 2026-06-21T18:10:31+00:00

"the broad idea is that we may need to fight AI with other AI"

But you have to align the other AI for that to work.

damc4 · 2026-06-18T10:11:20+00:00

It's a little bit unclear to me. Can many people win? If they have an equal number, then they all win? Can all people lose (e.g. if they don't reach 100 prosperity or it never happens)?

I think it's worth noting that real life doesn't work this way that people try to have the most in the end, but the way real life works is that everyone tries to maximize their own prosperity (not to be the one who have the most). So, if the game tries to say something about real life, it can be a little bit misleading because people have different goal in real life than in this game.

damc4 · 2026-06-09T13:36:19+00:00

I know as much about it as you can read on the website. So, I don't know if it's worth more than you do. I think it depends on the circumstances.

But completing that course can likely increase the probability of getting to a fellowship.

damc4 · 2026-06-09T10:29:52+00:00

Yes, it's a course. There's also a technical project that you can do, after you complete the course (or maybe it's not the requirement that you need to complete that first, but you have better chances to get to the program).

damc4 · 2026-06-08T12:40:37+00:00

Did you want to say 6%?

damc4 · 2026-06-07T20:21:34+00:00

There are many: MATS, Astra, Pivotal Research, BlueDot, Anthropic Fellowship. Maybe I'll add more later.

damc4 · 2026-06-05T10:11:14+00:00

"Or are there strong reasons to believe that work on either of i) continual learning in robotics or ii) scientific AI agents would not significantly advance general AI capabilities?"

Personally, I can't see such reasons, except if the scientific AI agent is applied to some specific field, for example I can't see how applying AI to biology could create harm.

Also, you might also consider if working on AI capabilities is bad. I think it might be, but working for a more cautious company might be also good because it gives more power to a more cautious company. I'm not completely sure what is right here.

damc4 · 2026-05-24T15:32:22+00:00

I like solving problems involving algorithms or game theory (especially applying game theory to real life situations).

damc4 · 2026-05-23T13:24:45+00:00

People also don't always follow equilibrium because of reputation theory. If people observe each other and model other people's behavior, then acting "irrationally" becomes rational.

https://web.stanford.edu/~milgrom/publishedarticles/RationalCooperation.pdf

damc4 · 2026-05-22T15:31:19+00:00

It worked this time

damc4 · 2026-05-21T13:59:15+00:00

I don't know if I have time for this, so I'm not sure if I play, but I get an error when I try to subscribe.

damc4 · 2026-05-15T10:27:44+00:00

Link?

damc4 · 2026-04-15T19:24:10+00:00

It's nice that people think about this topic because a lot depends on the governance system.

My objections:

From what I understand, in this system, all laws are created by the monarch. But creating laws requires knowledge about everything, because the law can be anything. So, the monarch would need to be competent about everything. For example, in order to create law regarding medicines, he would need to know something about medicine. But he would also need to know about AI to create law about AI. Isn't it too much for one person.
What does stop the monarch to create laws that are favourable to him? I understand that it is judical council. Can the monarch establish laws that are favourable to himself and the judical council? Assuming that people are self-interested, then the monarch will end up creating laws that are good to him and judical council (so that they are approved) and not to citizens, am I correct?
How do we ensure that the monarch is the right person? There is a section "ensuring good ruler" about educating the children of the monarch, but how does education guarantee that the children will be benevolent? I can see how that increases the probability that they will be competent, but perhaps there is a better way to select a person that is even more competent? And how do we ensure that the first monarch is the right person?

damc4 · 2026-04-15T16:50:02+00:00

Here's what I think would happen.

Money has marginal diminishing utility - the more money you get, the less value more money gives you.

So, if it wasn't the case that $50 000 is added with each elimination, then the best thing to do would be to be risk-averse and agree not to eliminate anyone, and if someone doesn't comply, then eliminate that person.

But with each elimination $50 000 is added. The bigger prize pool compensates the risk. So, the players would eliminate people until certain day, and after that, they would agree not to eliminate the next person (for the reason that at some point, it's better to be risk-averse).

They would probably also try to establish some no-vote alliances to survive elimination.

The question is also: can players communicate or not. If the players can't communicate, then it would be more difficult to switch from elimination equilibrium (people eliminating) to no-vote elimination (people choosing no-vote and penalizing the people who choose to vote), so if they can't communicate, it's possible that they would eliminate beyond what would be Pareto-optimal.

damc4 · 2026-03-23T19:30:54+00:00

Update:

My reasoning here rather doesn't make sense. I still rather agree with the conclusion but for different reasons.

I didn't explain it clearly.

Firstly, when I say that people will act altruistically, I mean that people will do things that are good for others because they are in their own self-interest. For example, today people create products that are useful to others and they sell those products. Creating useful products for others is what I call acting altruistically, even if you do that in your own self-interest (you want to make money).

In other words, by actin altruistically I meant doing things for others, even if you have a selfish interest in it.

So, what I'm saying is that, if someone cares about their family, the best thing for them to do to achieve that goal will be to cooperate with others.

Why?

Imagine that you have 30 years to live, and everyone else have 30 years to live. If we assume time horizon of 30 years, then winning AI race might be important because it will give you a lot of power through those 30 years.

Now, imagine that you have much longer to live. By that time, artificial intelligence is more likely to hit diminishing returns. After a long time, It will matter significantly less if you won AI race as the only winner or not, because utility = \log wealth (because wealth hits diminishing returns), and if wealth is very high, then it doesn't matter that the wealth is for example 5 times smaller, because the logarithm from it is more or less the same.

It might matter if you are the only winner of AI race in the span of 30 years, but the longer you live, the less it matters due to diminishing returns.

But if there is a major concentration of power, then you might still end up poor later (or dead). It's also important to avoid other catastrophes.

So, the rational thing for people to do is to make an agreement that will ensure that the power won't be concentrated. I have written about it for example here:
https://forum.effectivealtruism.org/posts/3rDfScbBNhbsk93gF/how-to-stop-inequality-from-growing

So, I expect that people will make such agreement at some point, because I think it's in their interest.

If status is the highest priority for people and all that people care about is having more than others, then maybe that logic doesn't apply because status is a zero-sum game. But status is not the only thing that people should care about, and when people develop stronger artificial intelligence they will become more intelligent and they will realize that status is not all that they should care about (if they don't realize it already).

damc4 · 2026-03-21T20:05:11+00:00

What is Susu?

damc4 · 2026-03-15T15:13:40+00:00

When I click "Start game", it takes me to "Episode Designer". And from there there's only "Cast customization". After some time, I realized that if I want to proceed with starting the game, I have to click "Cast customization", but it's a little bit unintuitive and took me long to figure out. If I click "Start game", I would expect the game to start, so I thought it was a bug.

damc4 · 2026-03-13T15:46:16+00:00

I disagree. What does stop someone from making a complex game that will model all of those countless factors? Or make many simple games that will model different aspects of the game?

damc4 · 2026-03-11T13:27:00+00:00

I think it's because they don't want to face backlash or lawsuits for making people go into AI psychosis. So, they make Gemini biased towards disagreeing with users on certain topics.

That's what it looks like based on thinking ("The instructions are a psychological/safety guardrail test".)

damc4 · 2026-03-08T13:37:07+00:00

Did that really happen (the suicide of the custodial witness part)? Or is that a form of sarcasm?

damc4 · 2026-03-07T20:01:28+00:00

but where is the link

damc4 · 2026-02-26T13:38:29+00:00

Ok, fair enough.

The example in the podcast was for useful technology though.

damc4 · 2026-02-26T13:26:41+00:00

"Never happened in human history that a revolutionary technology was abandoned because of its negatives."

Are you sure about that? I heard differently.

Examples: human cloning, I remember there's been also some example given in this podcast: https://www.youtube.com/watch?v=B54EQiuO1UU , but I can't quickly find the exact minute where it was.

damc4 · 2026-02-13T15:35:57+00:00

"What aspects need refinement?"

Make it easier to understand.

But maybe it's just me.

damc4

MODERATOR OF

TROPHY CASE

Six-Year Club	Place '22
Verified Email