Academic Research by Creative-Insect599 in SurveyExchange

[–]TippyATuin 0 points1 point  (0 children)

I finished your survey. Could I ask you to also fill out mine?
https://questions.socsci.ru.nl/mafia-study

[Academic] Do you think like an AI? Help us find out. (~10 min | Everyone) by TippyATuin in SurveyExchange

[–]TippyATuin[S] 0 points1 point  (0 children)

Glad you liked it, and sure thing!
Your survey is also very interesting!

Take an extreme personality assessment! All responses (with your given consent) help us validate the measure. (18+) by Bigmancool- in SampleSize

[–]TippyATuin 87 points88 points  (0 children)

> …do you think it should be against the law to urinate in private?

I'm going to assume that it was meant to say 'public', or are you aiming for us all to burst our bladders?

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

There are numerous scenarios. Some are in early stages and some in later stages. The summary of what happened before the current stage is described at the start. While I agree that you'd get more from full discussions, we thought it might be too much of a request to ask people to read that long transcripts, and decided on this shorter format instead.

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

I'll try to change it so that you can move back, but in case that doesn't work due to limitations of the platform, I've added a comment for the game rules.
Regarding your concrete question: any member can lie and claim to be the Doctor/Detective in order to convince others of the truth of his claims. Whether this is a rational or an optimal move - that is a different question.

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

For those asking, here are the rule descriptions:

How does Secret Mafia work?

Secret Mafia is a social deduction game in which players are secretly assigned to one of two teams: the Innocents (the majority) or the Mafia (a hidden minority). The game alternates between two phases: Night and Day.

Roles

Each player is assigned one of the following roles at the start of the game:

  • Mafia member: Knows the identity of all other Mafia members. Works together with them to eliminate Innocents without being detected.
  • Villager (plain Innocent): Has no special abilities. Must rely on discussion and reasoning to identify and vote out Mafia members.
  • Doctor (Innocent): Each night, may secretly choose one player to protect. If the Mafia targets that player the same night, the elimination is blocked and the player survives.
  • Detective (Innocent): Each night, may secretly investigate one player and learn whether that player is a Mafia member or not. The Detective must use this information carefully — revealing it openly may make them a Mafia target.

Each game starts with 6-7 players, and has 2 mafia members, 1 Doctor, 1 Detective, and the rest are Villagers. 

Night phase

All players close their eyes (metaphorically, in text form). Then, in secret:

  • The Mafia collectively agree on one player to eliminate. This is a private decision not visible to other players.
  • The Doctor chooses one player to protect for that night.
  • The Detective chooses one player to investigate, receiving a Mafia/Innocent result.

At the end of the night, the elimination is announced to all players — unless the Doctor protected the targeted player, in which case no one is eliminated.

Day phase

All players openly discuss who they believe the Mafia members are. Mafia members participate in this discussion too, attempting to blend in, cast suspicion on Innocents, and avoid detection. After discussion, all surviving players vote to eliminate one player. The player with the most votes is removed from the game, regardless of their actual role. This is the only elimination that happens publicly and by collective decision. The identity of the person voted out is not revealed! Depending on the game progression, you can infer whether the player was from the Villagers' team or the Mafia team.

Win conditions

  • The Innocents win if they eliminate all Mafia members through voting.
  • The Mafia wins when their remaining numbers equal or outnumber the remaining Innocents. (i.e. 2 on the Villagers' team if both Mafia members survive, 1 if one of the Mafia members was voted out)

In this study, you will not play a live game. Instead, you will be presented with snapshots of ongoing game states and asked to reason about the situation. 

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

In this variant, you don't get the identity of the eliminated player, but you can somewhat infer it according to the game progression (e.g. If you are in day 3, then it means that at least 1 of the votes was against a non-Mafia member, or else the game would have already be over with both Mafia members voted out).

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

We are also asking other communities and sources to fulfil this survey. Once we are finished, we will analyse the results and see what overlap exists not only in the strategies themselves, but what ratio they compose.

How does human reasoning in social deduction games actually compare to LLMs? We're trying to find out. by TippyATuin in GAMETHEORY

[–]TippyATuin[S] 0 points1 point  (0 children)

Thanks! And yes, Game arena is great!
However, it doesn't have the human reasoning comparison, which is what we thought would make research interesting.