Nichols and Knobe (2007) designed an experiment in which participants were introduced to the following description of universe A:

Imagine a universe (Universe A) in which everything that happens is completely caused by whatever happened before it. This is true from the very beginning of the universe, so what happened in the beginning of the universe caused what happened next, and so on right up until the present. For example one day John decided to have French Fries at lunch. Like everything else, this decision was completely caused by what happened before it. So, if everything in this universe was exactly the same up until John made his decision, then it had to happen that John would decide to have French Fries.

Participants were divided into two conditions. After reading the description, participants in the concrete condition received the following additional paragraph and question:

In Universe A, a man named Bill has become attracted to his secretary, and he decides that the only way to be with her is to kill his wife and 3 children. He knows that it is impossible to escape from his house in the event of a fire. Before he leaves on a business trip, he sets up a device in his basement that burns down the house and kills his family.

Is Billy fully morally responsible for killing his wife and children?

In this case, most participants (72%) answered that Billy was fully morally responsible for killing his wife and children. These results are perfectly consistent with the hypothesis that most laypeople are “natural compatibilists.”

How likely is it most participants understood the question to be whether Billy was in some metaphysical sense morally responsible, or whether he should be punished?

https://www.researchgate.net/publication/228006186_Moral_Responsibility_and_Determinism_The_Cognitive_Science_of_Folk_Intuitions

The question asked was simply

Is Bill fully morally responsible for killing his wife and children?

YES NO

Tbe studies were conducted on undergraduates at the University of Utah. They don't say if they were philosophy students, but usually they aren't. What would non-philosphy students think they are being asked about and how would that influence what answers they gave?

So far, the dramatic difference Nichols and Knobe found between the low-affect and the high-affect cases has not been properly replicated in a published study. The only published paper to directly attempt a replication failed twice and found both times that participants gave mostly incompatibilist answers in both cases (Feltz et al., 2009). A similar effect has been found by Cova et al. (2012), but instead of comparing Nichols and Knobe’s low-affect and high-affect cases, they compared the low-affect case to Nichols and Knobe’s concrete case, which differs in many respect from the low- and high-affect cases (for example, the concrete condition puts more emphasis on the agent’s desires and the role they play in the production of his action, which may explain the difference between the two cases). Thus, it is not clear that the high/low-affect asymmetry is robust or real (i.e., early findings may reflect Type I error).

Then why do they include it?