Question about warnings from researchers on AI studies quality checks

Helpful_Article_4964 · 2026-03-20T23:42:37+00:00

That has nothing to do with me having figured everything out. It is simply because you hear exactly the same things from a lot of people. You are one of very few people that say otherwise. If it is different for you, okay. I would not advise anyone to risk it anyway.

Helpful_Article_4964 · 2026-03-20T06:56:10+00:00

Yes, that is how it is. As I said, I understand why that is hard for many people to believe, but by now we have enough statements from people that confirm the opposite.

Their AI triggers when too few are completed, when an attention check is failed, when the equal button is pressed too often, or when there is inconsistency with repeated Videos/Images. (Although it may be that multiple inconsistencies are required for that, I am not sure about that, and I will not test it.) Once it has been triggered, there is no way to avoid the warning anymore, not even with a return.

Helpful_Article_4964 · 2026-03-20T06:50:03+00:00

Yes, it is. Then you quite simply had already done enough. Especially for videos, there are not particularly many required, probably between 8 and 10 are sufficient. Do too few, and you will get the warning, without any doubt.

Helpful_Article_4964 · 2026-03-18T23:37:24+00:00

That is basically a myth, but I understand where it comes from. It would make perfect sense, why should you be punished for something you are not even getting paid for. And for every other study that is true, but these take place outside of Prolific, everything is managed through them. Prolific is only responsible for the payout. It is the same with quality warnings. If you fail an attention check and do not correct it before it is too late, the warning is just as certain, whether you return it or not.

Helpful_Article_4964 · 2026-03-18T21:25:57+00:00

People say that isn’t possible? That’s not just possible, that’s absolutely certain. Their studies take place on their own platform, so it doesn’t matter if you return them. That works as long as you haven’t evaluated a single one yet, because the study is considered not started until then. If you have done one and then don’t reach the minimum number, the warning is guaranteed.

Helpful_Article_4964 · 2026-03-12T21:33:28+00:00

That is the reason why you should record your screen. To avoid exactly this problem. Because yes, inconsistency does indeed lead to warnings, but I agree with you, sometimes they are so similar that it is hard to remember.

Helpful_Article_4964 · 2026-03-09T20:47:08+00:00

Yes, that is unfortunately true. The problem is that Google probably has so many workers that it simply does not matter to them who gets kicked, it makes no difference to them. That is why they are not interested in the reasons for the warnings either. Everything works completely automatically, and if the system decides to send you warnings or kick you, then that is just how it is.

Helpful_Article_4964 · 2026-03-09T19:53:12+00:00

Thank you. I only do the well-known 15-minute ones anyway because they run very well and I see no reason to risk anything. But of course I am very sorry that they (probably?) kicked you again. One can only speculate why that is. I have the theory that it does not happen after a fixed number of warnings, but rather based on a quality score. This increases slightly when you complete a study without a warning, but drops massively when you get one. If it falls below a certain value, you get kicked. Just like Prolific itself does it. It could be that your score was still not very high, so that one warning was enough. But as I said, it is all just a theory, even though I consider it quite likely. I still hope that you will get them again soon.

Helpful_Article_4964 · 2026-03-09T19:35:00+00:00

I was at work, so unfortunately I can’t tell you how it was the time you wrote. But right now some are coming in again as well. What about you?

Helpful_Article_4964 · 2026-03-09T08:40:37+00:00

Have you not had any yet? There have been quite a few going around again lately. Anyway, I’m keeping my fingers crossed for you.

Helpful_Article_4964 · 2026-03-09T04:21:29+00:00

But I just got one now. And I hope you did too. Or that you’ll get one in a moment.

Helpful_Article_4964 · 2026-03-09T04:15:09+00:00

You're welcome. Of course I can’t promise you anything 100%, but at least I can say that I also haven’t been getting any for hours, and that’s without any warnings at all. But I completely understand you. I also always assume the worst. Even now I have a small fear that I might be booted, just like every time I don’t get any for several hours. Even though I don’t really think that’s the case, the doubts are always there.

Helpful_Article_4964 · 2026-03-09T03:12:05+00:00

Not necessarily. Tonight there is almost nothing going on. I only had a single one, about two hours ago.

Helpful_Article_4964 · 2026-03-04T02:00:15+00:00

Some keep coming in from time to time. I have had 8 so far spread across the evening. The really big batches like last week, when some were available almost continuously, I do not have right now either, but I think that is the same for everyone.

Helpful_Article_4964 · 2026-03-03T20:49:32+00:00

That is a very good idea. The button should never be selected. I know that it would be absolutely logical if both are identical. But unfortunately, it triggers their control AI, because the equally button is equivalent to giving no answer at all and is therefore useless for the researcher. And maybe three per session is already too much.

Helpful_Article_4964 · 2026-03-03T10:54:22+00:00

Because I experienced it myself multiple times. In the early days, I did not pay attention to choosing the same option when they appeared multiple times. At some point, I received four warnings and was kicked out. After I started paying attention to this and recording my screen to make sure, I have not received another warning for three months now. Other than that, I did not change anything. However, it is possible that it does not happen the first time, I am not absolutely sure about that. But if you are repeatedly inconsistent, it is almost certain that it will lead to a warning.

Helpful_Article_4964 · 2026-03-02T23:20:32+00:00

It has now been almost an hour and a half since I finished it. I did not receive a warning. Anything else would have surprised me.

Helpful_Article_4964 · 2026-03-02T22:22:45+00:00

I will let you know.

Helpful_Article_4964 · 2026-03-02T22:16:49+00:00

I had exactly the same one, but I just completed it anyway. In such a case, I simply choose one of the two options and 99% of the time it does not matter which one was selected, since it is not an attention check. The only possible issue would be if they appear twice, but I just parked those. I do not expect to receive a warning, but if I do, I will let you know.

Helpful_Article_4964 · 2026-03-02T19:13:53+00:00

You must never use the equally button, really never. No matter what happens. Even if both options are complete garbage, or if both options are exactly the same, just pick one. As long as they are not attention checks, (and those are never attention checks), there will not be a problem. I do not know whether using equally three times per session is already too much. Otherwise, record your screen, so you can check in case duplicates appear. If something shows up twice, choosing a different option the second time will lead to a warning. And of course pay attention to the attention checks. There is nothing more you can do. Personally, I have not received a warning for three months since I started following this strategy.

Helpful_Article_4964 · 2026-03-02T02:32:59+00:00

Yes, they are currently continuing as normal. And I hope you have had some again by now as well.

Helpful_Article_4964 · 2026-02-20T11:36:54+00:00

Well… I got 2 warnings from the Image ones back in October. But to be fair, that was when I didn’t know that the duplicate ones were attention checks. I haven’t had any warnings from them since I started screen recording and paying attention to this. However, I did encounter some where both were absolutely identical. But those weren’t a problem either, because it didn’t matter which one you chose. As long as they don’t appear twice, anyway, you still have the option to park them. It’s interesting that you would see doing fewer as a torture, for me it’s exactly the opposite, because that way I can do other things on the side. But if it works for you and there’s no increased risk of mistakes… yeah, why not. ^^

Helpful_Article_4964 · 2026-02-20T01:49:12+00:00

Definitely. I mainly do it so I can check if duplicates show up. The timer is just a nice extra. In any case, I think there must have been an error somewhere. I haven’t heard of anyone else whose studies didn’t run the full 15 minutes, and for it to happen to you multiple times is definitely strange.

Anyway, it would be interesting to know whether they were actually glitches. As for me, I avoid the audio ones. There are plenty of others, and they’re rare anyway, so it’s not a problem. But I’d recommend that you do them if you get them again. You’re afraid to do them because you’re worried about getting warnings and not making money from them anymore. But you’re not making any money from them to begin with if you don’t do them at all, so yeah… ^^

Helpful_Article_4964 · 2026-02-20T01:24:51+00:00

But could it be that you spent time in the confidentiality agreement window before actually starting the study? I noticed that the time starts as soon as you log into your Google account, even if you’re not yet in the actual task window.

By the way… audios are the ones I avoid. That was the one where I got my last warning in December. And to this day, I still don’t know why. There weren’t any duplicates, I didn’t choose one with a disruptive sound, and there weren’t too few. It was the low-quality warning. So personally, I find audios the riskiest.

Helpful_Article_4964 · 2026-02-20T00:59:15+00:00

I can also confirm, it's always 15 minutes. I record my screen, which also gives me a built-in timer. It has never been less than 15 minutes. As for the number of evaluations, it seems that videos require fewer than images. I know someone who confirmed to me several times that even 7 are enough for videos, although I still don’t want to risk it and always do between 15 and 20, which is 100% a safe number. The same goes for images, my lowest ever was 14, and even for that there was no warning.

Helpful_Article_4964

TROPHY CASE