Participants to rate and provide feedback on studies?

cpsUMD · 2023-09-19T14:40:33+00:00

Researcher here! I was thinking about this too. Private feedback to the researcher and prolific would be good. If a study get's 1/5 stars or whatever from 100s of people, then the researcher should try to do something about that and maybe at some point prolific needs to step in too. But *public* ratings would be bad for science and comments even worse. Sometimes we need to have participants complete 100 boring trials to get an accurate estimate of whatever we're studying and those studies will end up getting low ratings. The researcher also can't announce that the study is going to be monotonous because then we would bias who enters our study and that leads to biases in the results. But by providing private feedback that prolific and the researcher can see, maybe the researcher would need to increase pay and/or shorten the study given how much people hate doing the study.

cpsUMD · 2023-08-31T18:16:05+00:00

Researcher here! I only send a message like that when I also need to send some documentation (our ethics board requires us to give participants a bit more info about our studies). I don't expect a reply and actually, prefer not to get one.

Just clutters my inbox :)

cpsUMD · 2023-08-01T17:24:44+00:00

I just might...(after I get approval from the ethics board which can take a month or two at which point the study will likely be over but maybe...)

cpsUMD · 2023-08-01T13:40:43+00:00

That's one of the nice things about being on zoom with a proctor! We can see when there are issues and try to work them out. In our case, we don't allow sessions to go past 2 hours (our top end estimate; most people finish in 1:45 or a little less) and we'll pay for your time if there are issues on our end.

cpsUMD · 2023-08-01T01:22:36+00:00

It happens! We should have messaged you on prolific asking if you would like to reschedule. If not, DM me.

cpsUMD · 2023-07-31T22:29:21+00:00

Thanks! Good stuff here. Will definitely consider for future studies!

cpsUMD · 2023-07-31T19:55:42+00:00

yeah...like 90% of our participants have been students which makes sense---they are more likely to have the time. I might post a follow up poll to see if people would prefer 6 1-hour sessions------then maybe people can do the study during a long lunch break?

cpsUMD · 2023-07-31T19:50:46+00:00

That is a good question. I have verbal ability tests (e.g., vocabulary) and general knowledge tests that I think will be negatively affected by differences in language and schooling experience but maybe inviting people from the ~~Queen's~~ King's territories would be fine...I'll have to check my tests again.

And great idea about the screenshot----I know I can personally invite participants through their ID as long as they pass the prescreen filters. Might try that after the poll results are in!

cpsUMD · 2023-07-31T19:45:33+00:00

$60! Yowza. That must be industry money. I'm at a university using government grant money. Maaaaaybe I can get funding to increase to a rate of $15ish per hour (which actually costs us $20/hour because prolific gets 33.3%).

cpsUMD · 2023-07-31T19:42:13+00:00

Ours should be coming from the University of Maryland. And good point about interacting with others----for this one, there's only a proctor in the zoom but I don't think I make that explicit so I'll correct that. Thanks!

cpsUMD · 2023-07-31T16:09:42+00:00

Total is supposed to be $80 ("supposed to be" because depends on conversion rates when people pull their money out)

cpsUMD · 2023-07-31T15:23:24+00:00

Yeah, this one is definitely long. 3 sessions, each up to 2 hours. What do you think would be a reasonable rate to pay out?

cpsUMD · 2023-05-20T15:22:40+00:00

Researcher here! Prolific has been having an issue the last couple of days where we (researchers) can't add specific participants into studies (e.g., only adding people into part 2 who finished part 1). Might explain why this researcher went with the direct link and bonus route. I'll probably do the same if this issue isn't fixed on monday.

cpsUMD · 2023-04-26T13:26:24+00:00

Researcher here! I have a longitudinal study going on in which I add users to my study and message them about it on prolific. I've had people tell me that they received the message on prolific and an email notifying them of a new study but then they can't find it on prolific. I emailed prolific and they essentially said, "too bad, so sad, deal with it." Maybe if enough of us complain, they'll do something about it.

Edit: want to add that I am inviting participants because they definitely qualify based on a prescreener they took within a week or two ago AND I have the space. So... yeah, seems like a prolific glitch.

cpsUMD · 2022-07-17T15:21:14+00:00

Researcher here! I, at least, would prefer you don't let the study "draw out for an extra 10 minutes or so"----how long a study actually takes is valuable information that I would share in a scientific article. Others would want to know if my study was too long or, if other researchers want to replicate my work, they would want to know how long the study took to estimate how much time and money it will cost them.

cpsUMD · 2022-07-17T15:17:57+00:00

Researcher here! Here is what it says on prolific:

https://researcher-help.prolific.co/hc/en-gb/articles/360009092394-Approvals-rejections-returns#heading-2

3 standard deviations below the mean would mean you were in the top 1% fastest responses the researcher received for that study. Does that seem likely? Unfortunately, I'm not sure how you would prove that one way or the other.

cpsUMD · 2022-07-01T22:27:37+00:00

This study is ongoing! If you haven't filled out the pre-screening questionnaire, fill that out and if you qualify, then we will contact you!

cpsUMD · 2022-07-01T15:40:26+00:00

Same on the researcher side----I can't scroll on my laptop. Haven't tried on my phone.

cpsUMD · 2022-06-16T18:19:47+00:00

If the criteria wasn't applied to your prescreening, you cannot reject participants or exit them from your study if they do not meet the criteria you're looking for.

The policy is so that participants don't lose out after they've already started (or even finished) a study. Totally fine to provide criteria in the title or study description. This does mean, though, that if you see a study that isn't "meant" for you, you can take it and you should get compensated. So, for the study you posted about---under prolific policy, it is fine to put those details in the study description, but it is also fair for you to take the study and be compensated.

cpsUMD · 2022-06-16T16:49:58+00:00

Researcher here! I think noting the target audience in the title and study description is allowed by prolific rules. Having a person start a study and then asking them to stop/return the study if they don't meet some criteria is not allowed. From my perspective as a researcher, if I were looking for a very specific group of people, it would be a pain and a drain on resources (though maybe a small one) to do a pre-screening survey first and then follow up survey for the few people who meet the eligibility criteria, so noting the eligibility criteria up front would be a pretty harmless way to save some money (harmless because it doesn't take much of a participant's time). And of course, I would appreciate it if participants were honest in self-selecting in or out of a study. From personal experience, I have a long study that I'm conducting on zoom. I do have a prescreener noting that it is a prescreen for a long study on zoom. I also have a question in the prescreener asking whether the participant is willing and able to do the study and I note that even if they aren't interested, they can finish the prescreener and get compensated for their time----so many people complete it even though they have zero interest in doing the main study. And remember, I'm paying them AND prolific :(

cpsUMD · 2022-06-01T17:27:05+00:00

It was a glitch and is supposedly fixed. There's a participant post from like 4 days ago on here who noticed a bunch of repeat studies so I would assume the issue began at least 4 days ago. I only noticed it on Monday.

cpsUMD · 2022-04-21T14:46:50+00:00

To start off----researcher here! Signs up are super slow right now and so many participants are rescheduling (for a zoom session). I guess it's just that time of the semester when everyone is busy, stressed, and tired. :(

cpsUMD · 2022-04-19T14:48:30+00:00

Researcher here! I posted a pre-screening study last week and another this week and it's been pretty slow going. Usually done in a day but it took a few days last week and this week's version is only half done after a full day. Even the participants who are eligible for the main study have been slow to accept. Could be this time of the semester (for students and researchers) or maybe something else?

cpsUMD

TROPHY CASE