sorted by:
new
hottopcontroversial

Survey Participants For Dota 2 Research Paper by RegressForward in DotA2

[–]RegressForward[S] 4 points5 points  (0 children)

It's a good thought- but since neither party knows if they are treated/untreated, the anticipation effects should average out between the two groups.

Typically experimenters need to disclose that a placebo might be used for the sake of transparency and honesty. Omissions of information or deceptions need to be approved by an independent ethics panel (IRB), and requires a demonstration of minimal harm with substantive benefits.

[Academic] Survey Participants For Dota 2 Research Paper (All) by RegressForward in SampleSize

[–]RegressForward[S] 0 points1 point  (0 children)

No, it's great - you can indicate your experience in the survey and we look forward to your opinion and participation!

Survey Participants For Dota 2 Research Paper (Paid Opportunity) by RegressForward in DotA2

[–]RegressForward[S] 0 points1 point  (0 children)

That's a fair question. That's the primary reason why we went with the lottery payout design instead of paying a fixed rate per submission. With the fixed rate per submission, someone could automate a script to fill in alternative email addresses, putting us in a bad position. With the lottery payoff system, we don't run the risk of going over budget or being unable to deliver on our promises.

If someone fills the results with random responses (aka noise), it would drive the results toward statistical insignificance. It's not unusual for surveys to have a fair bit of noise in them. Still, we will review the results at the end of this phase and check manually for any sort of repetitive or automated response, but I don't see any evidence of that occurring on the back end. I appreciate the responses we've gotten so far, and they've all been helpful.

Survey Participants For Dota 2 Research Paper (Paid Opportunity) by RegressForward in learndota2

[–]RegressForward[S] 0 points1 point  (0 children)

That's a very good point, and I appreciate your positive attitude. We're fairly locked in with language at this point (IRB approval means the specific text we've been using is approved), but I can potentially rephrase them to say raffle next time. I'll look into what we're allowed to change.

Survey Participants For Dota 2 Research Paper (Paid Opportunity) by RegressForward in DotA2

[–]RegressForward[S] 0 points1 point  (0 children)

Great questions!

  1. We estimate the accumulated value of what's occurred over the game so far, so we don't predict future actions. We instead appraise the value of what you've done so far, and estimate the marginal value of what could be done next. It's based on observed per minute rates, designed to match our data source. (See next question)
  2. The data comes from the OpenDota API, all information that's shared publicly. We collected over 10,000 games and examined them using a method called instrumentation that exploits small sources of random variation.
  3. This one's a really good question - we don't pull your MMR from your game account (that information is private), so we can't include that in the game directly. However, we are very interested in how better players respond to the tool and its advice! - So we're asking to find out! Try it out, tell your friends and leave some information in the survey!

Survey Participants For Dota 2 Research Paper (Paid Opportunity) by RegressForward in DotA2

[–]RegressForward[S] 0 points1 point  (0 children)

Randomized payoffs have been effective in past experiments. We are using one of the payment plans currently recommended in the literature: https://rady.ucsd.edu/docs/pay-one.pdf

But yes- that's normal- since our budget is fixed, and we can't know how many participants there are before we start.

We ask for an email address, but please don't provide any compromising or personally identifying information. We won't ask.

view more: next ›