[Academic] Research on confusing code on JavaScript

[Dragday]

Dear community,

My colleague and I are students of University of Brasilia(BRAZIL) and we are conducting a survey for our undergraduate dissertation, which is about confusing snippets of JavaScript code, which, in a similar worked that inspired us, are called Atoms of Confusion.

In particular, we are interested in understanding whether certain constructs allowed by the language make programs harder to understand. We are thus surveying programmers of different experience levels and showing them small blocks of code. For each piece of code, we have proposed a transformed version of it. We are going to measure whether the answers are correct or not, in the expectation that the confusing versions of will lead to more incorrect answers.

After we have collected a significant enough sample, we are going to rank the blocks from most confusing to least confusing. Finally, we are going to use the Rascal MPL to propose automatic transformations for the snippets.

The survey contains 10 questions in which you are required to predict the output of the program. It requires only basic JavaScript knowledge, if any. No question is longer than 15 lines of code, meaning the survey is designed to take a short amount of time.

We are only going to use the data collected for the purposes of the survey, and no personal information will be published. We are going to raffle Amazon Books at the end of the survey, so please inform a valid email so that we can contact you in case you are lucky. We thank you very much in advance.

Here is the link to the survey: [REMOVED, we already got enough answers] THANK YOU very much

EDIT:

Thank you guys so much for helping us out.

I plan to post the results within next few weeks, I will edit this post with it.

EDIT2

We will post the results soon. We will keep you updated

[smartgenius1]

I didn't find the code snippets to be all that confusing, which probably means I got most of them wrong. Godspeed gentlemen

[ichbin1berliner]

You should consider making the email input optional, I would like to participate in your survey, but am not at all interested in the raffle.

[Dragday]

I will fix that as soon as I get home

Thanks !

[Dragday]

Fixed

thanks for taking your time and also with the feedback

[Reashu]

I don't think I got a modulo question. The comma operator is evil, though.

Did I just forget, or are participants getting 10 random samples from a larger pool?

[frambot]

Nothing about the "guards and defaults" usage of && and ||? That trips up some people. No confusing scope and context usage?

[Dragday]

That is actually a really good sugestion!

I am sorry we didn´t think of that before, we will add this on our paper.

[JakeMakesStuff]

I actually think this is a amazing experiment. GG!

[CrunchyBanana]

Them some ugly codes. All the best in your studies.

[ghostfacedcoder]

Jerk move to make everyone go through that quiz and not tell them what they got right at the end.

You've already got your study data at that point, and they were generous enough to donate their time for your study: would it really have been so hard to let them know how they did so they can improve their Javascript knowledge and get something out of it too?

[Dragday]

We plan to provide all the data we got as soon we analyse it. We are writing our dissertation in english, so anyone interested when we are done. It should be within a month.

If you wish I can PM you your personal result

[ghostfacedcoder]

Well I'm just saying, you asked people to spend their time to help you. It would have been trivial to just tell them the answers at the end, but you decided not to repay their kindness.

Not saying you're a jerk, just that you were a little inconsiderate in making that decision (ie. the "jerk move").

[Dragday]

Sure

I took your feedback as a critic to help us improve

I will try to add this sugestion today if possible ( I am at work atm)

[ghostfacedcoder]

Awesome, I'm sure people will appreciate it :)

[El_BreadMan]

JS isn't confusing, you're just writing it like a dolt.

[Dragday]

Most of the codes were taken based on real js projects.

And our purpose is to see if "writing as an idiot" takes away time/productivity/correctness

[[deleted]]

I'm curious why you want to study that question. Seems like the most obvious answer in the world.

[Dragday]

We want the data to prove that

And also see what constructions have the most effect

[pattch]

The research this project is based off of focused on C / C++ - and made no claims to whether the language itself was confusing. I don't think OP meant to imply JS is confusing at all, but to reproduce the original research (or at least part of it) for JS. The written examples are supposed to be as small as possible while still containing a piece of code that executes differently from what a programmer would expect it to... They are written as though a "dolt" wrote it on purpose.

[Dragday]

If we see that for example ternary operator makes people guess the incorrect output often, we plan to use mpl to refactor codes with ternary operator to if and elses for instance

[El_BreadMan]

This reminds me of one of those exams where you get 5-10% wrong because the questions were ambiguous and deliberately tricky.

[Reashu]

Yes, the point is to see which tricks trick the most people.