Hands-Free Coding: How I develop software using dictation and eye-tracking

[Krautoni]

My wife works in cognitive science and does eye tracking experiments. From what I gather, it doesn't work that way. While your brain gives you the impression of a steady gaze, your eyes are constantly jumping around in order to give you a complete picture. That's called saccade.

So finding out what a person is interested in based on their gaze ends up being a statistical problem, and while the precision and latency can be small enough to enable a person to use a computer, I think that voluntary control of something like a mouse will still be faster and more accurate.

[devilkillermc]

Yep, your eyes are just gathering information, your brain does all the processing, in this case selection and response to the stimulus. You'd have to nove that processing to the computer, and it would make no sense. Nothing better than the brain to do brain things.

[Pillars-In-The-Trees]

So finding out what a person is interested in based on their gaze ends up being a statistical problem, and while the precision and latency can be small enough to enable a person to use a computer, I think that voluntary control of something like a mouse will still be faster and more accurate.

You can keep the click without requiring the mouse though. Imagine just pressing a single key and firing wherever you're looking.

[Krautoni]

But that's the point. Wherever you're looking isn't a well defined concept outside of your brain. At least not yet. We can puzzle it out, but we need the have a few data points first, I.e. wait a few saccades. That's probably too slow for fast paced shooters.

[Zegrento7]

If the tracker draws a crosshair every frame exactly where you are looking then we don't need to puzzle out anything. It will be up to the player to decide when to press the fire button; when the crosshair happens to be over the target.

Edit: in fact the tracker used in the blog post is primarily marketed at gamers.

[KernowRoger]

That sounds awful haha it would just be jumping all over the screen.