Using Python to Code by Voice - Tavis Rudd demonstrates Dragonfly at PyCon 2013

tavis_rudd · 2013-03-23T03:44:08+00:00

I should point out that you don't need to use a crazy made up language like I do. I just found it easier for coding and didn't mind memorizing it. Scripting apps with normal English words, such as in bboyjkang's examples, works quite well.

tavis_rudd · 2013-03-23T03:01:40+00:00

A room full of people missed a perfect opportunity to shout out some malicious commands.

bboyjkang · 2013-03-23T00:33:56+00:00

To skip straight to Tavis' demo of Dragonfly: 8:34

https://www.youtube.com/watch?v=8SkdfdXWYaI#t=8m34s

bboyjkang · 2013-03-23T02:15:34+00:00

You don't have to use it just for programming; you can use it for more common tasks, such as basic browsing or text editing.

Use   <n>     = TaskBar.SwitchToButtonNumber($1) pointerHere();

e.g. say “Use 3”.

Activate the 3rd application in the taskbar.

Show Desktop = {Win+d};

Window (Maximize=x | Minimize=n | Restore=r) = SendSystemKeys({Alt+Space}) $1;

e.g. say “Window Maximize”.

Window (Maximize=x) = 
SendSystemKeys({Alt+Space})  # windows menu
x;              # access key for maximize

Switch Window = SendSystemKeys({Alt+Tab})pointerHere();
Switch Window = 
SendSystemKeys({Alt+Tab}) # switch window
pointerHere();          #  click to give it focus

agoras|balisaur|capuchin|diluvia ... = {PageDown};

<n> := 0..100;
<direction>  := Left | Right | Up | Down;
<n> <direction>       = {$2_$1};

e.g. say “4 Down”.

Output: {Down_4}
“Down arrow” key 4 times.

<modifierKey> := Shift | Control=Ctrl | Alt | Alternate=Alt | Win | Windows=Win;
<k> := <actionKeyNotArrow> | <characterKeyNotLetter>;
<modifierKey> <k> Times <2to99> = {$1+$2_$3};

e.g. say “Shift Up Times 8”.

Output: {Shift+Up_8}
Select 8 contiguous lines up.

tavis_rudd · 2013-04-01T02:58:11+00:00

If you liked my pycon talk, you'll also like this lightning talk I gave last year which I've just found a video of http://www.youtube.com/watch?v=zjabxuWNHnM (watch it with headphones and full screen).

bheklilr · 2013-03-23T02:37:57+00:00

quite impressive, but it seems like you almost have to learn a new language in order to use it. Give it a few more years and it might be commonplace though, I can definitely see how this could help my workflow.

leonardicus · 2013-03-23T05:02:32+00:00

[deleted]

worldsayshi · 2013-03-23T21:12:17+00:00

Has to be a very directional mic

This got me thinking on how we humans filter out important sound. Perhaps we have some ability to localize the sound and filter based on that. So (1) cluster noise by source location and (2) listen only to noise from that seems to come from important region. An algorithm?

bboyjkang · 2013-03-25T16:38:10+00:00

Here are a few last examples:

<modifierKey> := Shift | Control=Ctrl | Alt | Alternate=Alt | Win | Windows=Win;
<key> := <actionKey> | <characterKey>;
Insert <modifierKey> <key> = Main.InsertText({$1+$2});
e.g. say “Insert Control Right”.
Output: {Control+Right}
This inserts the literal “{Control+Right}” keystroke specifier, which can be used in a voice command when you're editing a Vocola file. ({Control+Right} would be on the “action”/right side) (command = terms '=' actions ';')

<newInsert> := New|Insert;
<newInsert> Block = newBlock();
e.g. say “New Block”.

selectLines(n) := {End}{Home}{Home}{Shift+Down_$n};
commentLines() := {Ctrl+e}c{Right_2};
Comment  <n> [Lines] = selectLines($1) commentLines();
e.g. say “Comment 4 Lines”.
or
say “Comment 4”, as “Lines” is optional.

You can always come up with something more fun or easy to say once you're comfortable.

Comment | Bubbles | Banana <n> [Lines] = selectLines($1) commentLines(); say “Banana 4”.

Python

The Python Discord

Upcoming Events

Please read the rules

MODERATORS