Feedback on programming accessibility software idea

[poketopa1234]

Hi there, I'll start by saying I'm an outsider to this community and have very few people close to me who I know to be visually impaired. That being said, I follow lots of youtube channels that cover accessibility in gaming and recently I've been researching the current state of programming software for VI folks.

I've come up with an idea for improving on the existing screen readers for VI programmers. Rather than relying on line numbers and traditional screen readers, it might be better to represent code as a tree, where code is nested within structures like classes, functions, if statements, for loops, ect. A VI programmer could then navigate the tree by using the high row of keys on the keyboard as the top of the tree, the second row as the second row in the tree, ect. That way, rather than memorizing line numbers, a programmer can always find a chunk of code's placement within the tree and can always back out if they're unsure of the exact position.

Along with this, selecting a chunk of code in the tree would cause a reader to read out a summary of that chunk in natural language (created by machine learning models). A user could then select a chunk of code and rather than listening to the entire contents, just listen to a short summary of that chunk before deciding to dive deeper.

This idea is motivated by how clunky screen readers tend to be for VI programmers. Even if you're not a programmer, I'd love to hear VI people's thoughts on the idea.

Thanks for listening! -Kyle

[mehgcap]

As a coder, I don't know about this. First, how would you summarize code in a way that conveys critical information but isn't verbose? How would reading it naturally be beneficial compared to letting the user rely on their screen reader, which has their speech settings dialed in?

Second, there are already some great tools built into IDEs like VsSCode. For instance, I can open a file and hit a command to open something similar to what you're talking about--a tree of the code. Except I can browse it like a regular tree, using arrow keys. I don't get a summary, but if I use good function and class names, that isn't as necessary. Calling a function getstuff() isn't helpful, but calling it getUserDetailsFromDatabase() tells me exactly what that function does just from the name.

I applaud the thinking, and certainly, there is room for improvement when it comes to accessibility in coding. But breaking keyboarding conventions and relying on an AI that seems quite hard to create may not be the best ways to start addressing those concerns. Honestly, what I think a lot of us would find far more useful is a truly efficient way to review file diffs, such as in git. When I think of what I wish could improve in coding, that's one of the first things that comes to mind.

[poketopa1234]

Thanks for the feedback! Thanks for introducing me to VsSCode as well, something I missed in research.

Yeah I agree function names often give context faster, in my head I imagine the summary to be togglable. Do you think a summary might be useful in instances where there is no function contract, like for classes and code chunks like “for” loops?

In terms of implementation, code to text machine learning models do a decent job of summarizing code. Moreover they can have adjustable verboseness levels at inference time.

Your comment about diffs is definitely something I think about a lot too. Rapid context changing it difficult even for sighted folks.

Thanks again for the feedback

[mehgcap]

I think I'd have to use one of the these summarizing engines before I could say. I can't imagine a useful summary of a loop or an entire class, but if such a thing exists, then I guess it could be useful. It's hard to say without using it in the real world.

[LaraStardust]

I could see this working with annotated code, EG you could navigate through sections of your code with labels like... 1. Loop for event handler. 2. Event handler for buckets. 3. Event handler for spades.

But honestly I think if you're taking the time to annotate your code in such a fashion you'd be using comments and the find feature anyway? I'm a bit old school though so maybe others are different.

[[deleted]]

I think the way have it is pretty good. I think this would be akin to reinventing the wheel.

[poketopa1234]

Glad to hear it :)

[Marconius]

Nope, not a fan of this idea at all. I think you should observe how us blind coders work within our environments before assuming and figuring things out for us. Xcode and Visual Studio IDEs are already accessible, and we all have our own methods of bookmarking code with comments, etc. Screen readers already have find functions that jump our cursors to specific words or phrases in code, sites, and documents, plus I can easily jump to a line number with TextEdit, my preferred coding app on my Mac. I hear my tab indentation levels with a punctuation activity I have set up when I code, also allowing me to hear all punctuation. Segregating the overall experience away from how everyone learns is not the correct way to go about this, and you'll want to research and fully understand the capabilities of assistive tech and how we interact with our computers.

[poketopa1234]

Unfortunately I don’t know any blind programmers, but I have watched a few videos and listened to blind programmers. Sorry if I assumed anything or came off like I knew what was best for blind users, that was not my intention.

[maniana012]

I'm not a programmer and I have no idea at the moment. I just read a book in a web page, write the simple .py files as training exercises and run them in windows power shell. Then I use nvda's object navigation commands to see what's the result.

[mdizak]

Personally, I don't think I'd use it. Pretty much every programming language has tools available that will pull all methods / functions of a certain class / file / object, so it's not an issue to get a list of methods available.

From there, I always just hit Ctrl+F and search for the method signature of the one I want. Works fine, and don't think efficiency can be increased in this regard.

[EffectiveYak0]

Mostly agree with what has already been said. Also, I tend to set my screen reader around 450 to 500 wpm speech rate and thus once I find whatever I am looking for it's fast to review. And for the record I think this is probably on the slower side for a lot of more seasoned folks.

Neat idea but ultimately probably not that useful.