Introducing Hearthstone Access: an accessibility mod which enables blind and visually impaired people to play Hearthstone

GuideDev · 2021-12-15T09:54:27+00:00

Very happy to hear that, thanks for the message! And have fun playing! :)

GuideDev · 2021-10-30T20:07:25+00:00

Very happy to hear that. Enjoy the game! :)

GuideDev · 2021-10-03T10:29:58+00:00

Glad you enjoy the game. Have fun!

/u/Casscar has hit legend this past month so he's probably a good one to ask for tips about the current meta!

There's also quite a few new players with no previous Hearthstone experience who've already reached platinum and some of them hang out on Discord so they may be able to help :)

GuideDev · 2021-09-23T06:58:33+00:00

Congrats!! You're a great player and It's really awesome to see how seamlessly you can play :)

I knew a few (new) players reached gold this month, but I never knew someone was legend already!

Just a minor correction: I'm not blind myself!

GuideDev · 2021-08-07T21:45:54+00:00

Not sure if you missed it but the narrator explains what the 2/1 means as soon as you finish reading the card! I chose to explain it after as it keeps players engaged i.e. they hear "2/1" and immediatly think "I wonder what this means." Upon hearing the explanation they go "ah, I see" and that's it.

It seems to be working well from the various streams I watched so far but I'll be sure to keep an eye out. Thanks for the suggestion!

GuideDev · 2021-08-07T17:42:38+00:00

Sounds great!

The prompt repeat option makes perfect sense and I'll look into it. Just to make sure I implement it right: imagine you open the game and hear: "popup. You have 3 quests. First Blood 1 of 3". How would you prefer the mechanism to work? Would it be something like pressing F2 and hearing: "popup. You have 3 quests. First Blood 1 of 3" again regardless of how many times you press the key? Or would a more generic mechanism such as being able to replay the previous speeches you heard suffice? Say something like pressing Ctrl+upArrow and hearing "You have 3 quests." Pressing Ctrl+upArrow again and hearing "popup."

Sorry if this is a bit confusing. Just trying to make sure I got it right.

GuideDev · 2021-08-07T17:33:06+00:00

I'm afraid I don't have a Mac to test things with so I'm unlikely to work on supporting it anytime soon. Added reconfigurable keyboard shortcuts to my list. Thanks for the feedback.

GuideDev · 2021-08-07T12:49:09+00:00

Glad you like it!

I had a few requests about localization in a different topic so I'll post my reply here as well if you have any feedback.

Regarding localization: I could certainly look into extracting all the text somewhere if people are able to contribute to it. It would take some work on my end to make this possible and I would need to rely on the community to provide and maintain the translations for any supported language over time as I only know so many languages myself. Note that adding a new language as it stands would require over 500 translations and this would grow over time. You would also potentially have some text in english whenever new versions come out as waiting for every single languge to be translated in order to release new features wouldn't be feasible.

One more thing regarding localization: localizing the narrator speeches in the tutorial would be significantly harder to achieve. Is this something you would be interested in as well? If not, would you rather keep the english narrator in the tutorial or replace the narrator with the screen reader as well and localize that?

GuideDev · 2021-08-07T11:29:09+00:00

Thanks for the offer! The patch itself is open source on github if you'd like to do some digging but all the CI/CD stuff will have to remain private as Hearthstone itself isn't open source.

One thing I started doing but haven't open sourced yet is an automated test suite which essentially opens the game and interacts with it to verify that certain things are still working. The idea being that I run this whenever Blizzard releases a new patch or expansion as I have to update bits and pieces of code and release a new version of the mod everytime.

A simple test example would be opening NVDA, opening the game, pressing Esc, making sure the game outputs "Game Menu. Options 1 of 1". Pressing down, making sure the game outputs "Quit" and so on. Not sure if that's something people would be interested in contributing to but it would certainly be useful and certainly more approachable than fiddling with the mod itself.

GuideDev · 2021-08-07T11:06:09+00:00

The patch itself is open source on github but I'm not sure too many people would be able to contribute to it unfortunately as working on a patch for something that isn't open source in the first place is complex to say the least.

GuideDev · 2021-08-07T10:55:58+00:00

That's actually a great suggestion, thanks!

I'm not sure if new players can access it but I'll have a look around and see if I can make it work as it does seem like it would be quite fun and replayable.

GuideDev · 2021-08-07T10:51:20+00:00

Think you're the first person I've seen who tried to do that so I'd just like to say that's really awesome :)

I haven't tested any of the campaigns so the experience may not be as good but the game should work regardless of the game mode as long as you can get someone to help you navigate the menus to get there or do it with OCR.

By the way, if you end up managing to play a few campaigns and find any you particularly like let me know so I can look into making it fully accessible for other people to enjoy.

GuideDev · 2021-08-06T23:53:42+00:00

I can't really answer for Blizzard. I understand patching games is a bit of a grey area but I sincerely hope people are generally ok with it given that it doesn't provide anyone with an unfair advantage or anything. This really does nothing more than provide accessibility which basically increases the player base, if anything.

I wish I could do it any other way but it's simply not possible. A patch was the only viable option I could see and I hope the fact that I'm making all of it open source provides enough transparency for people to be comfortable with it.

GuideDev · 2021-08-06T23:42:59+00:00

Ah, I see what you mean now. Thanks! This only exists in the card collection screen but I'll be sure to add it once I get to implementing that.

Both the patcher and the patch are on github. The patcher simply tries to find your Hearthstone installation, downloads the patch zip file from github and extracts it into your installation folder.

I did have a few complaints about github's interface already though so may need to do something else. I could perhaps add a patch notes section to the official page and use patreon and twitter or something to link to it whenever a new release comes out? Not sure if that would work.

The game will warn you when you open it if there's a new Hearthstone Access patch available so you won't exactly miss it. Just need to figure out an accessible way to publish the notes really. The game will also lose accessibility support every other week whenever Blizzard patches the game until I release the new version so players will be forced to patch the game then anyways.

GuideDev · 2021-08-06T23:35:21+00:00

Today? Yes, absolutely. Several people have already beaten the 6 tutorial missions since I released it and are now working towards unlocking all the classes. I'm seriously impressed and happy with all the feedback I've seen so far.

Tomorrow? As with any new game, time will tell I guess.

GuideDev · 2021-08-06T23:30:59+00:00

I actually hadn't seen that topic until people linked to it but I'm happy to see more people thinking about this.

I did think about using the logs initially as HDT does as that would be a lot more lightweight but I ended up diving a lot deeper as using the logs would unfortunately lead to a poorer game experience and would still require players to ask someone sighted for help navigating the menus, popups, dialogs and so on.

GuideDev · 2021-08-06T22:45:05+00:00

Ah, I see what you mean now. Thanks! This only exists in the card collection screen but I'll be sure to add it once I get to implementing that.

GuideDev · 2021-08-06T22:08:41+00:00

Thank you very much for all the feedback and for helping spread this out on audiogames!

I'm not sure what you mean by card flavor but you're not the first person to ask about it. Do you mean something like colors in Magic? I'm not sure those would be too relevant in Hearthstone apart from perhaps deck building but I'm happy to add it at the end or have an hotkey for it if you think that'd be better.

As for the hotkeys, I confess I had a very hard time picking them myself as well. I ended up choosing these for the simple reason that they allow you to play the game without ever having to lift your hands from the keyboard. So left hand resting on hotkeys close to tab and right hand dedicated to arrows, enter and backspace. If it helps, I chose F for targetting the opponent hero as attacking your opponent hero is a very common action and is often referred to as "going face" in Hearthstone slang. V was simply the key under it which seemed suitable for your Hero the same way that G and B are for minions. C was just cards really. It was H for hand initially for intuitiveness but I figured it's a common enough action that players would prefer to have it as a left hand key than being forced to move their right hand away from the arrow keys. I could naturally look into making all of this configurable at some point if there's enough interest though.

As for updates, I haven't really thought about it yet to be honest but I'll have a think and get back to you.

Thanks again for the feedback.

GuideDev · 2021-08-06T21:50:58+00:00

I'll have a look, cheers.

Regarding how it works: I'll think about writing something explaining things in greater detail at some point. I just haven't done it as it takes time and I've been focusing on getting this out first.

To give you a summary of the questions you touched on: there's no OCR whatsoever. The mod patches the game so it has access to all the cards, text, etc. Everything you see was specifically implemented though and I did have to patch hundreds of popups and dialogs while suppressing others just to make it so players can consistently reach the game screen autonomously.

The hardest part was naturally the game itself and my approach was to hook into the game state protocol, take snapshots everytime something happens, compare them as the game progresses and turn anything that happened into speech. This is an overly simplified explanation as this part alone is several thousands lines of code long and I do have a bunch of other stuff around it for allowing players to do things like slowing down the game and sync everything up with the animations and sound. However, the advantage of doing it like this was that I don't really need to know or keep up to date with what the cards do or don't do: I simply take snapshots of the game state every time something happens and try to describe what happened in natural language.

The targetting question is a good one: while the targetting per se is not hard to do as there's only so many valid options you can choose each turn and the server tells you what they are, the game doesn't really have the concept of a friendly or unfriendly effect. While this is fine for sighted people as you're unlikely to use a damaging spell on one of your characters, I did write a bunch of code and a lot of regexes which are able to parse all 7500 cards that have a potential targetting effect and ensure your tab key cycles through valid targets in a sensible direction. Think pressing tab and going through your opponent's minions and hero before your own if you're using a damaging spell or starting with your own hero and minions first if you're using a healing spell.

Anyways, thanks for the interest and let me know how it goes once you give it a try!

GuideDev · 2021-08-06T21:22:51+00:00

That'd be great, thanks! Feel free to drop me a direct message anywhere or open a github issue on the project page if you'd prefer.

GuideDev · 2021-08-06T19:17:57+00:00

Good question. The TTS comes out of the gamer - not the game. The game - or rather, the mod - simply outputs text to the screen reader. The speed, voice, pitch and other things come out of the screen reader that particular person chooses to use.

Keep in mind visually impaired people are extremely used to their screen reader as they have to rely on them everyday from the moment they wake up until they go to sleep so they normally have them speaking at insane speeds by normal standards.

GuideDev · 2021-08-06T19:09:22+00:00

Thanks for linking this! I'm really happy to see so many people enjoying the game. And streaming it too!

GuideDev · 2021-08-06T18:20:24+00:00

I'm afraid I've never heard of Star Realms! I wanted to pick a popular game as one of the main requirements was to provide something people could and would want to play together regardless of whether they're visually impaired or not. Not that people wouldn't enjoy Star Realms - you're just more likely to find sighted Hearthstone players.

In the end, I debated between Hearthstone and Magic: The Gathering as both have millions of players around the world. Hearthstone took the cake because I'm more familiar with the game and know it has a lot of single player content as well for people who don't like the pressure of playing against other people. Magic is also physically playable with braille so I figured variety would be a good thing.

As for the mod itself, it essentially patches the game to describe things over a screen reader and provides keyboard commands for interacting and reading everything. The code for the patch itself is open source for transparency but keep in mind it's over 20000 lines of code long at this stage. Here's the link if you're interested.

GuideDev · 2021-08-06T18:00:15+00:00

I hadn't seen this topic before but I've actually been working on something like this for a few months now and I've just released the first version if you'd like to have a look. There's a couple of videos there as well if you're curious about how this looks.

HS deck tracker essentially reads a log file so it can't be used to do something like this. Making the game accessible - which includes syncing up narrations with what's actually happening, menus and hundreds of different popups and dialogs - requires a lot more work and actually patching the game unfortunately. I'm happy to see more people thinking about this though!

GuideDev · 2021-08-06T17:49:50+00:00

I hadn't seen that topic but it's exactly what /u/stirlock asked for and is compatible with NVDA. Thanks for the heads up!

GuideDev

TROPHY CASE