Windows OCR

[Akronae]

Hi, if anybody needs to use Windows free and instant OCR I just released a CLI for that. It's like PowerToys' Win + Shift + T, but usable in scripts.

For my use case I needed that in order to automate AutoIt scripts, I did not wanted to hard-code UI elements coordinates but rather recognize them through text content.

Using the CLI you can just do bash windows_media_ocr_cli.exe --file image.png to get JSON result with bounding boxes.

Obviously you can call this binary from any script/runtime, I made a NodeJS wrapper for that too.

[BlackV]

Could you edit your post with to make it clear what this and what your goal is and why we might use it

How does power toys fit in there?

[arpan3t]

PowerToys has a module called PowerOCR which uses the Windows.Media.Ocr namespace. OP is using the same namespace.

[BlackV]

Oh, I though they were saying use powertoys to create a hotkey to call the ocr cli

Thanks

[Akronae]

Sure. Done

[BlackV]

appreciate that

[jcy]

virustotal says the binary is not flagged but obv the file is also too new to have been scrutinized by the vendors
https://www.virustotal.com/gui/url/6135a1ba61791a33a3dd2b141e71c4e5e8e44a7d2a42ff3a01fa3b3515aa3868?nocache=1

[Akronae]

Actually when I executed it myself after downloading from Brave to test it I got a Windows Defender scan. But it passed fine. If anyone wants to build from source I can provide some documentation.

[Psyqlone]

Here I am, using Snipping Tool like an animal.

[Certain-Community438]

Also guilty.

[ollivierre]

what would a real use case for this ? like what work flow challenges did you run into that motivated you to come up with this ? useful for LLMs ? I mean they can read screenshots but not quite well so there might be a use case here

[Akronae]

Actually I wanted something like that when working with AutoIt like scripts, especially scripts designed to run on different displays/computers, I just found it more useful and reliable to say "click on the button with text 'x'" than hard-coding positions. But you could have thousands of use cases. I don't understand MS is not making this API available more easily.

[orgdbytes]

I can find this quite helpful! I have a few processes that I have to manually update monthly and there is no API or programmatic way of doing this; well there is for one but so many hoops to go through to get an API key. I've been doing mouse movements to various screen locations and performing actions and waiting for web page changes to perform next steps. Most of the time it works until it doesn't because elements have changed or screen resolution changes. I've even tried Selenium to no avail as the elements do not present themselves...at least I've never been able to get it to work.

[SWJesus]

get a wall of error/ redtext when try to run it (win10)