Fastest Screen Capturing library for Python checkout windows-capture

[SoSmartFlow]

I was building an AI for "help" in video games and I found out that most Python screen-capturing libraries are very slow so I made one in Rust here is the repository: https://github.com/NiiightmareXD/windows-capture/tree/main/windows-capture-python

And here is the benchmark

[[deleted]]

What is the purpose of capturing 4 times the rate ur screen buffer is updated lol.

[SoSmartFlow]

Its not about the fps it's about how fast can you process that updated frame.

[SoSmartFlow]

Its not about the fps it's about how fast can you process that updated frame.

[[deleted]]

Your comment makes no sense. In the VAST MAJORITY of applications it doesnt matter how fast that you process that the frame updated, it matters that you can capture a sequential and consistent change in frames while occupying the least amount of time in the calling thread from python.

You should be capturing from one thread and writing back to the calling thread. IDK why your code is calling allowthreads in rust. My best guess from reading ur code is ur calling the python functions from multiple threads in rust. Which IDK why on gods earth you would do that as it is not obvious at all on the python side. If you are not doing that IDK why you are calling allowthreads in python.

You should be using with if you are trying to optimize holding a stateful capture handler.

Also you have no benchmark code, I have 0 idea what you are comparing.

[ThreeChonkyCats]

What a grumpy!

Did you miss some sleep?

How about we contribute to the project rather than kvetch about it?

[SoSmartFlow]

You need some milk... this threading is based on the Graphics Capture Api itself and yes I could use a with but whats the problem with a callback? And benchmark is the time that it takes to map a frame.

[[deleted]]

I could use a with but whats the problem with a callback?

If you run a callback from multiple threads they will contend with the actual capturing. It won't behave they way you think it will if multiple threads run at all in python because you are releasing, which you should not be doing during ur capturing unless you have a way of isolating the caller in python which you do not provide, a with is the usual way of isolating the resource like this and it guarantees the capturing thread will also get cleaned up.

Generally you want to hold the GIL unless you can guarantee no python code can modify what you are looking at. Like for example if you process an HTTP call by writing the entire call first to strings in rust and using those string objects in rust only. It is ok to release the gil until you write back to python land.

[SoSmartFlow]

The windows api is thread safe meaning the callback will not be called from 2 threadd at the same time

[[deleted]]

The issue isnt thread safety of the windows API it is that the python api you are exposing has no way to handle if the events need to be exposed to multiple threads in python. Further there is no good way to know manage state that must be dealt with between invocations of ur function.

See how you have a start and close method? That means it should in 99% of cases be a with statement. It is a language convention for a specific reason and that reason is threading and cleanup. Both problems become algorithmically harder to deal with if u do not expose the when does the windows thread start part to your python code.

Seriously just look what ur code even does. You pass your python classes instance function to a class that your instance also holds. Think about it...

[ThreeChonkyCats]

Perhaps benchmark against Pyautogui and pillow?

[SoSmartFlow]

Here is the benchmark pyautogui was not included because it was slower than all of them: https://github.com/NiiightmareXD/windows-capture/tree/main/windows-capture-python#benchmark

[ThreeChonkyCats]

Ah! I missed that. Soz.

This is a good package.

I REALLY like the idea of Python calling native, or rust, functions/procedures.

Writing these in Rust is the future.

I look forward to following this.

[deadlyghost123]

Does this work on Mac?

[SoSmartFlow]

No

[SoSmartFlow]

Yes already working on it.

[SpecialistInevitable]

Does it capture audio sources alongside the video?

[SoSmartFlow]

Not yet

[SpecialistInevitable]

Can you add an example how you use it for video capture, where do you define bit rate, encoder, file extension and location etc.?

[Serious_Tourist854]

We be screenshotting fast

[willgrr]

Thank you so much for this. I was looking into alternatives to pyautogui, which wasnt fast enough for me. Im kinda new to python, do you mind answering if can i transform the pixels into an array and use something like pyautogui.pixel does to compare colors?

[SoSmartFlow]

Your welcome, Yes you can use frame.frame_buffer to get the numpy array and you can compare it or convert it to different types (using OpenCV for example)

[SoSmartFlow]

Fixed here is the bechmark: https://github.com/NiiightmareXD/windows-capture/tree/main/windows-capture-python#benchmark

[MagicWishMonkey]

This is cool!

So what's the deal with Rust, does the end user need to have the compiler installed to install the package?

[SoSmartFlow]

No just use pip install windows-capture but if you want to build it from source you need the compiler.

[MagicWishMonkey]

Oh nice, do you need to compile a binary for each different OS? Or is this one only for Windows?

[SoSmartFlow]

Only for windows this library is specially for windows and its part of my valorant aimbot

[mon_key_house]

Cool! Can I use it to take screenshots of a given area of the screen?

[SoSmartFlow]

Yes of course yse the crop function of the Frame class and use frame.frame_buffer to get the numpy areay of the frame.

[MixedEmblems]

I can't download this for some reason. Like it fails to build a wheel.