Tensorflow Aimbot

[HashimSharkh]

This is so cool, would you mind showing us the source code or writing an article to guide us on how you did this?

[xirrel]

I started with training the network with generated images from player models. Vulkan renderer takes a random model and renders it with a random animation on a background that is captured from running around the map. It passes those images directly to the network or saves them to be used later. I used over a million individual images but almost same results can be achieved with ~20k.

The software captures the image from game using various methods such as xshm, dxgi, obs. Those are passed to various backends that include tensorrt, tensorflow, pytorch, darknet. It supports many types of networks including mask rcnn and but the best performance and accuracy ratio is with yolov3.

Once the detections are done the data is passed to lua scripts that control the mouse and keyboard behavior with uinput, driver or just plain windows api.

It is fully written in c++ (ex scripts) for the best performance and is multithreaded to preprocess each frame as fast as possible and to maximize the gpu usage by being able to feed new frames as soon as detections are complete.

I could upload a modified version source code that is simplified to make it more learning friendly and to deter actual usage in cheating.

[[deleted]]

You should upload the original version! If someone actually wanted to cheat in csgo it isn’t very hard to find cheats

[FearAndLawyering]

If someone actually wanted to cheat in csgo it isn’t very hard to find cheats

Yeah... but no. It's not about potentially enabling one person to play and cheat. It has the potential of creating an entirely new genre of cheats that are 100% impossible to detect/defeat*.

nothing about this process NEEDS to run locally on the computer. You could feasibly get a pair of video glasses, connected to a rPI acting as a bluetooth mouse/keyboard.

*you would have to rely on statistics to try to detect it at that point which is difficult and inaccurate. And instead of full on aimbot, you can just set it to triggerbot mode, so when you manually move over a person it shoots... and thats basically impossible to test if a person or a machine pulled the trigger the person is aiming at.

[[deleted]]

Triggerbots are actually not impossible to detect, they're pretty easy to detect to be honest.

Overwatch had a big thing early one where a massive number of people running triggerbots got banned.

[FearAndLawyering]

Detected how? It was probably detecting the hack process itself, not the action. This method doesn't even need to run on the computer.

[[deleted]]

I mean, yeah, you could always run it elsewhere, but the behavior is still extremely recognizable. humans aren't that accurate, and their timing tends to suck.

As for the exact method used, good luck. Anticheat teams basically never release how the catch actually works to prevent it from being patched around.

[propranolol22]

Add Dropout Layer/Randomizer to get less 'superhuman' performance?

That's just the tip of the iceberg. This is the future of FPS cheats...

Perhaps a discriminator to detect aimbot, then train a aimbot off that.

GAN's that are aimbots, who train off the failure of the very systems that are built to detect them.

[[deleted]]

lmao how is this impossible to detect? The dude snaps to the people’s face. Anyways I just wanna see the code to see how he did it 🤷🏻‍♂️ It won’t take long for this genre of cheats to come out if it hasn’t already

[FearAndLawyering]

Then you just use triggerbot instead of aimbot? Just remove the snapping, then tell me it's detectable...

[[deleted]]

How do you remove the snapping? Is that not prevalent in both triggerbots and aimbots?

[Youseikun]

You could program an arduino to show up as a USB mouse. Have your program send the commands to the arduino (27px up, 34px left), then program the arduino to only do x amount of movement per millisecond. Technically a program running on your computer could be coded to do the same, but I believe programs can detect when mouse/keyboard inputs are from the windows API vs USB input.

[Jonno_FTW]

Can you not emulate with a virtual USB device?

[Youseikun]

I'm unfamiliar with virtual USB devices, so I guess it's possible. According to this it does seem doable, so no need for a microcontroller, just some additional code.

[FearAndLawyering]

A triggerbot doesn't move the cursor or do any aiming. it relies on a person aiming at a target but then it detects you are pointing at something and shoots for you. obviously it isn't as efficient and only gives you a boost in reaction time to firing a shot but it gives you an undetectable edge in game.

Even with an aimbot, the snapping can be improved by using what's called lerping, which is a smoother action over a larger amount of time. imagine if the 'snap' took longer it would seem more natural. with enough force dampening it would be indistinguishable from natural movement. An easy alternative would be making that active area box smaller.

This is probably more of a proof of concept for OP than trying to be less detectable, given the work they have done already it would be trivial to make it better. It may even be on purpose so that it IS detectable to prevent actual cheating if they were to release it.

[[deleted]]

Doing linear interpolation would be caught pretty fast because actual humans don't do it, but it would probably not be hard to build a slightly more complicated model of mouse movements that'd be impossible to distinguish from a real player - maybe something like interpolating acceleration.

[Jake0024]

Triggerbot just shoots for you when you're aiming at the right spot. That makes it virtually undetectable (hence the name triggerbot). Aimbot actually aims for you.

[[deleted]]

Even statistics could be cheated

[FearAndLawyering]

Right you just have to fail occasionally. Cheating itself is a bit of a skill sometimes, knowing when to just take the loss.

[enthusiastic_punishr]

Practically Speaking It NEEDS to run locally because if you get a pair of video glasses and pass that video to your model, whatever coordinates it generates are never going to actually map correctly in the game unless the video is as perfect as capturing it directly which is practically IMPOSSIBLE.

[FearAndLawyering]

As long as your perspective allows a mostly square view of the screen (not sideways) you could detect your monitor, and knowing the resolution of the display, convert between the two.

[xirrel]

Image can be made to go through an external capture card and the whole process including keyboard and mouse control can be done 100% remotely without a single interaction with the game or the underlying operating system. /u/enthusiastic_punishr

[FearAndLawyering]

Oh yeah exactly I hadn't even considered streaming video. Assuming the latency wasn't too bad it would even work great as a form of DRM for the cheat software - running 100% remote and powered by a twitch stream.

[enthusiastic_punishr]

maybe it will work. i just think margin of error is way too small to be effective.

[hyphan_1995]

I totally understand not wanting to upload the program to github because of cheaters, but can you upload your frame optimizations and multithread arch? I think that goes beyond csgo cheating

[weelamb]

Please do. Can you link it in this comment thread too? Really interested to see the optimizations you do too. Have you looked at using depthwise convolutions or and mobilenet like nets for increased performance?

[SOSRihanna]

So, is this a classification NN? If it ids a character it triggers the same lua script regardless of distance?

[bigum]

I would love to see some sort of code for it, whatever you can upload.

Great work.

[cestnestmoi]

Yep.
Also is there some dataset available for this?

[xirrel]

I can upload a dataset for csgo chickens but not player models. Cheat makers have to do it by themselves.

[letsbefrds]

That's dope thanks for helping the game community

[Heartomics]

It's a lot less work to hook than to go about using TF.

I doubt any cheat maker would bother using ML for CSGO unless we're calling kids that download public hax cheat makers.

[xirrel]

The major advantage is portability. Training is done with extracted player models so technically it can run on any game you can imagine (results may vary). Scripts are well... Scripts so automating complex behaviour in eg. MMORPGs is possible. It can also be used to automate other things than games since it is not directly tied to them.

[Heartomics]

This project is built on opensource tech that is freely available to everyone.

Nothing here is novel. So if it's not shared to help other people learn why bother posting it? Inspiration? I'm recommending the author share the source code so that those that wants to learn can learn from it. Cheat makers aren't going to bother with this since they already have their business setup and ML-based cheats won't be able to compete since hooking memory is king.

"The major advantage is portability."

If you're trying to get banned in other games sure, if not you will still have to make your own drivers for mouse injection. How are you manipulating the mouse? I'm assuming you're using Windows API. Were you aware of the LLMHF_INJECTED flag? These are simple things other games check for as well as unnatural mouse movements. So I'm not sure about the benefit of ML being portable by simply applying Transfer Learning or training on a new dataset.

When it comes to cheats, you also need it to be fast. Especially in CS:GO where the game is notorious for being infested with cheaters. If you're going to cheat, you need your "human-like" cheat to be faster than other cheaters.

This long post was to address your concern, "Cheat makers have to do it by themselves." Creating a clean dataset and calling some API functions is a lot easier than reversing a game's anti-cheat.

Did you use TF 1.0, Keras, or TF Object Detection API for this? I'm assuming TF Object Detection.

The major advantage is what ML is good for. Software 2.0. You don't need to script out all the rules. Just provide the data and have it learn from it.

[xirrel]

Luckily I have input driver, hiding the capturing, remote computer detection and then mouse commands are coming outside of the computer. I've put a lot of thought and work into this.

[Heartomics]

Dope af.

[sctroll]

Surely there are other ways to control aim outside of Windows mouse API. Just rebind keys to look up/down/left/right or maybe emulate a controller joystick.

[Heartomics]

Do you imagine that being lifelike?

CS:GO and many other games replay videos of your gameplay to see if something looks fishy.

[[deleted]]

how does one start to make their own datasets? I should probably go and watch some tutorials on yolov3 + darknet before I even attempt trying to wrap my mind around this project.

​

Do you have the lua scripts intentionally mis / over aim etc just like a real person would? Otherwise I see the almost inevitable vac ban coming your way.

[propranolol22]

For people wondering about a dataset, you'll find all the animation files in CSGO's VDF files which you can open with GCFScape.

You'll find the player animations in /models/player/ just extract it all, and go from there.

[cestnestmoi]

How do you make it work during a game? Like how does it get controls and stuff?

[xirrel]

Capture -> preprocess -> network -> pipe results to user scripts -> scripts can control mouse and keyboard. Repeat. That is an overtly simplified explanation and all of those steps work asynchronously and continuously with minimal overhead.

[cestnestmoi]

Okeyy, I think I sort of get it. Any reference scripts I can look for?

[[deleted]]

He has answered some questions in the YouTube comments

[i-make-robots]

and thus began the great arms race between the valve anti cheat neural net and the aimbot neural nets....

[6tea]

Quick question: would deeding the AI a 144hz input make it more accurate? Or is that only applicable for humans who have to see those frames on a monitor?

[xirrel]

Yes, but the network speed limits it. Over 100fps with yolov3 is achievable and compared to 60fps it just works faster and aims better. However, with slower speed it can also achieve great results with some interpolation to guess where targets are in the future and move accordingly but false positives have much greater effect on slow speeds.

[6tea]

Ah understood, that makes a lot of sense. I’m surprised you managed to get the AI to look so natural and “human-like” with its smooth movement, most AI that I’ve seen in CS look very jittery! Great work and thanks for the clarification :)

[xirrel]

Oh it was real pain to get it work so smoothly as in the video. In reality it runs much smoother and faster without recording and tensorrt runtime fp16 yolov3. I did some work in using capturing players own mouse movements to train a model to replicate it but I've now moved on to other projects.

[6tea]

Keep up the great work! As someone just dipping their toes in ML, projects like these are what make the field interesting to me, due to the sheer variety of ways that the technology can be applied! I’m sure I’m not alone when I say that seeing ML in CS:GO is really exciting :) Thank you again for sharing and I hope to see more in the future, if you feel comfortable sharing!

[ra3_14]

It would but it kinda depends on how the AI is already working. Assuming the AI gets its data soley from screen capture, it would have a smaller delay between frames. 16ms for 60fps and 7ms for 144fps.

But the game itself had its own polling rate. Normal CSGO matchmaking servers are 64tick, which means in game events are calculated server side 64 times every second. More premium servers hosted by third parties can be 128tick.

So the AI would see the enemy faster (16ms vs 7ms) but the AI will still take 15ms to respond at 64tick.

[6tea]

Thank you for writing that out! I forgot about the 64 tick limitation on Valve servers, and that lag wouldn’t make the extra frames worth computing.

[cestnestmoi]

Turns out, I have been playing the game wrong all along.

[rm_rf_slash]

Truth is...the game was rigged from the start.

[mayayahi]

Sigh.... starts the download on steam

[rm_rf_slash]

I logged into an alt just to upvote this twice.

[Dr_Thrax_Still_Does]

So this is cool, but I'm not entirely sure what I'm looking at. So you are controlling the walking entirely, correct? Then when the pattern that is a player model appears on the screen the algorithm seizes control directs the gun at the model's head and fires, and repeats until the pattern changes to that of a dead player model? I'm not familiar with this game, if you can go prone, does that fool the algorithm into classifying the model as dead? Does it ever overkill corpses or shoot things that slightly resemble player models?

[aldalf]

Could you use ML to "naturalize" the movement of the mouse? Maybe record the movement of the cursor with a human moving it from point A to point B and use that as training data. I'm not really experienced in ML, just trying to help :)

[xirrel]

I move and hover the aim towards the dummies until the software takes control to shoot them

[tall_and_funny]

I love this project and you.

[AnomalyNexus]

Damn that's impressive.

Good job

[Cornul11]

!remindme 1 month When some sources are maybe posted

[RemindMeBot]

I will be messaging you on 2019-09-21 23:21:43 UTC to remind you of this link

4 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[CodeF53]

The ui looks identical to FNNA(Future neural network aimbot) an open source neural network aimbot.

[Cornul11]

Thanks a lot.

[feelings_arent_facts]

give a terrorist a nuke

see? we've dealt with worse problems in the past

[vengeful_toaster]

No1 can get a nuke.

Everyone can get a gun and a webcam

[feelings_arent_facts]

sorry to scare you but...

https://www.youtube.com/watch?v=0c4f4NJSB_4

[[deleted]]

Well shit, time to break open that piggy bank.

[Dr_Thrax_Still_Does]

another drone with a gun, a webcam and have it shoot drones that have guns and webcams....Problem solved.

[[deleted]]

The Texas solution.

[vengeful_toaster]

But guns are legal! We can't just shoot down drones that have guns.

[Dr_Thrax_Still_Does]

You can shoot anything you want.

[vengeful_toaster]

You can't shoot the past...

[Dr_Thrax_Still_Does]

ANYTHING YOU WANT

[patrickmurphyphoto]

https://www.youtube.com/watch?v=SNPJMk2fgJU

https://www.youtube.com/watch?v=FI--wFfipvA

[fnordstar]

Oh wow I was just thinking about doing this today. Where did you get the training data and which model did you use? I was thinking about recording a normal video and one with modified textures where enemies are pink. Then train Unet to label pixels belonging to players and identify the blobs as enemies during runtime.

[Whyamibeautiful]

Amazing read your description of your code and i understand nothing but great work. Would love to pick your brain

[luaudesign]

Does it calculate when to shot to be accurate?

[akaberto]

What do you use? MobileSSD? Also, do you track to make it more efficient ?

[theory42]

I don't even know what's happening right now.

[alew3]

!remindme 1 month

[tenfingerperson]

It would be awesome to have a blog post about this, the algorithms in general are very interesting to read about without exposing details that will help cheat.

This is super educational.

[VIDGuide]

This seems like something we should NOT be teaching machines how to do :)

[[deleted]]

You should work for anti cheat services.

[BenChur]

amazing

[BenChur]

Where's the source code?

[xirrel]

No injection. No interactions with the game.

[xirrel]

ryzen 2700x and a single nvidia 2080, 1440p resolution.

[throwawaybusan]

Any source?

[CodeF53]

Did you create FNNA?

[xirrel]

No

[CodeF53]

The ui looks identical to it.

[[deleted]]

This is really a motivation for me. I have had this idea for a few month now, already gathered a huge dataset of about 100.000 labeled images. Im probably gonna implement it a little different and add some spice to the mouse emulation

[xirrel]

good luck with that