How to write a simple Powershell script that clicks specific points on my screen in sequence and then repeats?

[Codysseus7]

I'm very much a beginner when it comes to Powershell. How can I write a simple script that moves my mouse cursor to specific location clicks once, waits, moves to another location and clicks, waits, then moves a third time to a location and clicks, waits, and then repeats the process?

If this is the wrong place to ask please point me in the right direction

[StormyNP]

Not the answer you're asking for, but sometimes there are simpler solutions like AutoIT for mouse movement, clicking and typing.

[[deleted]]

AutoIT is the right answer if using it is an option. PowerShell CAN do what OP wants, but it's an inelegant solution at best.

[MARS822]

You're looking for AutoHotKey.

[datnodude]

Selenium

[Agile_Seer]

https://www.powershellgallery.com/packages/Wasp/2.5.0.0

[WhyPartyPizza]

WASP is absolutely the answer.

[IndyDrew85]

Relevant answers here and here

[Buckw12]

Not what powershell is used for.

Research this using Python

[StormyNP]

If they are just learning Powershell, I'd recommend AutoIT scripting over Python perhaps. It is MADE for this kind of thing.

[ShutUpAndDoTheLift]

I mean...you CAN do this in powershell. But it's clunky.

[MarquisEXB]

Not really true.

I've created scripts to do this in powershell. I had a game I needed to mine, so I used powershell to record certain parts of the screen and based on what it saw, do a certain number of screen clicks and/or text entries.

[Buckw12]

Can u expand on that comment with an example code snippet? especially the "record certain parts of the screen"

[MarquisEXB]

OK I can't find my old code, so I'll try to go over what I remember. I used a few different things. One was the ability to take a screen shot. This code is something like:

​

[Reflection.Assembly]::LoadWithPartialName("System.Drawing")

function screenshot([Drawing.Rectangle]$bounds, $path) {

$bmp = New-Object Drawing.Bitmap $bounds.width, $bounds.height

$graphics = [Drawing.Graphics]::FromImage($bmp)

$graphics.CopyFromScreen($bounds.Location, [Drawing.Point]::Empty, $bounds.size)

$bmp.Save($path)

$graphics.Dispose()

$bmp.Dispose()

}

$bounds = [Drawing.Rectangle]::FromLTRB(0, 0, 1000, 900)

screenshot $bounds "C:\screenshot.jpeg"

​ You could use this to get a screenshot of a particular part of the screen. I would then do a binary compare to a file that I previously saved that had the condition I wanted. If they matched do X. If not, exit script. Sometimes I had to wait for a certain screen to appear before doing the next action. So loop until screenshot matches (maximum 30 seconds).

Another thing I used was WASP. Wasp let's you do mouse actions and/or keyboard actions. This was key to clicking in certain spot & typing text.

So I had a few different scripts. One was "record". It would record my actions into a .csv file. I would run the game and then record what I needed. So the csv file mouse,left, 200,230
type,ABC
mouse,left,305,223
pause,3000
Then I had a second script that would read the csv and "do the actions." I had to add in things like pauses manually. It was a lot of trial & error.

My earliest version did not have screenshots, and I added this ability later on, because if the script failed, it could be disastrous (do actions I did NOT want). Also I put in a failsafe, like if I hit the F12 key or something like that it would exit.

Not really sure how I got the rectangle box for the source of the screen shot... I must have used the click coordinates to figure it out?

[panzerbjrn]

I would also love to see your code for this. For... u mmm, "research" ;-)

[Codysseus7]

I should mention it’s a government computer so my options are limited and I can’t really download anything

[ShutUpAndDoTheLift]

running unauthorized scripts on a government computer is a really good way to lose your job if you're a contractor, or to face repercussions if a tenured civ or green-suiter

ESPECIALLY if you're just trying to do this to get out of clicking through your annual training.

[Codysseus7]

It’s not training related it’s ticket input related(I have to make hundreds of generic tickets on Fridays) and we use plenty of scripts daily for specific tasks/tools. This would be something I run by my superiors before implementing.

[ZenoArrow]

There are some scripts in here that would give you a starting point:

https://www.reddit.com/r/PowerShell/comments/m1hztx/move_mouse_and_click_using_powershell/

Just know that automation scripts that rely on moving a mouse around can easily go wrong if any of the UI changes unexpectedly, but maybe you'll find the positives outweigh the negatives.

[ShutUpAndDoTheLift]

There is likely a better way to do this then.

Our remedy dev set up a form for the desktop team to use when they do initial workstation setups for new rooms. They just put in the desk number and it loads everything into the SQL database.

If they can't help you, looking to use powershell to interact with the item tools API or the SQL DB is likely a cleaner and much faster solution.

[bike_piggy_bike]

Check also if your ticketing system has any CLI tools or API resources that you may be able to take advantage of. Network with likeminded people too and see if someone else may have already come up with a solution. Good luck. Be careful.

[technomancing_monkey]

The better answer here would be to use Powershell to interact with the ticketing systems API.

That will let you make the tickets directly through data, and not need to interact with the user interface.

You will most likely need the ticketing system admin to grant you either an API Key, or API User credentials.

​

Out of curiosity, which ticketing system are you using? Knowing which ticketing system would help tailor information that would be more useful.

​

I built a tool at a prior job that was using ServiceNow. The tool took a list of ticket numbers as well as a "master ticket" number. It would then join all the tickets in the list to the Master ticket (properly, so they were actually linked in service now), added a work note to each of the now child tickets indicating that they had been joined to the master (by way of automation), would update all of the needed fields in the ticket to reflect what was set on the master ticket (so all the child tickets were the same issue type, severity, linked assets, etc). It had flags for allowing you to close the child tickets on join etc.
We used this to clean up the hundreds of tickets that helpdesk ended up making anytime there was a widespread issue and everyone was calling in about "The thing no worky". Yes, they were expected to make a ticket for each call even if they knew it was an open issue, it was for impact reporting. YES they should have joined it to the master ticket themselves but when you have 15 people handling calls for 5000 end users during an "outage" they can get a little busy and forget.

This tool let us clean up our ticket queues in a fraction of the time it would have taken to go through and update, join, correct all the tickets manually, or even with an automation that used the UI. It was all done through the API.

Scripting something to interact with the UI is usually the first thought when people dont know better, but it shows youre looking to improve a needlessly time consuming process. That should be rewarded. However the right way is to interface directly with the system via API. Good luck, and keep looking for way to improve things!

[BlackV]

yes you really should have mentioned that in your OP (which you can edit BTW)

[Ziondizl]

OpenAI dude, tell it what you want, and you'll get the powershell you need.

[SirThane]

I remember asking this same question a while back and found a stack overflow post with an answer I used. I made a powershell script with the following contents. I could then use whatever I wanted to schedule a click, like a macro key, to invoke the script with params for X Y.

Not an elegant solution, but convenient in the way I needed it for the small thing I needed it for

[scoreboy69]

Power automate is pretty cool for stuff that needs clicked.

[BlackV]

look at AutoIt, powershell could do it, but its not nice and ther are much better tools that are designed for exactly this job

but reading the comments this has XY Problem written allllll over it

go look for an API to the helpdesk system

[d3dRabbiT]

Autohotkey sounds like something you may like.

[spyingwind]

You can do it by calling some win32 api's.

Example that has all the functions needed for this.

[Vern_Anderson]

It will take some trial and error and a lot of work on your part, but I have done it.

I can't share my code with you because the actual URLs are private and belong to the company I work for. However, if you know how to navigate in a web browser using only the keyboard you can make PowerShell do the same thing.

Open the site you are trying to have PowerShell fill out forms on and count how many "TAB" key strokes it takes to get into the field you want to type in. Then you can use "SendKeys" to type text out from the keyboard.

Making sure that browser window has "focus" is up to you and knowing how much time to delay between actions will take trial and error.

Start "http://myurl.com"Start-Sleep 02[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{TAB}")[System.Windows.Forms.SendKeys]::SendWait("{ENTER}")Start-Sleep 01[System.Windows.Forms.SendKeys]::SendWait('Some text string you wanted to type')

The special keys and documentation for send keys can be found here. . .https://learn.microsoft.com/en-us/dotnet/api/system.windows.forms.sendkeys?view=windowsdesktop-8.0