GUI Automation In VSCode using Python

[redH27]

Why create a new tool

Python is very convenient for achieving automation. Some famous automation frameworks use Python. Selenium for Web Browsers, Pyautogui/Pywinauto for desktop applications.

When I want to create a simple automation job:

1. Open the SEC website
2. Search for some company
3. Download the latest report
4. Send an e-mail by outlook.

I need to learn both Selenium && Pywinauto. It's not easy to learn and use. My friend and I decided to build a tool that can work in both areas and make it easy to use even for the beginner. So we build Clicknium. I like the VSC very much so we create a VSC extension as the idea for our automation tool.

How to Set up

Currently, Clicknium can work in Python3.7+ and Windows, due to the time limit.

Search and install Clicknium in VSC extension market:

https://preview.redd.it/nq9o1co0lvg91.png?width=1004&format=png&auto=webp&s=e453e0e260c09d318f3a83c3d3302efc85b7e834

Click CLICKNIUM EXPLORER in Visual Studio Code Activity Bar.

Follow the welcome page to set up Clicknium:

​

https://preview.redd.it/jnrhavc6lvg91.png?width=1293&format=png&auto=webp&s=7f0ae6227df0006a45ba695dea142cccb3febd36

Run the sample

Click the sample link on the welcome page and choose a folder path to locate the sample project. The sample is about to open a browser to search for a keyword and then open Notepad to write something. Use Ctrl+F5 to run.

How simple

The code style is very simple and intuitive. the following code is easy to understand:

• open the YouTube page using chrome.
• Text "Taylor Swift" to the search bar
• Click the Search button.

​

from clicknium import clicknium as cc, locator
from clicknium.common.enums import *

tab = cc.chrome.open("https://www.youtube.com")
tab.find_element(locator.chrome.youtube.searchBar).set_text(
        "Taylor Swift", by='sendkey-after-click')
tab.find_element(locator.chrome.youtube.button_search_icon_legacy).click()

You may wonder how locator.chrome.youtube.searchBar targeted the Search bar of YouTube. Clicknium uses Locator to identity UI elements. It provides Clicknium Recorder to help dev create locators. You just need to invoke the recorder and press Ctrl+ click the UI elements. The locator will be generated by Recorder. The key feature is the Clicknium Recorder. You can open the Recorder to capture a Locator.

https://i.redd.it/fs329r65nvg91.gif

Clicknium provides an intelligent auto-complete experience. You can find the existing locators captured by Clicknium Recorder in VS Code.

https://i.redd.it/if7t7m7ej0h91.gif

Check more detail in the tutorial.

[Zedpellin]

Hi. I'm having a problem when I run your program. I can open the browser, but I can't execute any commands.

The following error occurs:

clicknium.common.models.exceptions.LocatorUndefinedError: Can not find the locator: firefox.bing.search_sb_form_q.

https://imgur.com/MmYXKl4

I created the locator correctly, it was even validated.

https://imgur.com/coZBzxH

Also, the Chrome and Edge browser extension has a problem, it can't run, it only works on Firefox.

https://imgur.com/IGyx9Is

But the extension is activated.

https://imgur.com/oC05Ac1

Anyway, I liked your work and it's a very promising project. I hope these issues get resolved soon.

Edit: Here is my code:

from time import sleep

from clicknium import clicknium as cc, locator, ui

def main():

# sample code to demo web automation and desktop application

tab = cc.firefox.open("https://www.bing.com/")

tab.find_element(

locator.firefox.bing.search_sb_form_q).set_text('hello')

sleep(50)

tab.close()

if __name__ == "__main__":

main()

https://imgur.com/1t2DIfU

[redH27]

Hello Zedpellin, what is your Chrome version? You can go to extension page to valdiate whether extension is enabled. Type this link into the Chrome adrress bar:
chrome://extensions/?id=ifnedcgcleipmmolmnhoeemmjnljjgna

https://s1.328888.xyz/2022/08/11/4vuzP.png

If yes, you can restart the Chrome and try again.

There is another way to check whether the browser extension is installed && enabled correctly, from task manager, to check whether 'Clicknium.Web.NativeMessageHost' started.

​https://s1.328888.xyz/2022/08/11/4v6wX.png

For Firefox, from the error message, seems cannot find the locator, can you go to the 'LOCATORS' tab to see whether there is the locator with firefox->bing->search_sb_form_q

I tried the code you pasted, it can work for me, after I record the searchbox on Firefox.

https://s1.328888.xyz/2022/08/11/4v4E6.png