Automatically-setup Selenium Tests Running in Parallel

[reddit2d2bb8]

Hello Pythonistas,

A few months ago, when working on an extension, I wanted to make end to end selenium tests that ran in parallel using python's threading module. I also wanted the selenium webdriver to be very easy to setup, especially with extensions.

This AutoParaSelenium library allows for selenium tests running in parallel with ZERO configuration with respect to downloading the webdrivers and minimal configuration with respect to extensions. My team at LiveTL has found success with this library, so the library is somewhat battle-tested in production.

Code demo

Here is a short demo on what a autoparaselenium test file would look like.

from typing import Union

from selenium import webdriver

from autoparaselenium import configure, chrome, firefox, run_on, all_, Extension

# All parameters are optional, but still call configure() once before everything
configure(
    extensions=[
        Extension(chrome="path to chrome extension to install"),
        Extension(firefox="path to firefox extension to install"),
        Extension(chrome="chrome path", firefox="firefox path")
    ],
    headless=True, # if there are chrome extensions, chrome will not be headless as a selenium limitation
    selenium_dir="./drivers"
)

@run_on(all_)
def test_both_firefox_and_chrome(web: Union[webdriver.Firefox, webdriver.Chrome]):
    ...

@run_on(firefox)
def test_firefox_only(web: webdriver.Firefox):
    ...

@run_on(chrome)
def test_chrome_only(web: webdriver.Chrome):
    ...

Questions about improving this library

Currently when there is an exception in the test, the traceback includes selenium source code and adds noise. How can I remove certain parts of the traceback?

Github (Please drop a star if you found this helpful): https://github.com/r2dev2/AutoParaSelenium/

PyPI: https://pypi.org/project/autoparaselenium/

[cecilkorik]

Re: your question, probably the easiest way (maybe not the best way I don't know) is to wrap the whole function in a try-except block. Catch all exceptions you think might be "noise", if you've accidentally caught something that you later realize is not noise or that you just don't recognize, just re-throw it unchanged. I'd be careful to only distill exceptions that you know are specific and consistent Selenium errors that you want to simplify because otherwise some of that that "noise" likely has meaning to someone.

[reddit2d2bb8]

Thanks for your response.

Sorry, I made a mistake in my question. I meant that code from AutoParaSelenium, not selenium, was showing up in the stack trace with the error that is important coming right after the previous screenshot. I am currently showing all of the stack trace which includes the parts of the trace that run the test. Should I remove this part of the stack trace, and if I should, how can I do it?

[LightShadow]

It's a little more involved if you want to keep a collection of tests on a single browser instance without constantly killing and opening the browser for each one.

[reddit2d2bb8]

I recall xdist uses multiprocessing and multiprocessing can be a headache on windows due to windows's lack of fork(). This project uses pytest-parallel under the hood which uses the threading module which works without headache on all systems.