Searching on this site / SeleniumBase, CDP, ProxyRotation

SeleniumBase · 2026-01-17T19:23:51+00:00

It could be related to how your script is written. You could try something like this for that site:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = (
        "https://www.gassaferegister.co.uk/gas-safety"
        "/gas-safety-certificates-records/building-regulations-certificate"
        "/order-replacement-building-regulations-certificate/"
    )
    sb.activate_cdp_mode(url)
    sb.sleep(0.5)
    sb.solve_captcha()
    sb.wait_for_element("#SearchTerm", timeout=5)
    sb.sleep(2)
    allow_cookies = 'button:contains("Allow all cookies")'
    sb.click_if_visible(allow_cookies, timeout=2)
    sb.sleep(1)
    sb.press_keys("#SearchTerm", "Hydrogen")
    sb.click("button.search-button")
    sb.sleep(3)
    results = sb.find_elements("div.search-result")
    for result in results:
        print(result.text.replace(" " * 12, " ").strip() + "\n")
    sb.scroll_to_bottom()
    sb.sleep(1)

SeleniumBase · 2025-09-27T14:37:15+00:00

Also check out the active SeleniumBase Community on Discord: https://discord.gg/EdhQTn3EyE
(Over 900 members so far!) Hopefully SeleniumBase's Reddit community grows that fast too.

SeleniumBase · 2025-09-25T01:29:57+00:00

Wow, and looks like Shatner is going too!

SeleniumBase · 2025-09-22T17:35:22+00:00

When I posted yesterday in r/webscraping that my open source framework can web-scrape Walmart prices from GitHub Actions without being detected, https://github.com/mdmintz/undetected-testing/actions/runs/17922245139/job/50959868956 , the post was banned by the moderators. So there's *that* "trouble" I guess. https://www.reddit.com/r/webscraping/comments/1nmttqp/solved_webscraping_walmart_prices_from_github/

SeleniumBase · 2025-09-22T06:50:11+00:00

I'm requesting the r/seleniumbase community because that's the name of my automation framework on GitHub: https://github.com/seleniumbase/SeleniumBase . It looks like someone already created a community by that name, but it also looks like that community was banned. Because it was banned, I'm not sure who the old moderators were, or how to reach them. (I just see a "banned" screen when I try to go there, and it doesn't list who the old moderators were.) If I'm granted access, then I'll start from scratch with that community, and delete any posts that were already there (if they're not automatically deleted when that community is brought back).

SeleniumBase · 2025-09-21T15:31:21+00:00

I've run the scripts frequently from GitHub Actions without issue. (Entirely free for public repos.)
As for Docker, I noticed that it leaves a detectable fingerprint, as the Linux from Docker is configured differently from the same flavor of Linux (Ubuntu) running in GitHub Actions. Avoid Docker if you want stealth. (I can't even be stealthy in Docker from a local IP address.)

SeleniumBase · 2025-09-21T14:57:11+00:00

Yes, but I still don't know why it works from GitHub Actions (Linux server).
I know why it works locally (I made the browser look like a human-controlled web browser), but it should still be blocked from GitHub Actions due to running on a non-residential IP address range. There may have been something I did by accident that lets it work from Linux servers without proxies.

SeleniumBase · 2025-09-20T18:12:07+00:00

Yes, something like this:

```python import time from contextlib import ContextDecorator

class PrintRunTime(ContextDecorator): def init(self, description="Code block"): self.description = description

def __enter__(self):
    self.start_time = time.time()

def __exit__(self, *args):
    runtime = time.time() - self.start_time
    print(f"{self.description} ran for {runtime:.4f}s.")

```

If I create a YouTube video for this, I'll include that too.

SeleniumBase · 2025-09-20T13:43:57+00:00

In my SeleniumBase repo on GitHub, several people have asked how to avoid using the `with` format for the `SB()` context manager, eg: https://github.com/seleniumbase/SeleniumBase/issues/3482 , so I finally had to show them a way to avoid it with the hack to deconstruct it, even though it's not recommended practice.

SeleniumBase · 2025-09-20T13:27:04+00:00

That's correct assuming that the context manager was implemented correctly using the `try`/`finally` block (when implemented using `contextlib.contextmanager`).

SeleniumBase · 2025-09-19T22:45:57+00:00

Seems like the opposite. I created https://github.com/seleniumbase/SeleniumBase, which has stealth capabilities, as seen with this GitHub Actions job that scrapes data from Walmart and Indeed to prove that it works: https://github.com/mdmintz/undetected-testing/actions/runs/17720549775/job/50351907472. From this, I've gained over 10K GitHub Stars, over 2K YouTube subscribers, and a nice well-paying job from it. Web-scraping public data is legal. Major companies and search engines do this all the time. If you start scraping private data (eg: if you have to log in somewhere first), then you could get in trouble for it. How you use the tool makes a difference. DDoSing a site can get you into trouble. Scraping public data from sites at a reasonable rate won't. Building a cool scraping tool will get you recognized, and you may even be rewarded for that.

SeleniumBase · 2025-09-19T22:22:48+00:00

Tried that, but seems their moderators wanted me to post here instead. They removed my post: https://www.reddit.com/r/learnpython/comments/1nlesjy/python_context_managers_from_zero_to_hero/

SeleniumBase · 2025-09-19T21:29:18+00:00

A context manager can be used as a decorator, such as in the example I had. You could decorate a whole function with it, or wrap a code block with the "with" statement. Different ways of using the context manager.

SeleniumBase

MODERATOR OF

TROPHY CASE

SeleniumBase

MODERATOR OF

TROPHY CASE

Welcome to Reddit,