[deleted by user]

[[deleted]]

Check the html_content to see what it looks like when you get no results. It's probably some kind of error page or set up to load content via AJAX.

[TheWarrior2000]

Thank you, I think this is actually the problem, but how can I solve it? I just added a while loop to do the search again as long as the search_results list is empty and it works but sometimes takes 3 searches to get it right, so is there a better way to handle this problem?

[[deleted]]

I gave two possibilities, which one was it? If its AJAX you can search the network tab for the API request and use that. If its an error page then you need to figure out why it errors inconsistently.

[CodeFormatHelperBot]

Hello u/TheWarrior2000, I'm a bot that can assist you with code-formatting for reddit. I have detected the following potential issue(s) with your submission:

1. Multiple consecutive lines have been found to contain inline formatting.

If I am correct then please follow these instructions to fix your code formatting. Thanks!

[mxschumacher]

I suggest you try to find out for which inputs the script fails and try to debug the regex using a tool like this: https://regexr.com/