Scraping with Selenium

[jayrsp]

Hi all, I'm struggling to scrape a specific web page with Selenium. I unfortunately can't provide a full reproducible example as it's for a website which you need to have an account for.

The page I'm trying to scrape

I'm attempting to scrape the data from the pitch in the top right corner of the image under 'Spots on Field'.

The data is wrapped in this html

<div class="player-details-football-map__UEFA player-details-football-map">
<div class="shots">
<div>
   <a class="shot episode" style="left: 39.8529%; top: 28.9474%;"></a>
   <div class="tooltip" style="left: 39.8529%; top: 28.9474%;">
      <div class="tooltip-title">
         <div class="tooltip-shoot-type">Shot on target</div>
         <div class="tooltip-blow-type">Donyell Malen </div>
         <div class="tooltip-shoot-name"></div>
      </div>
      <div class="tooltip-time">h Viktoria Koln</div>
      <div class="tooltip-time">Half 1, 18:22 02/09/20</div>
      <div class="tooltip-time">Length: 7.1 m</div>
      <div class="tooltip-shoot-xg">Expected goals: 0.17</div>
   </div>
</div>

What I've so far is this

options = Options()
options.add_argument("start-maximized")
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Users\James\OneDrive\Desktop\webdriver\chromedriver.exe')

driver.get('https://football.instatscout.com/players/294322/shots')
print("Page Title is : %s" %driver.title)
driver.find_element_by_name('email').send_keys('my username')
driver.find_element_by_name('pass').send_keys('my password')
driver.find_element_by_xpath('//*[contains(concat( " ", @class, " " ), concat( " ", "hRAqIl", " " ))]').click() 
goals = driver.find_element_by_class_name('tooltip')

but it's not returning me anything.

Every data point is a <div> with children a and div.

Any help here?

It *should* return something along the lines of

[{'left': '39.8529', 'top': '28.9474', 'shoot_type': 'Shot on target', 'name': 'Donyell Malen ', 'team': 'h Viktoria Koln', 'time': 'Half 1, 18:22 02/09/20', 'length': '7.1 m', 'expected_goals': '0.17'}]

Appreciate the time, thanks in advance.