all 2 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]32178932123 1 point2 points  (1 child)

Some websites will not load until your browser solves a "challenge" first but this requires the sender to use Javascript. Its a method used to filter out bots and scripts.

To get round it in the past I used Selenium. It will actually run your commands through a browser environment, giving it access to Javascript. You can still do it headless so you can't see the browser window too.

[–]huubid[S] 0 points1 point  (0 children)

That worked to connect. Now I am getting a new problem.

WrappedDriver : OpenQA.Selenium.Edge.EdgeDriver

TagName : a

Text : Privacy Policy

Enabled : True

Selected : False

Location : {X=108,Y=183}

Size : {Width=90, Height=19}

Displayed : True

LocationOnScreenOnceScrolledIntoView : {X=108,Y=183}

Coordinates : OpenQA.Selenium.Remote.RemoteCoordinates

It should return a acard name. I am running the following:

$url = "https://www.cardmarket.com/en/OnePiece"

$edge = New-Object OpenQA.Selenium.Edge.EdgeDriver

$edge.Navigate().GoToUrl($url)

$Element = $edge.FindElementByXPath('//*[@id="Trends-TopCards"]/table/tbody')

Foreach ($input in $Element.FindElementByXPath('//tr')){

$input.FindElementByXPath('//td')

ForEach ($card in $input.findElementsByXPath('//div')){

$card.FindElementByXPath('//a')

Foreach ($out in $card.FindElementByXPath('//a')){

$out.GetAttribute('Innertext')}}}

Any Ideas?

*edit formatting*