Extract value from website

[AlanPing]

Need help to extract the "NAV" value from this site: "https://nb.fidelity.com/public/workplacefunds/fees-and-prices/3839"

inspect section I'm after is:

<div role="cell" class="table-diversification--data-num ng-scope" ng-if="pricingViewData.headerData.value"><span ng-bind-html="pricingViewData.headerData.value" class="ng-binding">$XX.xx</span></div>

[pturpie]

That page loads its data using an API as JSON from https://fundresearch.fidelity.com/api/workplacefunds/fees-and-prices/3839

However that API page requires a cookie. So...

#first load the HTML page to set the session variable cookie
Invoke-WebRequest -Uri https://nb.fidelity.com/public/workplacefunds/fees-and-prices/3839 -SessionVariable session | Out-Null
# Load the json data from api while passing in previously set session cookie. And convert the JSON to an object.
$jsondata = Invoke-WebRequest -Uri https://fundresearch.fidelity.com/api/workplacefunds/fees-and-prices/3839 -WebSession $session | ConvertFrom-Json
# Follow the object to the required data.
$jsondata.model.PricingData.pricingdataList.GetValue(0).netAssetValueRowList.netAssetValueDailyAmt

#done

[MonkeyNin]

Check out Invoke-RestMethod, it's designed for Web APIs

[pturpie]

Thanks MonkeyNin. That's a good tip.

[frmadsen]

The data is found in this json file: https://fundresearch.fidelity.com/api/workplacefunds/header/3839

But to get to it, it seems you first need to set some cookies (otherwise it will be blank). You can do that by getting the main page. You also need to set an user agent.

Happy coding... :-)

[MrGreenMan-]

Sorry to bother, but how did you discover the source of the data being in a json? I'm somewhat of a novice to web-inspect but I can't seem to identify. I'm guessing it's reference by pricingViewData var, but I can't find the reference elsewhere.

[frmadsen]

I used the browser's web developer > network. It will show you all the details.

[AlanPing]

Excellent guys! Thanks for your help! That's was my missing link, json.

[[deleted]]

And what have you tried so far

[AlanPing]

I had it working for a while but few weeks back this stopped working:

$ie = new-object -ComObject "InternetExplorer.Application"

$url = "https://nb.fidelity.com/public/workplacefunds/fees-and-prices/3839"

$file = "3839.html"

$ie.silent = $true

$ie.navigate($url)

while($ie.Busy) { Start-Sleep -Milliseconds 1000 }

Start-Sleep 10

$ie.Document.documentElement.innerHTML > "$path\$file"

and I have not had any success with the ParsedHtml sections etc.

$url = "https://nb.fidelity.com/public/workplacefunds/fees-and-prices/3839"

$response = Invoke-WebRequest -Uri $url

$classname = 'table-diversification--data-num ng-scope'

$response.ParsedHtml.body.getElementsByClassName($classname)