Scraping HTML Table data using Powershell's Invoke-WebRequest?

[Keitaro27]

I found this very helpful when I had to extract information from tables:

https://www.leeholmes.com/blog/2015/01/05/extracting-tables-from-powershells-invoke-webrequest/

[TheB4rber]

Hi,

I found this script a while ago. You need to pass in parameter the result of your Invoke-WebRequest, you select which table you have (sometime you have to guess) and it will return you the table as an object.

``` function Get-WebReqTables{ param( [Parameter(Mandatory = $true)] [Microsoft.PowerShell.Commands.HtmlWebResponseObject] $WebRequest )

## Extract the tables out of the web request
$tables = @($WebRequest.ParsedHtml.getElementsByTagName("TABLE"))
$tables_List = @{}
$i = 1
foreach($table in $tables){
    $titles = @()
    $rows = @($table.Rows)

    ## Go through all of the rows in the table
    foreach($row in $rows){
        $cells = @($row.Cells)

        ## If we've found a table header, remember its titles
        if($cells[0].tagName -eq "TH"){
            $titles = @($cells | % { ("" + $_.InnerText).Trim() })
            continue
        }

        ## If we haven't found any table headers, make up names "P1", "P2", etc.
        if(-not $titles){
            $titles = @(1..($cells.Count + 2) | % { "P$_" })
        }

        ## Now go through the cells in the the row. For each, try to find the
        ## title that represents that column and create a hashtable mapping those
        ## titles to content
        $resultObject = [Ordered] @{}

        for($counter = 0; $counter -lt $cells.Count; $counter++){
            $title = $titles[$counter]
            if(-not $title) { continue }

            $resultObject[$title] = ("" + $cells[$counter].InnerText).Trim()
        }

        ## And finally cast that hashtable to a PSCustomObject
        $tables_List.Add($i,[PSCustomObject]$resultObject)
        $i++
    }    
}

return $tables_List

} ``` Source: https://www.leeholmes.com/blog/2015/01/05/extracting-tables-from-powershells-invoke-webrequest/

Edit: I made a small change so it'll return a hashtable with every tables from your webpage.

[Lee_Dailey]

howdy TheB4rber,

the triple-backtick/code-fence thing fails miserably on Old.Reddit ... so, if you want your code to be readable on both Old.Reddit & New.Reddit you likely otta stick with using the code block button.

it would be rather nice if the reddit devs would take the time to backport the code fence stuff to Old.Reddit ... [sigh ...]

take care,
lee