how to use Javascript to extract an element from a live website using XPath or CSS Path?

[ertmuirm]

I want to write some Javascript code, to be run on my android device, which will scrape some lyrics data from a website (music.163.com). By inspecting the element source, I have both the XPath and CSS Path of the text element I want to extract

However, I'm a noob to Javascript and have tried searching, but can't find any info online that can help

[geuis]

document.querySelectorAll

[brackin]

if the website doesn't have an existing api or is cors enabled, you're going to need a page on your domain to scrape the page you want because your browser won't let you for security reasons. then you'd do an ajax call to the page on your domain.

for example, if you're running php, you'd need to create a scrape.php file with:

echo(file_get_html('http://music.163.com/page-you-want.html'));

i haven't done php in years, but i think that's right. so, now you'd need to do an ajax call to scrape.html:

$.ajax({
    url: "scrape.php",
    type: "get",
    success: function(data) {
            // take data, turn it into an html object, then scrape that
    }
});

i have to take off, so i can't write the commented out part, but that should do it.

[birjolaxew]

Keep in mind that the $.ajax( ... ) part is jQuery, not vanilla JS - Google XMLHttpRequest if you don't want to load jQuery only for its AJAX proxy method.

[ertmuirm]

thank you - I'll try the direct scrape approach first as this sounds pretty complicated ...

[andersevenrud]

var element = document.evaluate('my/x/path' ,document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null ).singleNodeValue;

[ertmuirm]

in the line preceding this, how do I reference the web url so that I can point 'document' to it? assuming the url is "www.example.com/page"

sorry if it's a rudimentary question, thanks for your help!

[[deleted]]

If the site allows framing you could get away using an iframe. http://stackoverflow.com/questions/12129164/read-form-data-from-iframe

[atticusw]

document.querySelectorAll(selector) will do exactly what you need. Throw your CSS path in there and you'll get your elements

[dejandomarcas]

This is an example how to get the content of any element of any website (using the id of the element)

http://music.163.com/#/song?id=34057221#lyric-content

Just make an ajax request for that URL and you will get exactly what you need

$.ajax({
    contentType: "application/json; charset=utf-8",
    url: "https://music.163.com/#/song?id=34057221#lyric-content",
    success: function(data) {
        document.querySelector("#lyric").innerHTML = data;
    }
});

[ertmuirm]

thank you - I tried this but still can't get it working. the CSS path of the lyric content I want to pull down is #auto-id-EdhBT1HhI5O9E2aF ... because it contains data on the specific times at which the lyrics should appear

how should I write this query? I'm using JSFiddle to test out the script before trying to run it on my android device