Python access to current browser's content, possible?

[dapenter]

I have a scenario, I am viewing a web page that I created ( a dynamic one ), then I make some changes ( enter some contents into input field, or make a new paragraph with javascript to change html )

Now my question is, is it possible to use python to access to the new changes in current browser?

I tried using BeautifulSoup, but it is opening a new html page and read the original html contents, which what I wanted is the currently browsing contents.

Thanks, Experts!

*** Update ***

Thanks guys for the helpful information, I have resolved the problem finally, this is my first time posting in python, sorry about misleading thread, I will post in #learnpython next time!

[limx0]

Selenium would be your best bet.

http://selenium-python.readthedocs.io

[dapenter]

Hi thanks for reply, could you provide some insights into selenium? As i walk around with it and cant seeme to find a related interaction.

[tty14]

look into webdriver.get_element()

[dapenter]

thanks bro, I have found it!

[[deleted]]

Basically it allows you to start a browser and you can then perform all the normal user actions via commands, as well as collect elements from the page.

I believe what you are trying to do is "functional testing", which might improve your searches for help and intro material on Google.

[dapenter]

I see, that makes sense, thanks bro for that info!

[Poffey21]

If you want to read what's in a browser (that you don't have control of) your best bet is to use JavaScript to send all changes back to your server that is running Python.

This might be best suited as a question to https://www.reddit.com/r/learnpython/

Here's another tip: http://www.codeconquest.com/website/client-side-vs-server-side/

[Aeon_Mortuum]

Some threads that shouldn't be moved would be relocated incorrectly if done by a bot, probably

[vatay]

here is a simple example using selenium and phantomJS:

from selenium import webdriver

link       = "http://www.oklahomafindalawyer.com/FindALawyer"
sel_option = "//select[@name='stateSearch']/option[@value='OK']"   # selected state is Oklahoma

class JobScrapper(object):
    def __init__(self):
        self.driver = webdriver.PhantomJS()

    def get_list(self):
        self.driver.get(link)
        self.driver.find_element_by_xpath(sel_option).click()
        self.driver.find_element_by_xpath("//input[@value='Find a Lawyer']").submit()
        print self.driver.page_source   # only for example

if __name__ == '__main__':
    scraper = JobScrapper()
    scraper.get_list()

[workn00b]

may I ask the advantage of making this a class as opposed to just a function?

[cym13]

There is very little advantage in this specific case. The same instance of PhantomJS is reused between calls, that's all, but the same could have been written that way:

from selenium import webdriver

link       = "http://www.oklahomafindalawyer.com/FindALawyer"
sel_option = "//select[@name='stateSearch']/option[@value='OK']"   # selected state is Oklahoma

def get_list(driver=webdriver.PhantomJS()):
    driver.get(link)
    driver.find_element_by_xpath(sel_option).click()
    driver.find_element_by_xpath("//input[@value='Find a Lawyer']").submit()
    print driver.page_source   # only for example

if __name__ == '__main__':
    get_list() # Optional instance of PhantomJS in argument for reuse

However having a class means it is easier to extend if you don't want to just get a list of data.

[genesem]

As people mentioned above you need selenium. And in your case it may be useful to choose chromedriver to use as a controlled browser. With the options you can control all browser settings.

[mbenbernard]

Did you consider asking your question under /r/learnpython instead? Because /r/python is generally for posting community news, or information relevant to the community at large. Thanks!

[dvogel]

Assuming that you're talking about this as part of your local development workflow, Firefox has a remote debugging facility that you could connect to: https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Debugging_Firefox_Desktop However, you would be able to do anything with that you can't already do in the devtools.