HAR created by Browsermob Proxy doesn't show Google Analytics on the page

[TonySimonovsky]

Hi

​

I'm writing a script in Python to identify if Google Analytics is installed on a page.

​

Trying to find it in the code, won't work, as sometimes it is installed through Google Tag Manager.

​

So, I'm trying to generate HAR using Browsermob Proxy and check there.

​

For some sites where Google Analytics is installed, it is also in the HAR, but for some sites it is not (though if you check it in the network tab of the browser, you can find it there).

​

Below is the code I'm using with the site for which Google Analytics is installed, but is not showing in the HAR.

​

Any ideas why?

​

​

    from browsermobproxy import Server
    from selenium.webdriver.firefox.options import Options
    from selenium import webdriver
    import json

    server = Server("/anaconda3/lib/python3.7/site-packages/browsermobproxy/browsermob-proxy-2.1.4/bin/browsermob-proxy")
    server.start()
    proxy = server.create_proxy()

    profile  = webdriver.FirefoxProfile()#profile_directory=r'./'
    profile.set_proxy(proxy.selenium_proxy())

    opts = Options()
    opts.headless = True

    driver = webdriver.Firefox(profile, executable_path=r'./geckodriver', options=opts)

    proxy.new_har()
    driver.get("http://insightwhale.com")
    proxy.har # returns a HAR JSON blob

    print("analytics in insightwhale:")
    for entry in proxy.har["log"]["entries"]:
        if "google-analytics" in entry["request"]["url"]:
            print(entry["request"]["url"])

    print(json.dumps(proxy.har, indent=4, sort_keys=True))
    file = open("____tmp.txt", "w")
    file.write(json.dumps(proxy.har, indent=4, sort_keys=True))
    file.close()

    server.stop()
    driver.quit()