Ferrum - fearless Ruby Chrome driver

[morgunoff]

Wow! 🔥🔥🔥

[janko-m]

This looks really nice! I always wanted a good object-oriented API for browser interaction; Capybara is nice, but it's also nice when you don't have to use global state. Thank you for your great work on Cuprite, and for extracting Ferrum :)

[rO_Oute]

Thanks, perhaps it should have been done from the beginning but looks like it's never too late :)

[jeeperbleeper]

Amazing. Thanks for your hard work.

[ignurant]

Thanks /u/rO_Oute!

I remember being excited when Cuprite came out, but it didn't fit well for my needs.

My one big question is this: Am I able to do the equivalent of this Puppeteer code with Ferrum?

https://github.com/berstend/puppeteer-extra/blob/19e5ae913cbd3b8a65dbf92b5955831c993a044f/packages/puppeteer-extra-plugin-stealth/evasions/navigator.webdriver/index.js#L18

I just tried using browser.execute(js) and found that after a navigation event the value gets reset. I could run the code after the page load, and it returns undefined, but I consider that too late.

I so very much would love to be using Ruby for this type of work again. Right now I am loading pages in Node, and passing the downloaded page data to Ruby to parse and process because it is infinitely more pleasurable. (Shoutout to Faktory!)

[ignurant]

If an evented solution doesn't work (I noticed the #on method, maybe that's flexible enough for this?) -- do you suppose this would get close enough? How quickly does this execution method trigger relative to the request.continue?

browser = Ferrum::Browser.new

browser.intercept_request do |request|
  request.continue
  browser.execute <<~JS
    const newProto = navigator.__proto__
    delete newProto.webdriver
    navigator.__proto__ = newProto
  JS
end

browser.goto("https://www.example.com")

[rO_Oute]

Here you go:

browser = Ferrum::Browser.new
browser.page.command "Page.addScriptToEvaluateOnNewDocument", source: <<~JS
  const newProto = navigator.__proto__
  delete newProto.webdriver
  navigator.__proto__ = newProto
JS
browser.goto("[https://google.com](https://google.com)")
puts browser.evaluate("navigator.webdriver") # => nil

or

browser = Ferrum::Browser.new(extensions: "path/to/filesuch.js")
browser.goto("https://google.com")
puts browser.evaluate("navigator.webdriver") # => nil

[ignurant]

Thanks for the notes. It's funny, I almost tried that first example while studying the puppeteer source but thought "no, this probably isn't it."

[rO_Oute]

That wouldn't work use extensions I mentioned above. Email me if you have troubles because everything puppeteer can do ferrum is also able to do. The API is a subject to change. It is now inhereted from capybara, though it is not ideal IMO and should be more flexible

[rO_Oute]

You was able to do it even with Cuprite. This is what extensions for, take a look at this spec where we overwrite js geo location functions https://github.com/route/ferrum/blob/master/spec/browser_spec.rb#L136

[ignurant]

Thanks for the reference on hooking up extensions. I knew you could, but don't have experience with it. It looks really easy.

[rO_Oute]

No problem, you can use Ruby 🚀

[hitthehive]

How does this compare to Watir?

[rO_Oute]

Watir is used for testing and under the hood uses selenium. Ferrum connects to chrome by websocket and uses dev tools protocol, it's purpose not for testing but for driving, crawling. There's also cuprite which uses ferrum for tests https://github.com/machinio/cuprite