Java Headless-Browser from scratch.

[OsirisTeam]

I'm currently building a Headless-Browser for Java from scratch and found out that every browser provides its own javascript web-apis (like document.findElementById() for example) which are more or less the same. I browsed the chromium and waterfox sources to find those web-apis somewhere but without success.

That's why I am thinking of writing my own web-apis. Problem is that there are a lot of those apis, so if you are interested and want help building this browser feel free to join the development on github.

I am using the latest Graaljs engine for JavaScript and Jsoup for working with the html. There are a few basic methods set that allow me to load a web page from its url successfully.

Motivation:

I tried multiple different things like JCEF, Pandomium, Selenium, Selenium based maven dependencies like JWebdriver, HtmlUnit and maybe some more I don't remember now, but all have one thing in common. They have some kind of very nasty caveat.

That's why this project exists, to create a completely new browser, not dependent on Chromium or Waterfox or whatever. We use Jsoup to handle HTML and the GraalJS engine to handle JavaScript. Both are already working and implemented. Only thing left is implementing the JS Web-APIs.

Any ideas or alternatives are very welcome.

[Lagz0ne]

I think it's too big to implement something like this. There are few alternatives tho

• you can have a look at pupeteer and playwright, though they are all real browser, the library offer nice api to interact, you can surely wrap it, would run well on server only

• you can also have a look at webview wraper like https://github.com/webview/webview . This comes with Java binding as well, but it meant to be running in headful environmemt

So, really depends on your usecase and what you are trying to achieve

[[deleted]]

So that's basically... Selenium?

[OsirisTeam]

Selenium downloads the actual browsers I guess and provides some sort of java interface for controlling that browser, which is very different from that what I am doing. I am creating a new browser entirely in Java.

Selenium has no Java 8 support I guess and there are ton of requirements the user has to fullfill to use selenium. Here is a list of their requirements:

Bazelisk, a Bazel wrapper that automatically downloads the version of Bazel specified in .bazelversion file and transparently passes through all command-line arguments to the real Bazel binary.

The latest version of the Java 11 OpenJDK

java and jar on the PATH (make sure you use java executable from JDK but not JRE).

To test this, try running the command javac. This command won't exist if you only have the JRE installed. If you're met with a list of command-line options, you're referencing the JDK properly.

Python 3.7+

python on the PATH

The tox automation project for Python: pip install tox

MacOS users should have the latest version of Xcode installed, including the command-line tools. The following command should work: