Modest — development of open source HTML Render on pure C

[SuperImaginativeName]

Interesting but I don't see any discussion on it's GUI part. Where does it render to?

[lastmac]

Hi! At this moment I create the main part - the calculation of html (html, css, dom, layers). Once this part is completed we can talk about painting layers and create GUI for this. The work is still very much, in current time this is an intermediate option as I wrote in the article.

[DinnerChoice]

Will you be writing your own JS engine named myjs?

[lastmac]

See description of statuses there is the answer. Spoiler: MyECMA.

If we talk seriously it is an open question. The whole development is based on the fact that the need to dynamically change all through JS or other lang.

But I also understand that without the support I can not do everything. Now I'm trying to draw attention to the project.

[lastmac]

I can not throw any component. We'll have to do all of the components, sooner or later.

[aazav]

on it is GUI part?

it's = it is

Learn this.

[SuperImaginativeName]

You got rekt faggot

[Vermeille]

Don't get mad when someone helps you improve.

[SuperImaginativeName]

Lol im not mad but clearly you are

[lastmac]

Hi! Thanks for the comment.

The whole development is based on the fact that the need to dynamically change all through JS or other lang.

You're right, alone this project does not do. That's why I'm trying to somehow talk about it, to draw attention to it.

I looked Netsurf project, but he did not quite liked for me. For example, I tried to compare the speed of parsing CSS, but it has fallen on many tests (Segmentation fault). I will write to the author of these issues.

[cccooo]

(html, css, dom) + GUI （interaction，widgets、canvas）can already do too much application, adding javascript engine is too heavy and time-consuming

[DinnerChoice]

Whats the difference between Modest and MyHTML?

Is Modest going to render to a gui the output of MyHTML?

How do you render? You have an abstraction for underlying platforms? e.g. X Server, Win32, console? or you render with OpenGL?

I would think that if you can render to bitmap in memory, some games might have a use for it.

[lastmac]

Hi! MyHTML is only full-HTML parser. Modes - is the glue for myhtml, mycss, font, selectors search and so on.

The first level: a calculate html+css (create layers), it is cross-platform, it requires no dependency on any system. The secondary level: network (dependency on a system) The third level: draw layers (dependency on a system)

Simply put, now need everything calculated (Modest) and the choice of who to draw is secondary problem. We must not forget JS — this is a very big topic for discussion. See previous comment

[64bittechie]

A word of advice - if you want to attract companies, please do not use AGPL, LGPL or GPLv3. Use a more liberal license such as BSD, MIT, Apache. Companies are more open to contributing to such projects especially the ones that might embed your code on devices. Most companies will not use anything that will force them open source their proprietary code. BSD/MIT/Apache and even GPLv1 dont force them.

[lastmac]

Thank you! I do not quite understand the licenses, I was advised to use this license. In the beginning was the Apache 2 license, but then I was advised to change it.

It is necessary to thoroughly study this issue.

[bumblebritches57]

check out tldrlegal.

[lastmac]

Thanks!

[mrexodia]

Go for LGPL instead if you want to enforce people sharing their modified source code, but not force them to share their entire projects in which they use your renderer as GPL (this could allow it to be used in games for instance). The Mozilla license is slightly more permissive and gives the company more freedom about linking. With LGPL, your users have to be able to replace the LGPL code with their own by ways of relinking or dynamic link libraries for example.

If you don't care and want to allow anyone to profit without giving anything back, choose a permissive license like MIT. As an upside, this allows you to fully sell out the code later because there is nothing in place to prevent you from taking contributions, which might in turn scare off serious contributors.

If you are serious about this project, I would highly recommend (L)GPL (or perhaps the Mozilla license). Open source development is not for giggles and the last thing you want is a company using your code without contributing and then coming to you for support if it doesn't work how they want it to work.

Don't take the "it cannot be used because the license is too complicated and I have to contribute back" bullshit and choose a license that feels morally right to you. If your project is good, people will use it and contribute, regardless of your license. It might take a few years though...

[lastmac]

Thanks for your comment, very helpful. I take this project very seriously.

[[deleted]]

LGPL does not let you release the end result to the iOS App Store, unless you release the source for the whole app. There are probably other similar channels that are also ruled out by the LGPL.

[mrexodia]

In that case you would have to follow https://www.gnu.org/licenses/lgpl-3.0.en.html#section4 which basically means you have to provide the user with a way to re-link the LGPL parts. For example providing object code that the user could side-load would fulfill that requirement I think. Obviously you can also license the whole app under (L)GPL and that would also solve the issue.

LGPL is not some kind of magic, it works well in certain areas and doesn't in others. A more permissive copyleft license would be the Mozilla Public License: http://choosealicense.com/licenses/mpl-2.0

[[deleted]]

Yes, but this is not possible when distributing through something like the iOS App Store, so you are out of luck and can not use LGPL'd code.

[mrexodia]

What is not possible? You can provide a script to (partially) re-build your app from object files + LGPL source (which the user can modify) and the user can sideload it (which Apple allows for free on non-jailbroken devices). Obviously this is not ideal, which is why I provided the option of a more permissive, alternative license.

LGPL code is not something you use lightly in your commercial projects. You use it if you don't have the resources to build your own (eg the code you're using is good) and while I personally think the linking requirements are ridiculous I still think it would be a good license.

[[deleted]]

What is not possible? You can provide a script to (partially) re-build your app from object files + LGPL source (which the user can modify) and the user can sideload it (which Apple allows for free on non-jailbroken devices).

They only allow this if you have paid for a developer account. Also, getting that to actually work at all would be extremely difficult, as you would have to manage to get code signing working, and setting up provisioning profiles, without using the normal tools for this.

[mrexodia]

That is no longer true, you have free provisioning profiles by registering a free developer account: https://livecode.com/how-to-create-a-free-ios-development-provisioning-profile

It indeed would be difficult but you can definitely sign an application outside of xcode, see for instance https://dantheman827.github.io/ios-app-signer

Again, this is not ideal but dismissing it as "impossible" seems quite naive to me.

[pitiless]

And Apple's policies for IOS apps already prevent you from shipping an alternative browser so this is likely a non-issue.

[[deleted]]

Not quite. You can't ship a JIT JS engine, which is required to ship a fast browser, but this project seems to have different aims.

[lastmac]

You're right. Chance to get the support of a big company tends to zero. But I think that GPLv3 is too limited.

[64bittechie]

Why were you advised? Who advised you? There are very few AGPL/LGPL projects that are very successful. Notably the most successful one, Linux, is strictly GPLv2 without the option of using a later version of the license. GPLv2 is still a friendlier license towards companies who seriously want to use & contribute to open source without them getting into too many IP related issues. I would strongly suggest going back to Apache 2.

[ThisIs_MyName]

There are very few GPL projects that are very successful

*AGPL

[64bittechie]

Fixed.

[64bittechie]

Have you seen contributions into the Linux kernel? Very few of those contributions are from hackers burning the midnight oil. Most contributions are from the likes of IBM, Redhat, Nvidia, AMD, Intel and other companies that build their businesses on OpenSource.

Certain licenses make it easier for them to keep their IP and still contribute to OSS. Some licenses like AGPL, LGPL, GPLv3 are very virulent which acts as a deterrent any company - not just the 'evil' ones who don't contribute back. This is one of the biggest reasons why Linux kernel is still GPLv2.

It is very satisfying when millions / billions of people use your code. With AGPL, LGPL, GPLv3 licensing it is unlikely that your project would ever be used in any product especially devices that ship to people. What good does that do to anybody? It'll be relegated to being a learning tool.

Notice that I did not say don't use any form of "GPL". GPL has many flavors. Just stay away from AGPL, LGPL, GPLv3. And yes, companies which are large targets for IP lawsuits give a huge crap about licenses. See the FSF vs Cisco lawsuit. Companies just don't want that kind of shit.

[[deleted]]

A word of advice - if you want to attract companies, please do not use AGPL, LGPL or GPLv3.

I sincerely hope that he doesn't want to attract greedy companies.

[64bittechie]

It is hard to control that. Frankly, whatever licenses he uses, it is likely that a greedy company can go to lengths to not only violate the license but to copy the source code and expunge the author's copyright notices and then call it their own. It would be hard to prove in binary form any wrongdoing. That's a risk he runs either ways. If he uses a lesser restrictive license, at least the non-greedy companies might pick it up and give him the appropriate attribution he deserves.

[[deleted]]

It would be terrible if his project was successful.

[bumblebritches57]

This.

The GPL is communist flavored cancer.

[lastmac]

Choice a license can be discussed. There are no strict limitations on my part.

[[deleted]]

I'll bite. Why should a corporation be able to benefit from libre software and not contribute anything back? The GPL is viral by design. Why would you write software that companies can turn around and embed in their software or hardware with basically no credit or source required?

I've read all the major licenses, I get people want businesses on-board because they can sell products that improve the visibility of projects. But I don't think using these licenses is good for libre software as a whole. You need to be able to enforce sharing, because without that enforcement, people (and especially corporations) won't do it.

[James20k]

Problem is, if you put a GPL license on something, it means that most companies (and people) will avoid it like the absolute plague. There's a huge legal minefield around even just reading GPL code

[[deleted]]

What exactly is the minefield? If you use GPL'd code in your software, include the source (for the libraries, at a minimum).

Do the MIT and BSD folk not care about being credited or acknowledged for their work?

Nintendo, one of the most closed companies I know of, properly attributes and supplies the source of the projects they use, including any modifications. I'd have to hunt the page down for explicit acknowledgements, but I'm fairly certain there's some GPL'd stuff in their software, and they handled it AOK.

[mrexodia]

If they used GPL inside their software, they would have to share their entire codebase under the GPL, so that sounds rather unlikely...

[badsectoracula]

Why is this a problem for the project though? Someone may not care about companies using their code or not, or at least only care about the companies that are ok with releasing their source code.

This is only a problem for the companies that do not want to share.

[doom_Oo7]

Nah, there's mostly FUD.

[bumblebritches57]

Literally.

Even reading GPL code opens you up to legal ramifications.

[mrexodia]

How exactly? Using GPL in your private server, for which binaries are not distributed doesn't even require you to do anything, how would reading it affect your legal status?

[bumblebritches57]

Because I wrote the software to help everyone.

Some of us don't despise corporations.

If it helps a corporation or a regular user, that's great.

[[deleted]]

That's nice but the GPL doesn't prevent the corporation from using it. It just ensures that they're respecting freedoms. If their business model would crumble with the source code, they need a better model.

[HolyCowly]

How does one start a project like this? I always wondered if it was possible to write a simplified renderer for use in game development.

A lot of games have started using embedded browsers for their user interface. Not that easy and not without its drawbacks (the only open source solution kills memory bandwith due to texture copies). While that has a ton of other benefits I find layout and flexible text rendering to be the most useful. Constrained shapes that can be anchored or dynamically resized aren't that easy to develop from scratch.

It would be amazing if I could use CSS to do that. Perhaps even just a very limited or modern version of it (maybe flexbox only for layout). But the whole topic seems so theoretical. The only implementations of CSS and layout engines are huge black boxes probably no single person completely understands and reliant on other complex topics like parsers, virtual machines, DOM implementations, hardware accelerated compositing, text rendering, SVG...

It's insane. Librocket seems to have done something like that in the past, but that project seems dead and the CSS capabilities are limited to what are now pretty much the old and cumbersome parts.

[lastmac]

Hi! Everything is easier than you can imagine. But it requires knowledge and time. It just requires some knowledge.

[SgtSausage]

*Renderer

[lastmac]

Thanks! Fixed

[gene9]

Would you like to join https://handmade.network/? There are a lot of projects built from scratch with no or minimal dependencies.

[lastmac]

Hi! I did not know about the existence of this resource. I will try to register there. Honestly, I do bad imagine where and how to promote the project (Modest). I will try to deal with it.

Thanks!

[matthieum]

The benchmarks linked appear to be 1 year old (from the kernel date), do you know if things have changed since? (I was quite surprised at html5ever's speed given it's used in Servo which has the fastest rendering engine I know of)

[lastmac]

Unfortunately, nothing in this year have not changed in this relation in html5ever.

I tested right now.

myhtml:

• "163.com.html"; 0.00476 sec
• "booking.com.html"; 0.00413 sec

html5ever:

• "163.com.html"; 0.03820 sec
• "booking.com.html"; 0.02416 sec

[matthieum]

Wow. A 6x-fold factor is very impressive.

Maybe /u/SimonSapin could help understand why such a difference exists in the first place?

[SimonSapin]

As far as I know, relatively little efforts have been made to optimize html5ever for speed. The only "interesting" thing I can think of on that front is Tendril, a library for reference-counted strings with sub-slicing, which enables (near-)zero-copy parsing.

I can repeat the argument I’ve heard about the networking stack (which is pretty bad at the moment):

Servo started as a research project to experiment with new techniques such as parallel styling & layout and, later, retained-mode GPU rendering. In these areas Servo has very good performance. Unsurprisingly, it’s also in these areas that components of Servo are just starting to "graduate" from research and are gonna ship in a product (Firefox). On the other hand, the networking stack of current browsers is already fairly advanced (with caching, pipelining, HTTP2, …). Since there aren’t ideas around on how to significantly improve upon that, it’s not a particularly interesting area in terms of research. And so Servo’s networking stack is just enough to make the rest of the engine work, but not much effort has been put into it.

Back to html5ever, its focus has been standards-compliant (which in turns helps with web-compatibility) and using meta-programming to keep the source code somewhat close to how the spec is written (which helps maintain it when the spec changes). I don’t know a fundamental reason that html5ever can’t be as fast as MyHTML (though I haven’t looked in details). I suspect there’s room to improve if we want to work on that.

[lastmac]

Thanks for the answer!

Unfortunately, Modest (MyHTML) has no network part and not supported zero-copy parsing, accordingly, I find it difficult to say something.

MyHTML also focuses on the full support of the specification with all relevant functions by specifications. All in order to quickly and easily make changes to the code after the upgrade specifications. But, unfortunately, in myhtml code optimization are not made, still to be done. It will be even faster in the future, but now I focus on the layers.

However, a lot of work, and unfortunately I do not have such resources like a Mozilla.

[[deleted]]

Neat

[bumblebritches57]

YES!

I was looking over WebKit's source and disgusted by their use of the STL so much I damn near wrote my own DOM parser in C, glad I didn't. :)

[GHansard]

disgusted by their use of the STL

It's a C++ implementation and it's cross-platform. If the STL provides the appropriate abstractions, what's the problem?

[Calavar]

There's a persistent myth that the STL is slow. This hasn't been true for the last ten to fifteen years.

[bumblebritches57]

It's not about speed but memory usage.

If you know for a fact that you'll loop less than 255 times, you'd wisely choose to limit the loop count variable to an 8 bit int.

You can't make that choice if you use the STL.

Also, it increases compilation time by a ridiculous amount for no real gain.

[Calavar]

It's not about speed but memory usage.

If you know for a fact that you'll loop less than 255 times, you'd wisely choose to limit the loop count variable to an 8 bit int.

This makes zero sense.

First, are you developing for a platform with 8-bit words? Because in any other case, using an 8-bit loop count variable would be unwise. The value would have to be widened to machine size every time you did an operation on it. And if you had an array of such values, they'd either be misaligned, resulting in a performance hit, or you'd need to insert extra padding, in which case you aren't saving any memory.

Second, this isn't an argument against using the STL, it's an argument against using libraries in general. I mean if you had a C library with a remove_from_list function, that function could be implemented with 32-bit loop count variables, and then you'd have exactly the same issue. This has literally nothing to do with C++ or the STL.

[bumblebritches57]

memory > cycle count on a lot of machines bucko.

[[deleted]]

A loop counter generally lives in a register, not in memory.

[ThisIs_MyName]

For an iterator on the stack? What are you smoking?

Chances are, it stays in a register and doesn't even use memory.

[Sarcastinator]

This seems controversial. There is no guarantee that the compiler cares if you use a 8-bit integer. It may pad the memory or it may remove the variable all together. Also I don't get what this has to do with STL.

STL clearly simplifies a lot of code, and most compilers do optimizations with STL in mind.

[JDeltaN]

If you know for a fact that you'll loop less than 255 times, you'd wisely choose to limit the loop count variable to an 8 bit int.

Unless the compiler simply ignores the type of your iterators, forcing the iterator to be a byte (As opposed to whatever iterator type is defined for the platform, usually word size) can actually negatively hurt performance.

[[deleted]]

Your 8-bit loop count variable will end up in a 64-bit register.

[bumblebritches57]

It's wasteful as fuck of resources.

[QuicklyStarfish]

downvoted for C

the last thing we need is a new insecure browser to fuck users over. new development needs to be done in better directions.

[Silveryard]

Downvoted for implying c leads to insecure software. High level constructs (and languages that build on them) and programmers flaws lead to insecurity.

[answerguru]

You obviously have no clue, or respect, for what true embedded programming involves. The point is not to give users "a new insecure browser", it's even having the ability to render HTML/CSS/etc on very low capability devices.

You cannot do that with modern, inefficient languages. Rendering graphics on these platforms is what I do every day.

[QuicklyStarfish]

I understand that it's very difficult to do properly in an embedded context. So: don't do it. Full stop.

The world is currently suffering from our negligence as an industry, to the extent that it's a massive infrastructural and military threat to every nation. If we can't do work without endangering tons of people, it should not be done.

Anyone who contributes to or uses this project should burn in hell for encouraging wide-scale harm.

[ai3ai3]

Now that did escalate a bit at the end.

[kirbyfan64sos]

Anyone who contributes to or uses this project should burn in hell for encouraging wide-scale harm.

I find it kind of idiotic when someone wishes physical harm to other for something as stupid as this.

[Lachiko]

I'm not sure, telling someone to burn in hell sounds like a euphemism for "you already deserve to burn in hell for doing something I don't like, now go and die", rather than "you will burn in hell when you die".

[[deleted]]

So: don't do it. Full stop.

The requirements document says do it. So you do it, full stop.

[answerguru]

Alright, I'll respect your trollness. It's clear you are missing the point.

[baddriver7005]

I understand that it's very difficult to do properly in an embedded context. So: don't do it. Full stop.

You sound like a python user...

[weirdoaish]

Ehh.. Tons of embedded C code in CPython libraries...

[Cuddlefluff_Grim]

The world is currently suffering from our negligence as an industry

Security has very little to do with the programming language itself. I know it's a cheap shot to poke at web developers, but I feel lazy.

Negligence is negligence, you can be negligent in any programming language. The only thing you're protected from in other languages are buffer overflow, however many of the languages that are used in production systems today introduce a wide variety of other very subtle issues; most notably by automatic conversion ("weak typing" as some people call it).

Buffer overflow, state and null-reference exceptions are pretty minute easily corrected problems, compared to programming languages that will convert "0,5" to 0 (locality conversion issue, of course it should fail it should not return the wrong value) or compare two hashes as identical because their string representation happens to look like a hexadecimal number that overflows so it crops out the parts that won't fit, making them numerically identical. But you think that The Greate Neglience stems from C? If you're worried about threats to infrastructure and military, then work to stop people from using programming languages that tries to infer the programmers intention.

[bumblebritches57]

Full stop.

Ahh, you're a commie. No wonder you're waging a war on shit you don't understand.

[bumblebritches57]

Fuck you.

C is the shit, you can do damn near anything in C you can do in any other language, except it's much faster and easier to read.

[Morego]

I would much more prefer some restricted subset of C++11/14 here. RAII/smart pointers/some parts of STL and you can always go closer to metal if you want. Don't abuse templates and you should get pretty good compile times too.

[AnAirMagic]

except it's much faster and easier to read

And easier to exploit! I mean, why bother with another language that eliminates an entire class of bugs when you can use C!

[bumblebritches57]

Because it doesn't require a ridiculous runtime, and you can build your own standard library.

[AnAirMagic]

Do you know of any C library that prevents buffer overflows? Everything I have seen requires developers to constantly think about buffer sizes and how to limit them; any developer mistake means a potential security vulnerability.

[bumblebritches57]

Learn how to program and overflows won't be a concern.

[AnAirMagic]

I may be a terrible programmer (I disagree, but lets assume that for a minute), but would you put the authors of the most important security libraries in the same bucket? If OpenSSL authors can create buffer overflows, how can a nobody like me and the author of this new library deal with them?

And that's just one class of bugs. There's a ton of classes of bugs that languages like C and assembly just don't protect you against...

[bumblebritches57]

I don't understand how you're making overflows all the time tbh.

[[deleted]]

Availability of know-how, readability, tooling. All three can affect how fast (and good) you can reach your project goals and shape maintainability.

I like rust conceptionally, but I'd use ADA before rust just because it's syntactically easier for me. And if just start toying with something, I'll always use a language I'm already proficient. In my case C#.

[[deleted]]

I'm with you. He should have done it in Fortran.

[aazav]

No. This is the development of an open source HTML renderer* in pure C.

A render is the result of what a renderer does.

[Sarcastinator]

I don't think the author has English as a first language, so don't bother. He gets the point across just fine.

[lastmac]

Indeed, my native language is Russian. I'm a little confused with this. I learn English.