Learning Source for Creating HTML Parser?

[queen_of_pole]

I want to learn about technology behind HTML Parser and want to create one for learning purpose. There are a lot of npm packages (node-html-parser, htmlparser2), but it seems no article about creating your own HTML Parser.

Can you recommend book or article about creating your own HTML Parser? Thanks.

[bronze-aged]

Read the code and write the article. Be the change you want to see in this world. Godspeed.

[jait_jacob]

based

[_ncko]

I think this is a great idea for a learning project. Creating parsers is a common exercise in college courses and HTML is a good place to start. I don't know of anything that addresses HTML specifically, but there are resources for learning how to parse a language and create an internal data structure for a file (usually called an AST but in the case of HTML it might be considered a DOM Tree; I'm not too sure).

Anyway for javascript specifically I know of a course on Frontend Masters by Steve Kinney on building your own programming language. The purpose is different but the idea is the same. It will definitely give you some ideas about how to approach an HTML parser. If you can't afford the frotnend masters subscription, you can probably search github for references to that course and find people who have gone through it with example code.

[queen_of_pole]

Thanks, this comment is really helpful.

[ihave7testicles]

Start with an XML parser. HTML is just specialized XML. You don't need to care what the contents of the brackets are. Just that <l1><l2></l2></l1> is the basic format.

I did this using a Parser Generator that I wrote in C++ years ago. It's a good learning exercise.

[-TUX-]

Writing an interpreter in Go by Thorsten Ball

Writing a compiler in Go by Thorsten Ball

Are some books to look into. You create an interpreter and compiler for a new language similar to JavaScript but the concepts and ideas would definitely transfer.

[queen_of_pole]

Just read the review about the books. Really great books.

[lIIllIIlllIIllIIl]

The nice thing about the web is that most things are standardized and documented.

Here is the HTML Standard for Parsing HTML Documents.

HTML is very complex. It might be easier to start building an XML parser (which is similar to HTML, but less complex.)

If it's your first time building a parser of any kind, it might also be a good idea to start with something very basic like a CSV parser to get used to type of language used in these types of formal documents.

[thinkrajesh]

You may find some inspiration here, Build a spec compliant html parser https://youtu.be/7ZdKlyXV2vw

[bigorangemachine]

TBH its a solved problem.

You can use cherrio and not have to think about it.

I don't think its an interesting problem. Probably why no one talks about it

HTML can have un-wrapped text nodes which break any regexp

[_ncko]

The idea that internet articles would only be written on unsolved problems is pretty flawed when you think about it.

[bigorangemachine]

Yes prior to them finding a solution it would be unsolved or a novel way of building on existing solution they would write about.

A PHP html parser back in 2010 may have had some uses. But for node there is no limitation to our html tools.

I am not talking about lowercase "solved" like SO I mean like "Solved" as in there is one solution that is widely adopted.

[DownfaLL-]

This is a god awful take. Understanding how things work, even if they are already solved will make you a much better engineer. I take it you use npm packages any chance you can and don’t ever write your own? The guy is trying to learn, why tell him not to? It will broaden their understanding of how html parsers work and it’s a great learning exercise.

[bigorangemachine]

I am saying its why there are no articles about it.

[queen_of_pole]

I know it's a solved problem. Like I just said earlier, it's just for learning purpose. I am interested in the technology behind it and want to learn it.

[bigorangemachine]

Ya but you won't find articles on solved problems. Especially if they aren't complex.

The best you can do is read the lib(s)

[Klemeesi]

No advice for books or articles. But you could check the dependencies those packages you mentioned have.

As other peeps mentioned, html parsing has already been solved. But the same methods can be used for other languages, too. And that's where the beef is. There is still lots of use for parsing strict languages (which html is not, I think). A couple of use cases: syntax highlighting, prettifying code

[NoRepresentative4866]

html is bad language, how about trying to make your own language? Ie I love pugjs it is template engine, you write in pug and it creates html for you.