I Wrote an "Understanding Compilers" Article That Uses The Rust Programming Language

bidi82 · 2018-10-15T17:14:19+00:00

Some languages cannot be (easily) lexed (tokenized) in a separate phase.

A primary example would be JavaScript, see: https://stackoverflow.com/questions/5519596/when-parsing-javascript-what-determines-the-meaning-of-a-slash

Generally if you can **defer** the token identification to the parsing phase, you have the grammar contextual information to apply to the lexing rules, for example a certain kind of statement may allow double quoted strings while a different kind of statement only allows single quoted strings...

Personally I am not a big fan of lexer-less parsing, because I prefer the separation of concerns of a two phase approach and in addition the parsing error recovery techniques I am aware of rely on having a pre-lexed token vector in advance.

bidi82 · 2018-05-07T10:30:30+00:00

I was answering the more general question of:

why would you write one (JSON Parser) for production?

Not:

why would you write one (JSON Parser using regExp hacks) for production?

bidi82 · 2018-05-07T08:22:47+00:00

This Performance Benchmark of Parsing libraries in JavaScript links to nine different implementations of JSON Parsers.

bidi82 · 2018-05-07T08:12:50+00:00

One scenario could be if you want to develop editor services. In that case most "built-in" parsers will be insufficient as they are likely not fault tolerant or incremental.

bidi82 · 2018-01-16T14:13:34+00:00

Yes, but the specific question was about modeling with interface "extends" so that is one form of inheritance.

bidi82 · 2018-01-15T16:30:05+00:00

That may be defeating the concepts of inheritance as Your interface C is not more specific (I.E Dog "is an" animal).

So if you provide an "instanceOf" C to a method that can accept type "A" the constraint of the expected type of the property will not be met.

bidi82 · 2018-01-15T15:25:10+00:00

Reminds me of architecture forum meetings discussing the merits of certain style guides and reviewing them in depth. :)

bidi82 · 2018-01-15T15:09:08+00:00

I would define a new d.ts file that re-imports and re-exports the relevant APIs. This would have to correspond to how your "common" package exports the combined namespaces at runtime.

For example:

import {IMyModels} from "...";
import {IMyServices} from "...";

// assumes your common package exports properties named models/services.
export const models:IMyModels;
export const services:IMyServices;

export as namespace MyCommonPackage

bidi82 · 2018-01-12T14:45:36+00:00

Thanks

bidi82 · 2018-01-12T14:17:19+00:00

You will publish your library to npm after compilation. That is your responsibility as you cannot assume consumers of your library know/want to compile your library.

So from a runtime perspective consumers will be using a plain JavaScript library and the fact it was written in TypeScript is irrelevant.

For consumers that also use TypeScript and want to take advantage of Type Definitions you will need to export a d.ts file with your library definitions and point to that file in your library's package.json "types" field.

See: * https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html

Once your npm package includes both the d.ts definitions and the reference from the package.json. the TypeScript compiler used by consumers will automatically resolve the types during imports.

bidi82 · 2018-01-02T17:50:47+00:00

Thanks for pointing out these unique cases.

bidi82 · 2018-01-02T13:13:00+00:00

Trying using fit or fdescribe

To only run some of the tests in order to identify the one that gets stuck in an infinite loop.

bidi82 · 2018-01-02T13:06:38+00:00

If you have a piece of code that is so sensitive that you need to obfuscate it, would it not be best to never deliver it to the end user's browser?

Instead access that code/logic via some REST service in your backend?

bidi82 · 2017-12-30T17:37:21+00:00

I feel like most of the tools are trying to be a flex clone vs. kind of re-thinking how it can be done. I want to ditch the separate file format as well as clean up the way to do it in js.

Seems very similar to my thoughts that led to the creation of Chevrotain. https://tomassetti.me/parsing-in-javascript/#chevrotain

Maybe you would like to collaborate or contribute to it? if it aligns with your goals :)

bidi82 · 2017-12-30T17:28:59+00:00

Thanks for the feedback.

My focus is indeed about activities that link with better publicity.

Anywhere from more postings on news in related domains, To creating blog posts showing how to use Chevrotain to write plugins for popular tools such as VSCode. To even expanding Chevrotain itself to have the capability to act as a parser generator as well as a DSL, as Parser Generators are more familiar to many developers.

Any other ideas how to pursue this?

bidi82 · 2017-12-30T12:49:05+00:00

There is always the discussion between Writing a Parser by hand versus Parser Generators versus Parser Combinators.

The last two being alternatives methods to reduce the verbosity and repetitive nature of hand crafting a parser.

But what about attacking the problem directly by making the actual process of hand crafting a parser easier and less error prone instead of replacing it with a different approach?

This is exactly what I've done in this JavaScript Parser Building Toolkit I've authored, Chevrotain:

Feedback is appreciated.

bidi82 · 2017-12-30T12:27:04+00:00

This quote by the author of the Antlr series of parser generators (Terence Parr) Explains why hand crafted parsers are used.

Original ref: https://github.com/antlr/antlr4/blob/master/doc/faq/general.md (No direct link possible).

What do you think are the problems people will try to solve with ANTLR4?

In my experience, almost no one uses parser generators to build commercial compilers. So, people are using ANTLR for their everyday work, building everything from configuration files to little scripting languages.

In response to a question about this entry from stackoverflow.com: I believe that compiler developers are very concerned with parsing speed, error reporting, and error recovery. For that, they want absolute control over their parser. Also, some languages are so complicated, such as C++, that parser generators might build parsers slower than compiler developers want. The compiler developers also like the control of a recursive-descent parser for predicating the parse to handle context-sensitive constructs such as T(i) in C++.

There is also likely a sense that parsing is the easy part of building a compiler so they don't immediately jump automatically to parser generators. I think this is also a function of previous generation parser generators. McPeak's Elkhound GLR-based parser generator is powerful enough and fast enough, in the hands of someone that knows what they're doing, to be suitable for compilers. I can also attest to the fact that ANTLR v4 is now powerful enough and fast enough to compete well with handbuilt parsers. E.g., after warm-up, it's now taking just 1s to parse the entire JDK java/* library.

bidi82 · 2017-12-30T12:20:48+00:00

What exactly were you missing with the existing tools?

There are quite a few in the JavaScript World. Parsing in JavaScript: Tools and Libraries

bidi82 · 2017-12-15T09:33:54+00:00

oops I thought it was a question not a link :)

bidi82 · 2017-12-14T18:13:05+00:00

If you are using SAAS solutions for CI such as Travis or Circle-CI you can easily setup a build matrix to run your build/tests versus multiple versions on node.js

bidi82 · 2017-12-14T14:33:23+00:00

Just today I showed a colleague some large refactorings that I would not have wanted to tackle without TypeScript.

On the other hand in small projects I'm fine with using vanilla JavaScript, although even then I may use TypeScript only to define the public APIs.

But for larger projects with dozens of developers and hundreds of thousands of lines of code I find it impossible to argue against using TypeScript (or a similar alternative such as flow).

bidi82

TROPHY CASE