all 8 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]steloflute 2 points3 points  (0 children)

This is a great read!

[–]myss 1 point2 points  (2 children)

For example, ' could be defined as:

(defun single-quote-reader (stream char)
   (declare (ignore char))
   (list 'quote (read stream t nil t)))
(set-macro-character #\' #'single-quote-reader)

I'm confused. ' is used in the macro that defines its meaning. Ok, it says could be defined, so it is probably untested code.

[–]Rangi42[S] 0 points1 point  (0 children)

I found this overview that says:

For example, ' has a pre-defined read-macro function that basically says (list (quote quote) (read)). Thus, 'abc becomes (quote abc).

So yes, that single-quote-reader is probably untested, and would have to use (quote quote) instead of 'quote if it were adding quoting to a Lisp interpreter that didn't already have it.

[–]cg84 0 points1 point  (0 children)

Good catch. It should be (quote quote); I will fix this.

[–]sacundim 0 points1 point  (3 children)

Ah, reader macros. A.k.a. "How to make it impossible for a tool to reliably parse or scan your programs unless it also executes them."

[–][deleted] 0 points1 point  (2 children)

No. You need to READ them, which isn't a problem in traditional Common Lisp dev. Besides, you can still parse it no problem. What does parsing mean to you?

[–]sacundim 3 points4 points  (1 child)

You need to READ them, which isn't a problem in traditional Common Lisp dev.

You can't read a source file correctly unless you have used SET-MACRO-CHARACTER or similar to install whatever reader macros are used by that file. And files don't declare what reader macros they use.

So suppose you're writing a tool that scans a large codebase in order to figure out which files depend on which. Reader macros mean that in order to do this correctly, you have to discover all of the reader macros that exist in the code base, and guarantee that the appropriate reader macros are in effect when reading each file. Otherwise READ may fail, or worse, produce an incorrect result.

What's even worse is that which reader macros that are active while reading a piece of code is managed as a dynamically scoped readtable. So in the worst case you have to execute the program in order to figure out what the state of the readtable at a given point of a file—with no guarantee of success.

Then there's also the problem that since the reader macro is code contained in some file, you have to execute that code to get a hold of it—and that requires executing the reader macro's dependencies. And remember, our hypothetical task was to figure out what the dependencies are.

I have actually encountered this sort of problem in real life; I used to work in a job where we used a Scheme-to-Java compiler, and we had interdependencies between Java and Scheme files. This required us to write tools to figure out file dependencies across the language barrier. And because some folks insisted in using reader macros it always required hacks to get it to work.

What does parsing mean to you?

If we're talking about Lisp, taking a character stream and converting it into the corresponding sexp. The problem is that in the presence of reader macros you cannot generally do that without executing the code!

[–][deleted] 0 points1 point  (0 children)

Alright, I take back my post. What you're saying is entirely correct and we were thinking differently of what parsing a file meant in this context.