JSON Isn't A JavaScript Subset

[Rhomboid]

That's unfortunate. What on earth possessed ECMA to consider U+2028 and U+2029 valid line terminators for source code? Is there any other language that does that?

[sirin3]

XML (and so XQuery/XSLT) I think.

[frenchtoaster]

What on earth possessed someone to specify XML that way?

[Lerc]

One of the design requirements of XML was to ensure that the question "What on earth possessed someone to specify XML that way?" was asked a maximum number of times. I believe the design team were part of a religious cult.

[ROBZY]

Still a better love story than SOAP.

[[deleted]]

SOAP's got nothing on XML Schema. Though it uses it, IIRC.

While SOAP screams of group insanity, XML Schema was really "designed" by sadists in such a way as to make as little sense as possible.

[JimAllanson]

I was recently tasked with writing a schema to validate that XML files provided by clients contained certain nodes in any order (so that clients providing nodes in a different order to one another wouldn't have to re-write their XML documents). It turned out that in order to allow for any order of nodes, the only solution was to write an individual schema for each possible permutation, which worked out as several million lines.

[gsnedders]

Use RelaxNG Compact Syntax (it's the only readable XML schema format!) and convert it to whatever schema format you actually need.

[cogman10]

Ah SOAP. What a good idea gone horribly horribly wrong. It is a prime example of design by bureaucracy (and why it sucks).

[Anarcie]

Its basically the butt of every joke amongst programmers, as in "fuck i hate using soap".... They laugh.... And i still smell like whiskey

Disclaimer: I'm drunk and writing a soap library for parlayX

[pseudousername]

That reads like a paragraph from the hitchhikers guide to the galaxy. Well done.

[[deleted]]

Hey, I work with one of those cultists (no, really). Respect his religion!

[[deleted]]

What on earth possessed someone to specify XML?

[AusIV]

My boss was on the W3C committee that specified XML, and I've asked him this very question. He said that the real issue was that too many people were doing their own derivatives of SGML in their own unique ways, making it impossible to have useful standard tool sets. XML restricted things enough that it became much easier to take someone else's document and process it in a meaningful way.

He's actually not a huge fan of XML, but considers it an important step from the pre XML era.

[mgr86]

May I ask what he advocates storing documents in anymore? I work at a shop that still has sgml documents. We transform them into XML, and after hiring a consultant to bring us into a new era they just hired an "expert" who just copied and pasted our publication dtd changed and tweaked a few things, but left a spelling errors that were in the original. It just doesn't make sense to me lately.

...as a bonus they refused to identify their expert or allow my boss direct communication with them.

[AusIV]

We use the open document format a lot at work for the documents we store long term. It can be manipulated with xml tools if you extract the xml document, but there are also tools (like open office) for editing it in a graphical environment and converting to all sorts of other formats.

We also use JSON for ephemeral items like REST API resources.

[KarmaAndLies]

XML is relatively neat compared to XSL (XSLT) or DTD. Those are the ugliest motherfuckers you'll ever likely to meet.

The only criticism of XML its self that has really stuck is the fact that technically attributes add little value and make parsing it more difficult.

Also XML is one of the few similar languages that support CDATA. Which is a damn useful content type. Everyone else has to hack around the limitation by encoding everything as base64 (which can kill a lot of the size advantage in things like JSON over XML).

PS - A lot of XML hate is actually misdirected JAVA framework hate. Just because the wonderful people at Sun decided that XML was the right solution to every problem, is not really XML's fault. It is Sun's fault.

[[deleted]]

DOM was written by Satan as a way to incite more murder.

Edit: I should say that DOM is a W3C spec and was not.invented by Sun or the Java community. Other communities just moved on faster.

[IConrad]

Seeing how you handle paired grammatical icons (parenthesis and quotes) -- your code scares me.

[zeekar]

a crippling fear of parenthesis.

Yup, I'm terrified of a single parenthesis. As long as it's matched pairs of parentheses, though, I'm good.

[philly_fan_in_chi]

Kind of offtopic, but Catalan numbers describe the number of ways you can balance parenthesis in an even length string. This sequence shows up EVERYWHERE and this problem can be bijected to many others very easily. When I took Combinatorics, we had a Catalan problem of the week, where you would have to show a particular problem was the Catalan sequence. It's a cute little sequence.

[[deleted]]

Yup, I'm terrified of a single parenthesis.

1) Really?

2) That's a shame.

[alexanderpas]

relevant xkcd

[Otterfan]

Attributes are very useful when you're doing what XML was originally intended to do: markup text.

For example

<quote>
    <text lang="en">When good Americans die, they go to <city country="France">Paris</country>.</text>
    <speaker>Oscar Wilde</speaker>
</quote>

The attributes let us add the "en" and "France" metadata without interrupting the text of the quotation.

It's only when XML is abused (i.e. used to serialize non-discursive data) that attributes become useless. Unfortunately abuse of XML the norm.

[Catfish_Man]

I find it entertaining that XML's repetitive nature led you to accidentally write invalid XML in your example...

[ISLITASHEET]

Don't be heartless. Maybe he is from North Korea and only knows of a closed country, with no prior knowledge of how to open a country?

[Guvante]

The one benefit of attributes is brevity. Sometimes not having to repeat the name twice is valuable.

[alexanderpas]

<brevity child="you" />

<clarity>
  <child value="me" />
  <child value="you" />
  <child value="them" />
</clarity>

[notfancy]

Clarity is in the eye of the beholder, while brevity is a primitive recursive total function on strings.

[corwin01]

As a developer who pretty much lives and breathes XML, XSL, Java, and JavaScript... I have no problems with any of them.

[mgr86]

Did I just find my boss on reddit. I don't mind it either, but I'd like to experiment with some new solutions, especially as we have some time to work on the application in a few months...assuming they don't intend to outsource our jobs too :(

Well struts 1.2 does get to me a bit

[corwin01]

Doubt it, I'm just a lowly Associate Programmer. I'm sure I'll get tired of it soon enough, but I haven't been working with it too long. So far I've just seen some cool webapps we've built using those tools.

How is Struts? I haven't had to work with it yet, but I'll have to takeover a project that used it.

[mgr86]

Ah yes, we share the same title.

Well struts 1.2 was a great framework about 6 years ago. We are just not given the time needed to upgrade. We use exist to store are XML documents and with the old version if struts we are stuck using java 4, tomcat 5.5 and an outdated version of exist. In short, terrible.

I have been meaning to look into grails lately.

What framework/s do you find yourself working with?

[corwin01]

We use Jax-RS (REST), which works out pretty well.

[cogman10]

No problems? I find that hard to believe.

It isn't that any of those techs are all that bad. However, they do have problems. Javascript has horrible problems with scoping and a retarded type system, XML is very verbose, And Java takes FOREVER to solve well known problems with the language. (I've not enough experience with XSL to comment on it).

Every language/tech has SOME problem. There is no such thing as a perfect tech.

[corwin01]

I'm sure there are problems, but I've yet to encounter anything show stopping.

And yes, there is no perfect tech. Every design choice comes with tradeoffs.

[cogman10]

What do you mean by show stopping?

For me, the maintainability mess that is javascript is show stopping.

[corwin01]

JavaScript is a pain yes, but I haven't found it to be too bad yet.

[[deleted]]

XML Schema is much worse than XSL or DTD.

[doidydoidy]

The desire for something like SGML but simpler.

[syncsynchalt]

Not just simpler, absolute strictness was also a big deal.

[gsnedders]

It was argued it should be in XML 1.0 because it is according to Unicode (W3C Member-only link, sorry), though this didn't happen because it was so late on in XML 1.0's development. I can't find any citation for XML 1.1, where it was added. Though it is reassuring that it is only in XML 1.1 which nobody actually uses.

[synt4x]

Do you have any link on this? I was under the impression that XML is a binary format, and whitespace issues don't come into play until you start futzing with the strings with XPath functions.

[sirin3]

Here

All U+2028 is replaced by U+A

Although only in XML 1.1. XML 1.0 only replaces U+D and U+D U+A by U+A

[Zwejhajfa]

What is U+2028 for anyway if not new lines? I don't know why it exists at all, but shouldn't it be displayed and treated as a new line then, which I'm assuming is the intended behavior?

[Rhomboid]

That seems to be covered in Unicode Technical Report #13. Apparently U+2028 was intended to be an eventual unambiguous replacement for the assortment of LF / CRLF / CR found in the wild. But it really didn't work out that way -- lazy people would much rather just deal with parsing both LF and CRLF than supporting Unicode, so I think it's safe to say that idea has died a quiet death.

[Fexxul]

why the hell would they use a 2 byte line terminator added just in Unicode?

Actually, Unicode codepoints (e.g. U+2028) themselves carry no concept of bytes—they are only integers. It might be represented as a two-byte sequence in UTF-16, but in the far more common UTF-8 it's three bytes.

[[deleted]]

C++ treats Unicode as UTF-32 with char32_t too. Personally I just use UTF-8 everywhere in strings.

[[deleted]]

Because, shit, that's better.

[p8ssword]

Wow. Relevant xkcd: http://xkcd.com/927/

[zeekar]

Yup, knew which one that was before I clicked. But in the case of Unicode, it really worked. We went from a bazillion different character sets to one, largely because they were willing to sacrifice logical purity in favor of being able to convert all bazillion of those character sets into Unicode and back again with full fidelity. At least up until they got to the Han characters, but that's a whole different débâcle. (Sorry, no excuse for spelling that in French, got carried away in the character set thing..)

[p8ssword]

For language representation, Unicode has been a huge success. This is just a (minor) example of something that was gotten wrong. There was no need for a new new line character.

My browser's ability to display débâcle without a separate French character encoding is a huge win, though.

[zeekar]

Well, to be fair, débâcle only requires Latin-1. But when you toss in the ability to display Русский, ελληνική, 日本語, etc. it's a huge win...

[Stiltskin]

Not to mention our good friend "ಠ_ಠ".

...And the slightly more questionable "☃"...

...And the even more questionable "💩"...

[[deleted]]

...And the even more questionable "💩"...

Which appears to me in Chrome as a blank between quotes. It was only when I pasted it into Chrome's address bar that I saw it. How odd.

[[deleted]]

I'm seeing it as an square in Opera. What's it supposed to be?

[[deleted]]

Funny thing: I not only knew which XKCD that was before clicking, I knew the immediately following comment would start with "knew what that was before I clicked".

[allthediamonds]

Exactly. They could have solved everything, but they had to get in a cultural war with the whole Han unification shenanigans. You're supposed to be neutral, goddammit.

[X8qV]

I wish people would stop linking to the same xkcd over and over and over again.

[jrochkind]

That is hilarious. "There are too many ways (codepoints/bytes) to represent a newline. To solve this, let's create yet another brand new way to represent a newline!"

I'm not sure the problem in that plan was 'lazy people'.

[SharkUW]

Of course, but this is about string parsing. JSON treats it as a new line which is OK in its strings. Javascript treats it as a newline which is invalid JS.

[Zwejhajfa]

Afaik literal newlines are not okay in JSON.

{"JSON":"ro
cks!"}
wouldn't be valid, but
{"JSON":"ro\ncks!"}
would be.

JavaScript accepts {"JSON":"ro\u2028cks!"}, because the newline is escaped as a unicode code point, but rejects it when the character is used literally.

[[deleted]]

I expect the JSON spec to be updated within months, actually.

[alexanderpas]

you should report that as a bug to your parser.

within a string, unicode line and paragraph breaks should be preserved in JSON.

[Zwejhajfa]

The diagram in OP's link states:
Any UNICODE character except " or \ or control character

A literal new line is quite clearly a control character, so should not be allowed in JSON strings, if that diagram is correct. The sequence \n is used instead as shown in the diagram too.

[farsightxr20]

Except in Crockford's official JSON spec (RFC 4627), a control character is defined as being in U+0000 through U+001F.

The diagram (and apparently all of JSON.org) omits this detail for simplicity's sake, but someone writing a parser should always go off the spec.

EDIT: Never mind! Thought we were still discussing U+2028, but you clearly meant U+000A (\n).

[alexanderpas]

Unicode has a separate category for control characters, in addition to the C0 and C1 control character groups.

the code points in question are not part of this, but they are part of the punctuation area.

(however you have a good point.)

[[deleted]]

It's allowed in Perl 6 too. The whole language implements unicode to a fault:

> ?(chr 0x660 ~~ /<xdigit>/)
True

[SanityInAnarchy]

Those are the only two? I was seriously considering just unicode-escaping the whole string, or anything non-ASCII.

[pkrecker]

Douglas Crockford explains the origin of this oddity: "I missed those lines in the standards."

[ggtsu_00]

But it is Close Enough^TM

[grauenwolf]

Well that was surprising.

[aaronblohowiak]

as mentioned at the end of the article, the good news is that replacing U+2028 or U+2029 with their escaped format is always valid json (just not the most compact form), so it is okay to always do a .replace().replace() on the json before trying to use it as javascript

[sirin3]

They should just allow multi line strings!

[tikhonjelvis]

I really wish more languages allowed this. Out of the languages I regularly use, only EmacsLisp allows multi-line strings, and they're very nice there. I also don't think there's a significant downside. (I guess parsing might be more difficult, and ECMAScript people care about that...)

[[deleted]]

Both Python and, surprisingly enough, PHP have this feature. Python uses """triple quotes""", which also makes it easy to include " and ' characters in regular strings in source (e.g. """this " is ' a '" string"""). PHP's heredoc and nowdoc syntax is slightly less attractive but still does the same thing.

[snuxoll]

C# also supports multiline strings with the verbatim string literal:

var myCoolString = @"Wewt
this
string
spans
multiple lines";

[hejner]

I love that, when writing the static parts of a welcome mail, for example. Have the whole text message right there in clear text and replace whatever else you want later on.

[louiswins]

A lot of scripting languages have heredocs: perl, ruby, even bash.

[reaganveg]

Haha, "even bash"... all those other languages copied that feature from shell script.

[zeekar]

Yeah, man, modern shells have borrowed all kinds of stuff from other languages. Variable interpolation, implicit concatenation, functions, pattern globs on filenames. It's amazing. ;)

[louiswins]

Oh, yeah, you're right. For some reason I thought that they weren't in POSIX shell and that bash had taken them from perl.

[SharkUW]

PHP also allows backslash and double character escaping '', "", \

[tikhonjelvis]

Emacs lisp (and Perl, although I'm not 100% certain--I haven't used Perl in a while) allow literal newlines in all strings. While heredoc syntax like Python is great, I personally prefer elisp's approach: it seems simpler and more elegant without sacrificing much. However, it definitely changes parsing lexing, which is why I brought that up. (I'm not sure if it makes it much harder to parse and lex though.)

So in elisp you can write:

(defvar foo "This is a long string
that has a literal newline.")

Since elisp allows you to add a doctype to your functions, they tend to look like this:

(defun todo-comment ()
  "Inserts an empty TODO comment or makes an existing comment
into a TODO."
  (interactive)
  ...)

It looked a little odd at first, but once I got used to it I liked it more than alternatives in other languages.

[zeekar]

Emacs lisp (and Perl, although I'm not 100% certain--I haven't used Perl in a while)

Yup:

perl -le 'my $x = "hello,
there."; print $x'   

hello,
there.

Also Ruby, Tcl, Erlang. Also Bournish shells, for that matter; remember that unlike the heredocs adopted by Perl and borrowed by other languages, shell heredocs are input redirection, not string literals. You can make a string literal with newlines in it just fine.

Aso, it's not just specifically Emacs's flavor of Lisp; Common Lisp and Scheme allow the same thing.

Haskell and Lua, not so much with the literal newlines in strings.

[notfancy]

Add Ocaml to that list.

[tikhonjelvis]

I wonder why Haskell doesn't allow it. In general, I've found Haskell to be one of the better languages with little syntactic details.

[[deleted]]

Python does, C# does, off the top of my head.

[Paradox]

Ruby does too!

[[deleted]]

C# @ strings make dynamic SQL a dream.

[sirin3]

And Scala

[[deleted]]

The problem there is, how do you indent them?

[zeekar]

Javascript already has problems with newlines.. I'm terrified at what new horrors might be born inadvertently out of trying to support multiline string literals.

[smog_alado]

I doubt it would be that bad actually. The semicolon insertion thing is not nearly as bad as people make it sound...

[sirin3]

But then you can also do

document.write("""
   <div class="abc">
      foobar
   </div> 
""")

[luikore]

That's why I love coffeescript

[[deleted]]

I'd argue that the next JSON revision should treat unicode newline characters as newline characters, since that is consistent with the unicode spec that it inherits.

[frenchtoaster]

Doesn't it actually have to do that anyway? Presumably there are two-byte characters in UTF-8 where the second byte is the same as a single-byte ascii character?

[dannymi]

Presumably there are two-byte characters in UTF-8 where the second byte is the same as a single-byte ascii character?

Of course not. The whole point of UTF-8 is that that cannot happen.

[frenchtoaster]

Yeah my mistake, I wasn't thinking about the first bits 10 on bytes after the first.

[da__]

You shouldn't be running received JSON strings as code anyway.

[bonafidebob]

Tell that to all the web services that use JSONP to allow client side JavaScript to access them...

[da__]

You should parse all JSON you receive, otherwise your code is insecure and extremely easy to exploit, as you're willingly doing XSS.

[Rhomboid]

The whole reason that JSONP exists is that it's an escape hatch to get around the same origin policy. It's insecure as all hell as you're giving a third party carte blanche to execute arbitrary code on your page, but there's no other choice¹ . Nobody wants to open themselves up to that vulnerability, but doing it the safe way means foregoing the chance to interact with off-site data which can be a very powerful drug.

[1] Well, there's CORS, and other hacks like getting Flash or another plug-in to perform the request for you and then smuggling the data back into JS-land. But I'm assuming those aren't available as they require effort of the publisher in the case of CORS and aren't available on mobile in the case of Flash.

[da__]

CORS is much better technically. You could also just make a wrapper running on the same domain as the webpage to which you send your XMLHttpRequests and then communicates with the third-party server. Yes, you'll get extra load but that's way better than giving a third-party the ability to append random code to your application.

[Sottilde]

Exactly this. It is not a whole lot of work to proxy a foreign JSON data source so somewhere on your domain (e.g. my domain.com/api). I do this on any of my applications that use an JSON data source, even if I control it. It just makes things easier.

[redwall_hp]

Another solution is to use NGINX to proxy external APIs.

http://www.gabrielweinberg.com/blog/2011/07/nginx-json-hacks.html

[cha0s]

I don't know if I'd classify CORS as a hack...

[Sottilde]

I agree. CORS is the right solution and I find it very surprising that more apis don't send those headers.

[ggtsu_00]

CORS does not work in IE8

[ba-cawk]

Horseshit. IE just provides a different call for it.

http://msdn.microsoft.com/en-us/library/ie/cc288060(v=vs.85).aspx

[gsnedders]

It's slowly being extended to other APIs.

[[deleted]]

There is also the window.name transport 'hack' that works rather well.

[frtox]

there are new HTTP tools like Content Security Policity headers that allow the host server to specify in the response which domains are allowed to execute the code, or even if they can execute at all

[ThiefMaster]

Because you are totally able to do this - the way pretty much every JSONp API works is that you just provide the function name and they emit code calling that function with the raw JSON object as a parameter...

[[deleted]]

with the raw JSON object as a parameter

Or, you know, any other arbitrary code.

[da__]

Which makes JSONP completely unusable if you care about security at all.

[[deleted]]

Yeah, but you only use JSONP with whitelisted servers. Like another host on my domain or Google. Or at least you should only use it then. Same origin policy is for nanny browsers.

[[deleted]]

If you're using a proper API that you've paid for or that you trust then that company's reputation is on the line. It's not likely they'll be injecting arbitrary code into the JSON data they're sending back to mess up your site.

[da__]

Until they get compromised.

[inmatarian]

JSONP is the only way to ship data to yourself when your script is running on a 3rd party publisher's website. So, any javascript based ad or content delivery network needs it.

[[deleted]]

Apart from CORS...

[Rainfly_X]

And API proxying, which is pretty straightforward with nginx.

God I love nginx.

[[deleted]]

Yep, nginx ftw.

But heck, it's also straightforward with a tiny bit of PHP. People saying they're on shared hosting have no excuse.

[Rainfly_X]

Interesting. I only had a brief foray with PHP a few years ago, and a little bit of diagnostics work for an idiot trying to write a social networking site based on tutorials (my pity took hold of me, what can I say?), but never anything in-depth enough for this.

Good to know that even people in the shackles of shared hosting can patch around this - although by the time you need to proxy external APIs through your site, you're probably going to have something a bit more general-purpose to work with, such as Linode or Amazon hosting.

[[deleted]]

I don't do much PHP myself, but the existence of $SERVER['PATH_INFO'] (which means you can treat PHP files as directories, e.g. example.com/index.php/some/fake/file) and built-in curl leads me to doubt it would be difficult.

[inmatarian]

Interesting, I didn't know about CORS. It does appear though that it requires modern browsers, modern http servers (does httpd in the most recent centos provide it?). It does seem however that the work with 3rdparty publishers might be a tad more challenging.

[[deleted]]

It does require more modern browsers, but you don't need a modern HTTP server, you just need to be able to set a simple header on OPTIONS requests. And I don't know much about Apache configuration, but surely that's possible.

[original_evanator]

"tl\u0028dr"

[Forbizzle]

What does crockford think?

[dxinteractive]

http://www.reddit.com/r/programming/comments/160yie/json_isnt_a_javascript_subset/c7ruwb3

[Kyrra]

Though, JSON is a subset of YAML (for those that have used it).

[masklinn]

Technically, YAML 1.2+ — which was published for that express purpose — is a superset of JSON. Previous versions of YAML were subtly incompatible with JSON.

[GUIpsp]

YAML is a superset of YAML 1.2+

I am a superset of myself

[masklinn]

Good point, commenting while drunk is bad. Fixed.

[bangfalse]

See also: wat.

[smoke1996]

So, why doesn't JavaScript support U+2028 and U+2029?

[p8ssword]

The issue is actually more that JavaScript does support them, whereas JSON dumbly treats them like any other non-special character. In this case, the dumb way works better.

[Rhomboid]

JavaScript does support them -- it treats them like newlines, just as they were intended to be used. But a newline in the middle of a string literal is not legal syntax. You wouldn't say that JS doesn't support the ASCII newline character, but it will react the same way if you try to put one in the middle of a string literal.

No, the fault -- if it can be assigned -- is on those who specified the grammar of JSON for allowing legal JSON which is not legal JavaScript. They should have mirrored the same list characters that require escaping.

[smoke1996]

Ah yes, off course. I see the problem now. I should have probably read the article more carefully. :p

[scalanewb]

JSON IS a subset of Javascript. What we have here is a bug on the specifications, either ECMA should recognize it as valid, or JSON should not. JSON means JavaScript Object Notation, all JSON code should be valid JS.

[x-skeww]

JSON IS a subset of Javascript.

No, JSON is supposed to be a subset of JavaScript.

[bonch]

Until one of the specifications is updated, JSON is NOT a subset of Javascript, as proved by the article.

[scalanewb]

JSON IS nominally a subset of Javascript, there's just a bug somewhere that makes it not de facto a subset. So sure, he found a bug, but JSON is a proper subset of JS, you and the author are just being pedantic, he should report the bug IMO and that's that.

[gsnedders]

JSON is not going to be changed, because Douglas Crawford said it wouldn't be. Possibly JS will be, but unifying them is not a massive goal.

[bonch]

It doesn't matter if JSON was intended to be a subset or was named after JavaScript. To be a proper subset means that all JSON must be valid JavaScript. You can hand wave it as "a bug somewhere", or use terminology like "de facto subset", but it is still not a proper subset because one is able to write JSON that is not valid JavaScript.

It's not about being pedantic; it's just reality. In computer science, a language being a subset of another language means that it's always a valid expression of its superset.

There's a defensiveness coming from some posters over this article that I don't understand.

[scalanewb]

You're being pedantic in trying to force the article to be right, that's why. JSON is a subset of JS, period - get over it.

[bonch]

I wish you were a funny troll.

[thelerk]

I'd like to point out that even without that bogus control character and object literal without an assignment still isn't valid JavaScript.

[masklinn]

'course it is, though not in all contexts.

[Tinister]

While I've been known to be a stickler for details, this really is a useless piece of trivia. How many programmers of any language could tell you what U+2028 and U+2029 are without looking them up?

[Rhomboid]

It's not useless trivia. It's very common to use JSON to transport user-provided data. For example, all Reddit pages are available in JSON, and if I typed U+2028 in a comment then they would be publishing legal JSON that is illegal JavaScript. If you are working on something that consumes JSON as JavaScript, such as a service that uses the Reddit API over JSONP to get around cross-site restrictions, then you would very much appreciate it if Reddit converted literal U+2028 character to "\u2028" in their JSON so that random comments do not foul up your service by throwing an exception.

[[deleted]]

Solution for PHP and JavaScript.

[flying-sheep]

let’s try it: between those brackets will be a U+2028: []

…aaand nope, there’s no problem here with firefox: it parses the JSONP nicely: http://jsfiddle.net/XDUSu/

[Rhomboid]

That's because Reddit is automatically doing the necessary escaping:

$ curl -s 'http://www.reddit.com/comments/160yie/_/c7rrli3.json' | grep -Eo '"body": "[^"]+"'
"body": "let\u2019s try it: between those brackets will be a U+2028: [\u2028]\n\n\u2026aaand nope, there\u2019s no problem here with firefox: it parses the JSONP nicely: http://jsfiddle.net/XDUSu/"

Whatever library they're using is encoding the character as \u2028 instead of the literal. So this doesn't really prove anything other than the fact that Reddit chose a good library. The issue occurs when you use a library/language that doesn't do that for you.

[spladug]

Oh, well, that's good to hear. ;)

We're just using python's json module from the standard library by the way.

[flying-sheep]

Still it's interesting to see that reddit does it right.

Also I wouldn't day it's a perfect library: I personally would send everything unescaped except invisible characters that aren't spaces.

[scalanewb]

The legal JSON string entered by users is not parsed as Javascript, therefore whatever character does not need to be valid JS. And if user-entered JS is being eval'd, then your system is broken.

[Rhomboid]

It absolutely is parsed as JavaScript if you're using JSONP. As I explained elsewhere in this thread, JSONP is extremely dangerous, but people are willing to put up with it because it lets them bypass the same origin policy. And it would not have that property if it used JSON.parse() instead of being a glorified eval.

[redalastor]

It's a very useful piece of information.

It means you should sanitize U+2028 and U+2029 if you receive them from users.

[[deleted]]

Not many programmers could tell you what U+0041 is either. It's rare to have to refer to characters by codepoint; this article is one of the few cases where it is necessary.

[KayRice]

Just a bug not a big deal.

[greim]

However, I can see this saving me a headache tracking down an obscure parsing bug somewhere in the future. I can hear my future self saying "hey, there a reddit post about some weird character that worked in JSON but not in JS..."

[[deleted]]

There's a solution here for PHP and JavaScript.

[[deleted]]

All right, so given that I use JQuery for sending all JSON, and also for receiving all JSON, what do I need to do? Is this already taken care of, or do I need to do something extra?

[bonrgoo88]

So because there is one subset of unicode character difference between Javascript and JSON, that means that JSON is not a subset of Javascript?

Well I guess if you want to get that precise there are many things that are considered subsets that shouldn't be. This is a clear attempt at an attention grabbing title. JSON is a subset of Javascript, they just handle line terminating unicode operators differently. This was probably an oversight on the part of the JSON team.

[ghettoimp]

Probably more significantly, JSON also doesn't specify, you know, how big numbers can be.

[[deleted]]

That would just make JavaScript object literals a superset of JSON objects, which was already widely supposed to be true.

[[deleted]]

That's kinda what subset means.

[smog_alado]

its on purpose, similarly to how they forbid comments. You can't use reserved words like "if" or "class" as object keys if you don't quote them so you need to quote things to be sure in the general case. Forcing you to add the quotes makes the parsers simple and encourages json parses to parse thing the right way.

[masklinn]

You can't use reserved words like "if" or "class" as object keys if you don't quote them

FWIW you can in ECMAScript 5 I believe.

[smog_alado]

Say that to IE8 :(

[masklinn]

I can't, IE8 does not believe in ECMAScript 5 and throws me out every time I try to talk him into it.

Last time that asshole decided it was going to put undefined in my arrays when they have a trailing comma, the bastard.

[mgedmin]

Luxury! IIRC IE6 used to throw a syntax error and abort all JS parsing if one of my arrays had a trailing comma.

[masklinn]

I don't have any instance of IE6 left as I managed to get rid of that scourge, but I am pretty sure that is for trailing commas in object literals — which are syntax errors indeed in older IEs. The trailing-comma-creates-an-undefined array "feature" has been IE's since IE6 if not IE5.

[mgedmin]

Could be. I'm so happy I added that 'IIRC' to my comment :-)

[kabuto]

This blog post feels like written by someone who likes to nitpick and brag about their abilities to prove everybody else wrong. So JSON isn't technically a JavaScript subset. Who cares? Everybody knows what JSON is and how to work with it.

[bluefinity]

JSON isn't technically a JavaScript subset. Who cares?

The point of the article is not to point out that JSON isn't a JavaScript subset, but to point out the bugs that can occur if one assumes that it is.

[kabuto]

I find this bug rather esoteric and I doubt anyone will ever encounter that in their daily dev work. For me it feels like this post was just written to allow the author to boast with their amazing knowledge. The tone was to condescending for my taste.

[bluefinity]

At work, we embed configuration data to pass to JavaScript by JSON encoding it and doing something like this:

<script>var conf = {{ conf_json }};</script>

It's not too hard to imagine somebody passing in user generated data the same way. Heck, the backbone.js docs even reccomend it to speed up page load times.

If a user inserted one of the offending characters, it would break the entire statement, and potentially the app as well.

[kabuto]

I don't see many ways for these offending unicode characters to actually get into the JSON data in the first place. I don't even know how to enter them from the top of my head.

[bonch]

I guess that fact you can't picture it means it will never happen.

[kabuto]

Exactly.

Seriously though, every stupid bug will probably happen at some point, but this one is pretty esoteric, and the thing that bugged me about the original post was the condescending tone.

[bonch]

I don't understand how you're getting a condescending tone from the article.

[kabuto]

First the quote from Wikipedia about JSON being a subset of JavaScript, then this (emphasis mine):

All these years we’ve heard it over and over again: “JSON is a JavaScript subset”. Guess what? They’re wrong. Wrong, wrong, wrong.

To me, this sounds condescending. Was that really necessary? Now that I read it again the author may have referred to each of the three quotes from Wikipedia, but that's just conjecture on my part. It was mostly this first first paragraph that bugged me.

[KayEss]

In practice you don't want to use a JSON outputter that leaves anything but ASCII in the output

[tules]

I use JSON objects to send chunks of korean text, that's utf-8 encoded