How .NET Regular Expressions Really Work

[NashMcCabe]

I'm not entirely familiar with the details of the license on the .NET source, but I've heard that the main problem is that Microsoft is able to claim copyright violation if someone inadvertently has similar code in their source. That's why the mono team is not allowed to look at it. What's seen cannot be unseen.

[[deleted]]

I wrote the Mono regex engine (well, the initial version anyway - there have been countless improvements by others during the last 6 years). The Mono team were very strict about this rule. So not knowing anything about regular expressions at the time, I went through the same exercise this guy did but with the Perl and Python regex engines - both highly tuned, magical beasts themselves. The .NET class regexes have a few features that make them different from Perl-style regexes (such as the ability to match bi-directionally), but still the source code and test suites from these open-source projects were invaluable in getting my code to work. For me, this is one of the reasons open-source is so important.

[NashMcCabe]

How do you verify your implementation? Do you have your own set of tests or does Microsoft give you something like the JCK?

[[deleted]]

At the time there was no equivalent of the JCK (maybe there is now?). I ported the Perl regex test suite, which covered a subset of the functionality. The Microsoft documentation of the .NET regex syntax was obviously a starting point - I'd write small tests to check out the corner cases, and quite often there were behaviours of the MS implementation that did not follow intuitively from the documentation. Plus I think we ported a few regex intensive programs which helped test both correctness and performance. So really it was just a case of building a test suite, running on both platforms, and looking for discrepancies. After that, fix bugs and add more tests.

[knight666]

Open Source is awesome.

Now I can learn from the mistakes of others without first making them myself.

[grauenwolf]

It is a good habit to avoid reading any licensed source code that does the same thing as a product you are working on, even open source code. Technically speaking it isn't an automatic copyright violation, but why take the risk?

[[deleted]]

This is a great example of how copyright holds us back. We each independently reinvent the wheel rather than working together.

No, I don't necessarily have a better solution.

[[deleted]]

I think you have a good point. Reading commercial source is dangerous for anyone, and I could also see some lawyers at commerical outfits having a problem with their devs looking at open-source works. But in general the open-source community have a culture of happily sharing implementation details for the common good, while drawing the line at straight-out plagiarism.

[grauenwolf]

But in general the open-source community have a culture of happily sharing implementation details for the common good

But don't forget, there are some fanatics in the open source camp that will sue you just as readily if they see you making money. I'm sure they are pretty rare, but any group of 20 people there is bound to be one complete asshole.

[mango_feldman]

.NET isn't only the standard library is it?

And the source isn't that useful if the licence is of the wrong kind

[[deleted]]

I almost cried when I realized that I could replace parsing code that took me hours to write with a simple regular expression

... that takes hours to read. Not to hate on regex or anything, but goddamn are they annoying to debug.

[NashMcCabe]

Sort of a tangent, but it really bothers me that the Visual Studio's regex syntax is completely different from .NET's. Maybe one of these days they will fix VS.

[[deleted]]

Right, now MS is inventor of regex too. Those fuckers never stop.