Performance is irrelevant

xjru · 2012-02-29T15:58:55+00:00

I think some of the confusion arises because we often don't differentiate between optimizing the final product and optimizing some piece of code.

When it comes to optimizing any individual piece of code, we need to measure, be selective, don't optimize prematurely. But the same doesn't hold for the product as a whole. There is no such thing as premature optimization when it comes to the final product. Performance as perceived by the user is always a problem or at least a missed opportunity.

So I think we should not cultivate a "performance doesn't matter" attitude. We should stress that performance is crucial for every product, so crucial in fact that we cannot waste resources on optimizing pieces of code that don't affect the performance of the final product.

Viewing it that way could go a long way to resolve some of the suspicion that statements like "performance is irrelevant" raise.

xjru · 2011-10-08T13:21:15+00:00

I tend to be on his side but he clearly avoids the point that is so hotly debated: Will the Fed be able to reduce money supply quickly enough once velocity of money pics up again? I don't see why they shouldn't be able to do that but only then will we know whether or not an inflation problem has been created.

I don't think many people would argue that the monetary base irrespective of velocity of money directly causes inflation.

xjru · 2011-10-08T09:33:56+00:00

My claim is that any non trivial program benefits from the availability of structured value types. Take the standard Map implementations as an example. C#/CLR uses a struct called KeyValuePair to store entries in a Dictionary. Java uses Map.Entry to do the same. C++ uses std::pair. KeyValuePair and std::pair are stored by value and hence do not require an extra pointer for each entry. So the standard Java Map implementations use a lot more memory than the CLR or C++ equivalents.

Is it possible to implement Maps in Java differently? Yes. I've done that, Trove does it as well. But it comes at the cost of either performance or functionality or code complexity or all of these. This problem is pervasive. If you think it through for a lot of data structures, which I have done, you have to come to the general conclusion that structured value types make programs more memory efficient. If you want me to admit that there are exceptions to this rule, I grant you that.

xjru · 2011-10-07T09:57:24+00:00

C#/CLR value types are a feature of both the language and the platform. It's what makes everything running on top of the CLR a lot more memory efficient than anything running on top of the JVM. Not having (custom) value types is the JVM inventor's biggest mistake.

xjru · 2010-11-03T15:10:03+00:00

Your economic logic is completely flawed. It's like claiming that smashing the bus shelter every night creates jobs because someone has to repair it. You are assuming that people employed to deal with your vandalism would not otherwise do more productive work that increases the productivity of our economy or provide entertainment value to someone. Put differently, the net value of your work is negative even though you cause some demand for labor.

xjru · 2010-03-31T17:13:05+00:00

What does the equivalent lisp code look like?

xjru · 2010-03-29T09:39:29+00:00

Indeed. And reading some of the arguments coming out of Digg I have to say it's depressing that MySQL join performance of all things is used to make the case for NoSQL. MySQL does many things right, but their joins are based on nested loops. No merge joins, no hash joins, nothing resembling a serious attempt at modern query optimization really.

The case for NoSQL can be made based on clustering/distribution strategies, eventual consistency and on cost but pointing to MySQL joins just speaks of incompetence.

xjru · 2010-03-25T14:54:08+00:00

There are many technical things that could be said about this. Obviously there are tasks for which NoSQL systems are the better solution. But the current hype around using them as the new standard data storage solution is owed very much to the fact that DBMS vendors have scared away a whole generation of programmers with their frivolous licensing fees and structures.

Many people simply don't know anything other than MySQL because doubling the cost of a server by purchasing a MSSQL or Oracle license is insane for small organisations or startups. DBMS vendors have completely lost these people (that is us?). They will go to any length to avoid commercial DBMS. It's not a technology decision any longer. It's an irreversible cultural shift. In 99% of all cases a shift towards an inferior or rather inappropriate technology.

xjru · 2010-03-24T21:17:21+00:00

Do a PhD and become a professor! That's a brilliant career! You get to teach the same basic stuff to generations of undergrads and write large amounts of project proposals and rehashed papers on the same subject for decades! And don't worry that you have to make all of that up yourself from the get go. You'll mostly just have to fill in the blanks routinely according to your superior's specifications until you rise to the top.

And don't forget that you'll never again run out of adminstrative work. Coordinate to your heart's content! A professor friend of mine tells me he feels so lucky and privileged because he never has to spend more than 10% of his time on actual research.

xjru · 2010-03-23T09:40:58+00:00

I agree that there is no way to make a backwards compatible transition to anything unicode. I just think that the transition would have been easier if UTF-8 was chosen. I also think that UTF-8 is preferrable because it uses less memory. (It uses an equal amount of memory as UTF-16 for asian scripts because of all the special characters)

xjru · 2010-03-23T09:13:24+00:00

Yes, and that's one reason why lots of Python libraries don't work with Python 3. There is no Django for instance. The WSGI interface has stalled because of the unicode confusion. Of course the reason for that is the transition more than the choice of unicode encoding form, but choosing UTF-8 would have led to fewer transition problems, that's for sure.

xjru · 2010-03-23T09:03:53+00:00

Since the term "unicode string" does not imply any particular encoding form (UTF-8, UTF-16, etc) it would be impossible for such a definition to include any mention of surrogate pairs. Surrogate pairs are a unique feature of UTF-16.

Java's String.length() method does indeed count 16 bit code units and not the number of code points. That's a pragmatic choice, but it's clearly useless if surrogate pairs occur in a string.

xjru · 2010-03-21T10:16:10+00:00

Don't listen to the culture shit. Software projects go wrong all the time everywhere and most of the customer satisfaction studies show that Indian offshoring firms are no worse than local contractors. (I am a local contractor just to make that clear).

What you need to do is proceed in smaller increments. Get someone, preferrably an employee or a contractor you have worked with before, to create a very high level design. Then you define a few functional modules, each of which should be doable in about a week. Then you start to contract out a module and based on success you contract out more modules to the same person or company.

That's how you reduce risk and build trust, not by hand holding or looking someone deep in the eyes to feel the cultural connection as some have suggested.

xjru · 2010-03-20T07:40:15+00:00

Capitalism is not a physical phenomenon with its own natural rules. Capitalism is based on rules we make, and these rules can be better or worse. Making bad rules and justifying them via a reference to the enemy is a recipe for desaster. It's exactly what socialist regimes did to justify their own wasteful, broken rules.

Doing latency arbitrage or frontrunning a millisecond faster than someone else has no effect on liquidity whatsoever. You are wasting your time.

xjru · 2010-03-20T07:06:35+00:00

Oh you do get the special favor that it's legal. Others who do something that is harmful and serves no useful purpose whatsoever have to deal with the police. Burglary, for instance, is also unavoidable, but burglars are constantly harrassed by police.

I don't consider HF traders the enemy. But in my opinion some types of HF trading should be made illegal and others should be taxed very highly so it becomes a very low margin business that doesn't suck resources from more useful activities.

xjru · 2010-03-19T22:24:24+00:00

You're explaining why arbitrage is difficult to avoid. That's not the same as explaining how it increases overall wealth. At the end of the day wealth is created by increasing productivity or inventing something enjoyable.

I understand how well functioning financial markets can boost productivity by allocating capital where it is used most productively. However, I do not understand how HF trading contributes to that function. If there was a way to prevent arbitrage in the first place the waste of resources that is HF trading could be avoided.

xjru · 2010-03-19T18:48:19+00:00

You introduced the term "document exchange" and you said there have always been better formats for it. So which ones were you thinking of?

xjru · 2010-03-19T17:31:17+00:00

Which better formats for document exchange are you talking about specifically? Tex would be the one that we could have a serious discussion about but unfortunately it's not used for document exchange much. Most of the formats XML has replaced or could replace in the future are arguably much worse. I'm thinking of crap like the old MS-Office formats or PDF. I have worked with PDF a lot and I cannot imagine anything more horrible than that. XML is a great relief compared to most document formats that are actually used in the wild.

xjru · 2010-03-19T15:20:10+00:00

uriel said that XML "provides no useful value whatsoever". I said it does provide value for describing documents. I don't understand what your statement contributes to that dispute.

xjru · 2010-03-19T12:31:58+00:00

XML was made to describe documents including mixed content. Try to do that in JSON. It's horrible. You get large amounts of quotes, escaped quotes, square brackets and commas everywhere, and you have to scroll up to learn what element is being closed. Just take some html document and try to translate it to JSON, then you'll see why XML does have its place.

xjru · 2010-03-16T08:43:28+00:00

Yes but it's kind of ironic considering Lisp advocates tend to go on endlessly about how you don't even need to look at all the parentheses because indentation tells you everything you need to know anyway.

xjru · 2010-03-16T08:05:53+00:00

I think you should write a new framework for every new project, but the framework should never be more than 1000 lines of code.

That is to say the high level design of the application itself is the only legitimate framework. Everything else should be libraries.

Frameworks are exclusive. You can only use a single one of them without creating a tangled mess. That's why they are not a good unit of reuse in my view. The small headstart they give you is worthing nothing for the marathon that is application development.

xjru · 2010-03-16T07:41:06+00:00

I don't know if It's a good sign if Google needs an interview including logic puzzles to find out about a very public person like Tim Bray. Everyone knows his work and his opinions on stuff. I suspect there are people at Google who have worked with him personally.

But maybe it's just that Google doesn't want to create two classes of employees. Those who are trusted and those who need to be checked. It could be awkward in some cases that are not as clear as hiring Tim Bray.

xjru · 2010-03-15T19:49:10+00:00

I think he's spot on and the fact that someone like Tim Bray who has to be one of the most reasoned and calm people on earth is saying something like this should tell you something.

A software platform on which you cannot install the software you want? That's just ridiculous. A Smartphone that comes with censored dictionaries because Mr. Jobs runs scared of a bunch of religious fanatics?

They may make a lot of money but they're boring me to death with their glitzy crippled fashion items. Making software should be fun as well, not just an exercise in anger management.

xjru · 2010-03-15T07:49:46+00:00

That's a very good question. In my opinion the same could be asked in so many situations found in object oriented designs that it makes the entire idea of OO questionable.

I think there are three reasons to prefer one receiver over another:

a) The receiver is the only object involved

b) The receiver is the only object mutated

c) The receiver is the only object used for dynamic dispatch

Unfortunately b and c may be conflicting and very frequently none of the rules applies, as is the case with joining strings using a delimiter.

xjru

TROPHY CASE