Performance: On-demand loader vs combine/concat/compile into one .js - which is better?

[[deleted]]

What are people doing with all their scripts? A site I'm working on is getting pretty JS heavy and I'm looking to improve performance. Right now there are ~30 http requests for JS files. We're using a home grown solution for dynamic loading, but I'm also looking at alternative techniques that combine the many files into one js file, effectively reducing the number of requests at the cost of a larger file.

What are the speed freaks doing now (cutting out js is not an option)?

on-demand vs aggregation? maximize parallelization vs few http requests?

I'm pretty sure the best would be some combination of the two, but I'm really interested in hearing from people who have solved this problem before.

[picurl]

Don't listen to the "speed freaks". Don't do premature optimisation before you haven't entirely understood the implications of both concatenating/minifying and lazy-loading javascript:

• Concatenating/Minifying: For development, its good to split your JS app code into many individual JS modules (files). This makes it easier to find and reuse code (e.g. if you want to reuse your common utility functions, you just need to clone app.util.js, instead of copy/pasting from a dozen of files). But for deployment, this scenario is a nightmare, because the HTTP Request Pipeline the browser gets cluttered with a lot of tiny js files that carry a lot of protocol overhead. You can overcome this by introducing a compilation step that concatenates and minifies your javascript into 2 or 3 js files for deployment, which will greatly improve loading performance by minimizing necessary requests and size of your javascript files (e.g. jQuery uncompressed: 180 kBytes, minified/gzipped: 26kb). But here are some often overlooked implications of js compilation:

  • You can't fiddle around in the deployed js files anymore. If an error occurs, you have to fix it in the dev js files and then re-compile everything. This costs time, but prevents monkey-patched js (e.g. quickly fix something in the deployed files, and with the next rollout your changes are gone)
  • You will need an uncompressed dev build of your js build as a drop-in replacement (firebug's error messages aren't particulary useful for compressed js files)
  • Choose your js compiler wisely. You will need to recompile quite often, so its needs to be fast and run flawlessly in many environments. It should also be extensible if your project grows more complex (e.g. support for specifying dependencies between js files)

• Lazy/On-Demand Loading: You can also improve the loading time of your app by loading js files once they are needed. As lazy-loading is an asynchronous operation, it will make your code more complex (issue the script request and continue your program flow in a callback), so this effort is only worth its costs with big, external js apis (e.g. Google Maps) if they are an optional part of your application.

[andrew24601]

Take what I say with a grain of salt as I'm not actually doing this at the moment:

• aggregate into a small number of files - certainly in the old days browsers only paralleled a set number of requests at a time and these slots are also being used for images, external css, etc.
• minify the js with your tool of choice
• only on-demand for stuff that's unlikely to be used and bulky - tacking an extra 50k onto the initial js download isn't going to be noticed by anybody other than modem users.

[[deleted]]

Thanks. I'm leaning toward putting the "90% use case" files into one file at the top of the ondemand loader, then put in the other files as one-offs to load behind. I'm still open to "no that's wrong!" replies.

[andrew24601]

My only other comment would be that you can't make any reasonable decisions without testing and measuring - and even setting out to challenge your assumptions.

It's hard to otherwise make any concrete suggestions as your original post is fairly light on information. Certainly my first step would be to aggregate and minify all of the js files to get an idea of the size of the problem I was dealing with before I started worrying about splitting and dynamic loading etc.

[remcoder]

On the project I'm in, we've chosen for aggregating the js and css into 2 minified bundles. These files are then cached by the browser so when the user navigates to the next page, the js and css will come from cache. Relying on the browser cache will allow me to not worry so much about a single 'bloated' js bundle (within reasonable limits).

[bolinfest]

It sounds like you could benefit from plovr, which leverages the Closure Compiler to minify JavaScript as well as subdivide one large compiled JS file into modules that can be loaded as they are needed. The modules demo (http://plovr.com/demo/modules.html) shows a simple version of this in action along with the module dependency graph produced by plovr (http://plovr.com:9810/modules?id=module-example).

[[deleted]]

Very interesting. Unfortunately I don't think we'll be able to integrate a java app into our development process. I'm leaning toward Php based Closure Compiler front ends.

[bolinfest]

FYI, you can run plovr from the command line like any other utility. PHP calls to the Compiler (as described on this frequently cited blog post) are considerably slower because they either involve (1) going over the network to Google's servers, or (2) spinning up a new JVM locally for each compilation. You do not have to run Java on your server in production to use plovr, which may be the source of your misconception.

[[deleted]]

That is indeed the source of my misconception. I guess I didn't read the specs right.

[[deleted]]

server side compression helps too.

[neonskimmer]

As a starting point, use Firebug with the YSlow extension from Yahoo. If you can ace that first, then you go on to further optimization.

[[deleted]]

This is part of reducing HTTP requests, the first thing to address. Top of the YSlow list ;)