We knew we were outgrowing this monolithic project because we had some clear problems...
- Slow initial page loads because of lacking server-side rendering. Twitter describes this problem well.
- Slow following client-side renders because of downloading large asset packages without clear ways to break them up.
- Poor mobile experience from trying to responsively scale down a large single page app with bloated and unused assets.
- Slow asset compilation, server boot, and general build times. Productivity suffered greatly as more code was added to the same monolithic project.
There's Got to Be a Better Way
A monolithic app that treats it's client-side code as a second class citizen was clearly not going to scale. Our poor mobile web experience was a good candidate to try something new. So we started building a separate mobile optimized website (m.artsy.net).
Some goals became clear:
- Share rendering code server/client to reduce duplication and optimize initial page load.
- Flexibility. We needed a way to divide our app into smaller chunks with smaller asset packages.
Node was a clear choice because it made sharing rendering code server/client possible where other languages and frameworks struggle to do so. There were some existing Node projects that accomplish this such as Derby and Rendr. However, adopting these had challenges of their own including being difficult to integrate with our API, learning unnecessary conventions, or being early prototypes with lacking documentation.
We open sourced this combination of tools and patterns into Ezel. Ezel is a light-weight boilerplate project using Express and Backbone for structure, and Browserify to compose modules that can be shared server/client.
Sharing and Rendering Server/Client
To share rendering code server/client we had to make sure our templates and objects being passed in to them could work the same server/client.
Sharing Objects (Backbone Models)
Browserify lets you write modules that can run in Node or the browser. Since Backbone is able to be required on the server out of the box, it's easy to write models and collections that can be required on both sides with Browserify. However, there are two main speed bumps in doing this:
Backbone uses AJAX for persistence.
We needed a Backbone.sync adapter that makes HTTP requests server-side, so we wrote one and it's open sourced.
Data from the server needed to be shared in modules that are used server/client.
All Together Now
With templates and models require-able server/client, sharing rendering code became much simpler. Below is an example using the same artwork model and detail template server/client.
Shared Backbone "Artwork" model to be required server/client:
1 2 3 4 5 6 7 8
Shared partial jade template used server/client:
Full server-side page template including the partial:
1 2 3 4 5 6 7
Route handler that uses the model server-side:
1 2 3 4 5 6 7 8 9 10 11 12
Client side code that requires the partial template and model:
1 2 3 4 5 6 7 8
With Browserify we were able to use npm as a package manager for server or client-side dependencies. There are other package managers for the client-side. However, because we were already using npm (and npm supports git urls), we could usually point to the project hosted on npm or Github without having to fork it.
For projects that don't support CommonJS modules (or npm), often one can still use npm and requires like so:
1 2 3
1 2 3 4 5 6
Testing is light-years ahead because you can test all of your code in Node headless. I wrote an article on this a while back, and now with Browserify it's even better.
Models, templates, and other modules that are shared server/client can be required into mocha and tested server-side without extra effort. For more view-like client-side code that depends on DOM APIs, pre-rendered HTML, etc., we open sourced a library called benv to help build a fake browser environment in Node for testing.
Instead, we borrowed a page from Django and broke up our project into smaller conceptual pieces called "apps" (small express sub-applications mounted into the main project) and "components" (portions of reusable UI such as a modal widget). This let us easily maintain decoupled segments of our project and build up smaller asset packages through Browserify's
requires and Stylus'
imports. For more details on how this is done please check out Ezel, its organization, and asset pipeline docs.
It's also worth noting, to avoid CSS spaghetti we followed a simple convention of name-spacing all of our classes/ids by the app or component name it was a part of. This was inspired by a blog post from Philip Walton.