Coding in the Crease

The importance of reducing deploy dependencies can never be overstated. On November 4th, the npm registry was unavailable for several hours. Leaving many developers without a way to obtain the node modules they need for development. And in some cases, the npm downtime kept people from deploying their applications.

As a result, many developers flocked to twitter to commiserate about how they would have a very unproductive afternoon. Even the npm.js twitter account was weighing in:

We, on the otherhand, were able to deploy all but one of our eight node.js apps without an issue. We were also able to get developers running our node apps on their local machines without relying on npm.

But how?

With a tiny little module called pac

With pac, our node module dependencies are gzipped and placed in a .modules directory in the project root. The entire directory is added to source control and we simply unzip them into the node_modules directory upon deploy with a simple bash script like this:

# from the app root
mkdir -p node_modules
for f in .modules/*.tgz; do tar -zxf "$f" -C node_modules/; done
npm rebuild

Using pac

First, you'll need pac installed globally via npm:

$ npm install -g pac

Then it's simply a matter of running pac from the command-line in your application's root directory. Here's the output of running pac in the pac project itself:

$ pac        
Adding async-v0.2.9
Packing async-v0.2.9
Packed async
Adding glob-v3.2.6
Packing glob-v3.2.6
Packed glob
Adding mkdirp-v0.3.5
Packing mkdirp-v0.3.5
Packed mkdirp
Adding rmrf-v1.0.1
Packing rmrf-v1.0.1
Packed rmrf
Adding tar.gz-v0.1.1
Packing tar.gz-v0.1.1
Packed tar.gz
Adding underscore-v1.4.4
Packing underscore-v1.4.4
Packed underscore

And the resulting contents of the .modules directory:

$ ls .modules/
async-v0.2.9.tgz  mkdirp-v0.3.5.tgz  tar.gz-v0.1.1.tgz
glob-v3.2.6.tgz   rmrf-v1.0.1.tgz    underscore-v1.4.4.tgz

Each tgz file contains all of the module's code and the code for the module's dependencies. That way, there is no reliance on the npm registry to do deploys. You simply need to keep your modules up to date in the .modules directory.

Didn't you say "all but one"?

Yep. One of our apps hadn't been updated to use modules included via pac. Oddly enough, this is an application that we deploy several times per day, so we quickly discovered that npm was having a bit of difficulty. Luckily, updating the app to use pac is only a 10 minute process. I quickly made the change, deployed to our staging environment to confirm that everything was working and the npm downtime was no longer an issue.

Other uses

$ pac install

Unzips all of the *.tgz files in the .modules directory into the node_modules directory. Handy for ensuring that all developers are using the same versions of each module. Don't forget to run npm rebuild afterward to build any native modules for your platform.

$ pac <module name>

Gzips just the module specified in <module name>. By default, running pac will gzip all of the installed modules listed in your package.json file's dependencies list.

Additional benefits

Not only did we reduce our deploy dependencies, our deploys are faster! Unzipping the modules is quite a bit faster than installing them each time. With a slightly more complex deploy shell script, we could unzip only the modules that have changed since our last deploy, saving even more time.

Happy noding!