Here you will find ideas and code straight from the Software Development Team at SportsEngine. Our focus is on building great software products for the world of youth and amateur sports. We are fortunate to be able to combine our love of sports with our passion for writing code.
The SportsEngine application originated in 2006 as a single Ruby on Rails 1.2 application. Today the SportsEngine Platform is composed of more than 20 applications built on Rails and Node.js, forming a service oriented architecture that is poised to scale for the future.
The importance of reducing deploy dependencies can never be overstated. On November 4th, the npm registry was unavailable for several hours. Leaving many developers without a way to obtain the node modules they need for development. And in some cases, the npm downtime kept people from deploying their applications.
As a result, many developers flocked to twitter to commiserate about how they would have a very unproductive afternoon. Even the npm.js twitter account was weighing in:
reminder: for deployments, don't depend on machines you don't own. check in deps for your deployed apps, or run local mirror, or local git.— npm (@npmjs) November 4, 2013
i am happiest when i am helping you develop joyfully. deployment time is scary and dangerous and i get scared with all that responsibility.— npm (@npmjs) November 4, 2013
We, on the otherhand, were able to deploy all but one of our eight node.js apps without an issue. We were also able to get developers running our node apps on their local machines without relying on npm.
With a tiny little module called pac
With pac, our node module dependencies are gzipped and placed in a
.modules directory in the project root. The entire directory is added to source control and we simply unzip them into the
node_modules directory upon deploy with a simple bash script like this:
# from the app root mkdir -p node_modules for f in .modules/*.tgz; do tar -zxf "$f" -C node_modules/; done npm rebuild
First, you'll need pac installed globally via npm:
$ npm install -g pac
Then it's simply a matter of running
pac from the command-line in your application's root directory. Here's the output of running
pac in the pac project itself:
$ pac Adding async-v0.2.9 Packing async-v0.2.9 Packed async Adding glob-v3.2.6 Packing glob-v3.2.6 Packed glob Adding mkdirp-v0.3.5 Packing mkdirp-v0.3.5 Packed mkdirp Adding rmrf-v1.0.1 Packing rmrf-v1.0.1 Packed rmrf Adding tar.gz-v0.1.1 Packing tar.gz-v0.1.1 Packed tar.gz Adding underscore-v1.4.4 Packing underscore-v1.4.4 Packed underscore
And the resulting contents of the
$ ls .modules/ async-v0.2.9.tgz mkdirp-v0.3.5.tgz tar.gz-v0.1.1.tgz glob-v3.2.6.tgz rmrf-v1.0.1.tgz underscore-v1.4.4.tgz
tgz file contains all of the module's code and the code for the module's dependencies. That way, there is no reliance on the npm registry to do deploys. You simply need to keep your modules up to date in the
Yep. One of our apps hadn't been updated to use modules included via pac. Oddly enough, this is an application that we deploy several times per day, so we quickly discovered that npm was having a bit of difficulty. Luckily, updating the app to use pac is only a 10 minute process. I quickly made the change, deployed to our staging environment to confirm that everything was working and the npm downtime was no longer an issue.
$ pac install
Unzips all of the *.tgz files in the
.modules directory into the
node_modules directory. Handy for ensuring that all developers are using the same versions of each module. Don't forget to run
npm rebuild afterward to build any native modules for your platform.
$ pac <module name>
Gzips just the module specified in
<module name>. By default, running
pac will gzip all of the installed modules listed in your package.json file's dependencies list.
Not only did we reduce our deploy dependencies, our deploys are faster! Unzipping the modules is quite a bit faster than installing them each time. With a slightly more complex deploy shell script, we could unzip only the modules that have changed since our last deploy, saving even more time.