skip navigation

Here you will find ideas and code straight from the Software Development Team at SportsEngine. Our focus is on building great software products for the world of youth and amateur sports. We are fortunate to be able to combine our love of sports with our passion for writing code.

The SportsEngine application originated in 2006 as a single Ruby on Rails 1.2 application. Today the SportsEngine Platform is composed of more than 20 applications built on Rails and Node.js, forming a service oriented architecture that is poised to scale for the future.

About Us
Home

NGIN Receives An Overhaul - Upgraded To Rails 3

04/15/2011, 2:00pm CDT
By Luke Ludwig, Andrew Stevens, Ian Ehlert, Doug Rohde, Zheng Jia

NGIN, the web platform powering TST Media and thousands of websites, is upgraded to Ruby on Rails 3.

NGIN Overview

At TST Media we upgraded NGIN to Rails 3.0.5 from Rails 2.3.2. The NGIN (pronounced “engine”) website platform empowers thousands of organizations around the world with its powerful, easy-to-use content management system. This upgrade does not include any new features for users of the NGIN platform, but instead is packed full of improvements to the underlying software architecture powering NGIN.

To understand the scope of this effort it helps to have a little background on the code base. NGIN originated in August 2006 running on Rails 1.1.2. NGIN has gone through several Rails upgrades, the most recent of which was in June of 2009 which brought NGIN to Rails 2.3.2 (see Upgrading NGIN to Rails 2.3.2). At that time NGIN had 36,000 lines of code, 322 models, and 108 controllers. Less than 2 years later, NGIN has roughly doubled in size to 72,000 lines of code, 571 models, and 187 controllers. The NGIN test suite has been growing and still needs improvement. The rcov gem reports that the test coverage is at 50%.

Upgrading to Rails 3.0.5 was much more than just upgrading Rails. NGIN depends on 60 gems and 9 plugins. Most of these gems and plugins were upgraded during this process as well. Considering the amount of code that we had to upgrade and the major changes between Rails 3 and Rails 2, we knew this was going to be an enormous task.

We had a single developer work on the Rails 3 upgrade for two weeks. Following this our entire development team focused on this effort for two more weeks. After one full week of QA activities, where we manually tested NGIN in our staging environment, we were finally satisfied enough with the upgrade to deploy it to our production environment. During this process we ran across a wide range of issues, many of which are documented here.


Bundler

The vey first step in getting Rails 3 to boot NGIN was to move all of our gems over to a Gemfile for Bundler. Bundler is sweeeeet!

Getting it runing was a bit of a chore. A handful of our gems need to build native extensions. They were not listed in our old environment so we had to track those down to include them. A few of the gems we use do not correctly list their dependencies, so we had to find what was missing and include them manually. Finally we had a few gems that just don't jive with bundler. RubyInline modules were the big offenders. We wound up having to create a hack that loaded in an initializer to get gems that used, or had dependencies on inline modules.

Now that it's all in place the days of keeping a list of gems and versions that need to be installed on a developer's or designer's workstation are a fading memory. And that is nice.

We did have to fork and update a few of the gems we use. Thank the Flying Spaghetti Monster for GitHub, making this a pretty smooth procedure. On a side note, Bundler's documentation for using the :path, :branch and :reference options is completely lacking.


Deprecations Everywhere

The Rails Upgrade plugin was a brilliant resource when taking the first steps of our upgrade. It caught and listed about a third of the changes we needed to make. It wasn't smart enough to catch the problems we had about passing in the (now) wrong data types to render functions, or ActiveRecord methods that had become private, but it did catch all the places where views had deprecated statements, and ActiveRecord queries would have problems.

We hadn't planned to upgrade all of the AciveRecord deprecation warnings early on in the upgrade process, but the number of them clogging up our test output made it necessary to get them out of the way. After that we got our tests up and running quite quickly, but it was a lot of effort to get them all passing properly.

Cutting down on the wads of deprecation warnings was an important step, we use the console log output in development to see what's going on with failures and SQL queries frequently. Having the important parts of that output separated by hundreds of lines ofwarnings added a lot of difficulty.


Caching for 0 Seconds!

The Rails.cache.write method takes an optional :expires_in parameter. Previously passing a value of 0 in for this parameter had the same effect as passing nil, meaning it would cause it to not expire automatically at a predetermined time. With Rails 3, :expires_in => 0 is now treated as 0 seconds, so setting :expires_in => 0 caused memcache to immediately expire the cached value, which clearly is not the desired behavior. We have a library wrapping all of our memcache related calls, which acts as an abstraction around Rails.cache. This wrapper library has a method called add with the following signature:

def self.add(key, value, expiry = 0) 

Most of our memcache calls do not set an expiration explicitly, so on upgrading to Rails 3 most of our memcache sets were being expired immediately! Changing the expiry variable here to default to nil fixes the problem.


AREL's Lazy Loading and Some Not-So-Clean Code Causes Data Loss

Part way into our QA effort on staging we began noticing that the root page of several of our websites was missing, which as expected completely broke these websites. After some detective work using mysqlbinlog to find the exact sql statement that deleted the missing pages, we found the code that was responsible.

nodes = page_nodes.non_trashed.public_status
nodes.delete(root_node)

With Rails 2, the above code made a sql call through the page_nodes has_many association combined with the two named scopes and loaded the ActiveRecords immediately into the nodes array. For the purposes of this method the root node of the site was not needed, and so this code got rid of it by calling delete on the array, which in Rails 2 simply removes the root_node from the nodes array. This code isn't very clean and at a glance should be refactored to use another scope instead of deleting the root node after the fact. Regardless, with Rails 3, this exact same code behaves dangerously different. Due to AREL's lazy loading, the sql is not executed with the first line of code. Instead of calling Array's delete method, ActiveRecord::Relation's delete method is called, which actually executes a sql call to delete the given page. The fix for this issue was simply to chain on another scope condition for not including the root page and getting rid of the delete call.

nodes = page_nodes.non_trashed.public_status.non_root

Prior to fixing this code, simply browsing to a specific type of page on any given website would cause the root page of the site to be deleted! We are happy we caught this issue before this code was rolled out to production.


Active Record Time Zone changes

In NGIN we generally store datetimes exactly as entered by the user and do any timezone conversions only when necessary. We noticed that the datetimes that were being displayed after we saved records were not the same as what we entered. On further investigation, it turned out that the datetimes saved in the database were not the same as what we entered or what was displaying in the view.

This all happened because the default setting for ActiveRecord::Base.time_zone_aware_attributes changed from false in Rails 2.3.2 to true in Rails 3. Setting this in an initializer to false fixed part of the problem. The datetime in the database now matched the datetime that was displaying in the views, but it still wasn't what we entered for the record!

Upon further investigation it turned out that ActiveRecord:Base.default_timezone by default was being set to :utc instead of :local like it used to be in Rails 2.3.2.

So, once we changed our timezone initializer file to the following, all our timezone related woes disappeared!

ActiveRecord::Base.time_zone_aware_attributes = false
ActiveRecord::Base.default_timezone = :local

More fun with Time Zones!

Once we got our time zone defaults back to what they were before Rails 3, we thought our datetime trials were over. Things are never so simple (not that it was all that simple in the first place), however.

We later noticed that, when filtering events down to a specific date range, certain events that should display in the range were not. Digging further we discovered that ActiveRecord::Base.connection.quote(...) was translating our datetime variables into the server's time zone. This was happening after we had already translated the datetimes into the time zone that we wanted it to be in. Here's an example:

time = TstDate.to_server_time(Time.now, "pst")
 => Tue Apr 12 15:50:36 UTC 2011 

ActiveRecord::Base.connection.quote(time)
 => "'2011-04-12 10:50:36'" 

time.to_s(:db)
 => "2011-04-12 15:50:36"

You can see that ActiveRecord::Base.connection.quote(...) converted the time to the server's timezone, which it would not do with Rails 2.3.2. All we really wanted was for the datetime to be put into a DB friendly format with quotes around it.

The simple solution was to put the single quotes around it ourselves and use the db friendly to_s, like so:

"'#{time.to_s(:db)}'"

write_attribute has gone private

There were a number of places in our codebase that used the write_attribute(attr_name, value) method. Generally we use it when we are overloading the attribute= method, but there were a few cases where we used it in controller actions. In these cases, where we couldn't just use the normal setter method, we replaced the write_attribute call with:

obj.attributes[:att_name] = value

There were also a few places in the model that were throwing errors as well. These were caused by the fact that we were calling:

self.write_attribute(...)

Did you spot the problem? That is right, you can't call write_attribute on self because of its private status. We changed those to simply use write_attribute(...) without the self and we were in the clear.


Mailers

NGIN has 11 mailers and 43 mailer actions. Emails are sent in the background using delayed_job. In development mode we don’t want to background emails for simplicity.

The Rails 3 way of sending emails is like:

UserMailer.password_reset(@user, @site).deliver

To use delayed_job, we can chain a delay method to the mailer class and specify a priority parameter.

UserMailer.delay(:priority => 1).password_reset(@user, @site)

It’s kind of a hassle to write delay(:priority => 1) every time and it’s easy to forget. We have a TstMailer module which handles the difference of development and production mode, sets a default header  'x-custom-ip-tag’ required by our hosting provider, and assigns a priority of 1 by default.

So we can just use:

UserMailer.send_email(:password_reset, @user, @site)

 For emails that need to be delivered immediately, we can specify a :foreground parameter:

UserMailer.send_email(:password_reset, @user, @site, :foreground => true)

 We can also specify a lower priority like:

UserMailer.send_email(:password_reset, @user, @site, :priority => 10)

 Having an abstraction around all outgoing emails in NGIN provides a single spot in our code to control email behavior.


Rendering with Layouts

In most cases, when calling render with a :layout => specified it just renders a normal rhtml view. Once in a while, however, you want to render a specific layout with a partial or js request. In Rails 2.3.2, it would just look for the normal layout file.

For example, render :layout => 'application' would look for a file called application.html.erb in your layouts directory.

With Rails 3, it actually looks for a layout in the same format as the file being rendered.

So, render :partial => 'foo', :layout => 'bar' would look for the file _bar.html.erb in the layouts directory.

One last example:

respond_to do |format|
  format.js { render :layout => 'bar' }
end

This example looks for a layout called bar.js.erb even if it ends up rendering an html view.

A simple way to maintain the same layout for different formats is to use symlinks to keep copies of the file in different formats. Here's how you would create a symlink for a partial version of the application layout.

ln -s application.html.erb _application.html.erb

A Subtle Active Record Change

Admittedly, this problem was pretty obscure, and only affected one place in our code. However it's the sort of thing you will inevitably run into when performing this sort of upgrade. 

We had a has_many association with an :include and an :order statement, like this:

has_many :question_elements, :include => :page_element, :order => "page_elements.position"

So far there is no problem with this, but using another feature in conjunction with this association was causing problems. Simply calling the  question_elements association returns all the QuestionElement objects, each with their associated PageElement object. However, calling "question_element_ids", which is a built-in ActiveRecord feature that will return the id column only, instead of the full object, no longer honors the :include statement. The page_elements table was not joined to the question_elements table in the underlying SQL.

Of course, this caused issues with the :order clause that is part of the association. Since the order clause is relying on the position column on the page_elements table, and it wasn't there, it caused the SQL to blow up.

The simple fix was to not use the built in _ids method on the consuming call. Instead, just get the full objects back and map the ids into an array manually.  

While this was a minor issue at best, one-off gotchas like this can add up to equal a serious debugging effort.


Using render_to_string in a view

While working on QA for the Rails 3 upgrade, we ran into a frustrating problem related to using render_to_string in a view and the new standard that Rails escapes all html. No matter how we tried to combine using raw and html_safe with render_to_string, we couldn't stop render_to_string from escaping the html output.

In the end, we ended up rendering the partial as normal rather than using render_to_string.

We still aren't exactly sure why calling render_to_string from a view ignores all attempts at leaving the html formatting intact. It's very possible that this is a bug with Rails 3, but we didn't come across anyone else with this particular problem.


Sport Ngin Pulse Subscribeto our newsletter

Tag(s): Home  Ruby