skip navigation

Here you will find ideas and code straight from the Software Development Team at SportsEngine. Our focus is on building great software products for the world of youth and amateur sports. We are fortunate to be able to combine our love of sports with our passion for writing code.

The SportsEngine application originated in 2006 as a single Ruby on Rails 1.2 application. Today the SportsEngine Platform is composed of more than 20 applications built on Rails and Node.js, forming a service oriented architecture that is poised to scale for the future.

About Us
Home

Fast Rails Mass Inserts

09/26/2012, 1:46pm CDT
By Jon Phenow

Trying to insert a bunch of join records in your Has And Belongs To Many relation could likely use a speed-up, see how we did it here.

Have you ever tried to do a large insert on a join-table using Ruby on Rails? Say you have a User who has and belongs to many posts. A new user has just signed up and you need to automatically link the user up with 2000 posts. In an attempt to make this a fast insert you collect the post IDs and do a @user.post_ids = found_post_ids. When you do an insert like *.relation_ids << new_ids Rails does a find on each of those IDs and then does an insert. This wouldn't seem like a big problem, but what if you have several thousand records to work with? That becomes an expensive SQL query (before an INSERT even happens) and that's a fat chunk of memory being used just to prepare an INSERT. Here’s a simple example:

That’s about 18.8 seconds just to enter a bunch of IDs.

With Rails, the convention is to not deal with SQL by hand and by loading each of those objects, their standard callbacks and validations may be run. Access to a quick insert like this would be nice, because when doing a big HABTM insert it should all just get in the database fast. Really, how many validations/callbacks are you running on a join table anyways?

To facilitate this little ActiveRecord extension a gem was conceived. m2m_fast_insert gives you the ability to do a fast ID insert on a has_and_belongs_to_many relation. Just add:

to your Gemfile. Now you get free access to do:

That basically builds a SQL insert and executes it. The nice thing is that, as said previously, its much faster than the standard Rails insert of loading tons of objects and then inserting them, although the callbacks and validations are lost.

So, how fast is this compared to the other Rails mass-insert?

Looking at “real” time (the last number), thats 18.708865s faster.

I hope you found this helpful, leave your questions here, and check out the repo here.

Tag(s): Home  Ruby  Performance