Give Real has great execution on technology concepts. We use Rails, Merb and other tools to integrate with a diverse set of APIs. Abstraction and planning for scale are our middle names.

ShardTheLove is a library we use in both Ruby frameworks, Rails and Merb, for scaling our MySQL databases. The code has been humming on our servers for over a year now. There is an alternative library named DataFabric that has similar functionality to ShardTheLove, but the configuration felt clunky and the codebase was getting less maintenance than Mike looks to be giving it today. ShardTheLove is built with support for migrations, testing/RSpec, a flexibility of partitioning patterns, a simple syntax, support for Rails & Merb, and a great name :-) .

Using ShardTheLove

ShardTheLove puts your databases into one of three buckets:

• System - The default database, and the one where non-sharded data lives.
• Directory - A database intended for fast queries directing you to the correct shard.
• Shards - Database servers storing partitioned data.

Partitioning your data by user is a common model. Here’s an example with ShardTheLove randomly choosing a server for your user’s data to live on, storing that location on the directory, then later finding that shard and data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

class User < ActiveRecord::Base
  acts_as_directory

  has_many :reviews

  before_create :choose_shard

private

  def choose_shard
    return unless self.shard.blank?
    self.shard = ShardTheLove.shards[rand(ShardTheLove.shards.length)]
  end

end

class Review < ActiveRecord::Base
  acts_as_shard

  belongs_to :user

end

# Now we can create some reviews.
#

@user = User.create( :email => 'test@example.com' )

ShardTheLove.with(@user.shard) {
  Review.create(
    :user => @user,
    :body => 'Man those aussie meat pies are great!'
}

# And read them back.
#

@user = User.find_by_email( 'test@example.com' )

ShardTheLove.with(@user.shard) {
  @user.reviews
}

If a model calls acts_as_directory or acts_as_shard it behaves as such. If nothing is called, it is assumed to be a model on the system database. This a great way to lazy-configure your app and add sharding without immediately refactoring all your models.

There is documentation up at the ShardTheLove github project page on migrations and testing integration. It’s exciting to get a tool we code with every day out in public! Check back for another post on how ShardTheLove can integrate smoothly with controller actions.