How to Set Up a Rails Search API With JSON and Sunspot/Solr

I’m going to show you how to set up search using Sunspot in your Rails application. Here’s what we’ll cover:

Install Sunspot

Add Sunspot to your Gemfile:

1
2

gem 'sunspot_rails'
gem 'sunspot_solr' # optional pre-packaged Solr distribution for use in development

Bundle:

1

bundle

Generate a default configuration file at config/sunspot.yml:

1

rails g sunspot_rails:install

Run the Solr server

If sunspot_solr was installed, start the packaged Solr distribution with:

1

bundle exec rake sunspot:solr:start # or sunspot:solr:run to start in foreground

For your test environment, start Solr like this:

1

RAILS_ENV=test bundle exec rake sunspot:solr:run

Make your models searchable

Let’s say your application has a User model that has_one :profile.

In your User class:

1
2
3
4
5

searchable do
  string :name
  integer :age
  time :last_sign_in_at
end

All the above symbols are simply methods on a User. The name and age methods actually live in Profile, but they’re defined on User by doing:

1

delegate :name, :age, to: :profile

By using delegate, you avoid breaking Law of Demeter and doing something silly like this:

1
2
3
4
5
6
7
8
9

searchable do
  string :name do
    profile.name # BAD
  end

  integer :age do
    profile.age # BAD
  end
end

See the Sunspot docs for more ways to index your data.

Index associations with touch and after_touch

You’re probably wondering how to index your associations. At first, you might think to open Profile and put a searchable block in there, but that’s not what you want to do.

Instead, you want to denormalize the data from User’s associated models into the User index. That means when a user edits their profile, we need a way to tell the user object to index itself.

That can be accomplished using touch on your belongs_to associations. In our example, in the Project class, change this:

1
2

# models/profile.rb
belongs_to :user

To this:

1
2

# models/profile.rb
belongs_to :user, touch: true

Now, when a profile gets saved, its associated user will get its timestamps updated, and its after_touch callback will be executed, so let’s define that in User:

1
2

# models/user.rb
after_touch :index

You could also pass in a block if you wanted to do other things in your after_touch callback.

1
2
3
4
5

# models/user.rb
after_touch do
  # other things
  index
end

With these changes, when a profile gets updated, its user will re-index itself.

You should modify your profile_spec.rb to ensure that the touch: true never gets removed. If you’re using shoulda_matchers, it’s as easy as:

1
2

# spec/models/profile_spec.rb
it { should belong_to(:user).touch(true) }

Unit tests with sunspot_matchers

sunspot_matchers gives you an easy way to unit test your search functionality.

After adding the gem to your Gemfile and bundling, in your spec_helper.rb, you’ll need:

1
2
3
4
5
6
7

# spec/spec_helper.rb
RSpec.configure do |config|
  config.include SunspotMatchers
  config.before do
    Sunspot.session = SunspotMatchers::SunspotSessionSpy.new(Sunspot.session)
  end
end

Now let’s test that User is indexing the fields that we expect it to.

1
2
3
4
5
6

# spec/models/user_spec.rb
describe 'searchable fields' do
  it { should have_searchable_field :name }
  it { should have_searchable_field :age }
  it { should have_searchable_field :last_sign_in_at }
end

We’ll add some more unit tests in the next section.

Creating a wrapper for searching for users

At this point, you could just search for users by throwing something like this in your controller: User.search name: params[:name], but that’s quickly going to get out of hand once you want to support more fields, pagination, and add other logic like search by location if lat and lon are provided. That would be too much responsibility for the controller and it would make it very difficult to test. A better idea is to assign the controller the responsibility of calling some class and simply returning its results as JSON, and then that class has the big responsibility of searching – we’re trying to follow the single responsibility principle here.

Let’s make a class called UserSearch. Its responsibility will be to search users so the controller can stay nice and slim.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# spec/models/user_search_spec.rb
describe UserSearch do
  describe '.search' do
    it 'searches for users' do
      UserSearch.search(name: 'John Doe')
      Sunspot.session.should be_a_search_for(User)
    end

    it 'accepts one field' do
      UserSearch.search(name: 'John Doe')
      Sunspot.session.should have_search_params(:with, :name, 'John Doe')
    end

    it 'accepts two fields' do
      UserSearch.search(name: 'John Doe', age: 20)
      Sunspot.session.should have_search_params(:with, :name, 'John Doe')
      Sunspot.session.should have_search_params(:with, :age, 20)
    end

    it 'does not explode if it recieves unknown params' do
      expect {
        UserSearch.search(unknown: 'param')
      }.to_not raise_error
    end
  end
end
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# spec/models/user_search.rb
class UserSearch do
  def self.search(q) do
    # These fields are special
    page = q.delete(:page) || 1
    per_page = q.delete(:per_page) || 50

    # Sanitize the rest of the fields
    q = sanitize_fields(q)

    # Perform the search
    s = User.search do
      paginate page: page, per_page: per_page
      all_of do
        q.each { |k,v| with(k, v) }
      end
    end

    # This is explained further down in the tutorial
    SunspotSearchDecorator.new(s)
  end

  private

  # Given a hash of fields to search, only accept the ones that User indexes.
  # Also accepts `page` and `per_page`.
  def self.sanitize_fields(q)
    whitelisted_fields = Sunspot::Setup.for(User).fields.map(&:name)
    whitelisted_fields.push(:page, :per_page)
    q.slice(*whitelisted_fields).with_indifferent_access
  end
end

Now that we have a UserSearch class that is fully tested, our controller specs will just have to test that they’re calling UserSearch with the correct parameters. Easy, and fast!

Creating a JSON API for searching users using active_model_serializers

The first thing we’ll do is define our routes.

1
2
3
4

# config/routes.rb
namespace :api, defaults: {format: 'json'} do
  resources :users, only: [:index]
end

For the JSON views, we’re using active_model_serializers (AMS) instead of something like jbuilder or rabl. Using AMS is dead simple and makes testing your JSON views trivial. Add gem 'active_model_serializers' to your Gemfile, create this initializer file, and restart your server.

1
2
3
4
5
6
7
8
9

# config/initializers/active_model_serializers.rb
# This disables the root element in JSON views *globally*.
# See https://github.com/rails-api/active_model_serializers#disabling-the-root-element.

# Disable for all serializers (except ArraySerializer)
ActiveModel::Serializer.root = false

# Disable for ArraySerializer
ActiveModel::ArraySerializer.root = false

Next, in your UsersController, let’s define an index action that simply passes parameters to UserSearch and returns a paginated list of User objects in JSON. If you’re using Rails 4 and/or using the strong_parameters gem, your controller will need a private method to permit all params. That’s okay in this instance because UserSearch handles sanitization for us; it only accepts parameters that User indexes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# spec/controllers/users_controller_spec.rb
describe UsersController do
  describe 'GET index'
    it 'is successful' do
      get :index, format: 'json'
      response.should be_success
    end

    it 'calls UserSearch.search with options' do
      UserSearch.should_receive(:search).with('name'=>'John Doe', 'format'=>'json', 'controller'=>'users', 'action'=>'index', 'user' => {})
      get :index, format: 'json', name: 'John Doe'
    end

    it 'accepts a page parameter and passes it to UserSearch' do
      UserSearch.should_receive(:search).with('name'=>'John Doe', 'format'=>'json', 'controller'=>'users', 'action'=>'index', 'page'=>'5', 'per_page'=>'100', 'user' => {})
      get :index, format: 'json', name: 'John Doe', page: '5', per_page: '100'
    end
  end
end
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# app/controllers/users_controller.rb
class UsersController < ApplicationController
  respond_to :json

  def index
    @user_search = UserSearch.search(params)
    respond_with @user_search, serializer: UsersSerializer
  end

  private

  # Permit all params here because UserSearch whitelists fields based on what User indexes
  def permit_all
    params.permit!
  end
end

The serializer: UsersSerializer part needs some explaining. As you may recall earlier in the tutorial when we defined UserSearch.search, it returns SunspotSearchDecorator.new(s). s is actually an instance of Sunspot::Search. UserSearch.search could have returned s.results which would simply return an array of user objects, but using a decorator means we can add some additional methods and it allows us to better customize our JSON output. This assumes you’re using draper.

1
2
3
4
5
6
7
8

# app/decorators/sunspot_search_decorator.rb
class SunspotSearchDecorator < Draper::Decorator
  include ActiveModel::SerializerSupport
  delegate_all
  delegate :total, to: :object
  delegate :page, :per_page, to: 'object.query'
  delegate :length, :first_page?, :last_page?, to: 'object.results'
end

According to the active_model_serializers docs, if the object you’re serializing doesn’t descend from ActiveRecord or include Mongoid::Document, then your object must include ActiveModel::SerializerSupport, which we’ve done above.

We also do some delegation to get some useful methods that our UsersSerializer will use.

1
2
3
4
5
6
7
8
9

# app/serializers/users_serializer.rb
class UsersSerializer < ActiveModel::Serializer
  attributes :total, :page, :per_page, :first_page?, :last_page?, :users
  attribute :length, key: :returned_count

  def users
    object.results.map {|u| UserSerializer.new(u)}
  end
end

The users JSON attribute will be an array of users from the search results that are serialized into JSON using UserSerializer.

1
2
3
4

# app/serializers/user_serializer.rb
class UserSerializer < ActiveModel::Serializer
  attributes :name, :age, :last_sign_in_at
end

Conclusion

And that’s it! You could use something like Postman to fire off GET requests to /api/users?name=John&age=20 and you should see some nicely formatted results. Hope that helps! Feel free to ask questions in the comments below.

Comments