Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minitest assertions and indexing controls. #396

Merged
merged 7 commits into from
Aug 6, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Chewy is an ODM and wrapper for [the official Elasticsearch client](https://gith
* [NewRelic integration] (#newrelic-integration)
* [Rake tasks] (#rake-tasks)
* [Rspec integration] (#rspec-integration)
* [Minitest integration] (#minitest-integration)
* [TODO a.k.a coming soon:] (#todo-aka-coming-soon)
* [Contributing] (#contributing)

Expand Down Expand Up @@ -1283,6 +1284,12 @@ rake chewy:update[-users,projects] # updates every index in application except s

Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features: See [update_index.rb](lib/chewy/rspec/update_index.rb) for more details.

### Minitest integration

Add `require 'chewy/minitest'` to your test_helper.rb, and then for tests which you'd like indexing test hooks, `include Chewy::Minitest::Helpers`.

### DatabaseCleaner

If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indexes data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:

```ruby
Expand Down
1 change: 1 addition & 0 deletions lib/chewy/minitest.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
require 'chewy/minitest/helpers'
80 changes: 80 additions & 0 deletions lib/chewy/minitest/helpers.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
require_relative 'search_index_receiver'

module Chewy
module Minitest
module Helpers
extend ActiveSupport::Concern

# Assert that an index *changes* during a block.
# @param (Chewy::Type) index the index / type to watch, eg EntitiesIndex::Entity.
# @param (Symbol) strategy the Chewy strategy to use around the block. See Chewy docs.
# @param (boolean) assert the index changes
# @param (boolean) bypass_actual_index
# True to preempt the http call to Elastic, false otherwise.
# Should be set to true unless actually testing search functionality.
#
# @return (SearchIndexReceiver) for optional further assertions on the nature of the index changes.
def assert_indexes index, strategy: :atomic, bypass_actual_index: true, &test_actions
type = Chewy.derive_type index
receiver = SearchIndexReceiver.new

bulk_method = type.method :bulk
# Manually mocking #bulk because we need to properly capture `self`
bulk_mock = -> (*bulk_args) do
receiver.catch bulk_args, self

unless bypass_actual_index
bulk_method.call *bulk_args
end

{}
end

type.define_singleton_method :bulk, bulk_mock

Chewy.strategy(strategy) do
test_actions.call
end

type.define_singleton_method :bulk, bulk_method

assert_includes receiver.updated_indexes, index, "Expected #{index} to be updated but it wasn't"

receiver
end

# Run indexing for the database changes during the block provided.
# By default, indexing is run at the end of the block.
# @param (Symbol) strategy the Chewy index update strategy see Chewy docs.
def run_indexing strategy: :atomic
Chewy.strategy strategy do
yield
end
end

module ClassMethods
# Declare that all tests in this file require real indexing, always.
# In my completely unscientific experiments, this roughly doubled test runtime.
# Use with trepidation.
def index_everything!
setup do
Chewy.strategy :urgent
end

teardown do
Chewy.strategy.pop
end
end
end

included do
teardown do
# always destroy indexes between tests
# Prevent croll pollution of test cases due to indexing
Chewy.massacre
end
end

end
end
end
81 changes: 81 additions & 0 deletions lib/chewy/minitest/search_index_receiver.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Test helper class to provide minitest hooks for Chewy::Index testing.
#
# @note Intended to be used in conjunction with a test helper which mocks over the #bulk
# method on a Chewy::Type class. (See SearchTestHelper)
#
# The class will capture the data from the *param on the Chewy::Type#bulk method and
# aggregate the data for test analysis.
class SearchIndexReceiver
def initialize
@mutations = {}
end

# @param bulk_params the bulk_params that should be sent to the Chewy::Type#bulk method.
# @param (Chewy::Type) type the Index::Type executing this query.
def catch bulk_params, type
Array.wrap(bulk_params).map {|y| y[:body] }.flatten.each do |update|
if body = update[:delete]
mutation_for(type).deletes << body[:_id]
elsif body = update[:index]
mutation_for(type).indexes << body
end
end
end

# @param index return only index requests to the specified Chewy::Type index.
# @return the index changes captured by the mock.
def indexes_for index = nil
if index
mutation_for(index).indexes
else
Hash[
@mutations.map { |a,b| [a, b.indexes] }
]
end
end
alias_method :indexes, :indexes_for

# @param index return only delete requests to the specified Chewy::Type index.
# @return the index deletes captured by the mock.
def deletes_for index = nil
if index
mutation_for(index).deletes
else
Hash[
@mutations.map { |a,b| [a, b.deletes] }
]
end
end
alias_method :deletes, :deletes_for

# Check to see if a given object has been indexed.
# @param (#id) obj the object to look for.
# @param Chewy::Type what type the object should be indexed as.
# @return bool if the object was indexed.
def indexed? obj, type
indexes_for(type).map {|i| i[:_id]}.include? obj.id
end

# Check to see if a given object has been deleted.
# @param (#id) obj the object to look for.
# @param Chewy::Type what type the object should have been deleted from.
# @return bool if the object was deleted.
def deleted? obj, type
deletes_for(type).include? obj.id
end

# @return a list of Chewy::Type indexes changed.
def updated_indexes
@mutations.keys
end

private
# Get the mutation object for a given type.
# @param (Chewy::Type) type the index type to fetch.
# @return (#indexes, #deletes) an object with a list of indexes and a list of deletes.
def mutation_for type
@mutations[type] ||= OpenStruct.new(indexes: [], deletes: [])
end

end

90 changes: 90 additions & 0 deletions spec/chewy/minitest/helpers_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
require 'spec_helper'
require 'chewy/minitest'

describe :minitest_helper do
class << self
alias_method :teardown, :after
end

def assert_includes haystack, needle, comment
expect(haystack).to include(needle)
end

include ::Chewy::Minitest::Helpers

before do
Chewy.massacre
end

before do
stub_index(:dummies) do
define_type :dummy do
root value: ->(o){{}}
end
end
end

context 'assert_indexes' do
specify 'doesn\'t fail when index updates correctly' do
expect {
assert_indexes DummiesIndex::Dummy do
DummiesIndex::Dummy.bulk body: [{index: {_id: 42, data: {}}}, {index: {_id: 41, data: {}}}]
end
}.to_not raise_error
end

specify 'fails when index doesn\'t update' do
expect {
assert_indexes DummiesIndex::Dummy do
end
}.to raise_error(RSpec::Expectations::ExpectationNotMetError)
end

specify 'SearchIndexReceiver catches the indexes' do
receiver = assert_indexes DummiesIndex::Dummy do
DummiesIndex::Dummy.bulk body: [{index: {_id: 42, data: {}}}, {index: {_id: 41, data: {}}}]
end

expect(receiver).to be_a(SearchIndexReceiver)

expect(
receiver.indexes_for(DummiesIndex::Dummy)
.map {|index| index[:_id]}
).to match_array([41,42])
end

specify 'Real index is bypassed when asserting' do
expect(DummiesIndex::Dummy).not_to receive(:bulk)

assert_indexes DummiesIndex::Dummy do
DummiesIndex::Dummy.bulk body: [{index: {_id: 42, data: {}}}, {index: {_id: 41, data: {}}}]
end
end

specify 'Real index is allowed when asserting' do
expect(DummiesIndex::Dummy).to receive(:bulk)

assert_indexes DummiesIndex::Dummy, bypass_actual_index: false do
DummiesIndex::Dummy.bulk body: [{index: {_id: 42, data: {}}}, {index: {_id: 41, data: {}}}]
end
end
end

context 'run_indexing' do
specify 'pushes onto the chewy strategy stack' do
Chewy.strategy :bypass do
run_indexing do
expect(Chewy.strategy.current.name).to be(:atomic)
end
end
end

specify 'allows tester to specify the strategy' do
Chewy.strategy :atomic do
run_indexing strategy: :bypass do
expect(Chewy.strategy.current.name).to be(:bypass)
end
end
end
end
end
121 changes: 121 additions & 0 deletions spec/chewy/minitest/search_index_receiver_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
require 'spec_helper'
require 'chewy/minitest'

describe :search_index_receiver do
def search_request item_count = 2, verb: :index
items = item_count.times.map do |i|
{
verb => {_id: i + 1, data: {}}
}
end

[
{
body: items
}
]
end

def parse_request request
request.map {|r| r[:_id]}
end

let(:receiver) do
SearchIndexReceiver.new
end

before do
stub_index(:dummies) do
define_type :fizz do
root value: ->(o){{}}
end

define_type :buzz do
root value: ->(o){{}}
end
end
end

context 'catch' do
specify 'archives more than one type' do
receiver.catch search_request(2), DummiesIndex::Fizz
receiver.catch search_request(3), DummiesIndex::Buzz
expect(receiver.indexes.keys).to match_array([DummiesIndex::Fizz, DummiesIndex::Buzz])
end
end

context 'indexes_for' do
before do
receiver.catch search_request(2), DummiesIndex::Fizz
receiver.catch search_request(3), DummiesIndex::Buzz
end

specify 'returns indexes for a specific type' do
expect(parse_request receiver.indexes_for(DummiesIndex::Fizz)).to match_array([1,2])
end

specify 'returns only indexes for all types' do
index_responses = receiver.indexes
expect(index_responses.keys).to match_array([DummiesIndex::Fizz, DummiesIndex::Buzz])
expect(parse_request index_responses.values.flatten).to match_array([1, 2, 1, 2, 3])
end
end

context 'deletes_for' do
before do
receiver.catch search_request(2, verb: :delete), DummiesIndex::Fizz
receiver.catch search_request(3, verb: :delete), DummiesIndex::Buzz
end

specify 'returns deletes for a specific type' do
expect(receiver.deletes_for(DummiesIndex::Buzz)).to match_array([1,2,3])
end

specify 'returns only deletes for all types' do
deletes = receiver.deletes
expect(deletes.keys).to match_array([DummiesIndex::Fizz, DummiesIndex::Buzz])
expect(deletes.values.flatten).to match_array([1, 2, 1, 2, 3])
end
end

context 'indexed?' do
before do
receiver.catch search_request(1), DummiesIndex::Fizz
end

specify 'validates that an object was indexed' do
dummy = OpenStruct.new(id: 1)
expect(receiver.indexed? dummy, DummiesIndex::Fizz).to be(true)
end

specify 'doesn\'t validate than unindexed objects were indexed' do
dummy = OpenStruct.new(id: 2)
expect(receiver.indexed? dummy, DummiesIndex::Fizz).to be(false)
end
end

context 'deleted?' do
before do
receiver.catch search_request(1, verb: :delete), DummiesIndex::Fizz
end

specify 'validates than an object was deleted' do
dummy = OpenStruct.new(id: 1)
expect(receiver.deleted? dummy, DummiesIndex::Fizz).to be(true)
end

specify 'doesn\'t validate than undeleted objects were deleted' do
dummy = OpenStruct.new(id: 2)
expect(receiver.deleted? dummy, DummiesIndex::Fizz).to be(false)
end
end

context 'updated_indexes' do
specify 'provides a list of indices updated' do
receiver.catch search_request(2, verb: :delete), DummiesIndex::Fizz
receiver.catch search_request(3, verb: :delete), DummiesIndex::Buzz
expect(receiver.updated_indexes).to match_array([DummiesIndex::Fizz, DummiesIndex::Buzz])
end
end

end