[Fix toptal#654] Mocking search support (toptal#794)

CHANGELOG.md

@@ -6,6 +6,12 @@
 
 ### Changes
 
+  * [#654](https://github.com/toptal/chewy/issues/654): Add helpers and matchers for testing ([@Vitalina-Vakulchyk][]):
+    * `mock_elasticsearch_response` helpers both Rspec and Minitest - to mock elasticsearch response
+    * `mock_elasticsearch_response_sources` helpers both Rspec and Minitest - to mock elasticsearch response sources
+    * `assert_elasticsearch_query` helper for Minitest - to compare request and expected query (returns `true`/`false`)
+    * `build_query` matcher for Rspec - to compare request and expected query (returns `true`/`false`)
+
 ### Bugs Fixed
 
 ## 7.2.1 (2021-05-11)

README.md

@@ -1117,7 +1117,16 @@ rake chewy:journal:apply["$(date -v-1H -u +%FT%TZ)",users] # apply journaled cha
 
 ### RSpec integration
 
-Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features: See [update_index.rb](lib/chewy/rspec/update_index.rb) for more details.
+Just add `require 'chewy/rspec'` to your spec_helper.rb and you will get additional features:
+
+[update_index](lib/chewy/rspec/update_index.rb) helper
+`mock_elasticsearch_response` helper to mock elasticsearch response
+`mock_elasticsearch_response_sources` helper to mock elasticsearch response sources
+`build_query` matcher to compare request and expected query (returns `true`/`false`)
+
+To use `mock_elasticsearch_response` and `mock_elasticsearch_response_sources` helpers add `include Chewy::Rspec::Helpers` to your tests.
+
+See [chewy/rspec/](lib/chewy/rspec/) for more details.
 
 ### Minitest integration
 
@@ -1127,6 +1136,14 @@ Since you can set `:bypass` strategy for test suites and manually handle import
 
 But if you require chewy to index/update model regularly in your test suite then you can specify `:urgent` strategy for documents indexing. Add `Chewy.strategy(:urgent)` to test_helper.rb.
 
+Also, you can use additional helpers:
+
+`mock_elasticsearch_response` to mock elasticsearch response
+`mock_elasticsearch_response_sources` to mock elasticsearch response sources
+`assert_elasticsearch_query` to compare request and expected query (returns `true`/`false`)
+
+See [chewy/minitest/](lib/chewy/minitest/) for more details.
+
 ### DatabaseCleaner
 
 If you use `DatabaseCleaner` in your tests with [the `transaction` strategy](https://github.com/DatabaseCleaner/database_cleaner#how-to-use), you may run into the problem that `ActiveRecord`'s models are not indexed automatically on save despite the fact that you set the callbacks to do this with the `update_index` method. The issue arises because `chewy` indices data on `after_commit` run as default, but all `after_commit` callbacks are not run with the `DatabaseCleaner`'s' `transaction` strategy. You can solve this issue by changing the `Chewy.use_after_commit_callbacks` option. Just add the following initializer in your Rails application:

lib/chewy/minitest/helpers.rb

@@ -42,10 +42,87 @@ def assert_indexes(index, strategy: :atomic, bypass_actual_index: true, &block)
       # Run indexing for the database changes during the block provided.
       # By default, indexing is run at the end of the block.
       # @param strategy [Symbol] the Chewy index update strategy see Chewy docs.
+      #
       def run_indexing(strategy: :atomic, &block)
         Chewy.strategy strategy, &block
       end
 
+      # Mock Elasticsearch response
+      # Simple usage - just pass index, expected raw response
+      # and block with the query.
+      #
+      # @param index [Chewy::Index] the index to watch, eg EntitiesIndex.
+      # @param raw_response [Hash] hash with response.
+      #
+      def mock_elasticsearch_response(index, raw_response)
+        mocked_request = Chewy::Search::Request.new(index)
+
+        original_new = Chewy::Search::Request.method(:new)
+
+        Chewy::Search::Request.define_singleton_method(:new) { |*_args| mocked_request }
+
+        original_perform = mocked_request.method(:perform)
+        mocked_request.define_singleton_method(:perform) { raw_response }
+
+        yield
+      ensure
+        mocked_request.define_singleton_method(:perform, original_perform)
+        Chewy::Search::Request.define_singleton_method(:new, original_new)
+      end
+
+      # Mock Elasticsearch response with defined sources
+      # Simple usage - just pass index, expected sources
+      # and block with the query.
+      #
+      # @param index [Chewy::Index] the index to watch, eg EntitiesIndex.
+      # @param hits [Hash] hash with sources.
+      #
+      def mock_elasticsearch_response_sources(index, hits, &block)
+        raw_response = {
+          'took' => 4,
+          'timed_out' => false,
+          '_shards' => {
+            'total' => 1,
+            'successful' => 1,
+            'skipped' => 0,
+            'failed' => 0
+          },
+          'hits' => {
+            'total' => {
+              'value' => hits.count,
+              'relation' => 'eq'
+            },
+            'max_score' => 1.0,
+            'hits' => hits.each_with_index.map do |hit, i|
+              {
+                '_index' => index.index_name,
+                '_type' => '_doc',
+                '_id' => (i + 1).to_s,
+                '_score' => 3.14,
+                '_source' => hit
+              }
+            end
+          }
+        }
+
+        mock_elasticsearch_response(index, raw_response, &block)
+      end
+
+      # Check the assertion that actual Elasticsearch query is rendered
+      # to the expected query
+      #
+      # @param query [::Query] the actual Elasticsearch query.
+      # @param expected_query [Hash] expected query.
+      #
+      # @return [Boolean]
+      #   True - in the case when actual Elasticsearch query is rendered to the expected query.
+      #   False - in the opposite case.
+      #
+      def assert_elasticsearch_query(query, expected_query)
+        actual_query = query.render
+        assert_equal expected_query, actual_query, "got #{actual_query.inspect} instead of expected query."
+      end
+
       module ClassMethods
         # Declare that all tests in this file require real indexing, always.
         # In my completely unscientific experiments, this roughly doubled test runtime.

lib/chewy/rspec.rb

@@ -1 +1,3 @@
+require 'chewy/rspec/build_query'
+require 'chewy/rspec/helpers'
 require 'chewy/rspec/update_index'

lib/chewy/rspec/build_query.rb

@@ -0,0 +1,12 @@
+# Rspec helper to compare request and expected query
+# To use it - add `require 'chewy/rspec/build_query'` to the `spec_helper.rb`
+# Simple usage - just pass expected response as argument
+# and then call needed query.
+#
+#   expect { method1.method2...methodN }.to build_query(expected_query)
+#
+RSpec::Matchers.define :build_query do |expected_query = {}|
+  match do |request|
+    request.render == expected_query
+  end
+end

lib/chewy/rspec/helpers.rb

@@ -0,0 +1,55 @@
+module Chewy
+  module Rspec
+    module Helpers
+      extend ActiveSupport::Concern
+      # Rspec helper to mock elasticsearch response
+      # To use it - add `require 'chewy/rspec'` to the `spec_helper.rb`
+      #
+      #   mock_elasticsearch_response(CitiesIndex, raw_response)
+      #   expect(CitiesIndex.query({}).hits).to eq(hits)
+      #
+      def mock_elasticsearch_response(index, raw_response)
+        mocked_request = Chewy::Search::Request.new(index)
+        allow(Chewy::Search::Request).to receive(:new).and_return(mocked_request)
+        allow(mocked_request).to receive(:perform).and_return(raw_response)
+      end
+
+      # Rspec helper to mock Elasticsearch response source
+      # To use it - add `require 'chewy/rspec'` to the `spec_helper.rb`
+      #
+      #  mock_elasticsearch_response_sources(CitiesIndex, sources)
+      #  expect(CitiesIndex.query({}).hits).to eq(hits)
+      #
+      def mock_elasticsearch_response_sources(index, hits)
+        raw_response = {
+          'took' => 4,
+          'timed_out' => false,
+          '_shards' => {
+            'total' => 1,
+            'successful' => 1,
+            'skipped' => 0,
+            'failed' => 0
+          },
+          'hits' => {
+            'total' => {
+              'value' => hits.count,
+              'relation' => 'eq'
+            },
+            'max_score' => 1.0,
+            'hits' => hits.each_with_index.map do |hit, i|
+              {
+                '_index' => index.index_name,
+                '_type' => '_doc',
+                '_id' => (i + 1).to_s,
+                '_score' => 3.14,
+                '_source' => hit
+              }
+            end
+          }
+        }
+
+        mock_elasticsearch_response(index, raw_response)
+      end
+    end
+  end
+end

spec/chewy/minitest/helpers_spec.rb

@@ -12,6 +12,10 @@ def assert_includes(haystack, needle, _comment)
 
   include ::Chewy::Minitest::Helpers
 
+  def assert_equal(expected, actual, message)
+    raise message unless expected == actual
+  end
+
   before do
     Chewy.massacre
   end
@@ -22,6 +26,112 @@ def assert_includes(haystack, needle, _comment)
     end
   end
 
+  describe 'mock_elasticsearch_response' do
+    let(:hits) do
+      [
+        {
+          '_index' => 'dummies',
+          '_type' => '_doc',
+          '_id' => '1',
+          '_score' => 3.14,
+          '_source' => source
+        }
+      ]
+    end
+
+    let(:source) { {'name' => 'some_name'} }
+    let(:sources) { [source] }
+
+    context 'mocks by raw response' do
+      let(:raw_response) do
+        {
+          'took' => 4,
+          'timed_out' => false,
+          '_shards' => {
+            'total' => 1,
+            'successful' => 1,
+            'skipped' => 0,
+            'failed' => 0
+          },
+          'hits' => {
+            'total' => {
+              'value' => 1,
+              'relation' => 'eq'
+            },
+            'max_score' => 1.0,
+            'hits' => hits
+          }
+        }
+      end
+
+      specify do
+        mock_elasticsearch_response(DummiesIndex, raw_response) do
+          expect(DummiesIndex.query({}).hits).to eq(hits)
+        end
+      end
+    end
+
+    context 'mocks by response sources' do
+      specify do
+        mock_elasticsearch_response_sources(DummiesIndex, sources) do
+          expect(DummiesIndex.query({}).hits).to eq(hits)
+        end
+      end
+    end
+  end
+
+  describe 'assert correct elasticsearch query' do
+    let(:query) do
+      DummiesIndex.filter.should { multi_match foo: 'bar' }.filter { match foo: 'bar' }
+    end
+
+    let(:expected_query) do
+      {
+        index: ['dummies'],
+        body: {
+          query: {
+            bool: {
+              filter: {
+                bool: {
+                  must: {
+                    match: {foo: 'bar'}
+                  },
+                  should: {
+                    multi_match: {foo: 'bar'}
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    end
+
+    context 'will be built' do
+      specify do
+        expect { assert_elasticsearch_query(query, expected_query) }.not_to raise_error
+      end
+    end
+
+    context 'will not be built' do
+      let(:unexpected_query) do
+        {
+          index: ['what?'],
+          body: {}
+        }
+      end
+
+      let(:unexpected_query_error_message) do
+        'got {:index=>["dummies"], :body=>{:query=>{:bool=>{:filter=>{:bool=>{:must=>{:match=>{:foo=>"bar"}}, :should=>{:multi_match=>{:foo=>"bar"}}}}}}}} instead of expected query.'
+      end
+
+      specify do
+        expect { assert_elasticsearch_query(query, unexpected_query) }
+          .to raise_error(RuntimeError, unexpected_query_error_message)
+      end
+    end
+  end
+
   context 'assert_indexes' do
     specify 'doesn\'t fail when index updates correctly' do
       expect do

spec/chewy/rspec/build_query_spec.rb

@@ -0,0 +1,34 @@
+require 'spec_helper'
+
+describe :build_query do
+  before do
+    stub_model(:city)
+    stub_index(:cities) { index_scope City }
+    CitiesIndex.create
+  end
+
+  let(:expected_query) do
+    {
+      index: ['cities'],
+      body: {
+        query: {
+          match: {name: 'name'}
+        }
+      }
+    }
+  end
+  let(:dummy_query) { {match: {name: 'name'}} }
+  let(:unexpected_query) { {match: {name: 'name'}} }
+
+  context 'build expected query' do
+    specify do
+      expect(CitiesIndex.query(dummy_query)).to build_query(expected_query)
+    end
+  end
+
+  context 'not to build unexpected query' do
+    specify do
+      expect(CitiesIndex.query(dummy_query)).not_to build_query(unexpected_query)
+    end
+  end
+end