Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore empty blank options on fields #778

Merged
merged 77 commits into from
Apr 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
c5ad207
Add geo_point location field to base_spec.rb
Vitalina-Vakulchyk Mar 25, 2021
b58356a
Add geo_point expect to spec
Vitalina-Vakulchyk Mar 26, 2021
1e0fa94
Add field `ignore_blank` flag with ability to skip fields with .blank…
Vitalina-Vakulchyk Mar 26, 2021
bbebecb
Add ignore_blank to geo_point field spec in /spec/chewy/fields/base_s…
Vitalina-Vakulchyk Mar 26, 2021
bdf6d67
Merge branch 'master' into ignore-empty-blank-options-on-fields
Vitalina-Vakulchyk Mar 26, 2021
ab8cca5
Reorganize base_spec objects and scopes section
Vitalina-Vakulchyk Mar 26, 2021
f7cf5d3
Add geo_point type without ignore_blank: true flag spec
Vitalina-Vakulchyk Mar 26, 2021
cb6ee07
Update base_spec without redundant variables
Vitalina-Vakulchyk Mar 26, 2021
2271ede
Add surname and description to cities test table
Vitalina-Vakulchyk Mar 26, 2021
8d8f1ae
Add fields/base spec without location field for geo_point type
Vitalina-Vakulchyk Mar 26, 2021
02ceacd
Split base #compose result check into 2 methods
Vitalina-Vakulchyk Mar 26, 2021
235f77b
Remove spec of default spec behavior send nil when no field at all
Vitalina-Vakulchyk Mar 26, 2021
ff81e4e
Refactor base spec and trigger CI
Vitalina-Vakulchyk Mar 26, 2021
0336f24
Add spec for parental field with ignore_blank: true flag to fields/ba…
Vitalina-Vakulchyk Mar 29, 2021
12fa1f1
Add spec for parental field without ignore_blank: true flag to fields…
Vitalina-Vakulchyk Mar 29, 2021
619e172
Refactor fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
3e59693
Refactor lib/chewy/fields/base #compose key/value returning
Vitalina-Vakulchyk Mar 29, 2021
14fc202
Assign hashes to variables in fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
9b71f1b
Use inline return for if statement for #compose in fields/base
Vitalina-Vakulchyk Mar 29, 2021
3dad3dd
Rubocop: update type/witchcraft range definition
Vitalina-Vakulchyk Mar 29, 2021
2e69c18
Rubocop: refactor fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
996eb10
Rubocop: refactor fields/base_spec hashes
Vitalina-Vakulchyk Mar 29, 2021
2481339
Rubocop: continue to refactor fields/base_spec hashes
Vitalina-Vakulchyk Mar 29, 2021
9ae19ee
Regenerate rubocop_todo to hide "Redundant begin block" offence for l…
Vitalina-Vakulchyk Mar 29, 2021
8e809f8
Regenerate rubocop_todo to hide Ruby 3.0.0 offences
Vitalina-Vakulchyk Mar 29, 2021
82c198e
Update fields/base_spec to test parental and children fields with ign…
Vitalina-Vakulchyk Mar 29, 2021
fe3fa88
Uncomment fields/base geo_point specs
Vitalina-Vakulchyk Mar 29, 2021
243cde9
Update double quotes with single in fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
a049eba
Avoid Style/RedundantBegin Rubocop check for Ruby 2.7.2
Vitalina-Vakulchyk Mar 29, 2021
7ee6832
Add lib/chewy/type/witchcraft.rb to Lint/Syntax in rubocop_todo
Vitalina-Vakulchyk Mar 29, 2021
ccfcf2d
Revert "Add lib/chewy/type/witchcraft.rb to Lint/Syntax in rubocop_todo"
Vitalina-Vakulchyk Mar 29, 2021
039b1f5
Add lib/chewy/type/witchcraft.rb Style/SlicingWithRange to rubocop_todo
Vitalina-Vakulchyk Mar 29, 2021
94a60f2
Describe error of sending {} as location in fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
1b5972a
Rubocop remove space in fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
73f183f
Comment 2nd way of local: {} definition for fields/base_spec
Vitalina-Vakulchyk Mar 29, 2021
54dc3d5
Fix arguments precedency
Mar 29, 2021
8f55209
Fix geo_point type with ignore_blank: true flag in fields/base_spec
Vitalina-Vakulchyk Mar 30, 2021
9735359
Update Readme
Vitalina-Vakulchyk Mar 30, 2021
1e6de4f
Remove redundant space
Vitalina-Vakulchyk Mar 30, 2021
9f8e744
Update CHANGELOG
Vitalina-Vakulchyk Mar 30, 2021
8423088
Replace field_with_empty_value? with result.empty? statement for fiel…
Vitalina-Vakulchyk Mar 30, 2021
16e4e91
Replace .empty? with .blank? statement for fields/base_spec
Vitalina-Vakulchyk Mar 30, 2021
4867524
Allow to pass ignore_blank params as string
Vitalina-Vakulchyk Mar 30, 2021
5c4f3fb
Add spec for ignore_blank false for geo_point
Vitalina-Vakulchyk Mar 30, 2021
f5353a0
Add spec for geo_point ignore_blank:false
Vitalina-Vakulchyk Mar 30, 2021
075cc81
Updare response spec order for geo_point ignore_blank:false
Vitalina-Vakulchyk Mar 30, 2021
934701e
Update cities' surname field with historical_name field
Vitalina-Vakulchyk Mar 30, 2021
ab89ca1
Remove adding of cities = [] to countries in fields/base_spec
Vitalina-Vakulchyk Mar 30, 2021
3cf59e5
Split into 2 contents countries with and without cities
Vitalina-Vakulchyk Mar 30, 2021
7854793
Remove pry from gems
Vitalina-Vakulchyk Mar 30, 2021
7346ee8
Rubocop spaces
Vitalina-Vakulchyk Mar 30, 2021
e1c92de
Add specs for ignore_blank flag as string type
Vitalina-Vakulchyk Mar 30, 2021
057b265
Update rubocop_todo to avoid Lint/SymbolConversion string check
Vitalina-Vakulchyk Mar 30, 2021
7f760b1
Update Readme Skip record fields during import section with default v…
Vitalina-Vakulchyk Mar 30, 2021
0e5940e
Update Changelog with ``
Vitalina-Vakulchyk Mar 30, 2021
4e29d64
Simplify #compose ignore_blank
Vitalina-Vakulchyk Mar 30, 2021
2abc18c
Reorganize fields/base_spec contexts for ignore_blank option
Vitalina-Vakulchyk Mar 30, 2021
0830cc7
Rubocop it
Vitalina-Vakulchyk Mar 30, 2021
d6fb747
Remove Rubocop Style/SlicingWithRange offence
Vitalina-Vakulchyk Mar 31, 2021
9e657d2
Remove Rubocop Style/RedundantFreeze offence
Vitalina-Vakulchyk Mar 31, 2021
901355f
Revert "Remove Rubocop Style/RedundantFreeze offence"
Vitalina-Vakulchyk Mar 31, 2021
172758d
Remove Rubocop Style/RedundantBegin offence in lib/chewy/type/syncer.rb
Vitalina-Vakulchyk Mar 31, 2021
4a9e804
Remove Rubocop Style/RedundantBegin offence lib/chewy/search/request.rb
Vitalina-Vakulchyk Mar 31, 2021
a30139d
Add spec for with ignore_blank: false
Vitalina-Vakulchyk Mar 31, 2021
f0dffe3
Ruboco inline disable Lint/SymbolConversion for fields/base_spec for …
Vitalina-Vakulchyk Mar 31, 2021
bcb4326
Regenerate rubocop_todo
Vitalina-Vakulchyk Mar 31, 2021
56b4361
Update target Ruby version in rubocop.yml
Vitalina-Vakulchyk Mar 31, 2021
8b3fb90
Remove Rubocop Style/RedundantBegin offence for 2.7 and Rubocop versi…
Vitalina-Vakulchyk Mar 31, 2021
ffe7a3c
Update changelog with gramma
Vitalina-Vakulchyk Mar 31, 2021
7c88fca
Update readme with gramma
Vitalina-Vakulchyk Mar 31, 2021
d472fde
Update gramma and remove redundant check from fields/base_spec.rb
Vitalina-Vakulchyk Mar 31, 2021
cdbf68a
Update Rubocop TargetRubyVersion to 2.6
Vitalina-Vakulchyk Apr 1, 2021
4e6e3f9
Change compatibility with MRI from 2.5-3.0 to 2.6-3.0
Vitalina-Vakulchyk Apr 1, 2021
61d67cc
Mark in CHANGELOG dropping of Ruby 2.5 support
Vitalina-Vakulchyk Apr 2, 2021
e7ba698
Update CHANGELOG letter to uppercase in changes section
Vitalina-Vakulchyk Apr 2, 2021
df3ac19
Update rubocop_todo with $ rubocop --auto-gen-config
Vitalina-Vakulchyk Apr 2, 2021
ef2ce52
Try to launch CI without new .ruby-version file
Vitalina-Vakulchyk Apr 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ inherit_from: .rubocop_todo.yml

AllCops:
NewCops: enable
TargetRubyVersion: 2.6

Layout/AccessModifierIndentation:
EnforcedStyle: outdent
Expand Down
23 changes: 11 additions & 12 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This configuration was generated by
# `rubocop --auto-gen-config`
# on 2021-03-11 12:44:08 UTC using RuboCop version 1.11.0.
# on 2021-04-02 12:44:05 UTC using RuboCop version 1.11.0.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
Expand All @@ -13,16 +13,15 @@ Gemspec/RequiredRubyVersion:
Exclude:
- 'chewy.gemspec'

# Offense count: 3
# Offense count: 2
# Configuration parameters: AllowedMethods.
# AllowedMethods: enums
Lint/ConstantDefinitionInBlock:
Exclude:
- 'lib/chewy.rb'
- 'lib/chewy/rspec/update_index.rb'
- 'spec/chewy/config_spec.rb'

# Offense count: 12
# Offense count: 10
# Configuration parameters: AllowComments, AllowEmptyLambdas.
Lint/EmptyBlock:
Exclude:
Expand All @@ -42,30 +41,30 @@ Lint/MissingSuper:
- 'lib/chewy/type/adapter/object.rb'
- 'lib/chewy/type/adapter/orm.rb'

# Offense count: 36
# Offense count: 35
# Configuration parameters: IgnoredMethods, CountRepeatedAttributes.
Metrics/AbcSize:
Max: 41

# Offense count: 4
# Configuration parameters: CountComments, CountAsOne.
Metrics/ClassLength:
Max: 269
Max: 267

# Offense count: 13
# Configuration parameters: IgnoredMethods.
Metrics/CyclomaticComplexity:
Max: 12

# Offense count: 44
# Offense count: 43
# Configuration parameters: CountComments, CountAsOne, ExcludedMethods, IgnoredMethods.
Metrics/MethodLength:
Max: 29

# Offense count: 2
# Configuration parameters: CountComments, CountAsOne.
Metrics/ModuleLength:
Max: 174
Max: 158

# Offense count: 18
# Configuration parameters: IgnoredMethods.
Expand All @@ -89,7 +88,7 @@ Style/DocumentDynamicEvalDefinition:
- 'lib/chewy/type/crutch.rb'
- 'lib/chewy/type/witchcraft.rb'

# Offense count: 63
# Offense count: 58
Style/Documentation:
Enabled: false

Expand All @@ -99,7 +98,7 @@ Style/EvalWithLocation:
Exclude:
- 'spec/chewy/index_spec.rb'

# Offense count: 208
# Offense count: 191
# Cop supports --auto-correct.
# Configuration parameters: EnforcedStyle.
# SupportedStyles: always, always_true, never
Expand All @@ -113,7 +112,7 @@ Style/GuardClause:
- 'lib/chewy.rb'
- 'spec/support/active_record.rb'

# Offense count: 12
# Offense count: 10
# Cop supports --auto-correct.
Style/IfUnlessModifier:
Exclude:
Expand All @@ -125,7 +124,7 @@ Style/IfUnlessModifier:
- 'lib/chewy/type/witchcraft.rb'
- 'spec/support/active_record.rb'

# Offense count: 64
# Offense count: 53
# Cop supports --auto-correct.
# Configuration parameters: AutoCorrect, AllowHeredoc, AllowURI, URISchemes, IgnoreCopDirectives, IgnoredPatterns.
# URISchemes: http, https
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@

### Changes

* [#778](https://github.com/toptal/chewy/pull/778): Add `ignore_blank` option to `field` method ([@Vitalina-Vakulchyk][]):
Vitalina-Vakulchyk marked this conversation as resolved.
Show resolved Hide resolved
* `true` by default for the `geo_point` type
* `false` by default for other types

* [#780](https://github.com/toptal/chewy/pull/780): **(Breaking)** Drop support for Ruby 2.5 ([@Vitalina-Vakulchyk][]):

* [#780](https://github.com/toptal/chewy/pull/780): Changing default console strategy ([@Vitalina-Vakulchyk][]):
* ability to set default console strategy with `Chewy.console_strategy`
* `:urgent` is used by default if `Chewy.console_strategy` is not set

* [#776](https://github.com/toptal/chewy/pull/776): **(Breaking)** Removal of unnecessary features and integrations ([@Vitalina-Vakulchyk][]):
* `aws-sdk-sqs` / `shoryuken`
* `mongoid`
Expand Down
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ Chewy is an ODM (Object Document Mapper), built on top of the [the official Elas
* [Witchcraft™ technology](#witchcraft-technology)
* [Raw Import](#raw-import)
* [Index creation during import](#index-creation-during-import)
* [Skip record fields during import](#skip-record-fields-during-import)
* [Default values for different types](#default-values-for-different-types)
* [Journaling](#journaling)
* [Types access](#types-access)
* [Index manipulation](#index-manipulation)
Expand Down Expand Up @@ -97,7 +99,7 @@ Or install it yourself as:

### Ruby

Chewy is compatible with MRI 2.5-3.0¹.
Chewy is compatible with MRI 2.6-3.0¹.

> ¹ Ruby 3 is only supported with Rails 6.1

Expand Down Expand Up @@ -660,6 +662,25 @@ By default, when you perform import Chewy checks whether an index exists and cre
You can turn off this feature to decrease Elasticsearch hits count.
To do so you need to set `skip_index_creation_on_import` parameter to `false` in your `config/chewy.yml`

### Skip record fields during import

You can use `ignore_blank: true` to skip fields that return `true` for the `.blank?` method:

```ruby
define_type Country do
field :id
field :cities, ignore_blank: true do
field :id
field :name
field :surname, ignore_blank: true
field :description
end
end
```

#### Default values for different types

By default `ignore_blank` is false on every type except `geo_point`.

### Journaling

Expand Down
10 changes: 10 additions & 0 deletions lib/chewy/fields/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ def mappings_hash
def compose(*objects)
result = evaluate(objects)

return {} if result.blank? && ignore_blank?

if children.present? && !multi_field?
result = if result.respond_to?(:to_ary)
result.to_ary.map { |item| compose_children(item, *objects) }
Expand All @@ -53,6 +55,14 @@ def compose(*objects)

private

def geo_point?
@options[:type].to_s == 'geo_point'
end

def ignore_blank?
@options.fetch(:ignore_blank) { geo_point? }
end

def evaluate(objects)
object = objects.first

Expand Down
8 changes: 3 additions & 5 deletions lib/chewy/search/request.rb
Original file line number Diff line number Diff line change
Expand Up @@ -988,11 +988,9 @@ def reset
def perform(additional = {})
request_body = render.merge(additional)
ActiveSupport::Notifications.instrument 'search_query.chewy', notification_payload(request: request_body) do
begin
Chewy.client.search(request_body)
rescue Elasticsearch::Transport::Transport::Errors::NotFound
{}
end
Chewy.client.search(request_body)
rescue Elasticsearch::Transport::Transport::Errors::NotFound
{}
end
end

Expand Down
30 changes: 13 additions & 17 deletions lib/chewy/type/syncer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,10 @@ def missing_ids
def outdated_ids
return [] if source_data.blank? || index_data.blank? || !@type.supports_outdated_sync?

@outdated_ids ||= begin
if @parallel
parallel_outdated_ids
else
linear_outdated_ids
end
@outdated_ids ||= if @parallel
parallel_outdated_ids
else
linear_outdated_ids
end
end

Expand All @@ -147,19 +145,17 @@ def index_data
end

def source_and_index_data
@source_and_index_data ||= begin
if @parallel
::ActiveRecord::Base.connection.close if defined?(::ActiveRecord::Base)
result = ::Parallel.map(%i[source index], @parallel, &SOURCE_OR_INDEX_DATA_WORKER.curry[self, @type])
::ActiveRecord::Base.connection.reconnect! if defined?(::ActiveRecord::Base)
if result.first.keys.first == :source
[result.first.values.first, result.second.values.first]
else
[result.second.values.first, result.first.values.first]
end
@source_and_index_data ||= if @parallel
::ActiveRecord::Base.connection.close if defined?(::ActiveRecord::Base)
result = ::Parallel.map(%i[source index], @parallel, &SOURCE_OR_INDEX_DATA_WORKER.curry[self, @type])
::ActiveRecord::Base.connection.reconnect! if defined?(::ActiveRecord::Base)
if result.first.keys.first == :source
[result.first.values.first, result.second.values.first]
else
[fetch_source_data, fetch_index_data]
[result.second.values.first, result.first.values.first]
end
else
[fetch_source_data, fetch_index_data]
end
end

Expand Down
2 changes: 1 addition & 1 deletion lib/chewy/type/witchcraft.rb
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ def replace_lvar(node, old_variable, new_variable)
def replace_send(node, variable)
if node.is_a?(Parser::AST::Node)
if node.type == :send && node.children[0].nil?
node.updated(nil, [Parser::AST::Node.new(:lvar, [variable]), *node.children[1..-1]])
node.updated(nil, [Parser::AST::Node.new(:lvar, [variable]), *node.children[1..]])
else
node.updated(nil, node.children.map { |child| replace_send(child, variable) })
end
Expand Down
Loading