Skip to content

MONGOID-5212 Support for Decimal128 type #5125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jan 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,10 @@ axes:
display_name: "Driver-min (JRuby)"
variables:
DRIVER: "min-jruby"
- id: bson-min
display_name: "BSON-min"
variables:
DRIVER: "bson-min"
- id: "rails"
display_name: Rails Version
values:
Expand Down Expand Up @@ -663,7 +667,19 @@ buildvariants:
run_on:
- rhel70-small
tasks:
- name: "test"
- name: "test"

- matrix_name: "bson-min"
matrix_spec:
driver: [min]
ruby: ["ruby-2.7"]
mongodb-version: "5.0"
topology: "standalone"
display_name: "${ruby}, ${driver}, ${mongodb-version}, ${topology}"
run_on:
- ubuntu1804-small
tasks:
- name: "test"

- matrix_name: "rails-6"
matrix_spec:
Expand Down
5 changes: 4 additions & 1 deletion .evergreen/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ elif test "$DRIVER" = "oldstable"; then
elif test "$DRIVER" = "min"; then
bundle install --gemfile=gemfiles/driver_min.gemfile
BUNDLE_GEMFILE=gemfiles/driver_min.gemfile
elif test "$DRIVER" = "bson-min"; then
bundle install --gemfile=gemfiles/bson_min.gemfile
BUNDLE_GEMFILE=gemfiles/bson_min.gemfile
elif test "$DRIVER" = "stable-jruby"; then
bundle install --gemfile=gemfiles/driver_stable_jruby.gemfile
BUNDLE_GEMFILE=gemfiles/driver_stable_jruby.gemfile
Expand Down Expand Up @@ -104,7 +107,7 @@ elif test -n "$APP_TESTS"; then
bash $HOME/n stable
export PATH=$HOME/.n/bin:$PATH
npm -g install yarn

bundle exec rspec spec/integration/app_spec.rb
else
bundle exec rake ci
Expand Down
11 changes: 11 additions & 0 deletions gemfiles/bson_min.gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
source "https://rubygems.org"
gemspec path: '..'

gem 'bson', '4.14.0'
# This configuration doesn't require a specific driver version. When bson-4.14.0
# was released, current driver version was 2.17.0
gem 'mongo', '2.17.0'

require_relative './standard'

standard_dependencies
3 changes: 3 additions & 0 deletions lib/mongoid/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ module Config
# Return stored times as UTC.
option :use_utc, default: false

# Store BigDecimals as Decimal128s instead of strings in the db.
option :map_big_decimal_to_decimal128, default: false

# Has Mongoid been configured? This is checking that at least a valid
# client config exists.
#
Expand Down
10 changes: 9 additions & 1 deletion lib/mongoid/criteria/queryable/extensions/big_decimal.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,15 @@ module ClassMethods
# @return [ String ] The big decimal as a string.
def evolve(object)
__evolve__(object) do |obj|
obj ? obj.to_s : obj
if obj
if obj.is_a?(::BigDecimal) && Mongoid.map_big_decimal_to_decimal128
BSON::Decimal128.new(obj)
elsif obj.is_a?(BSON::Decimal128)
obj
else
obj.to_s
end
end
end
end
end
Expand Down
35 changes: 30 additions & 5 deletions lib/mongoid/extensions/big_decimal.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@ def __to_inc__
#
# @return [ Object ] The object.
def mongoize
to_s
if Mongoid.map_big_decimal_to_decimal128
BSON::Decimal128.new(self)
else
to_s
end
end

# Is the BigDecimal a number?
Expand All @@ -46,19 +50,40 @@ module ClassMethods
#
# @return [ BigDecimal, nil ] A BigDecimal derived from the object or nil.
def demongoize(object)
object && object.numeric? ? BigDecimal(object.to_s) : nil
unless object.nil?
if object.is_a?(BSON::Decimal128)
object.to_big_decimal
elsif object.numeric?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like the numeric? condition to be dropped and this path be taken for non-numeric input also. I see that non-numeric input was already trashed previously but I think this is not good behavior.

Test with a time field:

  class Foo
    include Mongoid::Document

    field :a
  end

  Foo.delete_all

  Foo.create!(a: 'zz')

  class Foo
    field :a, type: BigDecimal
  end

  p Foo.first, Foo.first.a

produces:

NoMethodError: undefined method `getlocal' for "zz":String

Test with the code as proposed with BigDecimal field:

  class Foo
    include Mongoid::Document

    field :a
  end

  Foo.delete_all

  Foo.create!(a: 'zz')

  class Foo
    field :a, type: Time
  end

  p Foo.first, Foo.first.a

produces:

#<Foo _id: 61d76a95a15d5d4e9c038186, a: "zz">
nil

I think it would be better to always attempt to construct a BigDecimal, allowing that call to raise whatever exceptions on problems.

@comandeo thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. I think demongoize should return nil unless the object can be represented as a BigDecimal. Should check the behavior of demongoize on other classes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@p-mongo I kind of agree with @johnnyshields here... Here are some examples of invalid values to demongoize:

irb(main):004:0> Float.demongoize("1.2asda")
=> 0.0
irb(main):005:0> Integer.demongoize("1234rqwdd")
=> 0

Now, I'm not sure why 0 is the right return value here but the important thing is that it doesn't raise an error. Let me know what you think...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neilshweky is correct should make this return 0. Important thing is that it doesn't raise error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually after rereading your comment, I agree @p-mongo. Let's let BigDecimal throw an error

Copy link
Contributor

@johnnyshields johnnyshields Jan 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the two existing numeric types (Float and Integer) currently return 0, I would be inclined to say BigDecimal should return 0. Fine to raise a follow-up ticket/PR if we wish to change it later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for now it's better practice to return nil, and we should file a ticket to follow upon Float and Integer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created https://jira.mongodb.org/browse/MONGOID-5221 to address the demongoization issue across all types.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unresolving this discussion thread because it's referenced from https://jira.mongodb.org/browse/MONGOID-5221.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've crossed out my earlier comment here--BigDecimal correctly returns nil today for non-numeric string, we should do the same for Integer/Float. It's ready to merge in PR: #5220

BigDecimal(object.to_s)
end
end
end

# Mongoize an object of any type to how it's stored in the db as a String.
# Mongoize an object of any type to how it's stored in the db.
#
# @example Mongoize the object.
# BigDecimal.mongoize(123)
#
# @param [ Object ] object The object to Mongoize
#
# @return [ String, nil ] A String representing the object or nil.
# @return [ String | BSON::Decimal128 | nil ] A String or Decimal128
# representing the object or nil.
def mongoize(object)
object && object.numeric? ? object.to_s : nil
unless object.nil?
if object.is_a?(BSON::Decimal128)
object
elsif Mongoid.map_big_decimal_to_decimal128
if object.is_a?(BigDecimal)
BSON::Decimal128.new(object)
elsif object.numeric?
BSON::Decimal128.new(object.to_s)
else
object.mongoize
end
elsif object.numeric?
object.to_s
end
end
end
end
end
Expand Down
2 changes: 1 addition & 1 deletion mongoid.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Gem::Specification.new do |s|
# Therefore, usage of this gem looks like a reasonable solution at the moment.
s.add_dependency("ruby2_keywords", "~> 0.0.5")

s.add_development_dependency("bson", ['>=4.9.4', '<5.0.0'])
s.add_development_dependency("bson", ['>=4.14.0', '<5.0.0'])

s.files = Dir.glob("lib/**/*") + %w(CHANGELOG.md LICENSE README.md Rakefile)
s.test_files = Dir.glob("spec/**/*")
Expand Down
46 changes: 46 additions & 0 deletions spec/mongoid/config_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,52 @@
end
end

context 'when the map_big_decimal_to_decimal128 option is set in the config' do


before do
Mongoid.configure do |config|
config.load_configuration(conf)
end
end

context 'when the value is false' do

let(:conf) do
CONFIG.merge(options: { map_big_decimal_to_decimal128: false })
end

it 'sets the Mongoid.map_big_decimal_to_decimal128 option to the provided value' do
expect(Mongoid.map_big_decimal_to_decimal128).to be(false)
end
end

context 'when the value is true' do

let(:conf) do
CONFIG.merge(options: { map_big_decimal_to_decimal128: true })
end

it 'sets the Mongoid.map_big_decimal_to_decimal128 option to the provided value' do
expect(Mongoid.map_big_decimal_to_decimal128).to be(true)
end
end
end

context 'when the map_big_decimal_to_decimal128 option is not set in the config' do

before do
Mongoid::Config.reset
Mongoid.configure do |config|
config.load_configuration(CONFIG)
end
end

it 'does not set the Mongoid.map_big_decimal_to_decimal128 option' do
expect(Mongoid.map_big_decimal_to_decimal128).to be(false)
end
end

describe "#load!" do

before(:all) do
Expand Down
120 changes: 93 additions & 27 deletions spec/mongoid/criteria/queryable/extensions/big_decimal_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,55 +6,121 @@

describe ".evolve" do

context "when provided a big decimal" do
context 'when map_big_decimal_to_decimal128 is false' do

let(:big_decimal) do
BigDecimal("123456.789")
context "when provided a big decimal" do

let(:big_decimal) do
BigDecimal("123456.789")
end

it "returns the decimal as a string" do
expect(described_class.evolve(big_decimal)).to eq(big_decimal.to_s)
end
end

it "returns the decimal as a string" do
expect(described_class.evolve(big_decimal)).to eq(big_decimal.to_s)
context "when provided a non big decimal" do

it "returns the object as a string" do
expect(described_class.evolve("testing")).to eq("testing")
end
end
end

context "when provided a non big decimal" do
context "when provided an array of big decimals" do

let(:bd_one) do
BigDecimal("123456.789")
end

let(:bd_two) do
BigDecimal("123333.789")
end

it "returns the object as a string" do
expect(described_class.evolve("testing")).to eq("testing")
let(:array) do
[ bd_one, bd_two ]
end

let(:evolved) do
described_class.evolve(array)
end

it "returns the array as strings" do
expect(evolved).to eq([ bd_one.to_s, bd_two.to_s ])
end

it "does not evolve in place" do
expect(evolved).to_not equal(array)
end
end
end

context "when provided an array of big decimals" do
context "when provided nil" do

let(:bd_one) do
BigDecimal("123456.789")
it "returns nil" do
expect(described_class.evolve(nil)).to be_nil
end
end
end

let(:bd_two) do
BigDecimal("123333.789")
context 'when map_big_decimal_to_decimal128 is true' do

before do
Mongoid.map_big_decimal_to_decimal128 = true
end

let(:array) do
[ bd_one, bd_two ]
after do
Mongoid.map_big_decimal_to_decimal128 = false
end

let(:evolved) do
described_class.evolve(array)
context "when provided a big decimal" do

let(:big_decimal) do
BigDecimal("123456.789")
end

it "returns the decimal as a BSON::Decimal128 object" do
expect(described_class.evolve(big_decimal)).to eq(BSON::Decimal128.new(big_decimal))
end
end

it "returns the array as strings" do
expect(evolved).to eq([ bd_one.to_s, bd_two.to_s ])
context "when provided a non big decimal" do

it "returns the object as a string" do
expect(described_class.evolve("testing")).to eq("testing")
end
end

it "does not evolve in place" do
expect(evolved).to_not equal(array)
context "when provided an array of big decimals" do

let(:bd_one) do
BigDecimal("123456.789")
end

let(:bd_two) do
BigDecimal("123333.789")
end

let(:array) do
[ bd_one, bd_two ]
end

let(:evolved) do
described_class.evolve(array)
end

it "returns the array as BSON::Decimal128s" do
expect(evolved).to eq([ BSON::Decimal128.new(bd_one), BSON::Decimal128.new(bd_two) ])
end

it "does not evolve in place" do
expect(evolved).to_not equal(array)
end
end
end

context "when provided nil" do
context "when provided nil" do

it "returns nil" do
expect(described_class.evolve(nil)).to be_nil
it "returns nil" do
expect(described_class.evolve(nil)).to be_nil
end
end
end
end
Expand Down
Loading