Improve handling of monetary values

[michaelbromley]

Background

Currently in Vendure we store and transmit all monetary values as integers in the minor unit of the currency (cents, pennies etc). The reason for this is to avoid floating-point math errors when dealing with decimals, i.e. 0.1 + 0.2 = 0.30000000000000004.

This is a widely-used and recommended way of dealing with currencies and works well overall.

However, it does have a weakness in that when working with decimal multiplications or divisions - very common operations when applying taxes or promotional discounts - the lack of precision (fractions of cents) can result in unexpected results, off by one or a few cents.

Reference: Best way to store currency values in C++

Example: Calculating taxes

• Let's say we have sell a Pen for $0.31.
• Our tax rate is 20%.
• Customer buys 10 pens.

Customer might expect to pay a total of 0.31 * 1.2 * 10, i.e. $3.72

The actual calculation done by Vendure will be

31 * 1.2 = 37.2
Math.round(37.2) =37
37 * 10 = 370

So customer will end up paying $3.70. Now in this case the customer might be happy! But in other cases they will end up paying more than expected.

Example: syncing prices from external PIM

We have an existing PIM system that stores prices ex. VAT, in fractions of a penny. When we sync to Vendure, we send this ex. VAT amount and then in Vendure it must get rounded in order to be stored as an integer. This means that sometimes our inc. VAT prices differ from those in the PIM, due to the rounding error.

• we have a paint tube in the PIM listed at £5.37.
• Our tax rate is 20%
• The ex. vat price listed in the PIM is 5.37 / 1.2 = 4.475

So we cannot represent this in Vendure, as we would need to store it as 447.5, but we only work with integers so this gets rounded to 448. When we then calculate the inc. VAT price, we get

448 * 1.2 = 537.6
Math.round(537.6) = 538

So we now list the paint tube as £5.38, rather than £5.37 as it should be to match the PIM.

Solution

To solve these issues we need to be able to work with more precision when performing calculations.

Money storage data type

It is still desirable to expose integers via the API, but as some point lower in the stack we need to be able to have greater precision, which means either a non-integer data type, or adding extra significant figures to the integer representation, i.e. storing $1 as 100_000 rather than 100, giving us 3 extra digits of precision to work with.

Since we perform all these calculations on the server, it should still be safe to expose the same minor-unit integer via the GraphQL APIs, so there would not need to be any breaking change to client applications.

Money transmission data type

Currently we expose all monetary values as the Int GraphQL type. However, this leads the a problem of integer overflow since the max 32-bit integer is 2,147,483,647, which gives us an upper limit on representing a maximum of ~21 million major units. For certain industries and high-denomination currencies, this can become a problem:

• Edge case - increase limits of monetary columns #838

Conversion

Somewhere in the system there will be a boundary, where the internal more-precise representation gets converted into the existing minor-unit integer. For example, we could use a custom GraphQL Scalar to take care of this (though this will leave a problem for values exposed not via GraphQL, e.g. a REST endpoint).

Strategy

I'd like to encapsulate all of the above in a strategy interface, which allows the developer to define:

1. The level of precision required
2. How the conversion happens in both directions

The default strategy would exactly replicate the existing behaviour probably, to reduce the chances of unexpected changes in behaviour. Business requirements and laws vary from place to place, so it makes the most sense to allow this to be configured as needed.

Next steps

This would be a major change that will touch virtually every part of Vendure core. I want to spend some time gathering feedback on this and experimenting with different designs to solve this in a way that:

1. handles the problem
2. is minimally-breaking
3. does not significantly increase the complexity of working with monetary values for the developer building on Vendure

[martijnvdbrug]

Does this only occur with taxes, or are there other scenarios?

In my perception, the tax cases were handled by the channel.pricesIncludeTax setting:
pricesIncludeTax=true Means we only use prices including tax for calculations.
⬆️ This does still result in discrepancies in the priceWithoutTax. Most websites I see avoid this discrepancy this by:

• Not displaying item prices without tax for consumers (channel.pricesIncludeTax=true)
• Not displaying item prices with tax for B2B (channel.pricesIncludeTax=false)

✔️ I still agree that more precision is better though, as it prevents 90% of rounding errors that occur now 🥳 It's just that the change does sound very impactful...

Precision strategy
It feels like Vendure should decide this for its users, instead of a strategy:
Generally speaking, more precision is always better, right? There are no laws that prohibit correct calculations, I assume.
(dangerous assumption 🙃 ?)

I agree that migration of an existing project would be a pain, but maintaining another strategy also comes at a cost (support, questions, issues, testing, development)

[JMPJNS]

Databases like Postgres already support that with money types and Languages like c# have a native decimal type and c++ is getting one soon, sadly i think nobody has yet written a native nodejs extension that adds one to javascript, there is a stage 1 proposal but that probably won't be implemented in the next few years