proposal: spec: add decimal float types (IEEE 754-2008)

[typeless]

I know there was a proposal about decimal as an std package.
But according to https://en.wikipedia.org/wiki/Decimal_floating_point, there have already been multiple compliant hardware implementations, and supported by several commonly used programming languages. I think having decimal as first-class citizen like the other part of the IEEE-754 standard makes sense.

The upside is we don't have to spend the effort on API design upfront to make it useful like math.big. We can just limit the scope of the operations only within the Go's existing arithmetic operators. Designing a good numerical/arithmetic API beyond the conventional operators is harder and can be deferred later with no harm.

[bradfitz]

Related: #12127, #19770

[bronze1man]

I think this proposal can be implement outside golang runtime, builtin and builtin package.
Does any implement exist?

[typeless]

@bronze1man Yes, there are thirty-party packages and proposals for including a decimal package into std. But this proposal is about adding fixed-size decimal floating point types, along with the existing binary floating point types using the existing arithmetic operators, rather than introducing a new package.

[deanveloper]

I made my own post for this without seeing this one first. Here are the reasons I believe that a decimal float type would be good for Go - #26699

[deanveloper]

Adding onto this - If we had operator overloading, this could be implemented as a 3rd party lib instead

[bradfitz]

Some useful information in:
http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n3871.html

In particular, this reference was useful:

The need for support of exact decimal computations is recognized in many communities and supported in several systems, although different alternatives for the support are chosen. Below is a list of example programming languages with decimal support:

• The C committee is working on a Decimal TR as TR 24732. The decimal support in C uses built-in types _Decimal32, _Decimal64, and _Decimal128.
• Java provides decimal arithmetic by java.math.BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.
• Python provides decimal.Decimal which is a fixed point decimal representation. The number of decimal digits can be set globally.
• .Net provides System.Decimal which is a 128 bit decimal floating point. The details of this represntation are slightly different from the 128 bit decimal floating point in IEEE 754–2008. System.Decimal is accessible in C# as decimal.
• SQL provides a fixed point decimal representation where the number of digits and the number of fractional digits can be chosen for each context.
• Ruby provides BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.

[ianlancetaylor]

It would help to have a clear idea of who would want to use these new types.

It would also help if the proposal spelled out the new types and how untyped constants would be handled.

[ericlagergren]

It would help to have a clear idea of who would want to use these new types.

From manually polling people who use my (big) decimal package:

• Banks
• CAD
• Numerical analysis
• Databases (see: cockroachdb/apd)
• E-commerce sites
• Bitcoin sites

[ianlancetaylor]

Thanks! But we shouldn't confuse big decimals with this proposal, which is for fixed size decimals. It's not obvious to me that people who use the former will want to use the latter. For example, the fact that people use the math/big Rat type does not suggest that we should add a rational number type to the language.

[ericlagergren]

I think there's more overlap than it would seem. A 128-bit decimal gives you 34 significant digits which is plenty for most regular usage, like wanting to perform accurate monetary arithmetic.

And, as Brad's link mentioned, four of the six listed languages have only big decimals. So, I think it's somewhat germane :)

A point to consider (though I'm definitely in support of adding decimal type(s)): if decimals are added there will likely need to be support in the math/ package, like math and math/cmplx.

[djadala]

In this proposal, i expect only fixed size decimals, and i need them for financial computations.

[ianlancetaylor]

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

[deanveloper]

Section 1.1 discusses a few reasons here - http://speleotrove.com/decimal/IEEE-cowlishaw-arith16.pdf

[ericlagergren]

That PDF is good. His FAQ sections is good as well: http://speleotrove.com/decimal/decifaq1.html#inexact

TL;DR: floats are inaccurate because they can't exactly store base 10 numbers.

Integers (representing the smallest currency amount, e.g. 1 cent USD) work well, but can be annoying to use for anything other than basic arithmetic.

And for non-monetary calculations integers don't work. E.g., http://speleotrove.com/decimal/decifaq4.html#order

[djadala]

In my company, we used all 3 alternatives:
decimal in c/c++
int64 in go, c/c++,
float in c/c++, perl

floats are not exact numbers, making some elementary calculations can give surprising ( for financial people) results.
just storing and loading float64 from database (oracle use some sort of decimals for numbers) can lead to inexact value, and some financial applications are 90% database io.
int64s mostly works, but are cumbersome (IMO), in all IO to external systems (including databases) you must convert them to "right" form.

hope this is useful

[alanfo]

As I can personally testify, the 128 bit decimal type is great for writing financial applications in C# which need to be accurate to the cent, penny etc. as you don't have to worry about the rounding errors which can accumulate if you use their float or double types. The results are always exact even for the largest amounts likely to be encountered in practice.

The alternative is to always work in cents using their 64 bit long type and then div/rem by 100 when you need to display or store the final result. However, this is both tedious and error prone and may even not be large enough when you're dealing with some currencies.

The twin drawbacks are the extra memory required and performance - decimal is much slower than double which in turn is slower than long. However, for most financial programmers this is a price worth paying for the convenience they gain from using decimal.

Back in pre-.NET days, VB6 etc. had a 64 bit Currency type which (IIRC) was scaled to 4 d.p. This was more than adequate for applications which didn't need to process large amounts of money and I always thought it was a mistake not to support it in .NET as well.

So, yes, if you'd like Go to appeal to financial programmers, I think that adding language support for fixed point decimal types would certainly be worthwhile and I'd suggest that 64 bit and 128 bit types (perhaps named dec64 and dec128 respectively) would be adequate as I can't see much point nowadays in having a 32 bit type as well.

As far as untyped constants are concerned, I'd suggest that they should work in a similar fashion to what they do now for the other basic numeric types. So literals would accommodate themselves to whatever type was required, there'd be conversions to/from the other numeric types and so on. However, no untyped numeric constant would have a default type of either dec64 or dec128. So for example, we'd have:

    a := 123.45              // a is float64
    b := dec128(123.45)      // b is dec128
    var c = dec64(a)         // c is dec64
    d := b + 6               // d is dec128
    var e dec64 = 678.9      // e is dec64

[typeless]

I believe most people can agree that there is a need for decimals. The question is whether the standard library should provide it or whether it should be a primitive type.
The shortcoming of making decimal as a third-party package is that libraries working with it are hard to cooperate. For example, a package provides finance calculations based on a particular decimal type would not be naturally composable with another decimal type belonging to a time series data store. Then, it would create a situation where a financial application might have to include multiple decimal packages, and programmers have to write conversion code between them, which is clumsy and inefficient.

[alanfo]

As @deanveloper indicated earlier, the lack of operator overloading in Go would also be a problem if decimal support were provided as a library rather than built into the language.

In fact, I suspect most folks would prefer to stick with what they currently do rather than having to deal with named methods for the basic arithmetic operations. Whilst it's true that you have to do this already for 'big' number arithmetic, the difference is that you have no choice if you need to deal with such numbers.

[vitesse-ftian]

Just some background from data/database people's point of view. Decimal64/128 is really a big help. As said in previous posts, dec32 is probably not interesting.

Unfortunately IEEE allow two different formats for decimal, BID and DPD -- I believe one is from Intel and the other from IBM -- therefore, it is necessary to decide on one format and maybe code conversion routines between these two. Both formats have high quality open source implementations.

Decimal64/128 is slower than double, but not too slow, we measured 2 to 3 x compared to double, which is quite impressive. Databases, for example, PostgreSQL, have to implement their own decimal/numerical type. In case of PostgreSQL, it is a variable length data type, supporting greater precision (but, decimal128 is enough for almost everybody), but much slower.

We need a native decimal64/128 type instead of cgo binding to intel/IBM library because cgo cost may just killed it.

[vitesse-ftian]

https://github.com/gcc-mirror/gcc/tree/master/libdecnumber

What GCC has.

[tc-hib]

Here, we're working with a proprietary language that is made for quickly developing business applications.

Today, I've helped a colleague fix a bug that was due to using binary floats instead of a builtin monetary type. The latter is an 80 bits fixed precision number with 6 decimal places, and it has always been sufficient for us.
My colleague only had to replace the type name everywhere and the bug was fixed.
It wouldn't have been that easy if the monetary type didn't have support for arithmetic operators.

I plan to present Go to my team, but I am a bit afraid of their reaction when I tell them how to write calculations with an external lib.

I think any serious business application should use decimal types. Maybe some developers don't realize that until an accountant asks them why their report doesn't show exactly same numbers as his Excel sheet.

A builtin dec128 would completely fulfill our need.
Of course, the ideal solution would be operator methods.

[tc-hib]

I have just stumbled upon this striking example:
https://play.golang.org/p/w-e7BDgo734

Trying to round 35.855 to 2 decimal places gives unexpected result.
Can be embarrassing in a business app.

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

[ianlancetaylor]

@tc-hib Thanks. I understand that floating point numbers behave in ways that people find surprising (https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). That's not the question I was trying to ask.

[tc-hib]

Oh, sorry I didn't get it.

Decimal type is a true silver bullet for all of us who don't need to save every cpu cycle, yet have to follow strict business rules. We simply use it everywhere. Once we've done that, it is really hard to go back, so we would always prefer translating our formulas to method calls rather than using ints or floats.

There are many legacy business apps (in COBOL for example, quite old), Excel sheets, etc. using decimal arithmetic as fluently as integers. I think Go would be a good fit for modernizing all those commerce specific apps thanks to its simplicity, but it would require "translating" every calculation carefully.

I believe using integers and floats is trickier than it seems, because we often have varying number of decimal places (3 for purchases, 2 for sales, and some intermediate calculations not rounded, such as taxes and commercial agreements). If we're only adding and multiplying numbers, or dividing by multiples of 5 and 2, users expect exact results. In fact, they expect same results as Excel sheet or pen and paper.

I must admit i don't have much experience in workarounds, because I've always been working with decimal types, and we nearly never use binary floats, as strange as it may seem.

[dunglas]

The upcoming "Structured Field Values for HTTP" RFC will have a decimal type. To support structured headers and trailers properly in the stdlib, support for decimal floating point will be mandatory.

[typeless]

https://www.reddit.com/r/golang/comments/l4x9zs/introducing_gofinancial/
Hopefully, the upcoming compiler refactoring for type parameters would make this more likely to happen as well.

[Jalalx]

I am new to Golang and found it surprising that after more than a decade maintainers aren’t convinced yet that having native support for decimals is essential for many applications.
Is there a serious drawback for adding decimals?

[ianlancetaylor]

Well, it's a lot of work.

And having reread the comments, I see that there is still no clear statement of exactly what types should be added to the language. I have a feeling that different people on this issue are talking about somewhat different things.

[tc-hib]

This proposal is about floats as defined by IEEE 754-2008, and I think most of us would be happy with only a 128 bit float.
It's good enough for most uses, just like binary floats are good enough for most uses except for correctness in decimal (human) representation.

I wouldn't expect a built-in type that "works like an int" to support arbitrary precision.

[ianlancetaylor]

For a language proposal we need to be precise. You are suggesting adding decimal128, but not decimal32, decimal64, or float128?

[tc-hib]

I believe decimal128 would suffice, yes, and it would be a huge improvement for all business apps, avoiding a lot of subtle bugs, especially for beginners.
Having a one size fits all type has proven to be worth it in several languages.
C# has decimal, which is a decimal128. It is widely used in business apps.

Having decimal64 or decimal32 seems like premature optimisation to me. Once we use decimal floats, we've usually already accepted their cost. Most of the time we'd rather have fixed point decimals for best performance anyway.

Having a float128 doesn't solve the same issue, that is, correctness in some representation. And I don't think the need would be as common as for the decimal type.

[Al2Klimov]

Integers (representing the smallest currency amount, e.g. 1 cent USD) work well, but can be annoying to use for anything other than basic arithmetic.

@ericlagergren Ah, you mean e.g. accrued interest, math.Pow, etc.?

[myserg]

To whom it may concern: there is a new, decent decimal128 implementation: github.com/woodsbury/decimal128. I've tested it and it's probably the best. I don't understand why the author hasn't advertised this package anywhere yet.

[treeder]

To whom it may concern: there is a new, decent decimal128 implementation: github.com/woodsbury/decimal128. I've tested it and it's probably the best. I don't understand why the author hasn't advertised this package anywhere yet.

That any better than the shopspring one that's been around for years?

[djadala]

That any better than the shopspring one that's been around for years?

It is better because it is fixed size (128 bits), not arbitrary precision. If i want to do scientific calculations, i will use arbitrary-precision, but for financial i want fixed type.

[myserg]

To whom it may concern: there is a new, decent decimal128 implementation: github.com/woodsbury/decimal128. I've tested it and it's probably the best. I don't understand why the author hasn't advertised this package anywhere yet.

That any better than the shopspring one that's been around for years?

Yes. Just make benchmarks for your case, and you will see the difference.

[avdva]

To whom it may concern: there is a new, decent decimal128 implementation: github.com/woodsbury/decimal128. I've tested it and it's probably the best. I don't understand why the author hasn't advertised this package anywhere yet.

That any better than the shopspring one that's been around for years?

shopspring uses *big.Int under the hood, which makes it quite slow if you need to perform some calculations (e.g. on an order book).

[lrewega]

I agree that adding decimal128 is more practical than decimal32 or decimal64 or even float128. I would certainly love to have the full IEEE 754-2008 suite, especially since generics reduce some of the burden of supporting many numeric types in libraries, etc., but having some decimal type is of considerable value. And if you were to pick only one, then decimal128 is clearly the forerunner:

• This TigerBeetle article is a good example of where 128-bit decimals can shine actually about int128 support but touches on issues with native support and decimals: https://tigerbeetle.com/blog/2023-09-19-64-bit-bank-balances-ought-to-be-enough-for-anybody/ (there is also a reference to Go's lack of native support)

• It was pointed out earlier in this thread, but note that .NET's original (recent) decimal implementation is not entirely equivalent to the decimal128 in IEEE 754-2008, but is nonetheless useful. There is a relevant discussion within this .NET runtime proposal: [API Proposal]: Add Decimal32, Decimal64, and Decimal128 from the IEEE 754-2019 standard. dotnet/runtime#81376 (note also that the proposal has been approved!)

• _Decimal32, _Decimal64, _Decimal128 have also been included in C23.

• As mentioned elsewhere, support for decimal types is common in databases, is included in SQL and facilitated by the database/sql package (see also proposal: database/sql: define a Decimal decompose interface for decimal packages #30870).

• Apache Arrow supports Decimal128 and Decimal256, though the Go implementation is currently using math/big which is unfortunate.

• MongoDB and BSON support decimal128.

Initially my thinking was that #9455 should be a higher priority than this, but after reading through comments in both that thread and this, and after porting a fairly large code base off of big.Rat to the excellent github.com/woodsbury/decimal128 package, it has been so painful that I now feel strongly that some native decimal type is warranted. The math/big package is very useful but difficult to hold correctly and folks get tripped up by its strategy to avoid allocation. Packages like github.com/woodsbury/decimal128 provide a more natural interface, though allocations become harder to control. But for both approaches, they become an absolute chore to use and leak through libraries in uncomfortable ways -- at least for me.

complex64 and complex128 appear to be considered unusual but occasionally very useful, and have proven themselves warranted. I would expect decimal128 to prove to be far more popular and useful much more often.

I am left unsatisfied by the disarray of packages, sometimes compatible at the seams, but each never feeling whole. Working with currencies and combining big.Rat, x/text/... and github.com/woodsbury/decimal128, for example, certainly does not feel cohesive or straightforward. By comparison, complex128 is very pleasant to use. (There of course are a few issues to unpack here but the lack of builtin support is the most glaring by far)

Would a proposal to add a single decimal128 type be amenable? Is that effort more worthwhile independent of efforts to support the other IEEE 754-2008 types? I suspect such a separate proposal would be closed as a duplicate of this one. I certainly understand hesitation to introduce a new builtin type to the language as that has consequences that I struggle to comprehend completely, but I similarly struggle to understand what the bar is for this proposal.