Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: add support for int128 and uint128 #9455

Open
mei-rune opened this issue Dec 27, 2014 · 151 comments
Open

proposal: spec: add support for int128 and uint128 #9455

mei-rune opened this issue Dec 27, 2014 · 151 comments
Labels
LanguageChange Suggested changes to the Go language LanguageChangeReview Discussed by language change review committee Proposal
Milestone

Comments

@mei-rune
Copy link

No description provided.

@mikioh mikioh changed the title can add int128 and uint128 support spec: add support for int128 and uint128 Dec 27, 2014
@ianlancetaylor ianlancetaylor added LanguageChange Suggested changes to the Go language repo-main labels Dec 27, 2014
@ianlancetaylor
Copy link
Contributor

Can you provide a real-life use case?

@twoleds
Copy link

twoleds commented Feb 12, 2015

it's good for UUID, IPv6, hashing (MD5) etc, we can store IPv6 into uint128 instead byte slice and do some mathematics with subnetworks, checking range of IP addresses

@minux
Copy link
Member

minux commented Feb 12, 2015

These use cases are not strong enough to justify adding 128-bit types,
which is a big task to emulate it on all targets.

  1. MD5 is not secure anymore, so there is little benefit adding types to
    store its result.
  2. How often do you need to manipulate a UUID as a number rather than a
    byte slice (or a string)?
  3. The other use cases can be done with math/big just as easy.

Also note that GCC doesn't support __int128 on 32-bit targets and Go do
want consistent language features across all supported architectures.

@twoleds
Copy link

twoleds commented Feb 13, 2015

I agree with you there aren't a lot of benefits for int128/uint128, maybe a little better performance for comparing and hashing in maps when we use uint128 for storing UUID/IPv6 because for byte slices or string we need do some loops and extra memory but it isn't important I think

@mei-rune
Copy link
Author

I stat all interface flux of a device in one day.

@rsc rsc changed the title spec: add support for int128 and uint128 proposal: spec: add support for int128 and uint128 Jun 20, 2017
@rsc rsc added the v2 An incompatible library change label Jun 20, 2017
@the80srobot
Copy link

In addition to crypto, UUID and IPv6, int128 would be enormously helpful for volatile memory analysis, by giving you a safe uintptr diff type.

@iMartyn
Copy link

iMartyn commented Oct 2, 2017

It also just makes code that much more readable if you have to deal with large IDs e.g. those you get back from google directory API amongst others (effectively they're uuids encoded as uint128).
Obviously you can use math/big but it makes the code much harder to reason about because you have to parse the code mentally first, distracting you from reading the code.

@ericlagergren
Copy link
Contributor

Adding a data point: ran into a situation with a current project where I need to compute (x * y) % m where x*y can possibly overflow and require a 128-bit integer. Doing the modulus by hand for the high and low halves is needlessly complicated.

@jfesler
Copy link

jfesler commented Jan 6, 2018

Another +1 for both IPv6 and UUID cases.

@ianlancetaylor
Copy link
Contributor

The examples of UUID and IPv6 are not convincing to me. Those types can be done as a struct just as easily.

It's not clear that this is worth doing if processors do not have hardware support for the type; are there processors with 128-bit integer multiply and divide instructions?

See also #19623.

@ianlancetaylor ianlancetaylor added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Jan 9, 2018
@ericlagergren
Copy link
Contributor

@ianlancetaylor I do not think so. GCC seems to use the obvious 6 instructions for mul, 4 for add and sub, and a more involved routine for quo. I'm not how anybody could emulate mul, add, or sub that precisely (in Go) without assembly, but that prohibits inlining and adds function call overhead.

@ianlancetaylor
Copy link
Contributor

The fact that the current tools can't yet inline asm code is not in itself an argument for changing the language. We would additionally need to see a significant need for efficient int128 arithmetic.

If there were hardware support, that in itself would suggest a need, since presumably the processor manufacturers would only add such instructions if people wanted them.

@ericlagergren
Copy link
Contributor

ericlagergren commented Jan 10, 2018

If there were hardware support, that in itself would suggest a need

A need that—presumably—compilers couldn't meet by adding their own 128-bit types, which they have. I mean, for all but division it's a couple extra instructions. For most cases that's been sufficient.

I confess I'm not an expert on CPU characteristics, but my understanding is much of the driving force behind adding larger sizes was the ability to address more memory. That makes me think general 128-bit support is rather unlikely.

Yet major compilers have added support (GCC, Clang, ICC, ...) for C and C++. Rust has them because of LLVM. Julia has them as well.

Other languages and compilers having support isn't sufficient reason to make a language change, sure. But it's evidence there exists a need other than simply UUIDs.

Their domain seems to lie in cryptography and arbitrary-precision calculations, for now.

@FlorianUekermann
Copy link
Contributor

Additional usecases are timestamps, cryptographic nonces and database keys.

Examples like database keys, nonces and UUID represent a pretty large collection of applications where keys/handles can't ever be reused or number ranges can't overlap.

@ianlancetaylor
Copy link
Contributor

@FlorianUekermann People keep saying UUID, but I see no reason that a UUID could not be implemented using a struct. It's not like people use arithmetic on a UUID once it has been created. The only reason to add int128 to the language is if people are going to use arithmetic on values of that type.

@FlorianUekermann
Copy link
Contributor

FlorianUekermann commented Jan 11, 2018

It's not like people use arithmetic on a UUID once it has been created

They do. UUIDs don't have to be random. Sequential UUIDs are common in databases for example. Combine sequential UUIDs with some range partitioning and you'll wish for integer ops in practice.

Still, timestamps seem like the most obvious example to me, where 64bit is not sufficient and the full range of arithmetic operations is obviously meaningful. Had it been available, I would expect that the time package contained some examples.

How big of an undertaking is the implementation of div? The rest seems rather straightforward.

@ericlagergren
Copy link
Contributor

How big of an undertaking is the implementation of div?

The code for naïve 128-bit division exists in the stdlib already (math/big). The PowerPC Compiler Writer’s Guide has a 32-bit implementation of 64-bit division (https://cr.yp.to/2005-590/powerpc-cwg.pdf, page 82) that can be translated upwards.

@josharian
Copy link
Contributor

Use case: [u]int128 can be used to check for overflow of [u]int64 operations in a natural way. Yes, this could make you want int256, but since int64 is the word size of many machines, this particular overflow matters a lot. See e.g. #21588. Other obvious options to address this use case are math/bits and
#19623.

Somewhat related use case: #21835 (comment).

@lfaoro
Copy link

lfaoro commented Mar 16, 2023

I don't understand why it's such a big deal to just do it, you have 128 comments on this issue, it's been open since 2014, other mainstream languages already complied w/ the users need.

What else do you need to consider support for int128? We don't want to use big.Int anymore.

@gophun
Copy link

gophun commented Mar 16, 2023

@lfaoro It's been explained why it would be a big deal: #9455 (comment)

@phuclv90
Copy link

We have a path where we represent the size of an IP range, and IPv6 ranges can be larger than 64 bits. uint128 would have fit the bill.

@thockin an IPv6 address isn't a number and you don't do arithmetic on it. Only bitwise operations are needed. Therefore that's not a good reason to add the type. There are already lots of comments regarding IPv6 above

@thockin
Copy link

thockin commented Mar 16, 2023 via email

@chiro-hiro
Copy link

If anyone try to build a WebAssembly runtime with Go, int128 would help.

@JesseCoretta
Copy link

JesseCoretta commented May 11, 2023

Hi!

I have read the comments on this thread, and for the most part I see both points of view. But one thing occurs to me, and while it is somewhat related to UUIDs, unfortunately this can of worms is a little deeper when you consider ASN.1 Object Identifiers.

Per [X.667], there is precedent for UUIDs to be interpreted as actual integers in certain Object Identifiers. One that exists in the wild, just for those who are curious, is:

// {joint-iso-itu-t(2) uuid(25) ans(987895962269883002155146617097157934)}
2.25.987895962269883002155146617097157934

Now, many of you can look at that leaf arc and see quite clearly that it will overflow not just int64, but uint64 as well. By quite a bit, actually:

  • 340,282,366,920,938,463,463,374,607,431,768,211,455 for uint128
  • 18,446,744,073,709,551,615 for uint64
  • 9,223,372,036,854,775,807 for int64

Now, I know some of you might be biting your tongues, thinking "oh no ... if she's right, that would mean the encoding/asn1-provided ObjectIdentifier type is basically invalid too, right?".

Well, in a way it always been invalid (at least in my point of view). No offense to the Go team, its just that because int is used as the slice type, and int allows negative values -- something that should never, ever appear in an OID -- that's something else I need to account for manually in my code. And this isn't even taking the overflow issue into account.

That said, I'm not here to bash the asn1 package, nor am I directly requesting any changes to it. I know how the Go team feels about this package, but quite frankly I love ASN.1 and I am just grateful it is supported via my favorite language.

But I am voting in favor of uint128 support (and, I suppose, int128 by necessity), just from my obscure PoV. I can easily make my own OID type (e.g.: []uint128) and do what I need to do, never worrying about overflows involving legal UUID-based OIDs. So I guess, from where I stand, its sort of a compromise.

One counter-argument that I can foresee:

"OIDs have no set limit on the magnitude of individual arcs, so conceivably this would be an issue when someone registers an OID that has some ungodly long number that would overflow the largest supported uintXXX-type in the universe."

This is true. One can only do so much. I'm also certain that OID RAs (Registration Authorities) look for such questionable moves and perhaps mitigate them. But you're not wrong. And yet, [X.667] exists for a reason, so I would cite that as motivation for supporting uint128. At least those registrations are legitimate, if a little odd ...

Thank you for listening

Jesse 💜❤️

@Bjohnson131
Copy link

Is the team still looking for "strong use cases" for uint128?

@josharian
Copy link
Contributor

@Bjohnson131 more strong use cases are welcome. Please "load hidden" on the comments on this issue first, though, and take a pass through--it's easy to miss things that have already been discussed above.

I suspect that the most helpful thing at this moment might be to write a complete doc, paying particular attention to what packages might be impacted (search for the text "strconv" and "generics" in comments after loading all hidden), and what the answer there is. But that's a lot of work, with no guarantee of impact.

(It'd also be helpful to update the original post with a list of use cases proposed in the comments.)

@phuclv90
Copy link

phuclv90 commented Jul 19, 2023

FWIW .NET added support for 128-bit int not so long ago and you may want to check out the discussions

@josharian
Copy link
Contributor

@phuclv90 thanks! That conversation is strikingly similar to this one, including (perhaps) the denouement of using generics once they were available to solve the strconv problem.

@Bjohnson131
Copy link

Bjohnson131 commented Oct 15, 2023

I think that there's 2 things that people have not mentioned. (u)int128s allow us to write cleaner, more error-free software. period.

There's an air among communities that this isn't a valid reason, as people should write good code. Often though, (and we all know we're all guilty of this) our code isn't good or completely readable. (u)int128s would help everybody in this regaurd every time they're used to avoid messy code as well as code with bugs.

Intrinsically, this value-add should be obvious. more people writing more better bug-free code is good for the entire ecosystem.

@c-robinson
Copy link

It looks like the Go source indicates that it might be useful to have at least a uint128. There are multiple references to uint128 peppered around the crypto/aes package, as well as a type in crypto/internal/edwards25519 for uses that are waaaaay over my head:

// uint128 holds a 128-bit number as two 64-bit limbs, for use with the
// bits.Mul64 and bits.Add64 intrinsics.
type uint128 struct {
	lo, hi uint64
}

and then net/netip defines the same type with the same name, but adds a bunch of methods... so much so that the netip implementation seems like it would be entirely usable as a replacement for, say, a uint128-sized value stored in big.Int.

@minux's initial counter-argument to this proposal hinged on the statement that

The other use cases can be done with math/big just as easy.

But that would seem to be in conflict with the netip approach.

@Eisenwave
Copy link

I have written a proposal P3140: std::int_least128_t for C++.

The Motivation in that proposal is largely language-agnostic, so it might be helpful in this discussion.

@andreyvit
Copy link

andreyvit commented Apr 2, 2024

+1. My use cases: arithmetics — 128-bit fixed-point numerics (high-precision monetary values), bitwise ops — hashing, identifiers, larger bitmasks. Like others have noted, code simplification for bitwise operations is a noticeable benefit.

However, if the team wants an alternative proposal, we could allow arithmetics and bitwise ops on [n]uint64 types. This would take care of uint128, uint256 and similar, without extending the list of primary types. Then extending fmt, strconv, json and other packages would be an optional effort that doesn't have to happen in lockstep with compiler changes (or maybe doesn't need to happen at all).

@sylr
Copy link

sylr commented Apr 2, 2024

+1. My use cases: arithmetics — 128-bit fixed-point numerics (high-precision monetary values)

Same here, an int128 based fixed-point numeric could allow to be precise enough for cryptocurrencies such as bitcoin and be "large" enough to be also used for market capitalizations with fiat currencies.

@andreyvit
Copy link

I went through my codebase, and I'm more in favor of [2]uint64 proposal now (i.e. allow treating [n]uint64 as integers for arithmetics, bitwise ops, comparisons, maybe even literal initializers) than simply an uint128 type. This would be assigning new semantics to something that is already 99% used for bignums, won't break any existing code, and won't require any library changes (we can add extra strconv functions for these, but, most importantly perhaps, no sweeping reflection changes across all serialization libs).

People can add MarshalText etc on top of these as appropriate for their domains (which will probably produce hex, base64 or even dashed UUID formatting).

The reasoning for [n]uint64 over int128 etc is:

  • in at least two cases, I really want an uint256, not uint128
  • formatting and parsing is domain-specific (and materially differs in different places in my code)
  • I bet at some point in the future people will start asking for uint512, uint1024, etc
  • really the annoying part making code hard to read is operators; if these are solved by the compiler, everything else can be added trivially

Should I open a separate ticket for discussion, or is this a non-starter @ianlancetaylor @robpike?

@gaby
Copy link

gaby commented Apr 28, 2024

Why isnt https://github.com/golang/go/blob/master/src/net/netip/uint128.go exported ? It would solve part of this issue.

@Bjohnson131
Copy link

Bjohnson131 commented Apr 29, 2024

I'm more in favor of [2]uint64 proposal now

Allow me to play the consistency questioner.

  1. Will a float128 then be called a [2]float64?
  2. How will you express generic functions that accept []uint64s as well as []uint128s?

Let me say this now, if [2]uint64 worked for my use-case, I would not be here. What you're proposing is something which is already taught as a work-around for the lack of u128 support, and it is not a solution to many use-cases named here.

@andreyvit
Copy link

andreyvit commented May 1, 2024

@Bjohnson131 Could you please clarify about the use cases not covered by [n]uint64? I feel like I've read the entire thread, but cannot recall any that fit, would love to consider those.

Let's make sure we're talking about the same thing. I'm proposing that the following code should be legal (for any operator in place of +):

var a, b, c [2]uint64
a = b + c

Re: generics overall, I'd imagine initially there'd be no way to write generic code that works for [n]uint64 with any n, but that's not any worse than having an uint128 type. You can probably write generic code over something like ~[2]uint64 | ~[4]uint64 if needed.

But perhaps I misunderstand your question about generics. If we imagine that type uint128 = [2]uint64, then accepting either []uint64 or []uint128 looks like func xxx[T uint64 | [2]uint64](v []T).

Re: float128, I don't imagine this being applicable for floats. Long uints are truly often represented as [n]uint64, but that's not true about floats at all.

@ianlancetaylor ianlancetaylor added Proposal LanguageChangeReview Discussed by language change review committee and removed v2 An incompatible library change labels Aug 6, 2024
@dylan1951
Copy link

Why does it take 10+ years to add 128 bit ints despite enormous demand for it?

@DmitriyMV
Copy link
Contributor

@dylan1951 please refer to #9455 (comment)

@dylan1951
Copy link

@dylan1951 please refer to #9455 (comment)

The comment says it's not a zero change thing to do, and says it isn't arguing against the change. What's your point?

@c-chk

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LanguageChange Suggested changes to the Go language LanguageChangeReview Discussed by language change review committee Proposal
Projects
None yet
Development

No branches or pull requests