x/text/number: understands specific BCP-47 u-nu-extensions, but not general categories

[golightlyb]

Impacts x/text/language and x/text/number

What version of Go are you using (go version)?

go1.18, go1.19rc1,

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ben/.cache/go-build"
GOENV="/home/ben/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ben/go/pkg/mod"
GONOPROXY="github.com/tawesoft/privatetest"
GONOSUMDB="github.com/tawesoft/privatetest"
GOOS="linux"
GOPATH="/home/ben/go"
GOPRIVATE="github.com/tawesoft/privatetest"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19rc1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/ben/go/src/unicode/example/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build805290762=/tmp/go-build -gno-record-gcc-switches"

Also, Go playground

What did you do?

https://go.dev/play/p/PEXNpU0N-lf

What did you expect to see?

en: 123,456,789
en-u-nu-fullwide: １２３,４５６,７８９
ta: 12,34,56,789
ta-u-nu-native: ௧௨,௩௪,௫௬,௭௮௯
ta-u-nu-tamldec: ௧௨,௩௪,௫௬,௭௮௯
ta-u-nu-finance: 12,34,56,789

"For example, in Tamil the default numbering system is latn, the native numbering system is tamldec and the traditional numbering system is taml." (ref1)

Therefore, ta-u-nu-native should display the same result as ta-u-nu-tamldec

What did you see instead?

en: 123,456,789
en-u-nu-fullwide: １２３,４５６,７８９
ta: 12,34,56,789
ta-u-nu-native: 12,34,56,789 // <-- this line differs
ta-u-nu-tamldec: ௧௨,௩௪,௫௬,௭௮௯
ta-u-nu-finance: 12,34,56,789

ta-u-nu-native uses the default "latn", rather than tamldec.

Additional examples would look like "zh-u-nu-finance" but I couldn't get any "zh" locale strings to print in anything other than latn

Resolving

Ideally, this should be supported. If it isn't, an error would be good. But there is no general way to know that the specified number system extension in the locale string is invalid or ignored, so this relates to #53872 and the returned Confidence.

Also, for cases like "example-u-nu-finance" or "example-u-nu-traditio": "If the traditional numbering system is not defined, applications should use the native numbering system as a fallback. If the financial numbering system is not specified, applications should use the default numbering system as a fallback."
(ref2)

Additional references:

• Unicode data in cldr-41.0/common/bcp47/number.xml
• https://cldr.unicode.org/index/bcp47-extension

[cherrymui]

cc @mpvl

[gopherbot]

Change https://go.dev/cl/465735 mentions this issue: x/text/number: provide a way to query number system

[golightlyb]

Change https://go.dev/cl/465735 mentions this issue: x/text/number: provide a way to query number system

Apologies - ignore, wrong issue number.