Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config set fails if there is entry in config that is integer bigger than 1e6 #3354

Open
Kubuxu opened this issue Nov 4, 2016 · 6 comments
Open
Labels
exp/expert Having worked on the specific codebase is important help wanted Seeking public contribution on this issue kind/bug A bug in existing code (including security flaws) status/deferred Conscious decision to pause or backlog topic/api Topic api

Comments

@Kubuxu
Copy link
Member

Kubuxu commented Nov 4, 2016

I have BloomSize set to 1048576, if I then try to do any config call, example

ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["*"]'

I get:

Error: failed to set config value: Failure to decode config: json: cannot unmarshal number 1.048576e+06 into Go value of type int (maybe use --json?)

The integer is being transformed into floating point at some point.

@whyrusleeping
Copy link
Member

Thats.... annoying

@whyrusleeping whyrusleeping added the kind/bug A bug in existing code (including security flaws) label Nov 16, 2016
@Kubuxu Kubuxu added status/deferred Conscious decision to pause or backlog topic/api Topic api exp/expert Having worked on the specific codebase is important help wanted Seeking public contribution on this issue labels Nov 28, 2016
@Kubuxu Kubuxu added status/ready Ready to be worked and removed status/deferred Conscious decision to pause or backlog labels Jan 15, 2017
@Kubuxu
Copy link
Member Author

Kubuxu commented Jan 15, 2017

I will hit it if I set the bloom filter size on storage hosts to 128MiB.

@Kubuxu
Copy link
Member Author

Kubuxu commented Jan 15, 2017

The reason for it is our trickery with map[string]interface{} for replacing single variables and so on.

@whyrusleeping @lgierth any idea how to handle it?

@torresashjian
Copy link
Contributor

After doing some testing:
1.- BloomSize value <= 9223372036854775000
Result: [All good]

2.- 9223372036854775000 > BloomSize value < 9223372036854775295
Result: [BloomSize Value rounded and changed to 9223372036854775000]

3.- When BloomSize value > 9223372036854775295
Result: [Error: failed to set config value: Failure to decode config: json: cannot unmarshal number 9223372036854776000 into Go struct field Datastore.BloomFilterSize of type int (maybe use --json?)]

If we use in repo/fsrepo/serialize/serialize.go:

d := json.NewDecoder(f)
d.UseNumber()

Then the rounding happens between 9223372036854775000 and 9223372036854776000, little gain, but the error message is more in sync with the values since it fails at > 9223372036854776000.

Other options could be using RawMessage to delay the resolution of the values until is is used or set, this way each value is independent from each other and only the desired part gets decoded, with some other tradeoffs of course....

The number allowed seems large enough for a Bloom Filter, so not sure a fix is needed here.

@Kubuxu
Copy link
Member Author

Kubuxu commented Jun 13, 2017

Solution for the bloom filter would be migration for it to be string instead with unit suffix.

@Kubuxu
Copy link
Member Author

Kubuxu commented Oct 18, 2017

Solution for it also is using custom json decoder and https://golang.org/pkg/encoding/json/#Decoder.UseNumber

@Kubuxu Kubuxu modified the milestones: Ipfs 0.4.12, go-ipfs 0.4.13 Nov 6, 2017
@Stebalien Stebalien added status/deferred Conscious decision to pause or backlog and removed status/ready Ready to be worked labels Dec 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exp/expert Having worked on the specific codebase is important help wanted Seeking public contribution on this issue kind/bug A bug in existing code (including security flaws) status/deferred Conscious decision to pause or backlog topic/api Topic api
Projects
None yet
Development

No branches or pull requests

4 participants