query_values regressed with latest release

[crawlik]

Hi guys,
According this doc http://addressable.rubyforge.org/api/Addressable/URI.html#query_values-instance_method query_values has regressed with latest release.

Here is what get calling it
jruby-1.6.2 :001 > require "addressable/uri" => true jruby-1.6.2 :002 > Addressable::URI.parse( jruby-1.6.2 :003 > "?one[two][three][]=four&one[two][three][]=five" jruby-1.6.2 :004?> ).query_values => {"one[two][three][]"=>"five"}
2.2.8 version returns expected results
ruby-1.6.2 :001 > require "addressable/uri" => true jruby-1.6.2 :002 > require "addressable/uri" => false jruby-1.6.2 :003 > Addressable::URI.parse( jruby-1.6.2 :004 > "?one[two][three][]=four&one[two][three][]=five" jruby-1.6.2 :005?> ).query_values => {"one"=>{"two"=>{"three"=>["four", "five"]}}}

[sporkmonger]

This is correct. I haven't been able to update the docs due to a weird SSH error from RubyForge that I haven't figured out yet, but yes, this is a breaking change. The old style of parsing query parameters was non-standard, excessively complex, and worked very poorly in the case of repeated parameters. It's been abandoned in favor of deleting and simplifying the code drastically.

On the one hand, it's good to get the feedback that this functionality was getting used, but I feel very strongly at this point that this is a query parameter format that introduces a great deal of unnecessary complexity. I would recommend trying to transition to simpler query parameters and avoiding complex structured parameters if at all possible.

I believe Rails originally introduced this format, and I think it was a great disservice to everyone to enable this kind of thing instead of forcing the developer to simplify.

The original parsing code is still in source control, and there's no reason why you couldn't borrow from it if you determine that this is something you absolutely have to have, but honestly, I don't recommend it. Yes, it's very mature code, and extremely well-tested, but it's also slow, overly complex, and it causes a lot of problems and interface inconsistencies for much more important scenarios like repeated parameters.

[crawlik]

@sporkmonger thanks for explanation. There is one more thing I would like to mention. Take a look an this query string ?one[]=two&one[]=three. query_values returns {"one[]"=>"three"} Note [] is part of returned parameter name. Wonder if it is expected behavior. Used to be returned without []. Please let me know.
BTW is there an RFC that describes use of [] in URI query string. I checked RFC 3986 but could not find any specifics on [] use except that they are reserved. Thanks.

[sporkmonger]

This is intended behavior now. The use of [] was, I believe, introduced initially by Rails (and copied by Addressable for compatibility, but then also expanded somewhat) as a way to make URI query parameters more "ruby-like", but I believe this was a counter-productive objective, which is why this functionality was removed.

[Xylakant]

The use of [] as delimiter for nested query parameters predates rails for quite a while. PHP at least uses the same format for nested query parameters and has done so as far as I can remember (at least back to version 3). Nested query params are understood by rack itself, so any rack-based framework can handle that format.

I'd really love that functionality to stick around since it makes interoperability so much easier.

[sporkmonger]

I suspect this is going to continue being a controversial decision, but so far I haven't heard a sufficiently strong argument to reverse it. For absolute certain I will never accept "PHP did it!" as a good reason to follow suit. Happy to be corrected on my history of the usage, but the decision still stands. Nested query parameters are a mistake and Addressable will not support them natively, because there's no way to do so without either massively polluting the interface and causing developer confusion on which method to use, or overloading a single set of methods with a complex parser that ends up being awkward and ambiguous for assignment.

I am far, far more concerned about supporting this use case:

?thing=1&thing=2

Than I am with this use case:

?thing[]=1&thing[]=2

The reason is simple. One is a perfect superset of the other. I can implement the latter using the former, but not the other way around. So I will choose the more expressive option every single time, even if it occasionally makes the developer do a little more work. And especially since I have a great deal of reason at this point to believe that the developer shouldn't be doing this in the first place.

[Xylakant]

Doing as PHP did is not a reason in itself, but wouldn't it be nice for interoperability if everyone did query parameters in the same way? And PHP has been around a long time, so that it certainly had an influence on how other web-languages/frameworks are built. (like rack for example)

In any case: What's the argument against nested query parameters? I'm the author of https://github.com/Asquera/warden-hmac-authentication/ which uses an hmac signature to authenticate requests between servers. I need to add various parameters to the query string. To prevent polluting the parameter namespace, I use a single parameter name with nested sub-keys. I could do without, but I don't want to accidentally overwrite random parameters. I use addressable to generate the final URIs, and that's now broken for me in the latest release. I certainly can do without or use flat parameter names, but that means that on the receiving side, the parameter namespace is filled with a set of authentication-related parameters that are only of interest for my warden-authenticator. I can certainly clean that up, but that's not nice either.

I've seen similar ways of passing a set of related parameters in other use-cases. In general: Nested parameters are useful, since they allow grouping of query parameters. The use of [] as delimiter is not forbidden by the respective RFC. I'd love to see them supported in a way that's in line with how other frameworks and languages support them

(edit: clarified statement about RFC).

[sporkmonger]

Not sure what you mean that [] are covered by the RFC.

[Xylakant]

They're allowed as separators in the RFC, i.e. not reserved characters. Sorry about the confusion.

[bascht]

One quick question. I've stumbled across this thread, as I'm both using the Addressable as well as warden-hmac-authentication gem.

Like @sporkmonger said: »this is a breaking change.« - why does it come covert in a minor 2.2.8 to 2.2.9 version bump?

[sporkmonger]

@bascht That's 100% my fault, and mea culpa. To be honest, I didn't actually expect it'd even be noticed, and clearly I was wrong. For absolute sure, that should not have happened and won't happen again.

[sporkmonger]

Off topic and out of curiosity, why are people today picking warden-hmac-authentication over either OAuth 1 or OAuth 2?