Add optional iterator to _.uniq

[albemuth]

Hi,

I needed to be able to pass in an iterator to _.uniq like in ruby's Array.uniq to do things like:

_.uniq([{name:'moe'}, {name:'curly'}, {name:'larry'}, {name:'curly'}], false, function (value) { return value.name; })
=>[{name:'moe'}, {name:'curly'}, {name:'larry'}]

Added test and updated docs too, any feedback welcome.

Cheers!
Alfredo Mesén

[bradleybuda]

+1

[edtsech]

+1

[fgalassi]

+1

[jimschubert]

This looks excellent

[ngauthier]

+1

[jashkenas]

It's a shame about the api, having isSorted before iterator ... but I don't think it's worth breaking backwards-compatibility over this.

[ngauthier]

Alternatively, it could become uniqBy (like sortBy) and then you could deprecate uniq.

[andrewplummer]

@albemuth I'm curious about the use case here... uniquing on a property returns the first object for which that property uniquely occurs, yes? If that is the case, you have no guarantee that the other omitted data will be unique or not. I can understand needing a unique array of the properties themselves, but in that case you would simply map the properties into their own array first using _.map, no?

[ngauthier]

@adrewplummer no, uniquing on a property returns all the objects that are unique when compared using that property. So in the example code at the top, he is uniquing object by their name (as opposed to the default which would be equality between the objects themselves).

[andrewplummer]

I guess I'm not clear on the phrase "all the objects that are unique". What if the objects aren't unique by the name property but are unique by other properties? Examples are worth a thousand words:

var people = [ { name: 'John', age: 20 }, { name: 'Mary', age: 31 }, { name: 'Kevin', age: 20 }]; 
_.uniq(people, false, function(p){ return p.age; });

I was going to ask "what happens when I do this", but realized I should just try for myself... as I thought, it returns only "John" and "Mary". So I'm just curious what the use case is of needing just John here (or maybe a better example if mine is too contrived)? Now of course I could understand wanting the ages themselves uniqued, but in that case this would seem more appropriate:

_.uniq(_.map(people, function(p){ return p.age; }));

[ngauthier]

often it's not the case that the attributes differ but the primary key (name, in this case) do not. The usual use case is:

var people = [{ name: 'John', age: 20 }, { name: 'John', age: 20 }, { name: 'Kevin', age: 20 }];
_.uniq(people)
// =>  [{ name: 'John', age: 20 }, { name: 'John', age: 20 }, { name: 'Kevin', age: 20 }]
_.uniq(people, false, function(p){ return p.name; });
// => [{ name: 'John', age: 20 }, { name: 'Kevin', age: 20 }]

It's really useful to specify the key when uniquing objects because uniq uses === so two objects that are equivalent are not actually uniq. It would inefficient to do a deep compare when as a programmer we know what the primary key is.

[andrewplummer]

Ah I see. But this kind of case does make the assumption that the objects are identical if the keys they are uniqueing on are identical, is that not true? I suppose that would follow from the "primary key" paradigm. But in your experience, would it really be so common to have multiple identical entries in the same array?

[ngauthier]

Yes.

Although the true power here is allowing the programmer to define their own comparator so they can choose what is important and what is not.

Note that uniq calls include and include uses === to compare. In another language, we could redefine the comparator between objects, but in javascript we can't override ===. Meaning we can't define object comparison.

Allowing uniq to accept a function as a comparator lets us do this.

[andrewplummer]

Ok... interesting... that makes a lot more sense now... thanks!

[ngauthier]

no problem! I think I learned more about underscore and javascript comparison in the process. :-)

[davidgoli]

It's too bad this iterator is idiosyncratic and not a comparison iterator like Javascript's native sort iterator, otherwise it would be so easy to use underscore's own comparison methods like:

_.uniq([{'foo': 'bar', 'a':1}, {'foo': 'bar', 'a':1}, {'foo': 'bar', 'a': 2}], false, _.isEqual)

As it is, you have to "serialize" the object in order to have a single value type to return, eg.:

function (item) { return item.foo + item.a; }

Probably too late now, though you could use iterator.arguments.length to see if the iterator expects 1 or 2 arguments.

[jdalton]

@davidgoli I really dig that use case but it's trickier because currently the iterator for unique is passed 3 arguments, value, index, and array.

[CWSpear]

@davidgoli I was thinking that getting it to work with _.isEqual was how it was supposed to work, and so it totally threw me off my game. Finally, I found myself here and your idea to serialize it and it was kind of a lightbulb moment, haha. Thanks for your comment!

An implementation like you suggest, if possible, would make it a lot more powerful, because then you could compare any array of collections instead of having to know what the keys are, etc.

[jdalton]

Comparing like Array#sort would be expensive, also I'm not entirely sure how that would work. Uniqueness isn't between two values its uniqueness of elements in the entire array regardless of element order.

[CWSpear]

Oh yeah, I am aware of the cost. I just had a relatively small array with objects with three fields. The concatenation situation is perfectly adequate, and yes, faster. It would just be nifty with isEqual, haha.

[jdalton]

@CWSpear Ah! So like a _.uniqWhere instead of strict equality using object comparisons to simply like another object.