map for Dict

[filmackay]

I noticed there wasn't any map methods that generated Dicts. Which function/name - comments please:

map{K,V}(f::Callable, d::Associative{K,V}) = [ f(k,v) for (k,v)=d ]
map{K,V}(f::Callable, d::Associative{K,V}) = [ (k, f(k,v)) for (k,v)=d ] # or this?
map{K,V}(f::Callable, d::Associative{K,V}) = [ (k, f(v)) for (k,v)=d ] # ..or this?

To distinguish the above from the current map (which you still may want to use), they will need new names - suggestions?

I noticed that map for Arrays handles multiple arrays - presumably this is more of a join-by-key semantic for Dicts and therefore not needed in map for Dicts?

[mbauman]

Currently, map for dictionary arguments falls back to the general iterable case. I'm not sure I see a difference between the status quo and what you are looking for. Does this not work the way you'd like it to?

julia> f(x) = x;
julia> @which map(f,[1=>"1",2=>"2"])
map(f::Union(Function,DataType),iters...) at abstractarray.jl:1265

julia> map(f,[1=>"1",2=>"2"])
2-element Array{Any,1}:
 (2,"2")
 (1,"1")

This is very nearly your first suggestion, effectively like … = [ f(x) for x in d ]

[StefanKarpinski]

It would be more convenient if the key and value were provided as separate arguments instead of as a pair. Of course, then mapping with the identity function doesn't produce an equal Dict, but I think that may well be ok. Another issue is what to do about key collisions.

[mbauman]

Yeah, I see. And this was a little surprising to me:

julia> f(x1,x2) = (x1,x2);
julia> map(f,1:3,1:2)
ERROR: dimensions must match # as expected
julia> map(f,Dict(1:3,map(string,1:3)),Dict(11:12,map(string,11:12)))
2-element Array{Any,1}:
 ((2,"2"),(11,"11"))
 ((3,"3"),(12,"12"))

Perhaps the iters… map method should check to see if they're all done at the end?

Oh! I see this came from the mailing list: https://groups.google.com/forum/?fromgroups=#!topic/julia-users/naGp9DUdSq8

[filmackay]

How should we distinguish between a map-I-want-a-Dict-returned and map-just-a-tuple-array-is-fine?

Should map stay returning an Array, and amap return an Association? It seems in some cases you want either.

The more general problem is: what about functions where you want to specify the return type - but the parameter types are identical? Is there a Julian solution for this problem - one being the convert convention of specifying the target type. But in this case you don't necessarily want to verbosely specify the exact Dict{K,V} that it is to return - just distinguish between "array" and "association please".

[kmsquire]

I believe there is a dictionary constructor from an iterable of tuples now, so while the whole thing could be done more efficiently, constructing a Dict is pretty straightforward if that's what you want.

Might need better docs (I haven't checked).

[filmackay]

So we definitely need a specific associative method, since currently the type information gets dropped:

function f() # to avoid global-induced Any types
       local a = ["a"=>1, "b"=>2] # Dict{ASCIIString,Int64}
       which(map, kv->kv, a) # map(f::Union(DataType,Function),iters...) at abstractarray.jl:1265
       local m = map(kv->kv, a) # Array{Any,1}
end

I was actually expecting map to return an Array{(ASCIIString,Int64},1}. Why not? Answer: Associative is not an AbstractArray. Why is there not an Iterable abstract type that is a parent of both Associative and AbstractArray Should we have a typed iterator to handle both cases?

(edit: it seems that anonymous functions are another reason the type information gets dropped)

Would Associative support be required in map anyway? We could do:

Dict(map(kv->(k[1], v[2]+1), ["a"=>1, "b"=>2]))
Dict(map(k->k, v->v+1, ["a"=>1, "b"=>2]))
Dict(map(v->v+1, ["a"=>1, "b"=>2]))

[filmackay]

Is it possible to constrain Function parameters? Looking at doing a map for AssociativeArray I want to be able to constrain the mapping function signature:

Base.map{K,V,KR,VR}(f::((K,V)->(KR,VR)), d::Associative{K,V}) # produces an Associative{KR,VR}

Note: not saying this specific example is a good idea, just asking generally about constraining functions.

[filmackay]

This function would at least get typed correctly (as opposed to the iter-based function)

map{K,V}(f::Callable, d::Associative{K,V}) = [f(k,v) for (k,v) in d]

Then you wrap the return in a Dict if that's what you want.

Of course anonymous functions have issues with preserving type information in comprehensions, which seems why map is not currently implemented with one.

My leaning is to implement the above with a comprehension, and use inline functions (not anonymous syntax). Then I have a hope of getting correctly types out even when an empty array is fed in. The current map can't do this.

[bryaan]

I just ran into this problem. Given the functional nature of Julia and its native collections, this should be implemented ASAP:

mapValues(dict::Dict{T, U}, fun::Function{U}) # should return a new Dict{T, V}.
# also implement mapValues!(...)

map(dict::Dict{T, U}, fun::Function{(T,U)}) # should return a new Array{V}

I don't think the syntax is perfect, but you get the idea. You should look to Scala, as those guys have gotten it down, Map (equivalent to Julia's Dict) in particular. The other available functions that are missing in Julia should also be added, like flatMap which excludes all null value Nullable objects (Scala's equivalent is Option).

http://www.scala-lang.org/api/current/scala/collection/Map.html

http://www.scala-lang.org/api/current/index.html#scala.Option

[kmsquire]

Thanks for the input, @BryanARivera. Any chance you could start a pull request? That would be the best way for this to get implemented ASAP. Most people working on Julia right now aren't focused on this area, but most are willing to review pull requests.

Also, consider opening an issue (and/or pull request) at DataStructures.jl. This is rather less organized (currently) than Scala's Collections, but it would initially be a more appropriate home for some of the other dictionary types.

Cheers!

[bryaan]

@kmsquire Hopefully I will have the chance to delve into it soon. I will try to implement everything I can as Scala's collections are quite central to the language. I envision it would accelerate Julia dev as well.

Should be relatively straightforward. Scala has got the naming and function conventions down, we just need to rebuild it in Julia. Would DataStructures.jl be the equivalent of Collections (meaning this is where PRs should be targeted) ?

[kmsquire]

@BryanARivera, that sounds fantastic. I'm also a huge fan of Scala's Collections framework.

DataStructures is probably the right place, but if what you want to implement is very different than what is already there, it might turn out that a new package would be better. But let's start in DataStructures.jl and see where that goes. Cheers!

[andyferris]

@BryanARivera I'm very happy to have read this since I too have been looking for a mapvalues function. The current implementation of map seems quite bizarre after you see the example with filter! which can return an intuitively filtered Dict, but I suppose it may be undesirable to many if we changed map! to behave more like filter!, as in:

x = ["a" =>1, "b"=>2, "c"=>3]
map!((k,v)->v+1, x)  # x = ["a" =>2, "b"=>3, "c"=>4]

since it could break existing code (unless someone like @StefanKarpinski thinks this is a good move?). In any case, functions which return Arrays and functions which return Dicts may both be needed.

We'll probably need both mapvalues and mapvalues! (note no capital 'V') (or possibly mapdict?? probably less generic...). Did you make a start on an implementation?

[bryaan]

I would be happy to write a PR. Does this look OK:

extractValue(fun) = (pair) -> fun(pair.second)

mapvalues(fun, dict) = map(extractValue(fun), dict)

It won't have the best performance until Jeff is done with his anon functions improvements.

[tkelman]

We don't usually camel-case function names in base.