-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convolutions for DL #52
Comments
That seems to be an important operation, so I'm totally willing to support it here. To my mind the most unfortunate part is the fact that the color dimension is not the fastest dimension (why did they do that?), but this seems to be standard so I guess we just have to live with it. I'm pretty swamped with Julia 0.7 changes right now (including one that I think you will be very happy with), but I'll get to this eventually. |
Different deep learning frameworks use different choices of whether the spatial or channel dimension is the innermost/fastest; |
In Julia there are likely to be performance advantages stemming from the fact that ColorTypes can act as fixed-size vectors, so the compiler can unroll loops automatically. One could surely do that by hand for channel-slow representations, but it's more of a pain in the neck. |
The only place in a convolutional NN where the "channel" dimension actually corresponds to colors and has a small fixed size is in the input layer; in all other layers there are typically many more channels (from dozens to thousands) and unrolling may be counterproductive from a compilation time standpoint. |
Yeah, I think there's a small mindset difference going on here: in ML we tend to see the channels as a stack of related images, rather than as a single image with N-dimensional pixels. It's somewhat more like a frame. |
Yes, when the channel dimension is that large, it's definitely better to use a loop. Thanks for clarifying! |
To add to this, the output is frequently computed with a stride in the spatial dimensions, producing a smaller image. That is important to optimize for. |
I thought I'd give a bit of an update on this for some people that have expressed interest in approaching this issue. We have https://github.com/FluxML/NNlib.jl for neural network filters in Julia. If we find a path forward to compatibility with it we would get GPU support for NN filters and more easily hook into a lot of the machine learning libraries in Julia. I'm not sure if it will be possible to get native GPU code to handle/optimize colorant type calculations. It may be best to just decide at what point in the convolution it is worth converting to a Float32 array and back. Also take a look at https://github.com/JuliaGPU/CuArrays.jl for some GPU interop. I don't think there's currently a way to fully interact with AMD GPUs using Julia's compiler yet but that's being worked on https://github.com/JuliaGPU/AMDGPUnative.jl. |
I'd like to use this excellent package for the filters commonly used in deep learning. I tried a naive implementation here, but performance isn't great and I'm wondering if we need support in this package to improve that.
To give a rough outline of the semantics, we have a
D×C×N
image array where:D
are the data dimension(s) (e.g. width and height),C
is the channel dimension (e.g. colour) andN
is a batch dimension (so we can convolveN
independent images at once). This is convolved with aK×C×F
filter (K
being the filter dimension(s) andF
being the number of filters / output channels) to produce aD′×F×N
output array. Each slice alongF
is treated as a separate "filter" that spans the channel dimension, and the filter outputs are concatenated.Hope that's clear and I'm happy to give more references if it's useful. Does this looks like something that's easy and appropriate for this package to support, or would it require an entirely new implementation?
The text was updated successfully, but these errors were encountered: