"Same" padding for conv layers?

[DrChainsaw]

Many popular DL frameworks out there support automatically computing the padding needed to make inputsize == outputsize (* stride).

As far as I can see this is not supported in Flux, or is there some secret util method for it somewhere?

I can make a PR for this. I just want to give someone a chance to stop me in case it is already implemented or if it is not wanted for some reason.

Early feedback on the API would also be useful. I'm thinking of just using the pad keyword for this in the existing method signatures (e.g. pad=Same() means calculate "same" padding).

I would prefer to make a type for this and then just dispatch on it. Afaik it is not possible to dispatch on kwargs, but I guess one can just call some determine_padding function (which ofc is a noop in case padding is given explicitly).

[prashantramnani]

Hi, I am new to julia and would like to take up this issue as my first contribution. I couldn't find any PR on this, should I get started with this?

[DrChainsaw]

Feel free to do so. I got bogged down in a "just need to finish this thing" => "crap! I need to redesign half my library" thingy.

Might be useful to be aware of this issue, although I don't think it is something which explicitly needs to be considered:
https://github.com/JuliaGPU/CuArrays.jl/issues/356

Here is the code I added on my side if you want to save a few minutes. I'm also kinda new, so I don't guarantee that this is the best way to do it. Note that it is probably only correct for vanilla conv layers.

I would have changed the functor-thingy below into a normal function (e.g. pad_conv(::SamePad, ks, dilation)) and added one new function for each type of conv layer which requires its own calculations and then have functions like pad_conv(pad, ks, dilation) = pad to handle the case when the user inputs padding manually without having if-statements in the code since all the cool kids no longer use if-statements.

"""
    SamePad

Generates padding parameters so that outputshape == inputshape ./ stride.

Formula from Relationship 14 in http://deeplearning.net/software/theano_versions/dev/tutorial/conv_arithmetic.html
"""
struct SamePad end
function (::SamePad)(ks, dilation)
    # Effective kernel size, including dilation
    ks_eff = @. ks + (ks - 1) * (dilation - 1)
    # How much total padding needs to be applied?
    pad_amt = @. ks_eff - 1
    # In case amount of padding is odd we need to apply different amounts to each side.
    return Tuple(mapfoldl(i -> [ceil(Int, i/2), i ÷ 2], vcat, pad_amt))
end

    @testset "SamePad" begin
        # Might wanna do this by asserting that input size == output size for conv layers created with SamePad instead
        @test SamePad()(2, 1) == (1, 0)
        @test SamePad()(3, 1) == (1, 1)
        @test SamePad()(5, 2) == (4, 4)
        @test SamePad()((4,6), 2) == (3, 3, 5, 5)
        @test SamePad()((3,7), (2,4)) == (2,2,12,12)
    end