Callable structures split over multiple lines break parsing.

[dcelisgarza]

I just saw the JuliaCon talk and it made me excited to try the formatter. However, it ended up breaking some code, I don't have any insight into how to fix it. It happens when structs have long names and types, and then these structs can be called with separate arguments.

For example this code will not run. Something like this happens when I run runic on my code for some functions. The function is not split by in this way by JuliaFormatter.jl

abstract type VeryLongAbstractAlgorithmNameForProcessOne end
abstract type VeryLongAbstractAlgorithmNameForProcessTwo end

struct VeryLongConcreteAlgorithmNameForProcessOneA <: VeryLongAbstractAlgorithmNameForProcessOne end
struct VeryLongConcreteAlgorithmNameForProcessOneB <: VeryLongAbstractAlgorithmNameForProcessOne end

struct VeryLongConcreteAlgorithmNameForProcessTwoA <: VeryLongAbstractAlgorithmNameForProcessTwo end
struct VeryLongConcreteAlgorithmNameForProcessTwoB <: VeryLongAbstractAlgorithmNameForProcessTwo end


struct MyStructWithQuiteALongAndVeryDescriptiveName{
    T1 <:VeryLongAbstractAlgorithmNameForProcessOne,
    T2 <:VeryLongAbstractAlgorithmNameForProcessTwo,
}
    a::T1
    b::T2
end

function (
        ::MyStructWithQuiteALongAndVeryDescriptiveName{
            <:VeryLongConcreteAlgorithmNameForProcessOneA,
            <:VeryLongConcreteAlgorithmNameForProcessTwoA
        },
    )
    (
        a::AbstractVector, b::AbstractVector, c::AbstractVector,
        d::AbstractMatrix, e::AbstractMatrix, d::AbstractMatrix,
    )

...

end

The solution is to keep it like so:

function (
        ::MyStructWithQuiteALongAndVeryDescriptiveName{
            <:VeryLongConcreteAlgorithmNameForProcessOneA,
            <:VeryLongConcreteAlgorithmNameForProcessTwoA
        },
    )(
        a::AbstractVector, b::AbstractVector, c::AbstractVector,
        d::AbstractMatrix, e::AbstractMatrix, d::AbstractMatrix,
    )

I'd love to be able to use runic, I find the style much nicer than anything i've managed to hack together using JuliaFormatter.jl.

[fredrikekre]

Thanks for the report. Can you provide a source example that fails? I can't come up with an example that formats to your broken case.

[dcelisgarza]

Sure thing, there's a large type hierarchy so i gotta hunt it down. Bear with me for a sec while i do that.

I realised i don't need to do that cause you only want to format it.

# This is the function that gets formatted badly
function (r::LowOrderMoment{<:Any, <:Any, <:Any,
                            <:LowOrderDeviation{<:Any, <:SecondLowerMoment{<:SqrtRiskExpr}}})(w::AbstractVector,
                                                                                              X::AbstractMatrix,
                                                                                              fees::Union{Nothing,
                                                                                                          <:Fees} = nothing)
    val = min.(calc_moment_val(r, w, X, fees), zero(eltype(X)))
    return Statistics.std(r.alg.ve, val; mean = zero(eltype(val)))
end

Ok this got really weird, I just tried to format this file on its own and it worked perfectly. But when I applied it earlier to the entire codebase it didn't.

[dcelisgarza]

So i managed to break it again. It appears that if I format the file individually, it works as expected.

function (
        r::LowOrderMoment{
            <:Any, <:Any, <:Any,
            <:LowOrderDeviation{<:Any, <:SecondLowerMoment},
        }
    )(
        w::AbstractVector,
        X::AbstractMatrix,
        fees::Union{
            Nothing,
            <:Fees,
        } = nothing
    )
    val = min.(calc_moment_val(r, w, X, fees), zero(eltype(X)))
    return Statistics.var(r.alg.ve, val; mean = zero(eltype(val)))
end

If I do it on the entire folder containing the file. I get this:

function (
        r::LowOrderMoment{
            <:Any, <:Any, <:Any,
            <:LowOrderDeviation{<:Any, <:SecondLowerMoment},
        },
    )
    (
        w::AbstractVector,
        X::AbstractMatrix,
        fees::Union{
            Nothing,
            <:Fees,
        } = nothing,
    )
    val = min.(calc_moment_val(r, w, X, fees), zero(eltype(X)))
    return Statistics.var(r.alg.ve, val; mean = zero(eltype(val)))
end

And once it breaks, it doesn't go back.

[dcelisgarza]

And it gets weirder. If you apply the formatting to this again, either by saving (VSCode auto formatting with runic), or by running runic again manually. This:

function (
        r::LowOrderMoment{
            <:Any, <:Any, <:Any,
            <:LowOrderDeviation{<:Any, <:SecondLowerMoment},
        }
    )(
        w::AbstractVector,
        X::AbstractMatrix,
        fees::Union{
            Nothing,
            <:Fees,
        } = nothing
    )
    val = min.(calc_moment_val(r, w, X, fees), zero(eltype(X)))
    return Statistics.var(r.alg.ve, val; mean = zero(eltype(val)))
end

turns into this:

function (
        r::LowOrderMoment{
            <:Any, <:Any, <:Any,
            <:LowOrderDeviation{<:Any, <:SecondCentralMoment},
        },
    )
    (
        w::AbstractVector,
        X::AbstractMatrix,
        fees::Union{
            Nothing,
            <:Fees,
        } = nothing,
    )
    val = calc_moment_val(r, w, X, fees)
    return Statistics.var(r.alg.ve, val; mean = zero(eltype(val)))
end

[dcelisgarza]

Perhaps the solution is if it encounters:

 <some multiple of 4 spaces>)(\n

skip applying the formatting to the expression?

Or maybe check that the result is a valid julia expression, and skip formatting if it's not?

[fredrikekre]

Thanks, I can reproduce it.

[dcelisgarza]

No worries, I look forward to using it once it's working.

[fredrikekre]

I think this is an issue in JuliaSyntax which will be fixed by JuliaLang/JuliaSyntax.jl#580 but I haven't confirmed yet.