Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resize does not seem to work with augmentbatch! #56

Closed
DrChainsaw opened this issue Apr 1, 2020 · 5 comments
Closed

Resize does not seem to work with augmentbatch! #56

DrChainsaw opened this issue Apr 1, 2020 · 5 comments

Comments

@DrChainsaw
Copy link

Maybe I just don't understand how to use it...

Makes most sense for me as a user to only specify heigh and width:

julia> augmentbatch!(zeros(6,6,3,2), ones(4,4,3,2), Resize(6,6))
ERROR: MethodError: no method matching applyaffine(::Resize{2}, ::ImageTransformations.InvWarpedView{Float64,3,Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}},CoordinateTransformations.AffineMap{UniformScaling{Float64},SArray{Tuple{3},Float64,1,3}},Tuple{UnitRange{Int64},UnitRange{Int64},UnitRange{Int64}},CoordinateTransformations.AffineMap{UniformScaling{Bool},SArray{Tuple{3},Float64,1,3}},Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}}}, ::Nothing)
Closest candidates are:
  applyaffine(::Augmentor.AffineOperation, ::AbstractArray, ::Any) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:77
  applyaffine(::Either, ::AbstractArray, ::Any) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\either.jl:203
  applyaffine(::Augmentor.Operation, ::AbstractArray) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:60  
  ...
Stacktrace:
 [1] applyaffineview(::Resize{2}, ::ImageTransformations.InvWarpedView{Float64,3,Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}},CoordinateTransformations.AffineMap{UniformScaling{Float64},SArray{Tuple{3},Float64,1,3}},Tuple{UnitRange{Int64},UnitRange{Int64},UnitRange{Int64}},CoordinateTransformations.AffineMap{UniformScaling{Bool},SArray{Tuple{3},Float64,1,3}},Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}}}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:72
 [2] applylazy(::Resize{2}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\resize.jl:74
 [3] applylazy at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:60 [inlined]
 [4] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\codegen.jl:76 [inlined]
 [5] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [6] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [7] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:126 [inlined]
 [8] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Tuple{Resize{2}}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:112
 [9] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Resize{2}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:109
 [10] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:105 [inlined]
 [11] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:94 [inlined] (repeats 2 times)     
 [12] augmentbatch!(::Array{Float64,4}, ::Array{Float64,4}, ::Resize{2}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:85
 [13] top-level scope at none:0

This is more consistent with Crop, but what should happen if dimension 3 is resized?

julia> augmentbatch!(zeros(6,6,3,2), ones(4,4,3,2), Resize(6,6,3))
ERROR: MethodError: no method matching +(::CartesianIndex{3}, ::CartesianIndex{2})
Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...) at operators.jl:529
  +(::CartesianIndex{N}, ::CartesianIndex{N}) where N at multidimensional.jl:110
Stacktrace:
 [1] applyaffineview(::Resize{3}, ::ImageTransformations.InvWarpedView{Float64,3,Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}},CoordinateTransformations.AffineMap{UniformScaling{Float64},SArray{Tuple{3},Float64,1,3}},Tuple{UnitRange{Int64},UnitRange{Int64},UnitRange{Int64}},CoordinateTransformations.AffineMap{UniformScaling{Bool},SArray{Tuple{3},Float64,1,3}},Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}}}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\resize.jl:83
 [2] applylazy(::Resize{3}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\resize.jl:74
 [3] applylazy at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:60 [inlined]
 [4] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\codegen.jl:76 [inlined]
 [5] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [6] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [7] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:126 [inlined]
 [8] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Tuple{Resize{3}}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:112
 [9] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Resize{3}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:109
 [10] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:105 [inlined]
 [11] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:94 [inlined] (repeats 2 times)     
 [12] augmentbatch!(::Array{Float64,4}, ::Array{Float64,4}, ::Resize{3}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:85
 [13] top-level scope at none:0

This does not make sense at all, but I tried it just in case:

julia> augmentbatch!(zeros(6,6,3,2), ones(4,4,3,2), Resize(6,6,3,2))
ERROR: MethodError: no method matching applyaffine(::Resize{4}, ::ImageTransformations.InvWarpedView{Float64,3,Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}},CoordinateTransformations.AffineMap{UniformScaling{Float64},SArray{Tuple{3},Float64,1,3}},Tuple{UnitRange{Int64},UnitRange{Int64},UnitRange{Int64}},CoordinateTransformations.AffineMap{UniformScaling{Bool},SArray{Tuple{3},Float64,1,3}},Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}}}, ::Nothing)
Closest candidates are:
  applyaffine(::Augmentor.AffineOperation, ::AbstractArray, ::Any) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:77
  applyaffine(::Either, ::AbstractArray, ::Any) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\either.jl:203
  applyaffine(::Augmentor.Operation, ::AbstractArray) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:60  
  ...
Stacktrace:
 [1] applyaffineview(::Resize{4}, ::ImageTransformations.InvWarpedView{Float64,3,Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}},CoordinateTransformations.AffineMap{UniformScaling{Float64},SArray{Tuple{3},Float64,1,3}},Tuple{UnitRange{Int64},UnitRange{Int64},UnitRange{Int64}},CoordinateTransformations.AffineMap{UniformScaling{Bool},SArray{Tuple{3},Float64,1,3}},Interpolations.Extrapolation{Float64,3,Interpolations.BSplineInterpolation{Float64,3,SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true},Interpolations.BSpline{Interpolations.Linear},Tuple{Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}},Interpolations.BSpline{Interpolations.Linear},Interpolations.Flat{Nothing}}}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:72
 [2] applylazy(::Resize{4}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Nothing) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operations\resize.jl:74
 [3] applylazy at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\operation.jl:60 [inlined]
 [4] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\codegen.jl:76 [inlined]
 [5] macro expansion at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [6] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:130 [inlined]
 [7] _augment_avoid_eager at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:126 [inlined]
 [8] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Tuple{Resize{4}}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:112
 [9] augment!(::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::SubArray{Float64,3,Array{Float64,4},Tuple{Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Base.Slice{Base.OneTo{Int64}},Int64},true}, ::Resize{4}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augment.jl:109
 [10] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:105 [inlined]
 [11] augmentbatch! at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:94 [inlined] (repeats 2 times)     
 [12] augmentbatch!(::Array{Float64,4}, ::Array{Float64,4}, ::Resize{4}) at E:\Programs\julia\.julia\packages\Augmentor\kkcKe\src\augmentbatch.jl:85
 [13] top-level scope at none:0
@barucden
Copy link
Collaborator

The problem does not seem to be with batches. For example, this works:

using Augmentor
img = testpattern()
batch = cat(img, img, dims=3)
augmentbatch!(similar(batch, 500,600,2), batch, Resize(500, 600))

Whereas, this does not work:

augment!(zeros(6,6,3), rand(4,4,3), Resize(6,6))

Resize does not work with images represented as 3D arrays (where the 3rd dimension is channel).

@johnnychen94, is this possibly an issue in general? Allowing only Colorant input would help. (Slightly related to what was discussed in #97)

@barucden barucden mentioned this issue Aug 13, 2021
Closed
@DrChainsaw
Copy link
Author

Ah, never occured to me to try images as the Array -> RGB -> Array seemed like an unnecssary hassle. Thanks a bunch for pointing it out as it is far less of a hassle than the other workarounds I did.

Is this the intended API for augmentbatch! though?

The docstring says it does multi-dimensional arrays (and it indeed does if they are 3D, i.e no channel dim) which is what threw me off.

@johnnychen94
Copy link
Collaborator

johnnychen94 commented Aug 14, 2021 •
edited
Loading

Both of these works:

augmentbatch!(zeros(RGB,6,6,2), fill(RGB(1,1,1), 4,4,2), Resize(6,6))
augmentbatch!(zeros(6,6,3,2), fill(RGB(1,1,1), 4,4,2), Resize(6,6)|>SplitChannels())

Augmentor.jl builds on top of JuliaImages and for more generic support to any image format (not just RGB), it's deliberately required to use RGB here. Any plain numerical array Array{Float32, 3} would be interpreted as a 3D grayscale image. By saying multi-dimensional arrays, it literately means multi-dimensional images...

julia> augment(rand(RGB, 4, 4, 4), Resize(6, 6, 6))
6×6×6 Array{RGB{Float64},3} with eltype RGB{Float64}:
...

The stacktrace you got when trying to call augmentbatch!(zeros(6,6,3,2), ones(4,4,3,2), Resize(6,6,3)), I think, is basically because there're some bugs here and there for augmentbatch!'s N-dimensional support.

@barucden
Copy link
Collaborator

We are currently working on improving the documentation. Please, see this example that explains some of the questions raised here.

@DrChainsaw
Copy link
Author

Sorry for extremely long responsen time. Explanation above and the excellent docs makes it perfectly clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@johnnychen94 @barucden @DrChainsaw