`scale!!`, `add!!` in VectorInterface.jl not implemented for InfinitePEPS

[Yue-Zhengyuan]

I was trying the gradient optimization for t-J states with FermionParity ⊠ U1Irrep (spin) symmetry. I load iPEPS peps and CTMRGEnv env obtained from simple update and run the following:

peps, env, E, = fixedpoint(
    H,
    peps,
    env;
    optimizer_alg=(; tol=1e-4, maxiter=120),
    gradient_alg=(; iterscheme=:diffgauge),
)

Then I get some warning at the beginning of the optimization, about scale!! and add!! not implemented in VectorInterface.jl:

[ Info: LBFGS: initializing with f = -17.443556413922, ‖∇f‖ = 7.1036e-01
┌ Warning: The function `scale!!` is not implemented for (values of) type `Tuple{InfinitePEPS{TensorMap{Float64, GradedSpace{ProductSector{Tuple{FermionParity, U1Irrep}}, TensorKit.SortedVectorDict{ProductSector{Tuple{FermionParity, U1Irrep}}, Int64}}, 1, 4, Vector{Float64}}}, Float64}`;
│ this fallback will disappear in future versions of VectorInterface.jl
└ @ VectorInterface ~/.julia/packages/VectorInterface/J6qCR/src/fallbacks.jl:91
[ Info: LBFGS: iter    1, time  636.23 s: f = -17.450254372687, ‖∇f‖ = 6.3079e-01, α = 1.00e+00, m = 0, nfg = 1
┌ Warning: The function `add!!` is not implemented for (values of) type `Tuple{InfinitePEPS{TensorMap{Float64, GradedSpace{ProductSector{Tuple{FermionParity, U1Irrep}}, TensorKit.SortedVectorDict{ProductSector{Tuple{FermionParity, U1Irrep}}, Int64}}, 1, 4, Vector{Float64}}}, InfinitePEPS{TensorMap{Float64, GradedSpace{ProductSector{Tuple{FermionParity, U1Irrep}}, TensorKit.SortedVectorDict{ProductSector{Tuple{FermionParity, U1Irrep}}, Int64}}, 1, 4, Vector{Float64}}}, Int64, VectorInterface.One}`;
│ this fallback will disappear in future versions of VectorInterface.jl
└ @ VectorInterface ~/.julia/packages/VectorInterface/J6qCR/src/fallbacks.jl:163

But anyway the optimization can still finish.

[lkdvos]

This is a bit strange, normally the optimization should be using our own retraction functions, where we do retractions that don't alter the norm of the tensors. Is there any way you could obtain a stack trace to see where exactly this is coming from?

[Yue-Zhengyuan]

I just find that the warning is already encountered in all examples that involves AD (you can search for the error message in docs/src/examples), so it's not a fermionic tensor thing.

I obtain the stack trace by manually inserting an error in VectorInterface.jl here:

https://github.com/Jutho/VectorInterface.jl/blob/8c009e900ca133c57b87d0f5b8ebff63fc9ea2b2/src/fallbacks.jl#L8

Then I run examples/xxz/main.jl, and keyboard-interrupt it when I get enough error messages (for scale!! and add!!). Below is what I got:

Details

[ Info: CTMRG init:     obj = -2.356413456811e+03 +3.307968169629e+02im err = 1.0000e+00
[ Info: CTMRG conv 30:  obj = +6.245129734283e+03 -4.009370968561e-08im err = 5.3638616519e-09  time = 12.03 sec
[ Info: LBFGS: initializing with f = -0.034628402377, ‖∇f‖ = 3.0490e-01
┌ Error: Exception while generating log record in module VectorInterface at /Users/zhengyuanyue/GitHub/VectorInterface.jl/src/fallbacks.jl:92
│   exception =
│    Break point
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] _warn_message(fun::Function, type::Type)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:8
│      [3] macro expansion
│        @ ./logging/logging.jl:383 [inlined]
│      [4] scale!!(x::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Float64)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:92
│      [5] _scale!(v::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Float64)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/OptimKit.jl:22
│      [6] optimize(fg::PEPSKit.var"#571#574"{LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, PEPSOptimize{EigSolver{:diffgauge}}, Vector{Float64}, Vector{Matrix{Float64}}, Vector{Float64}, Vector{Float64}}, x::Tuple{InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}}, alg::OptimKit.LBFGS{Float64, OptimKit.HagerZhangLineSearch{Rational{Int64}}}; precondition::typeof(OptimKit._precondition), finalize!::typeof(OptimKit._finalize!), shouldstop::OptimKit.DefaultShouldStop, hasconverged::OptimKit.DefaultHasConverged{Float64}, retract::Function, inner::typeof(PEPSKit.real_inner), transport!::typeof(PEPSKit.peps_transport!), scale!::typeof(OptimKit._scale!), add!::typeof(OptimKit._add!), isometrictransport::Bool)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/lbfgs.jl:0
│      [7] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}, alg::PEPSOptimize{EigSolver{:diffgauge}}; finalize!::Function)
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:237
│      [8] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}; finalize!::Function, kwargs::@Kwargs{boundary_alg::@NamedTuple{tol::Float64, alg::Symbol, trscheme::@NamedTuple{alg::Symbol}}, gradient_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, iterscheme::Symbol}, optimizer_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, ls_maxiter::Int64, ls_maxfg::Int64}, verbosity::Int64})
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:197
│      [9] top-level scope
│        @ ~/GitHub/PEPSKit.jl/examples/xxz/main.jl:88
│     [10] include(mod::Module, _path::String)
│        @ Base ./Base.jl:557
│     [11] exec_options(opts::Base.JLOptions)
│        @ Base ./client.jl:323
│     [12] _start()
│        @ Base ./client.jl:531
└ @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:92
┌ Warning: Linesearch not converged after 1 iterations and 4 function evaluations:
│ α = 2.50e+01, dϕ = -6.00e-03, ϕ - ϕ₀ = -1.13e-01
└ @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/linesearches.jl:148
[ Info: LBFGS: iter    1, time  592.72 s: f = -0.147314472246, ‖∇f‖ = 9.2219e-01, α = 2.50e+01, m = 0, nfg = 4
┌ Error: Exception while generating log record in module VectorInterface at /Users/zhengyuanyue/GitHub/VectorInterface.jl/src/fallbacks.jl:92
│   exception =
│    Break point
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] _warn_message(fun::Function, type::Type)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:8
│      [3] macro expansion
│        @ ./logging/logging.jl:383 [inlined]
│      [4] scale!!(x::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Float64)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:92
│      [5] _scale!(v::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Float64)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/OptimKit.jl:22
│      [6] optimize(fg::PEPSKit.var"#571#574"{LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, PEPSOptimize{EigSolver{:diffgauge}}, Vector{Float64}, Vector{Matrix{Float64}}, Vector{Float64}, Vector{Float64}}, x::Tuple{InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}}, alg::OptimKit.LBFGS{Float64, OptimKit.HagerZhangLineSearch{Rational{Int64}}}; precondition::typeof(OptimKit._precondition), finalize!::typeof(OptimKit._finalize!), shouldstop::OptimKit.DefaultShouldStop, hasconverged::OptimKit.DefaultHasConverged{Float64}, retract::Function, inner::typeof(PEPSKit.real_inner), transport!::typeof(PEPSKit.peps_transport!), scale!::typeof(OptimKit._scale!), add!::typeof(OptimKit._add!), isometrictransport::Bool)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/lbfgs.jl:180
│      [7] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}, alg::PEPSOptimize{EigSolver{:diffgauge}}; finalize!::Function)
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:237
│      [8] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}; finalize!::Function, kwargs::@Kwargs{boundary_alg::@NamedTuple{tol::Float64, alg::Symbol, trscheme::@NamedTuple{alg::Symbol}}, gradient_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, iterscheme::Symbol}, optimizer_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, ls_maxiter::Int64, ls_maxfg::Int64}, verbosity::Int64})
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:197
│      [9] top-level scope
│        @ ~/GitHub/PEPSKit.jl/examples/xxz/main.jl:88
│     [10] include(mod::Module, _path::String)
│        @ Base ./Base.jl:557
│     [11] exec_options(opts::Base.JLOptions)
│        @ Base ./client.jl:323
│     [12] _start()
│        @ Base ./client.jl:531
└ @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:92
┌ Error: Exception while generating log record in module VectorInterface at /Users/zhengyuanyue/GitHub/VectorInterface.jl/src/fallbacks.jl:164
│   exception =
│    Break point
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] _warn_message(fun::Function, type::Type)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:8
│      [3] macro expansion
│        @ ./logging/logging.jl:383 [inlined]
│      [4] add!!(y::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, x::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Int64, β::VectorInterface.One)
│        @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:164
│      [5] add!!
│        @ ~/GitHub/VectorInterface.jl/src/interface.jl:141 [inlined]
│      [6] _add!(vdst::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, vsrc::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, α::Int64)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/OptimKit.jl:23
│      [7] optimize(fg::PEPSKit.var"#571#574"{LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, PEPSOptimize{EigSolver{:diffgauge}}, Vector{Float64}, Vector{Matrix{Float64}}, Vector{Float64}, Vector{Float64}}, x::Tuple{InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}}, alg::OptimKit.LBFGS{Float64, OptimKit.HagerZhangLineSearch{Rational{Int64}}}; precondition::typeof(OptimKit._precondition), finalize!::typeof(OptimKit._finalize!), shouldstop::OptimKit.DefaultShouldStop, hasconverged::OptimKit.DefaultHasConverged{Float64}, retract::Function, inner::typeof(PEPSKit.real_inner), transport!::typeof(PEPSKit.peps_transport!), scale!::typeof(OptimKit._scale!), add!::typeof(OptimKit._add!), isometrictransport::Bool)
│        @ OptimKit ~/.julia/packages/OptimKit/G6i79/src/lbfgs.jl:180
│      [8] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}, alg::PEPSOptimize{EigSolver{:diffgauge}}; finalize!::Function)
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:237
│      [9] fixedpoint(operator::LocalOperator{NTuple{8, Pair{Tuple{CartesianIndex{2}, CartesianIndex{2}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 2, 2, Vector{ComplexF64}}}}, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}}, peps₀::InfinitePEPS{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 4, Vector{ComplexF64}}}, env₀::CTMRGEnv{TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 1, 1, Vector{ComplexF64}}, TensorMap{ComplexF64, GradedSpace{U1Irrep, TensorKit.SortedVectorDict{U1Irrep, Int64}}, 3, 1, Vector{ComplexF64}}}; finalize!::Function, kwargs::@Kwargs{boundary_alg::@NamedTuple{tol::Float64, alg::Symbol, trscheme::@NamedTuple{alg::Symbol}}, gradient_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, iterscheme::Symbol}, optimizer_alg::@NamedTuple{tol::Float64, alg::Symbol, maxiter::Int64, ls_maxiter::Int64, ls_maxfg::Int64}, verbosity::Int64})
│        @ PEPSKit ~/GitHub/PEPSKit.jl/src/algorithms/optimization/peps_optimization.jl:197
│     [10] top-level scope
│        @ ~/GitHub/PEPSKit.jl/examples/xxz/main.jl:88
│     [11] include(mod::Module, _path::String)
│        @ Base ./Base.jl:557
│     [12] exec_options(opts::Base.JLOptions)
│        @ Base ./client.jl:323
│     [13] _start()
│        @ Base ./client.jl:531
└ @ VectorInterface ~/GitHub/VectorInterface.jl/src/fallbacks.jl:164

[Yue-Zhengyuan]

I see that scale!!, add!! are already redefined in src/environments/ctmrg_environments.jl, but it seems that they are not used at all.

[pbrehmer]

Thanks for creating an issue on that. I was running into the warnings already quite a while ago but was busy with somethings else so I just ran with the fallback functions. Perhaps this is a good time to fix things :-)

Note that the warnings are about adding and scaling two InfinitePEPSs - the VectorInterface for CTMRGEnvs should be fine. I'll open up a PR and add the corresponding add!! and scale!! functions to get rid of the warnings.