Bug: Derivative of transposed-vector times matrix is incorrect.

[Red-Portal]

Hi,
Found a correctness bug that seems only to affect ReverseDiff.

julia> A = [1 2; 3 4]; x = [5, 6];

julia> [ReverseDiff.gradient(y -> sum(y'*A), x) ;; 
        ReverseDiff.gradient(y -> sum(A'*y), x) ;;
             Zygote.gradient(y -> sum(y'*A), x)[1] ;;
             Zygote.gradient(y -> sum(A'*y), x)[1] ]
2×4 Matrix{Float64}:
 4.0  3.0  3.0  3.0
 6.0  7.0  7.0  7.0

[mohamed82008]

Thanks for reporting this. Do you have any time to dig deeper into this?

[zelong-yin]

Hi, I also found that the memory allocation benchmarks are no longer correct.

When I tried the example codes it now shows spurious memory allocations

It seems that these allocations are associated with the use of transposes. When I delete the adjoint operators in the definition they goes away.

[Red-Portal]

@mohamed82008 Hi Mohamed! Unfortunately, my AD knowledge is insufficient to debug the AD frameworks. I do feel that I need to learn this stuff though...

[Red-Portal]

@zelongyin2000 I think it would be better to open a separate issue for that

[Red-Portal]

Thanks @willtebbutt !