Help! Does this work at all?

[spectrograph-uk]

Hi, I am sorry that I am so confused. I gather that most high level tensor operations have to be done using the single function einsum, I'm OK with that, for instance the square of a matrix x is einsum("ij,jk->ik",x,x). Also that grad() is a function that takes as input a function from tensors to tensors and outputs another one. So my "Hello World" for tensorflow.js would be

f=function(x){return tf.einsum("ij,jk->ik",x,x)};tf.grad(f)(tf.tensor([[1,2],[3,4]]))

which defines f to be the function on tensors of type (2,2) otherwise known as squaring the matrix, and it does not give any problem evaluating grad(f) on tf.tensor([[1,2],[3,4]]) but it says "gradient function not found for Einsum"

How can this be, that essentially the ONLY function operating on tensors which is provided is einsum, and grad ONLY operates on such functions, but it cannot "FIND" the gradient function for einsum????

I see this was asked earlier in 2019 with no answer except that someone checked einsum exists, come on, I am so frustrated. HOW CAN IT BE POSSIBLE THAT THE ONLY TENSOR OPERATION ALLOWED IS EINSUM, AND grad() CAN ONLY ACT ON TENSOR OPERATIONS, WHILE grad of einsum CANNOT BE FOUND!

I am so frustrated by this. It is like trying to do a 'hello world' and being told you must use the command print() but print() is not yet implemented for strings, while it is only implemented for strings.

Sorry bo be so dumb but did you guys really make an implementation of tensorflow with no other tensor operator but einsum, and grad only acting on tensor operators, but not yet defined for einsum?

I won't attach a file, but the error is caused by a script with just this line

f=function(x){return tf.einsum("ij,jk->ik",x,x)};tf.grad(f)(tf.tensor([[1,2],[3,4]]))

[spectrograph-uk]

Here is a screenshot showing the error message in the simplest possible context (the file tfjs.html is just an otherwise blank html file with your most recent minimized tensorflow.js script between script tags so I can call the tf,grad() function from a browser developer console to exhibit the error message)

[gaikwadrahul8]

Hi, @spectrograph-uk

Apologize for the delayed response and I was able to replicate the issue from my end and it's giving same error message Cannot compute gradient: gradient function not found for Einsum. For your reference, I have added screenshot below so we have to dig more into this issue and we'll update you soon. I hope you've gone through the limitations of tf.einsum Thank you for noticing this issue and I really appreciate your efforts and time. Thank you!

[spectrograph-uk]

I hope you've gone through the limitations of tf.einsum

We're not exceeding any limitations, but this gives me an idea what might be wrong. The function f= x => tf.einsum("ij,jk->ik",x,x) could be understood as acting on tensors of any shape (n,n) and so tf.grad(f) is required also to be ambiguous in the same sense. When I evalate grad(f) at tf.tensor([[1,2],[3,4]]) we see that we're talking about the case of shape (2,2) but it is perfectly reasonable -- and might be necessary --- to tell grad which shape of tensor we're restriting f to even before we try to evaluate it. I am guessing this is what I'd doing wrong...that I'm needing to give grad some extra information about the shape of tensors it's dealing with?

The examples in the documentation for grad do not enslighen us because they aren't actually a general type of tensor operation, they are things like squaring every entry of a matrix of a particular shape. To really use this one has to be able to use it with things like einsum, but I can see that the way einsum does not include anything about tensor shapes in its invocation could make this difficult.

[gaikwadrahul8]

Hi, @spectrograph-uk

Apologize for the delayed response, thank you for your pointers/findings and we don't have gradient function defined for Einsum at the moment so we'll update you soon. Thank you!

[r-it-snow]

Hi, @spectrograph-uk

Apologize for the delayed response, thank you for your pointers/findings and we don't have gradient function defined for Einsum at the moment so we'll update you soon. Thank you!

Any update ontf.einsum's grad function implementation? The issue still persists today. This function is needed to implementation MultiHeadAttention and transformers.