TensorTypeExtensions: Added conversion between Tensor to primitive C# types instead of throwing NotSupportedException #4290
Conversation
…g NotSupportedException - Fixed proper reading of TF_STRING - Added overloads that support TF_STRING
|
|
Nucs
changed the title
| %supported_numericals_TF_DataType = ["TF_BOOL","TF_UINT8","TF_INT16","TF_UINT16","TF_INT32","TF_UINT32","TF_INT64","TF_UINT64","TF_DOUBLE","TF_FLOAT"] | ||
| %supported_numericals_TF_DataType_full = ["TF_DataType.TF_BOOL","TF_DataType.TF_UINT8","TF_DataType.TF_INT16","TF_DataType.TF_UINT16","TF_DataType.TF_INT32","TF_DataType.TF_UINT32","TF_DataType.TF_INT64","TF_DataType.TF_UINT64","TF_DataType.TF_DOUBLE","TF_DataType.TF_FLOAT"] | ||
| #endif | ||
|
|
||
| namespace Microsoft.ML.Transforms | ||
| { | ||
| [BestFriend] | ||
| internal static class TensorTypeExtensions | ||
| { | ||
| public static void ToScalar<T>(this Tensor tensor, ref T dst) where T : unmanaged | ||
| { |
There was a problem hiding this comment.
Can this handle string tensors? #Closed
There was a problem hiding this comment.
I was sure I had String-Tensor -> T conversion implemented, Added it. #Closed
| } | ||
| } | ||
|
|
||
| public static void CopyTo<T>(this Tensor tensor, Span<T> destination) where T : unmanaged |
There was a problem hiding this comment.
public static void CopyTo(this Tensor tensor, Span destination) where T : unmanaged [](start = 8, length = 89)
There is a lot of unsafe code in here that does memory copy/manipulation, have you considered using Span.CopyTo(Span) and Span.Slice(int,int) to achieve the same?
There was a problem hiding this comment.
I don't see a point, I perform all the checks necessary beforehand. The code is safe. Any use of Span will result in unnecessary overhead.
Plus, is there a way to perform cast from one dtype to an other using only Span?
There was a problem hiding this comment.
@codemzs I can replace:
var sdst = (bool*) Unsafe.AsPointer(ref destination.GetPinnableReference());
With:
fixed (byte* sdst = MemoryMarshal.Cast<T, byte>(destination))
They do a similar logic just with more overhead.
- CopyTo<T>: Added `fixed` on destination to prevent GC from moving. - Fixed wrong argument passing to Converts.ChangeType<T> - Reformat
|
@Nucs Thanks for the change but I'm curious why are we making this change? Where exactly will it be used in the code base? This is an internal class btw. |
|
@codemzs, I assumed at this comment that the upvote was an answer to my question there. Should've asked for a more explicit answer 😅. Eitherway I'll probably add this code to |
ghost
locked as resolved and limited conversation to collaborators
unmanaged