Regardless of any additional types we may want to add to System.Numerics.Tensors, we would like to expand the set of APIs exposed on the TensorPrimitives static class in a few ways (beyond the work done in .NET 8 in #92219):

• Vectorize TensorPrimitives operations that are currently scalar only #97193
• Alignment improvements for ConvertXx, CosineSimilarity, IndexOfMin, IndexOfMax, IndexOfMinMagnitude, IndexOfMaxMagnitude
• Investigate precision of the vectorized TensorPrimitives implementations. #98861
• Additional operations defined in BLAS / LAPACK that don't currently have representation on TensorPrimitives
• Perform a broader scan of ML.NET APIs, seeking more methods that should be on the post-GA backlog - @michaelgsharp
  • We've already covered all of the shared methods, but there are one-off implementations that might be worth productizing into TensorPrimitives
• Additional operations that would enable completely removing the internal CpuMath class from ML.NET, e.g. Add (with indices), AddScale (with indices), DotProductSparse, MatrixTimesSource, ScaleAdd improvement via AddMultiply or MultipleAdd overloads, SdcaL1UpdateDense, SdcaL1UpdateSparse, and ZeroMatrixItems (might exist in System.Memory).
• Double-check the flow of XML docs -> https://github.com/dotnet/dotnet-api-docs -> docs.microsoft.com/
• Add conceptual docs for TensorPrimitives, maybe near https://github.com/dotnet/docs/blob/main/docs/standard/numerics.md
• Generic overloads of all relevant methods, constrained to the appropriate numerical types
• Get benchmarks added into dotnet/performance
  • Collect baseline results from the time between RC2 and GA right before our alignment improvements went in
  • Collect new results from main after all of the alignment
• Improve performance of Min, Max, MinMagnitude, MaxMagnitude with relation to NaN handling
• Determine for lengths of 0 if we want to throw or return NaN (we consistently throw today when non-0 is required; ML.NET apparently returns 0?) - @tannergooding
  • We currently throw; if we decide not to throw, this could be changed in a minor release in a non-breaking way.
• Additional operations from Math{F} that don't currently have representation on TensorPrimitives, e.g. CopySign, Reciprocal{Sqrt}{Estimate}, Sqrt, Ceiling, Floor, Truncate, Log10, Log(x, y) (with y as both span and scalar), Pow(x, y) (with y as both span and scalar), Cbrt, IEEERemainder, Acos, Acosh, Cos, Asin, Asinh, Sin, Atan. This unmerged commit has a sketch, but it's out-of-date with improvements that have been made to the library since, and all of the operations should be vectorized.
• Refactor the generic TP implementation into multiple source files.
• Additional operations defined in the numerical interfaces that don't currently have representation on TensorPrimitives, e.g. BitwiseAnd, BitwiseOr, BitwiseXor, Exp10, Exp10M1, Exp2, Exp2M1, ExpM1, Atan2, Atan2Pi, ILogB, Lerp, ScaleB, Round, Log10P1, Log2P1, LogP1, Hypot, RootN, AcosPi, AsinPi, AtanPi, CosPi, SinPi, TanPi

We plan to update the System.Numerics.Tensors package alongside .NET 8 servicing releases. When there are bug fixes and performance improvements only, the patch number part of the version will be incremented. When there are new APIs added, the minor version will be bumped. For guidance on how we bump minor/major package versions, see this example.