Skip to content

Softmax implementation in OneVersusAllModelParameters returns NaN even for scores of ~90.0f #3648

Closed
@rauhs

Description

@rauhs

version: 0.11

This implementation:

private void NormalizeSoftmax(float[] scores, int count)
{
float sum = 0;
for (int i = 0; i < count; i++)
{
scores[i] = (float)Math.Exp(scores[i]);
sum += scores[i];
}
for (int i = 0; i < count; i++)
scores[i] = scores[i] / sum;
}

will result in NaN as the output even for moderate scores of around 90f:

      var foo = new[] { 90f, 2, -2f };
      NormalizeSoftmax(foo, 3);

Seen very easily in the debugger (the float cast will result in Inf). I'm getting back a score of roughly 102 for some of my data sets and models. Though they're extremely rare.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions