Skip to content

Unexpected default for nn.softmax #69

@JackTemaki

Description

@JackTemaki

I just lost a couple of hours on debugging because I forgot that softmax_over_spatial, which is what nn.softmaxmaps to, does something completely different than the old "softmax" layer. Because it does not do the softmax per default over "F", but something different (defaulting to the time axis). This is really dangerous when you expect that you can use nn.softmax as an activation function.

I am not sure how to solve this best, I would say"softmax_over_spatial" is okay in its behavior (so no RETURNN changes), but nn.softmax should definitely not default to that behavior.

Maybe this issue is already resolved if nn.softmax needs an explicit dimension tag in the future, but if not, it needs to be fixed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions