Dim tag var names in serialized code are too long

[albertz]

E.g. for Transformer test case:

extern_data_data_dim_tags_1_time_dim = SpatialDim('time')
extern_data_data_dim_tags_2_input_dim = FeatureDim('input', 10)
network_encoder_subnetwork_layers_0_subnetwork_self_attn_subnetwork_self_attn_subnetwork_qkv_split_dims_dims_0_num_heads_dim = SpatialDim('num_heads', 2)
network_encoder_subnetwork_layers_0_subnetwork_self_attn_subnetwork_self_attn_subnetwork_k_new_dim_set_dim_tags_time_encoder_layers_0_self_attn_self_attn_self_att_expand_dim_dim = SpatialDim('encoder/layers.0/self_attn/self_attn:self_att_expand_dim')
network_encoder_subnetwork_layers_1_subnetwork_self_attn_subnetwork_self_attn_subnetwork_k_new_dim_set_dim_tags_time_encoder_layers_1_self_attn_self_attn_self_att_expand_dim_dim = SpatialDim('encoder/layers.1/self_attn/self_attn:self_att_expand_dim')
network_loop_unit_state_decoder_0_self_attn_k_accum_state_out_spatial_dim_loop_decoder_layers_0_self_attn_self_attn_self_att_expand_dim_dim = SpatialDim('loop/decoder/layers.0/self_attn/self_attn:self_att_expand_dim')
network_loop_unit_state_decoder_1_self_attn_k_accum_state_out_spatial_dim_loop_decoder_layers_1_self_attn_self_attn_self_att_expand_dim_dim = SpatialDim('loop/decoder/layers.1/self_attn/self_attn:self_att_expand_dim')
network_loop_unit_target_embedding_subnetwork_dot_reduce_target_dim = FeatureDim('target', 7)
network_loop_unit_decoder_subnetwork_layers_0_subnetwork_self_attn_subnetwork_self_attn_subnetwork_qkv_split_dims_dims_0_num_heads_dim = SpatialDim('num_heads', 2)
network_loop_axis_loop_dim_dim = SpatialDim('loop-dim')

This could be way shorter.

[albertz]

@JackTemaki @mmz33 Do you have some suggestion about better naming?

The current algorithm basically takes the path in the net dict or extern data which points first to the dim tag. This is then extended by the dim tag description (which is non-unique).

This makes it always unambiguous which is nice and simplifies the logic (no extra effort to make var names unique). And also, the var name shows where it comes from, which is also nice.

However, it's really too long (network_encoder_subnetwork_layers_1_subnetwork_self_attn_subnetwork_self_attn_subnetwork_k_new_dim_set_dim_tags_time_encoder_layers_1_self_attn_self_attn_self_att_expand_dim_dim).

I have several ideas in mind but I'm unsure.

E.g. we could strip away parts as long as it stays unique.

I'm also thinking about diffing one config to another. Currently when some other part in the network is extended or modified, this would not have any influence on previous unrelated dim tags. With some of the shorting solutions, this is not the case anymore. E.g. once the dim tag was the only dim tag, it could simply be named dim but as soon as other unrelated dim tags are introduced, dim should be renamed so that it stays unique, e.g. to time_dim. Now, in a diff, you would see the change dim to time_dim, although this dim tag was not changed. But maybe this is really not so relevant...

[albertz]

Actually, while dim tag description are not required to be unique, it is a good idea to have them unique anyway, for better debugging. This is up to the user. But let's assume the case they are unique: Then we can use the description (properly escaped or invalid chars replaced or removed) as var name.

Because there is no requirement for them to be unique, we still need the logic to make them unique but that's ok, and this logic would not be effective so often.

[albertz]

With the pushed change, it looks now like:

time_dim = SpatialDim('time')
input_dim = FeatureDim('input', 10)
num_heads_dim = SpatialDim('num_heads', 2)
encoder_layers_0_self_attn_self_att_expand_dim_dim = SpatialDim('encoder/layers.0/self_attn:self_att_expand_dim')
encoder_layers_1_self_attn_self_att_expand_dim_dim = SpatialDim('encoder/layers.1/self_attn:self_att_expand_dim')
target_dim = FeatureDim('target', 7)
loop_decoder_layers_0_self_attn_self_att_expand_dim_dim = SpatialDim('loop/decoder/layers.0/self_attn:self_att_expand_dim')
num_heads_0_dim = SpatialDim('num_heads', 2)
loop_decoder_layers_1_self_attn_self_att_expand_dim_dim = SpatialDim('loop/decoder/layers.1/self_attn:self_att_expand_dim')
loop_dim_dim = SpatialDim('loop-dim')

It's still not perfect but already better I think.

[albertz]

Small update, due to more recent changes, it now looks like:

time_dim = SpatialDim('time')
input_dim = FeatureDim('input', 10)
num_heads_dim = SpatialDim('num_heads', 2)
encoder_layers_0_self_attn_history_dim = SpatialDim('encoder/layers.0/self_attn:history')
encoder_layers_1_self_attn_history_dim = SpatialDim('encoder/layers.1/self_attn:history')
target_dim = FeatureDim('target', 7)
loop_decoder_layers_0_self_attn_history_dim = SpatialDim('loop/decoder/layers.0/self_attn:history')
num_heads_0_dim = SpatialDim('num_heads', 2)
loop_decoder_layers_1_self_attn_history_dim = SpatialDim('loop/decoder/layers.1/self_attn:history')
loop_dim = SpatialDim('loop-dim')

[albertz]

Another update, now it looks like:

time_dim = SpatialDim('time')
input_dim = FeatureDim('input', 10)
num_heads_dim = SpatialDim('num_heads', 2)
encoder_layers_0_self_attn_history_dim = SpatialDim('encoder/layers/0/self_attn:history')
encoder_layers_1_self_attn_history_dim = SpatialDim('encoder/layers/1/self_attn:history')
target_dim = FeatureDim('target', 7)
decoder_layers_0_self_attn_history_dim = SpatialDim('decoder/layers/0/self_attn:history')
num_heads_0_dim = SpatialDim('num_heads', 2)
decoder_layers_1_self_attn_history_dim = SpatialDim('decoder/layers/1/self_attn:history')
loop_dim = SpatialDim('loop-dim')

[albertz]

Another update:

time_dim = SpatialDim('time')
input_dim = FeatureDim('input', 10)
num_heads_dim = SpatialDim('num_heads', 2)
encoder_layers_0_self_attn_history_dim = SpatialDim('encoder/layers/0/self_attn:history')
encoder_layers_1_self_attn_history_dim = SpatialDim('encoder/layers/1/self_attn:history')
target_dim = FeatureDim('target', 7)
decoder_layers_0_self_attn_history_dim = SpatialDim('decoder/layers/0/self_attn:history')
decoder_layers_1_self_attn_history_dim = SpatialDim('decoder/layers/1/self_attn:history')
loop_dim = SpatialDim('loop-dim')