fail with parameters of size 1, tuple issue

[John-Boik]

I apologize in advance for creating an issue with little documentation to support it. But I have seen this problem with a new install of version 1.0.0 as recently as a week ago.

In essence, code in gluon/parameters.py seems to choke when a parameter is of size (1,). For example, a call to net.params.get() like the following will throw an error:
net.params.get('some_name', init= mx.init.Zero(), shape=(1,))

In these cases, in parameters.py, self._shape is not a tuple, so subsequent calls to zip() and other functions fail.

The fix I used was to make the following changes in gluon/parameters.py:

near line 151:

# the following two lines replace self._shape with a tuple
if isinstance(self._shape, tuple)==False:
    self._shape = (self._shape,)
assert len(self._shape) == len(new_shape) and \

similarly near line 196:

if isinstance(self._shape, tuple)==False:
    self._shape = (self._shape,)            
for self_dim, data_dim in zip(self.shape, data.shape):

similarly near line 121:

def __repr__(self):
    if isinstance(self._shape, tuple)==False:
        self._shape = (self._shape,)        
    s = 'Parameter {name} (shape={shape}, dtype={dtype})'

John

[szha]

shape=(1,) seems fine on my end. We use vector shapes quite a lot in gluon.rnn. https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/rnn/rnn_cell.py#L334-L339

Would you mind producing a small snippet to help understand the issue you're facing?

[Roshrini]

@sandeep-krishnamurthy Can you please add labels:
Gluon,
Pending Requester Info

[John-Boik]

I created a small application to replicate one of the problems. In the code below, the error occurs due to the print statement as noted.

import mxnet as mx
from mxnet import nd, gluon
from mxnet.gluon import Block

class myBlock(Block):
    
    def __init__(self, **kwargs):
        super(myBlock, self).__init__(**kwargs)

        with self.name_scope():
            self.p1 = self.params.get('param1', init= mx.init.Constant(1), shape=(1), grad_req = 'write') 
            self.p2 = self.params.get('param2', init= mx.init.Constant(1), shape=(2,3), grad_req = 'write') 

class App(object):
    def __init__(self):
        net = gluon.nn.Sequential()
        
        self.p3 = net.params.get('param3', init= mx.init.Constant(1), shape=(1), grad_req = 'write') 
        with net.name_scope():
            net.add(myBlock()) 
        
        net.collect_params().initialize(ctx=mx.gpu())
        params = net.collect_params()
        
        print(params)  # the error occurs due to this print statement
        
        net.save_params('test_params')
        net.load_params('test_params', ctx=mx.gpu())
        print("\ndone\n")

app = App()  

The error is:

  File "/usr/local/lib/python3.5/dist-packages/mxnet/gluon/parameter.py", line 127, in __repr__
    return s.format(**self.__dict__)
KeyError: 'shape'

This is one of the errors that I saw with my real code. It occurs near line 127 in parameters.py. A fix is pasted below.

    def __repr__(self):
        s = 'Parameter {name} (shape={shape}, dtype={dtype})'
        s2 = 'Parameter {name} (shape={_shape}, dtype={dtype})'  # added this line
        return s2.format(**self.__dict__)  # changed s.format() to s2.format()

I am not seeing the additional errors with this short app that I saw with my real code, the errors due to self._shape not being a tuple. To take care of these errors, I altered parameters.py in the three locations mentioned. In all three locations, I added this:

if isinstance(self._shape, tuple)==False:
            self._shape = (self._shape,)    

[piiswrong]

The cause here is (1) vs (1,). (1) is just 1, which is not a tuple.

I think we should support shape=1. Basically check if shape is a integer then convert it to a tuple in Parameter.init

@szha could you submit a fix?