python 2.7 string issue in plugins

[bpinsard]

with recent updates, I get the following error using SGE plugin:

TraitError: Each key of the 'environ' trait of a CommandLineInputSpec instance must be a newstr or None, but a value of 'MDMSESSION' <type 'str'> was specified.

It seems that the traits definition uses the future newstr which does not match with what os.environ contains (old str). It is certainly the same for other plugins, and this is the only places where environ is fed to CommandLine in whole nipype.

Any idea how to fix this?
Thanks

[oesteban]

Ok, I'll look at this now

[bpinsard]

Thanks but in fact in seems something more global, the future newstr seems to interact with all traits string type checking. Each string that comes from os.environ causes crash.

n_tesselate = pe.Node(
    freesurfer.MRITessellate(label_value=1),
    name='tesselate')

TraitError: Each key of the 'environ' trait of a MRITessellateInputSpec instance must be an implementor of, or can be adapted to implement, newstr or None, but a value of u'SUBJECTS_DIR' <type 'unicode'> was specified

I do not know if there is a simple way to change this except removing all future

[oesteban]

That's the thing: without unicode_literals from __future__ and the newstr we break python 3 compatibility. We've dealt with this issue in most of the situations, but of course some are not tested (our coverage is around 70% now, we want to raise that).

[bpinsard]

ok, I am just surprised that it got merged in master considering that it breaks a lot of nipype interfaces/plugins when used in 2.x environment. Or am I the only one to experience problems with 2.x?

[oesteban]

It doesn't break a lot nipype interfaces. We are constantly checking the circleCI examples for both 2.7 and 3.5 (https://circleci.com/gh/nipy/nipype/1475).

The alternative would be stick with 2.7 forever.

[bpinsard]

I have been running nosetest on nipype in my freshly cleaned conda 2.7 environment, and in fact it seems that any inputs that requires string from os.environ (most inheriting from CommandLine) crashes, which is a huge problem for most pipelines that I use.
I am preparing a PR to change DictStrStr

-DictStrStr = traits.Dict(str, (bytes, str))
+DictStrStr = traits.Dict((bytes,str,unicode), (bytes, str,unicode))

This will solve my case for environ problem, nosetests crashes for unicode string in other places, but I am less concerned about that for now.

[chrisgorgo]

This is weird... Are you testing current master? Travis was suppose to run tests in python 2.7, 3.4 and 3.5 to ensure all of those are supported. See: https://travis-ci.org/nipy/nipype

[bpinsard]

yes I am testing the current master

[bpinsard]

Since I updated, I have tested on my regular python and also anaconda.

[chrisgorgo]

I wonder if this could have something to do with the version of futures/six on your system.

[bpinsard]

six==1.10.0
future==0.15.2

[chrisgorgo]

It's the same as on Travis :/ https://s3.amazonaws.com/archive.travis-ci.org/jobs/159706130/log.txt