fix: unicode problem with MRTrix2TrackVis

[mgxd]

due to #1802

@satra @chrisfilo @oesteban is there a better solution than just ignoring the problematic byte?

[oesteban]

I'd say that the trk file should be opened as bytes ('rb'). Then, decode the strings (line.decode()). WDYT @satra ?

I would not advise skipping errors, that way we cannot reconstruct what really happened. It'd be better wrap it into a try catch that logged a warning, at the very least.

BTW, the import of open at the top should be from io, instead of builtins.

[satra]

@oesteban - i agree on both accounts.

[satra]

@mgxd - now the file object is a byte-stream, as @oesteban suggests wherever it's read, you will likely have to do read().decode()

[mgxd]

@satra When I tried that I was running into an error saying ascii codec can't decode byte - I'm assuming the read() method is flexible enough to take in the byte-stream as it's not throwing an error without the trailing .decode()

[mgxd]

still running into libicui18n.so.58 error on travis build, despite adding icu === 56.1 in #1794

[satra]

@mgxd - did you merge with master?

also i'm not getting the error in py3 when i just load the file directly using:

In [1]: from io import open

In [2]: with open('/Users/satra/Downloads/LeftTempSingleSeed_tracks.tck', 'rb') as fp:
   ...:     data = fp.read()
   ...:     

In [3]: data[:619].decode().split('\n')
Out[3]: 
['mrtrix tracks    ',
 'downsample_factor: 3',
 'fod_power: 0.25',
 'init_threshold: 0.100000001',
 'lmax: 8',
 'max_angle: 60',
 'max_dist: 243.00621',
 'max_num_attempts: 200000',
 'max_num_tracks: 2000',
 'max_seed_attempts: 1',
 'max_trials: 1000',
 'method: iFOD2',
 'min_dist: 12.1503105',
 'mrtrix_version: 0.3.15-285-g68589079',
 'output_step_size: 1',
 'rk4: 0',
 'samples_per_step: 4',
 'sh_precomputed: 1',
 'source: WMfod.mif',
 'step_size: 1',
 'stop_on_all_include: 0',
 'threshold: 0.100000001',
 'timestamp: 1486486988.2982976437',
 'unidirectional: 0',
 'roi: seed LeftTempSingleSeed.mif',
 'roi: mask bmask.nii.gz',
 'roi: mask bmask.nii.gz',
 'datatype: Float32LE',
 'file: . 640',
 'count: 2000',
 'total_count: 2925',
 'END',
 '']

[satra]

however i see the error trying to decode beyond the END marker:

In [4]: data[:1000].decode().split('\n')
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-4-cd5b79147f7f> in <module>()
----> 1 data[:1000].decode().split('\n')

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf5 in position 640: invalid start byte

i believe the rest is binary and shouldn't require decoding. @mgxd - could you take a look at the functions and the mrtrix file format to see how they represent things?

[codecov-io]

Codecov Report

Merging #1804 into master will decrease coverage by <.01%.
The diff coverage is 10%.

@@            Coverage Diff             @@
##           master    #1804      +/-   ##
==========================================
- Coverage    72.6%   72.59%   -0.01%     
==========================================
  Files        1070     1070              
  Lines       54456    54460       +4     
  Branches     7868     7868              
==========================================
  Hits        39537    39537              
- Misses      13686    13690       +4     
  Partials     1233     1233

Flag	Coverage Δ
#smoketests	72.59% <10%> (-0.01%)	⬇️
#unittests	70.15% <10%> (-0.01%)	⬇️

Impacted Files	Coverage Δ
nipype/interfaces/mrtrix/convert.py	21.27% <10%> (-0.47%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a17af6b...29512c7. Read the comment docs.

[mgxd]

@satra this should be good to go