Streamline failures on Power PC (?big-endian)

[matthew-brett]

These failures

======================================================================
ERROR: test_save_complex_file (nibabel.streamlines.tests.test_streamlines.TestLoadSave)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tests/test_streamlines.py", line 267, in test_save_complex_file
    tfile = nib.streamlines.load(filename, lazy_load=False)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/__init__.py", line 96, in load
    return tractogram_file.load(fileobj, lazy_load=lazy_load)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 153, in load
    streamlines = ArraySequence(tck_reader)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 93, in __init__
    self.extend(iterable)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 238, in extend
    for e in elements:
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 417, in _read
    "Cannot find a streamline delimiter. This file"
DataError: Cannot find a streamline delimiter. This file might be corrupted.

======================================================================
ERROR: test_save_simple_file (nibabel.streamlines.tests.test_streamlines.TestLoadSave)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tests/test_streamlines.py", line 229, in test_save_simple_file
    tfile = nib.streamlines.load(filename, lazy_load=False)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/__init__.py", line 96, in load
    return tractogram_file.load(fileobj, lazy_load=lazy_load)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 153, in load
    streamlines = ArraySequence(tck_reader)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 93, in __init__
    self.extend(iterable)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 238, in extend
    for e in elements:
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 417, in _read
    "Cannot find a streamline delimiter. This file"
DataError: Cannot find a streamline delimiter. This file might be corrupted.

======================================================================
ERROR: test_write_simple_file (nibabel.streamlines.tests.test_tck.TestTCK)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tests/test_tck.py", line 165, in test_write_simple_file
    new_tck = TckFile.load(tck_file)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 153, in load
    streamlines = ArraySequence(tck_reader)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 93, in __init__
    self.extend(iterable)
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/array_sequence.py", line 238, in extend
    for e in elements:
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tck.py", line 417, in _read
    "Cannot find a streamline delimiter. This file"
DataError: Cannot find a streamline delimiter. This file might be corrupted.

======================================================================
FAIL: test_load_write_file (nibabel.streamlines.tests.test_tck.TestTCK)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildslave/debian-ppc-64/nibabel-py2_7-debian-ppc/build/venv/local/lib/python2.7/site-packages/nibabel/streamlines/tests/test_tck.py", line 197, in test_load_write_file
    assert_equal(tck_file.read(), open(fname, 'rb').read())
AssertionError: 'mrtrix tracks\ncount: 0000000003\ndatatype: Float32LE\nfile: . 67\nEND\n\x00\x00\x00\x00?\x80\x00\x00@\x00\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x00\x00\x00\x00?\x80\x00\x00@\x00\x00\x00@@\x00\x00@\x80\x00\x00@\xa0\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x00\x00\x00\x00?\x80\x00\x00@\x00\x00\x00@@\x00\x00@\x80\x00\x00@\xa0\x00\x00@\xc0\x00\x00@\xe0\x00\x00A\x00\x00\x00A\x10\x00\x00A \x00\x00A0\x00\x00A@\x00\x00AP\x00\x00A`\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x7f\xc0\x00\x00\x00\x00\x80\x7f\x00\x00\x80\x7f\x00\x00\x80\x7f' != 'mrtrix tracks\ncount: 0000000003\ndatatype: Float32LE\nfile: . 67\nEND\n\x00\x00\x00\x00\x00\x00\x80?\x00\x00\x00@\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\x00\x00\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\xa0@\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\x00\x00\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\xa0@\x00\x00\xc0@\x00\x00\xe0@\x00\x00\x00A\x00\x00\x10A\x00\x00 A\x00\x000A\x00\x00@A\x00\x00PA\x00\x00`A\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\xc0\x7f\x00\x00\x80\x7f\x00\x00\x80\x7f\x00\x00\x80\x7f'

https://nipy.bic.berkeley.edu/builders/nibabel-py2.7-debian-ppc/builds/120/steps/shell_6/logs/stdio

Debugging, by error has existed since at least commit 72d146a . @MarcCote - any ideas?

[MarcCote]

This is clearly a problem between little-endian and big-endian architecture. I got an idea of what is causing the problem but am I able to run some tests on that machine (or another one using big-endian)?

I think it has something to do with how we check for end of file and end of streamline delimiters
(e.g. https://github.com/nipy/nibabel/blob/master/nibabel/streamlines/tck.py#L415 ). Or maybe it is related to how we save the data (wrong dtype?). I'll have to investigate.

[matthew-brett]

Send me your SSH public key somehow? - I'll give you a login to the machine where the failure is happening.