extra empty header/column in logfile

[CerenB]

Describe your problem in detail

I cannot pass the bids-validator with the logfile of visual motion localiser output.

Describe what you got instead

Bids validator throws an error

Error message

File Path: All rows must have the same number of columns as there are headers.

	Type:		Error
	File:		sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
	Location:		raw/sub-001/ses-001/func/sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
	Reason:		All rows must have the same number of columns as there are headers.
	Line:		2
	Evidence:	row 1: 5.543655	1.200499	static	-1.000000	0.000000	0.000000	1.000000	1.000000	n/a	

	Type:		Error
	File:		sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
	Location:		raw/sub-001/ses-001/func/sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
	Reason:		All rows must have the same number of columns as there are headers.
	Line:		2
	Evidence:	row 1: 5.320559	1.217551	static	-1.000000	15.000000	0.000000	1.000000	1.000000	n/a	


======================================================


File Path: Empty cell in TSV file detected: The proper way of labeling missing values is "n/a".

	Type:		Error
	File:		sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
	Location:		raw/sub-001/ses-001/func/sub-001_ses-001_task-auditoryLocalizer_run-001_events.tsv
	Reason:		Missing value at column # 10
	Line:		2
	Evidence:	row 1: 5.543655	1.200499	static	-1.000000	0.000000	0.000000	1.000000	1.000000	n/a	

	Type:		Error
	File:		sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
	Location:		raw/sub-001/ses-001/func/sub-001_ses-001_task-visualLocalizer_run-001_events.tsv
	Reason:		Missing value at column # 10
	Line:		2
	Evidence:	row 1: 5.320559	1.217551	static	-1.000000	15.000000	0.000000	1.000000	1.000000	n/a	

[CerenB]

heya! Since both @iqrashahzad14 and @fcerpe will start acquiring data, I want to ask this issue again.

@Remi-Gau @marcobarilari and whoever is using cpp-bids currently, when you try passing the bids validator, do you get error messages for the .tsv files? For the TMS project, I had issues with the last column being empty (see above). So I was deleting it. I wonder if someone else faced with the same issue and/or it is something to be changed in our cpp-bids repo?
In my projects, I created a small script to read and write .tsv files, which was doing the trick. nothing complicated.

[iqrashahzad14]

I got the same error. But I am using @ceren Battal ***@***.***> code to re-organise my tsv. Iqra

…

On Thu, Oct 28, 2021 at 2:38 PM CerenB ***@***.***> wrote: heya! Since both @iqrashahzad14 <https://github.com/iqrashahzad14> and @fcerpe <https://github.com/fcerpe> will start acquiring data, I want to ask this issue again. @Remi-Gau <https://github.com/Remi-Gau> @marcobarilari <https://github.com/marcobarilari> and whoever is using cpp-bids currently, when you try passing the bids validator, do you get error messages for the .tsv files? For the TMS project, I had issues with the last column being empty (see above). So I was deleting it. I wonder if someone else faced with the same issue and/or it is something to be changed in our cpp-bids repo? In my projects, I created a small script <https://github.com/CerenB/motionTMS_fMRI_analysis/blob/master/src/organiseTsvFile.m>to read and write .tsv files, which was doing the trick. nothing complicated. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#174 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASBKONC66BD6GIRU5WW6IJ3UJFG6XANCNFSM46LQ3PUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Remi-Gau]

Ok I ran this test and indeed the validator does give error code 22 and 23 on the output of this.

https://github.com/cpp-lln-lab/CPP_BIDS/blob/dev/tests/manualTests/test_makeRawDataset.m

I need to check but this seems due to the fact that the tsvwrite adds an extra tab at the end of the line.

If that's the case then this is a bids matlab bug not a CPP BIDS bug.

Will investigate.

[Remi-Gau]

This seems to rules out a pure bids-matlab issue

logFile.onset = 5;
logFile.duration = 3;

bids.util.tsvwrite(fullfile(pwd, 'tmp.tsv'), logFile)

FID = fopen(fullfile(pwd, 'tmp.tsv'), 'r');
content = fread(FID, 19, 'uint8=>char')

content =

  19×1 char array

    'o'
    'n'
    's'
    'e'
    't'
    '	'
    'd'
    'u'
    'r'
    'a'
    't'
    'i'
    'o'
    'n'
    '↵'  < ---- no extra tab at the end of the line
    '5'
    '	'
    '3'
    '↵'

[Remi-Gau]

definitely an extra is added by CPP_BIDS

    cfg.subject.subjectNb = 1;
    cfg.subject.runNb = 1;

    cfg.task.name = 'testtask';

    cfg = createFilename(cfg);

    logFile = saveEventsFile('init', cfg);
    logFile = saveEventsFile('open', cfg, logFile);

    logFile.onset = 2;
    logFile.duration = 3;

    logFile = saveEventsFile('save', cfg, logFile);

    saveEventsFile('close', cfg, logFile);
    
    funcDir = fullfile(cfg.dir.outputSubject, cfg.fileName.modality);
    eventFilename = cfg.fileName.events;
    
    FID = fopen(fullfile(funcDir, eventFilename), 'r');
    content = fread(FID, 49, 'uint8=>char')


content =

  49×1 char array

    'o'
    'n'
    's'
    'e'
    't'
    '	'
    'd'
    'u'
    'r'
    'a'
    't'
    'i'
    'o'
    'n'
    '	'
    't'
    'r'
    'i'
    'a'
    'l'
    '_'
    't'
    'y'
    'p'
    'e'
    '↵'
    '2'
    '.'
    '0'
    '0'
    '0'
    '0'
    '0'
    '0'
    '	'
    '3'
    '.'
    '0'
    '0'
    '0'
    '0'
    '0'
    '0'
    '	'
    'n'
    '/'
    'a'
    '	'  <--- this extra tab should not be there
    '↵'

[Remi-Gau]

#179 should have fixed this. Will close for now but reopen if this is not fixed.

I will update the submodule in the different repos we have (I really need to automate this).

I think that your localizers repos are diverging from the one the upstream repo of CPP_LAB: so you might have to update your submodule manually.

We will have to find a way to avoid our different repos diverging that much: I don't like it because it makes it hard to maintain.

[Remi-Gau]

@all-contributors please add @iqrashahzad14 for bug test

[allcontributors]

@Remi-Gau

I've put up a pull request to add @iqrashahzad14! 🎉