std::stoi error from MVDTool:open_combo_tsv() for reading mvd3 files

[WeinaJi]

In BlueConfigs, some tests contain tsv files with "layer" data as alphabetic letters not numbers, so that open_combo_tsv() throw errors.
E.g.
For blueconfigs/quick-hip-projSeed/BlueConfig:
MEComboInfoFile
/gpfs/bbp.cscs.ch/project/proj42/entities/emodels/20190402/mecombo_emodel.tsv

The error is

 File "/jenkins/07/workspace/sim.Neurodamus2/INSTALL_HOME/install/linux-rhel7-x86_64/intel-19.0.4/py-neurodamus-develop-w6vjoy/lib/python3.7/site-packages/neurodamus/cell_readers.py", line 203, in load_nodes
    node_reader.open_combo_tsv(combo_file)
ValueError: stoi

As far as I know, the layer value is not used in the simulation and there seems to be no format spec for this column. In MVDTool, could MEComboEntry::layer be string instead of int ?

[WeinaJi]

I see that in mvd3 file the /cells/properties/layer data type is H5T_STD_I32LE (int), so I guess here it is not good to change to string in MVDTool . Then shall we catch the std::stoi exception in case of alphabetic layer values from the tsv file?

[matz-e]

Hi Weina,

I think your first suggestion of using std::string maybe worthwhile, it does not seem to be correlated with the layers from actual MVD files?

In the simulator, the layer is not converted to a numerical value with the legacy method: https://bbpcode.epfl.ch/source/xref/sim/neurodamus-py/neurodamus/metype.py#136

[WeinaJi]

Hi Matthias, and @iomaganaris

I think they are correlated.

inline std::vector<int32_t> MVD3File::getLayers(const Range& range) const {
    return getDataFromTSVorMVD<int32_t>(did_cells_layer, did_lib_NONE, TSVColumn::Layer, range);
}

template <typename T>
inline std::vector<T> MVD3File::getDataFromTSVorMVD(const std::string& did_ds,
                                                    const std::string& did_lib,
                                                    const TSVColumn& col,
                                                    const Range& range) const {
    if (_tsv_file) {
        return getDataFromTSV<T>(col, range);
    }
    return getDataFromMVD<T>(did_ds, did_lib, range);
}

If the tsv file is opened, the value is retrieved from tsv file, otherwise from MVD3. It doesn't seem to work to have "string" for tsv layer and "int32" for mvd3 layer.

I am thinking to catch the stoi exception in case of tsv layer values not a number and to assign it an invalid value such as -1. Anyhow, this value from tsv is not used. What do you think ?

[matz-e]

Hi, you're right! This is… ugly…

Maybe we should document the spec first and then work from there? From my circuit building perspective, the solution you propose would also work fine, since I have no use for TSV files.

[WeinaJi]

PR is merged.