How to write CA block

[young147]

Hi! Your work has helped me a lot, but there few examples shows how to write CA or SD block in mdf files, I would be appreciate if you cloud resolve my problem.

[ihedvall]

Let us start with the Signal Data (SD) block. It is used to store Variable Length Signal Data (VLSD) which are string or byte arrays. In this implementation of MDF Writers it does store the fixed length values in the Data Groups (DG) data block (DT) and the VLSD bytes in a SD block.

The is a flaw in the above design as the fixed length values can be appended to the end of the files, the VLSD must be temporary stored in primary memory and flushed onto the end of the file, when the measurement is stopped. There is a risk of out of memory exception.

To solve the problem with the primary memory, you may add a Channel Block (CG) block without any channels and flag it as a CG-VLSD block. You now the VLSD records in this group while the original group stores the offset into the CG-VLSD group. This technique is used for bus loggers and application that store a video/sample or similar.

I will add the CA comments to the next comments.

[ihedvall]

The current implementation does not support writing of Channel Array blocks. It is however quite straight forward to implement this. There is one bottle neck that the C++ standard doesn't have something like the Python numpy i.e. support for multi-dimensional arrays. Boost have this support but the MDF library has stripped out the Boost support (user requirements). Maybe there exist a simple numpy C++ library out there in GitHub?

The current solution have an extra array index argument which makes it possible to get channel array values. It is not the best solution but it is relative simple solution. The IChannel::SetChannelValue function doesn't have this extra argument but should be possible to fix.

What is your requirements regarding an array interface?

[young147]

Thanks for your reply. The reason why I raised these question is that the recording CURVE array, which can be plotted in CANApe tools, and I implemented this when I used python library of asammdf which just like you said using numpy to handle this.
And thank you again for helping me with my questions, I will try your suggestion.

[ihedvall]

The A2L characteristic type may require up to 5 dimensions excluding the X-axis. I'm not sure how the CANApe actual figure out, in the MDF file, which adjustable object it is referring to. I do have an ASAP2 parser (ihedvall/a2llib) in case you feel unchallenged. A2L file appended in the MDF file?

I browsed around and preliminary the xtensor C++ library might replace the numpy functionality. It looks like a header only include.

I propose the following tasks.

1. Make the CA block writable. Some issues with the complex settings.
2. Add an array index to the IChannel::SetValue function.
3. Add support for xtensor library. Other library ?

I'm currently working with the mdf2csv application which I want to finalize first. Note that it exist a new MDF writer type MdfConverter which is a better choice if your application is of converter type.

[young147]

Thank you very much for your response, and I will try MdfConverter with my code.

[ihedvall]

I have added basic support for CA block writing. I have done point 1 and 2 above. I and start working with the xtensor library support. Note that the xtensor may not be the best choice why your input may be valuable.

[ihedvall]

@young147
My initial design to use the XTensor/Eigen multidimensional C++ libraries doesn't work. The libraries header file must be exposed in the mdf/include headers. This mess up the build for most users.

I noted that XTensor can convert a (1-D) vector of values to an array by using the array shape and the value list. My plan is to add a simple GetEngValues() function that return all array values in a vector. It's not faster that the current but simpler. The same for setting values.

[ihedvall]

@young147
I have added support for writing array blocks. Including any third-party array library will not work as it require that all users of the MDF library to do the same. Instead I have added functions so you can get/set the one-dimensional array easier. Most array library have functions that fold/unfold the multi-dimensional array to an one-dimensional array.

If you have time, I would appreciate if you can give me some feedback if the interfaces are good or need to be modified/improved.

Best Regards
Ingemar Hedvall