Description
Hello guys,
this is just a question came up to my mind. In Simulation (and other Areas) developers are always confronted with different departments (in a Company) having different data sources (SQL, XML, Excel, CSV, HDF5, MAT, MDF (Format in automotive), ....). All we developer want are the Arrays / datasets with their meta data. So honestly I was thinking "ok - it is the same Situation like table based SQL data - so many different sources from Oracle, IBM, Microsoft, etc. but one Framework to Access and represent all different sources : EF Core".
So now my question : you think it would be beneficial for ML.NET to have a EF Core like Framework but with Focus on Access Arrays? Every data source Format could implement an Interface / Provider. Maybe it would bring most benefit for heterogeneous Environments. I know for SQL etc. we have EF Core and EF Core could be also used maybe for this - but I think EF is focused on ORM and not searching for Arrays in HDF5, MDF, CSV etc.
To be honest : I did not find such a Framework in Python, Java etc. Thats why I ask how others think about this. If every source have a specific Provider and this Provider can search for Arrays and meta data : you even could think about a Query Language. Sth like "Select Arrays where Unit == "°C"" or sth in this way.