Skip to content

Commit

Permalink
Frontend part of JSON backend implementation
Browse files Browse the repository at this point in the history
Implement JSON backend to dummy state

Misses the actual implementation of AbstractIOHandlerImpl

Declare IOHandlerImpl for JSON and integrate it with other places

Misses the implementation.

Undebugged minimum implementation for JSON writing

First basically runnable version of JSON writing

To address:
No reading or deleting yet.
Datatypes are currently ignored and the data is assumed to be int64_t.
Attribute values are ignored and replaced with a dummy value.
If a subgroup name can be parsed as a nonnegative string, the JSON API
will create a JSON array rather than a JSON object (associative array) as intended.

Correctly handle groups that can be parsed as int

See last commit's description.

Fix index calculation with offsets in WriteData

Fix some mistakes in JSON writing

Correctly handle overwriting files:
-> overwritten files should not be possible to access any longer and be
clearly distinguished from the newly-created file
Make some verifications execute independent of compiler options.

Full implementation of JSON writing

Respects all datatypes now.

Format code according to Clion Stylesheet

https://github.com/ComputationalRadiationPhysics/contributing/blob/master/IDESettings/CLion/CRP_CLion2016_1.xml

Add generic branching over an openPMD datatype

First runnable version of JSON Reading

Cleanup and implementation of dataset extension

Undebugged version of JSON deletion

Properly (de)serialize datatypes

Instead of casting the Datatype enum to and from int (which is likely to break
when altering the enum), serialize to and from String values.

Fix a number of mistakes in JSON reading and writing

Cleanup

Add JSON tests and fix bugs found thusly

Add further tests and fix a further bug

The JSON library does not understand long double values (i.e. 128bit
floats), represent them as a char array.

Handle floating point special values

JSON represents +/-Infinity and NaN values as null. The JSON library
will correctly serialize those values *to* JSON, implement
(semi)-correct handly for deserialization. As it is unclear which exact
value a null represents, deserialize it to NaN.
Take notice that large floating point values (128 bit) might be
serialized to null as well.

Use std::is_floating_point to distinguish them from other types

Additionally write the byte width of the underlying type

Not yet used in reading

Mark the writable written after successfully extending a dataset

Remove support for absolute paths from openPath
  • Loading branch information
franzpoeschel committed Sep 13, 2018
1 parent 126a283 commit 60f1c58
Show file tree
Hide file tree
Showing 16 changed files with 2,924 additions and 28 deletions.
84 changes: 58 additions & 26 deletions include/openPMD/Datatype.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
*/
#pragma once


#include <array>
#include <cstdint>
#include <iosfwd>
Expand Down Expand Up @@ -76,38 +77,40 @@ enum class Datatype : int
template<
typename T,
typename U
>
struct decay_equiv :
>
struct decay_equiv :
std::is_same<
typename std::remove_pointer<
typename std::remove_cv<
typename std::decay<
typename std::remove_all_extents< T >::type
>::type
>::type
>::type,
typename std::remove_pointer<
typename std::remove_cv<
typename std::decay<
typename std::remove_all_extents< U >::type
>::type
>::type
typename std::remove_pointer<
typename std::remove_cv<
typename std::decay<
typename std::remove_all_extents< T >::type
>::type
>::type
>::type,
typename std::remove_pointer<
typename std::remove_cv<
typename std::decay<
typename std::remove_all_extents< U >::type
>::type
>::type
>::type
>::type
{ };
{
};

#if __cplusplus >= 201402L
template<
typename T,
typename U
>
constexpr bool decay_equiv_v = decay_equiv< T, U >::value;
template<
typename T,
typename U
>
constexpr bool decay_equiv_v = decay_equiv< T, U >::value;
#endif

template< typename T >
inline

template< typename T >
inline
#if __cplusplus >= 201402L
constexpr
constexpr
#endif
Datatype
determineDatatype()
Expand Down Expand Up @@ -150,7 +153,7 @@ determineDatatype()
template< typename T >
inline
#if __cplusplus >= 201402L
constexpr
constexpr
#endif
Datatype
determineDatatype(std::shared_ptr< T >)
Expand Down Expand Up @@ -517,6 +520,35 @@ isSame( openPMD::Datatype const d, openPMD::Datatype const e )

return false;
}

/**
* Generalizes switching over an openPMD datatype. Will call the functor passed
* to it using the C++ internal datatype corresponding to the openPMD datatype
* as template parameter for the templated <operator()>().
* @tparam ReturnType The functor's return type.
* @tparam Action The functor's type.
* @tparam Args The functors argument types.
* @param dt The openPMD datatype.
* @param action The functor.
* @param args The functor's arguments.
* @return The return value of the functor, when calling its <operator()>() with
* the passed arguments and the template parameter type corresponding to the
* openPMD type.
*/
template<
typename ReturnType = void,
typename Action,
typename ...Args
>
ReturnType switchType(
Datatype dt,
Action action,
Args && ...args
);

std::string datatypeToString( Datatype dt );

Datatype stringToDatatype( std::string s );
} // namespace openPMD

#if !defined(_MSC_VER)
Expand All @@ -541,4 +573,4 @@ namespace std
{
ostream&
operator<<(ostream&, openPMD::Datatype);
} // namespace std
} // namespace std
1 change: 1 addition & 0 deletions include/openPMD/IO/Format.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ enum class Format
HDF5,
ADIOS1,
ADIOS2,
JSON,
DUMMY
}; //Format
} // openPMD
27 changes: 27 additions & 0 deletions include/openPMD/IO/JSON/JSONFilePosition.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#pragma once


#include "openPMD/IO/AbstractFilePosition.hpp"
#include <stdint.h>
#include <nlohmann/json.hpp>


namespace openPMD
{


struct JSONFilePosition :
public AbstractFilePosition
{
using json = nlohmann::json;


JSONFilePosition( json::json_pointer id = json::json_pointer( ) ) :
id( id )
{}


json::json_pointer id;
};

} // openPMD
26 changes: 26 additions & 0 deletions include/openPMD/IO/JSON/JSONIOHandler.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#pragma once


#include "openPMD/IO/AbstractIOHandler.hpp"
#include "openPMD/IO/JSON/JSONIOHandlerImpl.hpp"


namespace openPMD
{
class JSONIOHandler :
public AbstractIOHandler
{
public:
JSONIOHandler(
std::string path,
AccessType at
);

virtual ~JSONIOHandler( );

std::future< void > flush( ) override;

private:
JSONIOHandlerImpl m_impl;
};
} // openPMD
Loading

0 comments on commit 60f1c58

Please sign in to comment.