Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADIOS2 Bool: Dataset Support #921

Open
ax3l opened this issue Feb 4, 2021 · 1 comment
Open

ADIOS2 Bool: Dataset Support #921

ax3l opened this issue Feb 4, 2021 · 1 comment

Comments

@ax3l
Copy link
Member

ax3l commented Feb 4, 2021

Describe the bug
It looks like bool types are currently only supported for attributes with our ADIOS2 backend.
Adding support for types with the same convention should be added, too.

To Reproduce

#include <openPMD/openPMD.hpp>

#include <array>
#include <iostream>
#include <memory>
#include <numeric>
#include <cstdlib>


using std::cout;
using namespace openPMD;

int main(int argc, char *argv[])
{
    // user input: size of matrix to write, default 3x3
    size_t size = (argc == 2 ? atoi(argv[1]) : 3);

    // matrix dataset to write with values 0...size*size-1
    std::array<bool, 3*3> global_data;
    //std::iota(global_data.begin(), global_data.end(), 0.);

    cout << "Set up a 2D square array (" << size << 'x' << size
         << ") that will be written\n";

    // open file for writing
    Series series = Series(
        "../samples/3_write_serial.bp",
        Access::CREATE
    );
    cout << "Created an empty " << series.iterationEncoding() << " Series\n";

    MeshRecordComponent rho =
      series
          .iterations[1]
          .meshes["rho"][MeshRecordComponent::SCALAR];
    cout << "Created a scalar mesh Record with all required openPMD attributes\n";

    Datatype datatype = determineDatatype<bool>();
    Extent extent = {size, size};
    Dataset dataset = Dataset(datatype, extent);
    cout << "Created a Dataset of size " << dataset.extent[0] << 'x' << dataset.extent[1]
         << " and Datatype " << dataset.dtype << '\n';

    rho.resetDataset(dataset);
    cout << "Set the dataset properties for the scalar field rho in iteration 1\n";

    series.flush();
    cout << "File structure and required attributes have been written\n";

    Offset offset = {0, 0};
    rho.storeChunk(shareRaw(global_data), offset, extent);
    cout << "Stored the whole Dataset contents as a single chunk, "
            "ready to write content\n";

    series.flush();
    cout << "Dataset content has been fully written\n";

    /* The files in 'series' are still open until the object is destroyed, on
     * which it cleanly flushes and closes all open file handles.
     * When running out of scope on return, the 'Series' destructor is called.
     */
    return 0;
}
Set up a 2D square array (3x3) that will be written
Created an empty groupBased Series
Created a scalar mesh Record with all required openPMD attributes
Created a Dataset of size 3x3 and Datatype BOOL
Set the dataset properties for the scalar field rho in iteration 1
terminate called after throwing an instance of 'std::runtime_error'
  what():  [ADIOS2] Trying to access dataset with unallowed datatype: BOOL
Aborted (core dumped)

Expected behavior
Let's apply the same convention as for attributes.

In order to avoid naming collisions, I would propose keep using __is_boolean__/ as prefix for the identifier for attributes and use another one, __is_boolean_var__/ for ADIOS variables.

Software Environment

  • version of openPMD-api: 0.13.2
  • installed openPMD-api via: from source
  • operating system: Ubuntu 20.04
  • machine: laptop
  • name and version of Python implementation: N/A
  • version of HDF5: N/A
  • version of ADIOS1: N/A
  • version of ADIOS2: 2.6.0
  • name and version of MPI: N/A

Additional context

In PIConGPU, we always just mapped bools to adios_unsigned_byte in ADIOS1 (BP3): ComputationalRadiationPhysics/picongpu#1756
This naturally means we cannot round-trip the type (we had no identifier).

I believe that for @PrometheusPi's particle mask flags with openPMD-api (radiation plugin), we do the same work-around at the moment. cc @franzpoeschel

@ax3l
Copy link
Member Author

ax3l commented Feb 4, 2021

Cross-linking: RFE in ADIOS2 for a distinct boolean type: ornladios/ADIOS2#2606

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant