pmemobj_open failing to open file after it has been copied?

[mdorier]

QUESTION: pmemobj_open fails to open file after it has been copied to another path?

Details

I have a code that creates a pmem pool using pmemobj_create, then writes and persists some data in it. It then copies the file to another path and use pmemobj_open to open the file at its new path. I can confirm that, before the call to pmemobj_open, the copy has exactly the same content as the original, and the same file permissions. The only thing that changes is its path. The call to pmemobj_open to open the copied file fails, with pmemobj_errormsg() returning "pool initialization failed". Additionally, after the call to pmemobj_open, the content of the file is different (some bytes have been changed). Any idea what the problem is?

[grom72]

Hi Matthieu (@mdorier) ,
Can you check the copied pool file with pmempool check <file_name> command? https://github.com/pmem/pmdk/blob/master/doc/pmempool/pmempool-check.1.md
Please attach the code that is used to open the pool.
What version of PMDK do you use?

[mdorier]

After executing the program, pmempool check doesn't find any error in both the original file and the copy. However if I run it at the same time my program is running (after pmemobj_open failed to open the copy but before pmemobj_close is called on the original file), pmempool check reports errors with the the original file (not with the copy):

error: Resource temporarily unavailable
error: checking consistency failed

Is it possible that pmemobj_open fails on the copy because the original file is still open?

EDIT: for the version of PMDK, I'm using 1.12.1. I can't attach the code as it's embedded in a large codebase with lots of dependencies, but I can try to write a small reproducer if needed.

[mdorier]

Here is a reproducer:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libpmemobj.h>

#define PMEM_FILE "/tmp/a.dat"
#define PMEM_FILE_COPY "/tmp/b.dat"
#define REGION_SIZE 256
#define DATA_SIZE 256

  /* Persistent structure definition */
  typedef struct {
      char data[DATA_SIZE];
  } persistent_data;

  int main() {
      PMEMobjpool *pop;
      persistent_data *pmem_data;

      // Create a new pool and allocate space for it
      if ((pop = pmemobj_create(PMEM_FILE, "CProgExample", 10 * 1024 * 1024, 0666)) == NULL) {
          perror("pmemobj_create");
          exit(1);
      }

      // Allocate a persistent object
      PMEMoid root = pmemobj_root(pop, sizeof(persistent_data));
      pmem_data = (persistent_data*)pmemobj_direct(root);

      // Write the letter 'A' 256 times
      memset(pmem_data->data, 'A', REGION_SIZE);
      pmemobj_persist(pop, pmem_data->data, REGION_SIZE);

      // Copy the file using standard file I/O
      FILE *src_file = fopen(PMEM_FILE, "rb");
      FILE *dst_file = fopen(PMEM_FILE_COPY, "wb");

      if (src_file == NULL || dst_file == NULL) {
          perror("File open error");
          exit(1);
      }

      char buffer[1024];
      size_t nread;

      while ((nread = fread(buffer, 1, sizeof(buffer), src_file)) > 0) {
          fwrite(buffer, 1, nread, dst_file);
      }

      fclose(src_file);
      fclose(dst_file);

      // Open the copied file using pmemobj_open
      PMEMobjpool *pop_copy = pmemobj_open(PMEM_FILE_COPY, "CProgExample");
      if (pop_copy == NULL) {
          fprintf(stderr, "%s\n", pmemobj_errormsg()) ;
          exit(1);
      }

      pmemobj_close(pop);
      pmemobj_close(pop_copy);

      return 0;
  }

I can confirm that if I call pmemobj_close(pop) before calling pmemobj_open on the copy, the latter works fine. So now my question is: how can I make pmemobj_open work properly even when the original is still open?

[grom72]

Please start your application with env PMEMOBJ_LOG_LEVEL=2. You should see the following line:
<libpmemobj>: <1> [obj.c:965 obj_runtime_init] critnib_insert to pools_ht: File exists which means you cannot open two pools with the same UUID.
The file you use for pmemobj operation is identified internally (in runtime) by UUID, not by a filename. The UUID is stored in the header section of the file and when you copy the file, you also copy the UUID.

[mdorier]

Is there a way to change the UUID of a file?

[grom72]

None of the PMDK tools and libraries support this directly.
We have not seen any value in such functionality.
What use case do you want to support with such "copied" data?

[mdorier]

I have a tool that copies a PMEM pool from a server to another. In typical scenarios it doesn't pose any problem as the copy is received by a different process, but in the unit tests of the tool, the copy is received by the process that sends it and this is where the problem appears. I was just wondering if there was a way I could make that test work without having to fork a new process (which is often difficult in unit tests). In practice though I don't think I have a use for opening the copy on the same process.

[grom72]

@mdorier , the best way would be to build some logic to copy data from one pool to another as part of the test setup.