File-related syscalls, like readdir, can not handle empty directories properly

[hly2019]

Please include the following in your bug report:

Version of emscripten/emsdk:
I'm using emcc 3.1.54, node version v20.11.0.

Failing command line in full:

emcc test_emptydir.cpp --preload-file ./dir@./dir  -o test_emptydir.js
node test_emptydir.js

Full link command and output with -v appended:
Even for runtime issues it helps a lot if you can include the full link command.
Adding -v to the link command will show all of the sub-commands run which
can help us diagnose your issue.

The description of issue

I'm trying to print all the files and directories under the ./dir directory using readdir. Here inside the ./dir I have a empty directory with path ./dir/empty_dir. It seems this empty directory is not captured and handled by readdir.

Here's the test_emptydir.cpp

#include <iostream>
#include <dirent.h>

int main() {
    dirent* _entry;
    DIR* _dir = ::opendir("./dir");
    if(!_dir) {
        std::cout << "dir is null" << std::endl;
    }
    while((_entry = ::readdir(_dir)) != nullptr) { 
        std::cout << _entry->d_name << std::endl;
    }
    return 0;
}

Here's the content inside the ./dir directory:

./dir/test.txt
./dir/empty_dir/
./dir/non_empty_dir/
./dir/non_empty_dir/test_2.txt

Where ./dir/test.txt is an empty file, ./dir/empty_dir/ is an empty directory, and ./dir/non_empty_dir/ is an non-empty directory

By compiling with emcc test_emptydir.cpp --preload-file ./dir@./dir -o test_emptydir.js and run node test_emptydir.js, I get the result:

.
..
non_empty_dir
test.txt

As the result shows, the empty directory ./dir/empty_dir is not printed.

Similar problem happens on other file-related syscalls like lstat. For example, if I compile and run the following code with the same command line emcc test_lstat.cpp --preload-file ./dir@./dir -o test_lstat.js and node test_lstat.js:

#include <iostream>
#include <sys/stat.h>
#include <stdio.h>

int main() {
    struct ::stat st;
    auto result = ::lstat("./dir/empty_dir", &st);
    std::cout << "is dir?: " << S_ISDIR(st.st_mode) << std::endl;
    return 0;
}

I get the result:

is dir?: 0

Actually, it seems emscripten --preload-file cannot solely load an empty directory. For example, if I run following command:
emcc test_emptydir.cpp --preload-file ./dir/empty_dir/@./dir/empty_dir -o test_emptydir.js
I get compilation error:

Nothing to do!
emcc: error: '/data/emsdk/upstream/emscripten/tools/file_packager test_emptydir.data --from-emcc --preload ./dir/empty_dir/@./dir/empty_dir' failed (returned 1)

So it seems the problem could be derived from the strategy that emscripten does not preload empty directories during compilation time, and such operation will be restricted.

However, if we load an empty directory within a non-empty directory (or just preload it together with other non-empty files/directories), then it will cause bugs during runtime like mentioned above.

[sbc100]

Yes, it looks like the file packager currently ignores empty directories. If you create the empty directory at runtime (e.g. using mkdir from C or FS_createPath from JS) then I assume it will work.

Is it possible for you to work around this by using FS_createPath before your program starts or should we look into extending the file packager to create empty directories?

[hly2019]

Yes, it looks like the file packager currently ignores empty directories. If you create the empty directory at runtime (e.g. using mkdir from C or FS_createPath from JS) then I assume it will work.

Is it possible for you to work around this by using FS_createPath before your program starts or should we look into extending the file packager to create empty directories?

Thank you very much! Your advice is very helpful! Actually I think it would be great if the file packager could be extended so that we don't need to consider the issue at runtime. Though it's helpful, it's not always possible or convenient to use FS.mkdir or FS_createPath.

Thanks a lot and best regards!