Directory.listSync sometimes not returning files

[gmackall]

Context: flutter/flutter#139378

We have an integration test in the flutter/packages repo that creates a file, reads it, and then checks also that the directory contains the file (actual contents of the test here):

  final File file = File('${directory.path}/$name');
  ...
  file.writeAsStringSync('Hello world!');
  expect(file.readAsStringSync(), 'Hello world!');
  expect(directory.listSync(), isNotEmpty);

This test is flaking from time to time on the last expect. I don't believe that this is a timing issue (it shouldn't be because these are synchronous operations, but also I've tried adding a sleep just in case and still saw it flake) nor a permissions issue (the read for the file still succeeds, and also it flakes even locally on my machine where the permissions aren't changing).

I've also tried looping for some time and comparing the output of directory.listSync to Process.runSync('ls', [directory.path]), and even when the file is missing in the former it is still there in the output of the latter.

Is there something that I'm missing that is wrong with the test here, or might there be an issue with Directory.listSync itself?

---

If you aren't sure, file the issue here, and we'll find the right home for it.
In your issue, please include:

• Dart version and tooling diagnostic info (dart info)

If providing this information as part of reporting a bug, please review the information
below to ensure it only contains things you're comfortable posting publicly.

#### General info

- Dart 3.3.0-198.0.dev (dev) (Wed Dec 6 04:03:18 2023 -0800) on "macos_arm64"
- on macos / Version 14.1 (Build 23B74)
- locale is en

#### Process info

| Memory |  CPU | Elapsed time | Command line                                                                               |
| -----: | ---: | -----------: | ------------------------------------------------------------------------------------------ |
|  22 MB | 0.0% |  01-18:08:37 | dart language-server --client-id=Android-Studio --client-version=AI-222.4459.24 --protocol=analyzer |
|  14 MB | 0.0% |  01-18:08:36 | dart language-server --client-id=Android-Studio --client-version=AI-222.4459.24 --protocol=analyzer |
|  10 MB | 0.0% |  02-23:10:18 | dart language-server --client-id=Android-Studio --client-version=AI-222.4459.24 --protocol=analyzer |
|  20 MB | 0.2% |  01-18:08:39 | flutter_tools.snapshot daemon                                                              |
|  26 MB | 0.1% |     19:54:16 | flutter_tools.snapshot daemon                                                              |
|  38 MB | 0.3% |        24:11 | flutter_tools.snapshot daemon 

• Whether you are using Windows, macOS, or Linux (if applicable)
  Locally macOS, but flake is also happens on linux in CI.

Missing some or all of the above might make the issue take longer or
be impossible to act on.

If you simply have a question, please consider using the listed Dart
developer communities on the following page:

https://dart.dev/community#join-the-conversation

[mraleph]

cc @brianquinlan

[mraleph]

(this is most likely some Android specific wonkyness around permissions for external storage)

[brianquinlan]

I tried reproducing this with:

void _verifySampleFile(Directory? directory, String name) {
  if (directory == null) {
    return;
  }
  final File file = File('${directory.path}/$name');

  if (file.existsSync()) {
    file.deleteSync();
    assert(!file.existsSync());
  }

  file.writeAsStringSync('Hello world!');
  assert(file.readAsStringSync() == 'Hello world!');
  directory
      .listSync()
      .where((f) => FileSystemEntity.identicalSync(f.path, f.path))
      .first;
  file.deleteSync();
}

void test() async {
  for (var i = 0; i < 1000; ++i) {
    final List<StorageDirectory?> allDirs = <StorageDirectory?>[
      null,
      StorageDirectory.music,
      StorageDirectory.podcasts,
      StorageDirectory.ringtones,
      StorageDirectory.alarms,
      StorageDirectory.notifications,
      StorageDirectory.pictures,
      StorageDirectory.movies,
    ];

    for (final StorageDirectory? type in allDirs) {
      final List<Directory>? directories =
          await getExternalStorageDirectories(type: type);

      for (final Directory result in directories!) {
        _verifySampleFile(result, '$type');
      }
    }
  }
}

And ran this several time (each run tests the file creation 1,000 times) on the Android emulator, using various Android versions and could not reproduce.

Is this happening consistently? What happens if you change your test code like:

void _verifySampleFile(Directory? directory, String name) {
  expect(directory, isNotNull);
  if (directory == null) {
    return;
  }

  final File file = File('${directory.path}/$name');

  if (file.existsSync()) {
    file.deleteSync();
    expect(file.existsSync(), isFalse);
  }

  file.writeAsStringSync('Hello world!');
  expect(file.readAsStringSync(), 'Hello world!');
- expect(directory.listSync(), isNotEmpty);
+ expect(file.existsSync(), isTrue);
  file.deleteSync();
}

?

[brianquinlan]

Marking this as P2 because there is a workaround on the Flutter side.

[gmackall]

Is this happening consistently?

It is a relatively common flake, maybe 1 in 5 runs of that test shard?

What happens if you change your test code like:

void _verifySampleFile(Directory? directory, String name) {
  expect(directory, isNotNull);
  if (directory == null) {
    return;
  }

  final File file = File('${directory.path}/$name');

  if (file.existsSync()) {
    file.deleteSync();
    expect(file.existsSync(), isFalse);
  }

  file.writeAsStringSync('Hello world!');
  expect(file.readAsStringSync(), 'Hello world!');
- expect(directory.listSync(), isNotEmpty);
+ expect(file.existsSync(), isTrue);
  file.deleteSync();
}

?

I looped the tests with this modification 60 times and didn't see any flakes (did the same without the change and saw 9). I was just lazily using bash to loop so it was kind of slow, I can modify the tests to loop more times faster if we want more data

[brianquinlan]

Hmmm.... that's interesting so the file exists but listSync doesn't see it.

Is it possible that there is some sort of timing issue? If you delay one second before calling listSync then do you see this issue?

[brianquinlan]

Also, what's the intent of the test?