Recursive Directory list causes dart to abort

[bsutton]

I have the following code which I'm using to do a scan of the complete file system under linux.

It appears that when the recursive list iterates through /proc (or in that area) that an unexpeted st_mode (384) is returned and the dart aot aborts.

To reproduce the problem:

Save the following code to listing.dart

import 'dart:io';

void main(List<String> args) {
  var lister = Directory('/').list(recursive: true);
  lister.listen((entity) {
    print(entity.path);
  });
}

Compile the script:

dart2native listing.dart
Generated: /tmp/temp/listing.exe

Run the code using sudo

sudo ./listing.exe

....
/proc/slabinfo
/proc/softirqs
../../runtime/bin/directory_linux.cc: 204: error: Unexpected st_mode: 384

/proc/zoneinfo
/proc/buddyinfo
/proc/diskstats
/proc/key-users
....

System information:

dcli doctor
DCli version      0.21.0                                                 

OS                linux                                                  
OS Version        Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
Path separator    /                                                      

dart version      2.9.1                                                  

dcli path         /<HOME>/.pub-cache/bin/dcli                            
dart exe path     /usr/bin/dart                                          
dart path         /usr/bin/dart                                           which: /usr/bin/dart                                   
dart2Native path  /usr/lib/dart/bin/dart2native                           which: /usr/lib/dart/bin/dart2native                   

pub path          /usr/lib/dart/bin/pub                                   which: /usr/lib/dart/bin/pub                           
Pub cache         /<HOME>/.pub-cache                                     

Package Config    /tmp/temp/file:/<HOME>/git/dcli/.packages              

PATH
                  /usr/local/sbin                                        
                  /usr/local/bin                                         
                  /usr/sbin                                              
                  /usr/bin                                               
                  /sbin                                                  
                  /bin                                                   
                  /usr/games                                             
                  /usr/local/games                                       
                  /snap/bin                                              
                  /usr/lib/dart/bin                                      
                  /<HOME>/apps/flutter/bin                               
                  /<HOME>/.pub-cache/bin                                 
                  /<HOME>/.dcli/bin                                      

$SHELL            /bin/bash                                              
Detected SHELL    bash                                                   
Shell Start Scrip Not Found                                              

Dart location(s)
                  /usr/bin/dart                                          

Permissions
HOME              rwxr-xr-x <user>:<user> /<HOME> 
.dcli             rwxrwxr-x <user>:<user> /<HOME>/.dcli 
cache             rwxrwxr-x <user>:<user> /<HOME>/.dcli/cache 
dependencies.yaml rw-rw-r-- <user>:<user> /<HOME>/.dcli/dependencies.yaml 
templates         rwxrwxr-x <user>:<user> /<HOME>/.dcli/templates 

[bsutton]

I managed to produce a variation on the above error without the use of sudo.

void main(List<String> args) {
  var lister = Directory('/').list(recursive: true);

  lister.listen((entity) {
  })
   .onError((Object e, StackTrace st) {
    print(e);
    print(st);
  });

Yields:

FileSystemException: Directory listing failed, path = '/lib/ssl/private' (OS Error: Permission denied, errno = 13)

FileSystemException: Directory listing failed, path = '/proc/tty/driver' (OS Error: Permission denied, errno = 13)

../../runtime/bin/directory_linux.cc: 204: error: Unexpected st_mode: 384

version=2.9.1 (stable) (Unknown timestamp) on "linux_x64"
pid=574321, thread=574327, isolate_group=(nil)((nil)), isolate=(nil)((nil))
isolate_instructions=0, vm_instructions=559e77f5ff80
  pc 0x0000559e781e9eac fp 0x00007fc063efaf90 dart::Profiler::DumpStackTrace(void*)+0x7c
  pc 0x0000559e77f600b2 fp 0x00007fc063efb070 dart::Assert::Fail(char const*, ...)+0x82
  pc 0x0000559e78089d5a fp 0x00007fc063efb1f0 /usr/lib/dart/bin/dart+0x1a57d5a
  pc 0x0000559e78087d0c fp 0x00007fc063efb240 dart::bin::Directory::List(dart::bin::DirectoryListing*)+0x7c
  pc 0x0000559e7808894f fp 0x00007fc063efb270 dart::bin::Directory::ListNextRequest(dart::bin::CObjectArray const&)+0x11f
  pc 0x0000559e77f4f891 fp 0x00007fc063efb2c0 dart::bin::IOServiceCallback(long, _Dart_CObject*)+0x3a1
  pc 0x0000559e78164bf3 fp 0x00007fc063efbc30 dart::NativeMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)+0xd3
  pc 0x0000559e78162836 fp 0x00007fc063efbca0 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x146
  pc 0x0000559e78162eea fp 0x00007fc063efbd00 dart::MessageHandler::TaskCallback()+0x1da
  pc 0x0000559e7826d8c8 fp 0x00007fc063efbd80 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x148
  pc 0x0000559e7826dd9c fp 0x00007fc063efbdb0 dart::ThreadPool::Worker::Main(unsigned long)+0x5c
  pc 0x0000559e781e605d fp 0x00007fc063efbe70 /usr/lib/dart/bin/dart+0x1bb405d
-- End of DumpStackTrace
Aborted (core dumped)

[devoncarew]

@bsutton, thanks for the report!

I see that you can reproduce this when the repro case is compiled via AOT; does this also reproduce when just run via dart (using JIT)?

[bsutton]

The second example i provided in the comments occurs without compiling and without Sudo.

…

On Wed, 26 Aug 2020, 6:03 am Devon Carew, ***@***.***> wrote: @bsutton <https://github.com/bsutton>, thanks for the report! I see that you can reproduce this when the repro case is compiled via AOT; does this also reproduce when just run via dart (using JIT)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG32ODOEVCBYN5YLW7ROFLSCQKHXANCNFSM4QKTMVNA> .

[zichangg]

It broke due to unrecognized mask. I don't think this is a bug of us.

st_mode: 384 (which is 0600) doesn't fall into any valid category (Reg file, links, directories, sockets and so on). Checkout the man page: https://man7.org/linux/man-pages/man7/inode.7.html

Can you identify the file (It is probably not even a file though) that causing the problem? @bsutton

[bsutton]

Its definitely a bug.
You can't have something as simple as a Directory.list causing dart to abend.

I believe the /proc directory is where linux mounts hardware.

https://man7.org/linux/man-pages/man5/proc.5.html

I have a vague recollection that proc uses additional bits but this isn't my area of expertise.

For what its worth I'm running a zfs file system but I'm sceptical that this is the issue.

I can't tell which file it is given as the above logs fail to report the filename.

If I run the stat command across the directory it reports no errors.

I've had a go at running a stat command over the directory but I'm not quite certain what I'm looking for.

If you have a command you would like me to run, happy to do so.

[zichangg]

I don't think there is an easy way to debug. You have to write a simple c program. Here is a pseudo code version, which using almost the same system calls the VM uses.

// Open the directory
int dir_fd = opendir(pathname);

// Iterate all elements in the directory.
dirent* entry = readdir(dir_fd);

// entry->d_name is the file name

// Use fstatat() or stat() to check the result
int result = stat(path, &stat);
if (stat->st_mode == 0600) {
  // This is the entity we are looking for
  // In the VM, we throw the error.
}

[bsutton]

Looks like some files in the /dev directory will yield the same error.

[bsutton]

I've just reproduced the problem on a second pc.

The PC is running mandaro and ext4.

The crash also happens in the /proc directory on the above system.

So I think its safe to conclude:
This isn't a problem with the zfs file system.
This isn't a problem with a corrupt file system.

[bsutton]

Looks like some files in /snap and /run show the same symptoms.