Unknown error calling "eio_open"

[hotrush]

I faced an issue with eio adapter. Simple snippet to reproduce:

$loop = \React\EventLoop\Factory::create();

$contents = '123';

$filesystem = \React\Filesystem\Filesystem::create($loop);
echo 'Using ', get_class($filesystem->getAdapter()), PHP_EOL;

$j = 0;

$loop->addPeriodicTimer(1, function () use ($filesystem, $contents, &$j) {
    for ($i = $j; $i <= $j+9; $i++) {
        $filename = __DIR__ . '/test/'.$i.'.txt';
        $filesystem->file($filename)->putContents($contents)->then(function () use ($filename) {
            echo $filename.' written'.PHP_EOL;
        }, function (Exception $e) {
            echo $e->getMessage(), PHP_EOL;
            echo $e->getTraceAsString(), PHP_EOL;
        });
    }
    $j += 10;
});

$loop->run();

First tick written ok, others cause an error:

php test.php 
Using React\Filesystem\Eio\Adapter
/root/www/test/1.txt written
/root/www/test/2.txt written
/root/www/test/3.txt written
/root/www/test/4.txt written
/root/www/test/5.txt written
/root/www/test/6.txt written
/root/www/test/7.txt written
/root/www/test/8.txt written
/root/www/test/9.txt written
/root/www/test/10.txt written
/root/www/test/11.txt written
Unknown error calling "eio_open"
#0 /root/www/vendor/react/filesystem/src/Eio/Adapter.php(385): React\Filesystem\Eio\Adapter->executeDelayedCall('eio_open', Array, -1, Object(React\Promise\Deferred))
#1 /root/www/vendor/react/filesystem/src/InstantInvoker.php(39): React\Filesystem\Eio\Adapter->callFilesystem('eio_open', Array, -1)
#2 /root/www/vendor/react/filesystem/src/Eio/Adapter.php(271): React\Filesystem\InstantInvoker->invokeCall('eio_open', Array)
#3 /root/www/vendor/react/promise/src/FulfilledPromise.php(25): React\Filesystem\Eio\Adapter->React\Filesystem\Eio\{closure}(NULL)
#4 /root/www/vendor/react/filesystem/src/Eio/Adapter.php(273): React\Promise\FulfilledPromise->then(Object(Closure))
#5 /root/www/vendor/react/filesystem/src/Node/File.php(123): React\Filesystem\Eio\Adapter->open('/root/www/avtos...', 'cw', 496)
#6 /root/www/vendor/react/filesystem/src/Node/File.php(161): React\Filesystem\Node\File->open('cw')
#7 /root/www/test.php(16): React\Filesystem\Node\File->putContents('123')
#8 /root/www/vendor/react/event-loop/src/Timer/Timers.php(89): {closure}(Object(React\EventLoop\Timer\Timer))
#9 /root/www/vendor/react/event-loop/src/StreamSelectLoop.php(177): React\EventLoop\Timer\Timers->tick()
#10 /root/www/test.php(26): React\EventLoop\StreamSelectLoop->run()
#11 {main}

Opened files number growing and when reach system limits causing other errors e.g. DNS query failed or ssl handshake failed.

Here is another strange thing - eio_open causes an error, but ls -l /proc/23566/fd shows opened files, e.g. (but no real files created):

l-wx------ 1 root root 64 Aug 27 20:08 10 -> /root/www/data/13703540.json
l-wx------ 1 root root 64 Aug 27 20:07 11 -> /root/www/data/13493179.json

It works fine at my local deepin linux (based on debian 8), but can not make it work at any server with ubuntu 16.04/18.04. Any ideas how to debug it?

php -v
PHP 7.2.9-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Aug 19 2018 07:16:12) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.9-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies

composer info
aws/aws-sdk-php                           3.64.12 AWS SDK for PHP - Use Amazon Web Services in your PHP project
cakephp/core                              3.6.10  CakePHP Framework Core classes
cakephp/utility                           3.6.10  CakePHP Utility classes such as Inflector, String, Hash, and Security
campo/random-user-agent                   1.3.0   Generate real random user-agents.
clue/buzz-react                           v2.3.0  Simple, async PSR-7 HTTP client for concurrently processing any number of HTTP requests, built on top of ReactPHP
clue/http-proxy-react                     v1.3.0  Async HTTP proxy connector, use any TCP/IP-based protocol through an HTTP CONNECT proxy server, built on top of ReactPHP
clue/socks-react                          v0.8.7  Async SOCKS4, SOCKS4a and SOCKS5 proxy client and server implementation, built on top of ReactPHP
doctrine/inflector                        v1.3.0  Common String Manipulations with regard to casing and singular/plural rules.
doctrine/instantiator                     1.1.0   A small, lightweight utility to instantiate objects in PHP without invoking their constructors
evenement/evenement                       v3.0.1  Événement is a very simple event dispatching library for PHP
guzzlehttp/guzzle                         6.3.3   Guzzle is a PHP HTTP client library
guzzlehttp/promises                       v1.3.1  Guzzle promises library
guzzlehttp/psr7                           1.4.2   PSR-7 message implementation that also provides common utility methods
hotrush/scrapoxy-react-client             1.1.0   Async client for Scrapoxy and ReactPHP
hotrush/stealer                           0.1.9   Async website scrapper tool like scrapy built with reactphp.
hotrush/stealer-scrapoxy-client           0.1.2  
indigophp/hash-compat                     v1.1.0  Backports hash_* functionality to older PHP versions
josegonzalez/dotenv                       2.1.0   dotenv file parsing for PHP
m1/env                                    2.1.2   Env is a lightweight library bringing .env file parser compatibility to PHP. In short - it enables you to read .env files with PHP.
monolog/monolog                           1.23.0  Sends your logs to files, sockets, inboxes, databases and various web services
mtdowling/jmespath.php                    2.4.0   Declaratively specify how to extract elements from a JSON document
nikic/fast-route                          v1.3.0  Fast request router for PHP
paragonie/random_compat                   v2.0.17 PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
psr/http-message                          1.0.1   Common interface for HTTP messages
psr/log                                   1.0.2   Common interface for logging libraries
react/cache                               v0.5.0  Async, Promise-based cache interface for ReactPHP
react/child-process                       v0.5.2  Event-driven library for executing child processes with ReactPHP.
react/dns                                 v0.4.15 Async DNS resolver for ReactPHP
react/event-loop                          v0.5.3  ReactPHP's core reactor event loop that libraries can use for evented I/O.
react/filesystem                          v0.1.1  Asynchronous filesystem abstraction.
react/http                                v0.8.3  Event-driven, streaming plaintext HTTP and secure HTTPS server for ReactPHP
react/http-client                         v0.5.9  Event-driven, streaming HTTP client for ReactPHP
react/promise                             v2.7.0  A lightweight implementation of CommonJS Promises/A for PHP
react/promise-stream                      v1.1.1  The missing link between Promise-land and Stream-land for ReactPHP
react/promise-timer                       v1.5.0  A trivial implementation of timeouts for Promises, built on top of ReactPHP.
react/socket                              v0.8.12 Async, streaming plaintext TCP/IP and secure TLS socket server and client connections for ReactPHP
react/stream                              v0.7.7  Event-driven readable and writable streams for non-blocking I/O in ReactPHP
ringcentral/psr7                          1.2.2   PSR-7 message implementation
symfony/css-selector                      v3.4.14 Symfony CssSelector Component
symfony/dom-crawler                       v3.4.14 Symfony DomCrawler Component
symfony/polyfill-ctype                    v1.9.0  Symfony polyfill for ctype functions
symfony/polyfill-mbstring                 v1.9.0  Symfony polyfill for the Mbstring extension
tivie/php-os-detector                     1.1.0   A small utility library that detects the OS the server is running on
wyrihaximus/cpu-core-detector             1.0.1  
wyrihaximus/json-throwable                2.0.0   JSON encode and decode throwables and exceptions
wyrihaximus/json-utilities                1.0.0   Utilities for php-json-* packages
wyrihaximus/react-child-process-messenger 2.8.2   Messenger decorator for react/child-process
wyrihaximus/react-child-process-pool      1.4.2  
wyrihaximus/react-child-process-promise   2.0.1   Wrapping ticks into a promise
wyrihaximus/react-guzzle-http-client      4.0.1   Asynchronous adapter for different Guzzle versions powered by react/http-client
wyrihaximus/react-guzzle-psr7             2.1.0   Asyncronous Guzzle (PSR7) adapter powered by react/http-client
wyrihaximus/ticking-promise               1.6.3   Wrapping ticks into a promise

[WyriHaximus]

EIO needs a little care some times, you can set a call invoker (React\Filesystem\CallInvokerInterface) when you try the call invokers in this package to see if it the errors go away.

[hotrush]

@WyriHaximus unfortunately can not make it works. Try my snippet with eio and periodic timer)

[WyriHaximus]

@hotrush will do it together with #18 (comment)

[lucasnetau]

I'm experiencing a similar issue with a simple create temp file, write, rename cycle within a periodic timer. Every few iterations eio_open returns with -1. I have trier various Invokers without luck. As per the initial reporter, I can see file handles been open in lsof output and eventually the number of open files limit is hit.

If I run a similar loop calling eio_* functions directly I cannot reproduce the issue.

Issue occurring on AWS EC2 Linux template, running the same code on MacOSX does not show the issue.

Pecl package eio version 2.0.4 and ev 1.0.4 installed on both systems. Running php 7.2.12

Failing code snippet:

$loop = React\EventLoop\Factory::create();

$filesystem = \React\Filesystem\Filesystem::create($loop, ['invoker' => 'React\Filesystem\QueuedInvoker']);
$saveHandler = $loop->addPeriodicTimer(4, function() use ($filesystem) {
        $filename = tempnam("/tmp", ".foobar.tmp");
        $file = $filesystem->file($filename);
        echo "Created Temp file" . PHP_EOL;
        $file->putContents(date("c"))
             ->then(function () use ($file, $filename, $filesystem) {
                 echo "Written to Temp file" . PHP_EOL;
                $file->rename("/tmp/foo.bar")
                     ->then(function (\React\Filesystem\Node\FileInterface $newfile) use ($filename, $filesystem)
                     {
                         echo "Renamed tmp file" . PHP_EOL;
                         }, function ($error) {
                         throw $error;
                        });
                     }, function ($error) {
                 throw $error;
             });
            });

/** GO! */
$loop->run();

EIO Code working snippet:

<?php

while(1)
{
    $path = tempnam("/tmp", "eio.tmp");
    eio_open($path, 65, 496, 0 , function ($path, $fd) {
        if ($fd === -1) {
            echo "fail";
        }
        else {
            eio_write($fd, "Hello");
            eio_close($fd);
            eio_rename($path, "/tmp/eio.txt");
            echo "done";
        }
    });
    eio_event_loop();

    usleep(10000);
}

[lucasnetau]

I realised my eio example wasn't the best and I re-wrote it to work the same way as react filesystem and can replicate the issue.

The call to eio_open is occasionally returning FALSE instead of the resource but then successfully calling the callback when it opens the file. In the basic eio example this then can continue to write to the file. However react filesystem cannot because the open promise has been rejected due to the failed call to eio_open().

Reading the doco http://php.net/manual/en/function.eio-open.php I'm not sure the check for the return value from eio_open is correct. The documentation doesn't define what the return result should be from the function, it only specifies that a failure would be a -1 in the result code sent to the callback

https://github.com/reactphp/filesystem/blob/master/src/Eio/Adapter.php#L407

[lucasnetau]

An update on this, I've been running for the last 5 days with

        if (!call_user_func_array($function, $args)) {
            $name = $function;
            if (!is_string($function)) {
                $name = get_class($function);
            }
            $exception = new RuntimeException('Unknown error calling "' . $name . '"');
            $exception->setArgs($args);
            $deferred->reject($exception);
        };

replaced by

call_user_func_array($function, $args)

So far, no leaking open file handles, files are getting successfully created, opened, written to, and renamed with not error seen.

The only downside to this is that you can't get a reference to the request if you wanted to cancel the request. Not sure if anyone actually want's to do such a thing (eg cancelling an open file request, there is a good chance the file will be opened before the request is cancelled). However if libeio sometimes returns failure on eio_open() but then successfully completes the action you have to handle it the best way.

For completeness I compiled pecl eio with the current version of libeio instead of the version from 2012 currently in the codebase, no change.

Eio is quite a diva. I've been running for a few days react/fs with eio, with a periodic timer (2mins interval) to write data to a single file (always overwriting). Every 3rd or 4th time I got an unknown function eio_open error, all day and night long. To not get my logs clogged, I've switched to the child process adapter for now.

In my newish project I'm using pthreads and thus am using my pthreads adapter (with an improved react/fs (#45)).

[Logioniz]

I have the same error: "Unknown error calling "eio_open".

From documentation

eio_open() returns file descriptor in result argument of callback on success; otherwise, result is equal to -1.

In my case eio sometimes return false instead of descriptor of file.

I also found an error when not the whole file is read:

1. create file

php -r 'foreach(range(0, 1000000) as $i) { print $i . PHP_EOL; }' > static/test.txt

2. create script

<?php

use \React\Promise\Deferred;

$loader = require __DIR__ . '/vendor/autoload.php';

#$loop = React\EventLoop\Factory::create();
$loop = new React\EventLoop\ExtEvLoop();

$filesystem = \React\Filesystem\Filesystem::create($loop);
    $file = $filesystem->file('static/test.txt');
    $file->exists()
        ->then(function () use ($file) {
            return $file->open('r');
        })
        ->then(function ($stream) use ($loop) {
            $d = new Deferred();
            $stream->on('data', function ($data) {
                print $data;
            });
            $stream->on('end', function ($data) use ($stream, $d) {
                $stream->close();
                $d->resolve('finish');
            });

            return $d->promise();
        })
        ->then(function () use ($loop) {
            $loop->stop();
        })
        ->otherwise(function ($e) {
            var_dump($e);
        });
$loop->run();

3. run

php7.3 3.php
1
2
3
...
375148
375149
3751root@7cf1e7fada2c:/app/adv# 

I create separate topic #73 about this bug.