Skip to content

Change Compressor and Decompressor to use more efficient streaming compression context (requires PHP 7+) #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,6 @@ dist: trusty

matrix:
include:
- php: 5.4
- php: 5.5
- php: 5.6
- php: 7.0
- php: 7.1
- php: 7.2
Expand Down
20 changes: 3 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ supporting compression and decompression of GZIP, ZLIB and raw DEFLATE formats.
* [Usage](#usage)
* [Compressor](#compressor)
* [Decompressor](#decompressor)
* [Inconsistencies](#inconsistencies)
* [Install](#install)
* [Tests](#tests)
* [License](#license)
Expand Down Expand Up @@ -162,17 +161,6 @@ $input->pipe($decompressor)->pipe($filterBadWords)->pipe($output);
For more details, see ReactPHP's
[`DuplexStreamInterface`](https://github.com/reactphp/stream#duplexstreaminterface).

### Inconsistencies

The stream compression filters are not exactly the most commonly used features of PHP.
As such, we've spotted some inconsistencies (or *bugs*) in different PHP versions.
These inconsistencies exist in the underlying PHP engines and there's little we can do about this in this library.

* All PHP versions: Decompressing invalid data does not emit any data (and does not raise an error)

Our test suite contains several test cases that exhibit these issues.
If you feel some test case is missing or outdated, we're happy to accept PRs! :)

## Install

The recommended way to install this library is [through Composer](https://getcomposer.org).
Expand All @@ -188,9 +176,7 @@ $ composer require clue/zlib-react:^0.2.2
See also the [CHANGELOG](CHANGELOG.md) for details about version upgrades.

This project aims to run on any platform and thus does not require any PHP
extensions besides `ext-zlib` and supports running on legacy PHP 5.4 through current
PHP 7+.
It's *highly recommended to use PHP 7+* for this project.
extensions besides `ext-zlib` and supports running on current PHP 7+.

The `ext-zlib` extension is required for handling the underlying data compression
and decompression.
Expand All @@ -200,8 +186,8 @@ builds by default. If you're building PHP from source, you may have to
[manually enable](https://www.php.net/manual/en/zlib.installation.php) it.

We're committed to providing a smooth upgrade path for legacy setups.
If you need to support legacy PHP 5.3 and legacy HHVM, you may want to check out
the legacy `v0.2.x` release branch.
If you need to support legacy PHP versions and legacy HHVM, you may want to
check out the legacy `v0.2.x` release branch.
This legacy release branch also provides an installation candidate that does not
require `ext-zlib` during installation but uses runtime checks instead.

Expand Down
3 changes: 1 addition & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,8 @@
}
],
"require": {
"php": ">=5.4",
"php": ">=7.0",
"ext-zlib": "*",
"clue/stream-filter": "~1.3",
"react/stream": "^1.0 || ^0.7 || ^0.6"
},
"require-dev": {
Expand Down
36 changes: 31 additions & 5 deletions src/Compressor.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,18 +31,44 @@
* For more details, see ReactPHP's
* [`DuplexStreamInterface`](https://github.com/reactphp/stream#duplexstreaminterface).
*/
final class Compressor extends ZlibFilterStream
final class Compressor extends TransformStream
{
/** @var ?resource */
private $context;

/**
* @param int $encoding ZLIB_ENCODING_GZIP, ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
* @param int $level optional compression level
*/
public function __construct($encoding, $level = -1)
{
parent::__construct(
Filter\fun('zlib.deflate', array('window' => $encoding, 'level' => $level))
);
$context = @deflate_init($encoding, ['level' => $level]);
if ($context === false) {
throw new \InvalidArgumentException('Unable to initialize compressor' . strstr(error_get_last()['message'], ':'));
}

$this->context = $context;
}

protected function transformData($chunk)
{
$ret = deflate_add($this->context, $chunk, ZLIB_NO_FLUSH);

if ($ret !== '') {
$this->emit('data', [$ret]);
}
}

protected function transformEnd($chunk)
{
$ret = deflate_add($this->context, $chunk, ZLIB_FINISH);
$this->context = null;

if ($ret !== '') {
$this->emit('data', [$ret]);
}

$this->emptyWrite = $encoding;
$this->emit('end');
$this->close();
}
}
43 changes: 39 additions & 4 deletions src/Decompressor.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,50 @@
* For more details, see ReactPHP's
* [`DuplexStreamInterface`](https://github.com/reactphp/stream#duplexstreaminterface).
*/
final class Decompressor extends ZlibFilterStream
final class Decompressor extends TransformStream
{
/** @var ?resource */
private $context;

/**
* @param int $encoding ZLIB_ENCODING_GZIP, ZLIB_ENCODING_RAW or ZLIB_ENCODING_DEFLATE
*/
public function __construct($encoding)
{
parent::__construct(
Filter\fun('zlib.inflate', array('window' => $encoding))
);
$context = @inflate_init($encoding);
if ($context === false) {
throw new \InvalidArgumentException('Unable to initialize decompressor' . strstr(error_get_last()['message'], ':'));
}

$this->context = $context;
}

protected function transformData($chunk)
{
$ret = @inflate_add($this->context, $chunk);
if ($ret === false) {
throw new \RuntimeException('Unable to decompress' . strstr(error_get_last()['message'], ':'));
}

if ($ret !== '') {
$this->emit('data', [$ret]);
}
}

protected function transformEnd($chunk)
{
$ret = @inflate_add($this->context, $chunk, ZLIB_FINISH);
$this->context = null;

if ($ret === false) {
throw new \RuntimeException('Unable to decompress' . strstr(error_get_last()['message'], ':'));
}

if ($ret !== '') {
$this->emit('data', [$ret]);
}

$this->emit('end');
$this->close();
}
}
96 changes: 19 additions & 77 deletions src/TransformStream.php
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ public function write($data)

return true;
} catch (Exception $e) {
$this->forwardError($e);
$this->emit('error', [$e]);
$this->close();
return false;
}
}
Expand All @@ -53,7 +54,8 @@ public function end($data = null)
}
$this->transformEnd($data);
} catch (Exception $e) {
$this->forwardError($e);
$this->emit('error', [$e]);
$this->close();
}
}

Expand Down Expand Up @@ -107,109 +109,49 @@ public function pipe(WritableStreamInterface $dest, array $options = array())
return $dest;
}

/**
* Forwards a single "data" event to the reading side of the stream
*
* This will emit an "data" event.
*
* If the stream is not readable, then this is a NO-OP.
*
* @param string $data
*/
protected function forwardData($data)
{
if (!$this->readable) {
return;
}
$this->emit('data', array($data));
}

/**
* Forwards an "end" event to the reading side of the stream
*
* This will emit an "end" event and will then close this stream.
*
* If the stream is not readable, then this is a NO-OP.
*
* @uses self::close()
*/
protected function forwardEnd()
{
if (!$this->readable) {
return;
}
$this->readable = false;
$this->writable = false;

$this->emit('end');
$this->close();
}

/**
* Forwards the given $error message to the reading side of the stream
*
* This will emit an "error" event and will then close this stream.
*
* If the stream is not readable, then this is a NO-OP.
*
* @param Exception $error
* @uses self::close()
*/
protected function forwardError(Exception $error)
{
if (!$this->readable) {
return;
}
$this->readable = false;
$this->writable = false;

$this->emit('error', array($error));
$this->close();
}

/**
* can be overwritten in order to implement custom transformation behavior
*
* This gets passed a single chunk of $data and should invoke `forwardData()`
* This gets passed a single chunk of $data and should emit a `data` event
* with the filtered result.
*
* If the given data chunk is not valid, then you should invoke `forwardError()`
* or throw an Exception.
* If the given data chunk is not valid, then you should throw an Exception
* which will automatically be turned into an `error` event.
*
* If you do not overwrite this method, then its default implementation simply
* invokes `forwardData()` on the unmodified input data chunk.
* If you do not overwrite this method, then its default implementation
* simply emits a `data` event with the unmodified input data chunk.
*
* @param string $data
* @see self::forwardData()
*/
protected function transformData($data)
{
$this->forwardData($data);
$this->emit('data', [$data]);
}

/**
* can be overwritten in order to implement custom stream ending behavior
*
* This may get passed a single final chunk of $data and should invoke `forwardEnd()`.
* This may get passed a single final chunk of $data and should emit an
* `end` event and close the stream.
*
* If the given data chunk is not valid, then you should invoke `forwardError()`
* or throw an Exception.
* If the given data chunk is not valid, then you should throw an Exception
* which will automatically be turned into an `error` event.
*
* If you do not overwrite this method, then its default implementation simply
* invokes `transformData()` on the unmodified input data chunk (if any),
* which in turn defaults to invoking `forwardData()` and then finally
* invokes `forwardEnd()`.
* which in turn defaults to emitting a `data` event and then finally
* emits an `end` event and closes the stream.
*
* @param string $data
* @see self::transformData()
* @see self::forwardData()
* @see self::forwardEnd()
*/
protected function transformEnd($data)
{
if ($data !== '') {
$this->transformData($data);
}
$this->forwardEnd();

$this->emit('end');
$this->close();
}
}
65 changes: 0 additions & 65 deletions src/ZlibFilterStream.php

This file was deleted.

14 changes: 14 additions & 0 deletions tests/CompressorTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?php

use Clue\React\Zlib\Compressor;

class CompressorTest extends TestCase
{
/**
* @expectedException InvalidArgumentException
*/
public function testCtorThrowsForInvalidEncoding()
{
new Compressor(0);
}
}
Loading