HTTP client: Simplify HTTP file uploads and submitting forms

[clue]

This library should provide better support for HTTP file uploads and submitting forms. Right now, both are supported just fine, but require manually constructing the outgoing HTTP request body. This is relatively easy for simple forms (see https://github.com/reactphp/http#post), but more challenging for forms with file uploads.

Handling file uploads is already supported on the server side (see #252, #220, #226 and others), so adding better support for this also on the client side makes a lot of sense. Implementing this is non-trivial, but doable. The major challenge right now would be evaluating possible API options and once settled, sponsoring/funding for this feature.

On top of this, we would have to consider possible API options for adding file uploads. In particular, non-blocking filesystem access remains challenging (fstat + fopen + fread), see also experimental reactphp/filesystem.

We welcome contributions, reach out if you want to support this project 👍

[clue]

For the reference, submitting forms currently works like this (see also https://github.com/reactphp/http#post):

// currently: form data without file uploads (application/x-www-form-urlencoded)
// arguably good enough for most simple cases
$http->post($url, ['Content-Type' => 'application/x-www-form-urlencoded'], http_build_query(['name' => 'Alice']));

// currently: form data without file uploads (multipart/form-data)
// reasonable, but non-trivial even for simple cases
$boundary = 'foo';
$body = "--$boundary\r\nContent-Disposition: form-data; name=\"name\"\r\n\r\nAlice\r\n--$boundary--\r\n";
$http->post($url, ['Content-Type' => 'multipart/form-data; boundary=' . $boundary], $body);

// currently: form data with file uploads (multipart/form-data)
// this should be easier: same idea as above, but much harder to construct streaming request body
// major challenge: need to make sure not the entire file is kept in memory at once
// bonus challenge: non-blocking filesystem access (fstat + fopen + fread)
$body = new ThroughStream();
$size = $sizeOfAllMultipartBoundaries + $sizeOfAllFiles;
$http->post($url, ['Content-Type' => 'multipart/form-data; boundary=' . $boundary, 'Content-Length' => "$size"], $body);

Here are some possible options what this could look like in the future (brainstorming here, any input is welcome!):

// suggestion 1: accept body as array and assume form data?
// challenge: always assume form data, what about JSON?
// bonus: similar to curl
$http->post($url, [], ['name' => 'Alice', 'avatar' => $file]);

// suggestion 2: new `withFormData()` method to prefill request body
// challenge: post accepts `$body` as string, should this overwrite body?
// bonus: future `withJson()` possible
$http->withFormData(['name' => 'Alice', 'avatar' => $file])->post($url);

// suggestion 3: pass `FormData` object as request body
// challenge: post accepts `$body` as string type, should this be Stringable?
// bonus: similar to JavaScript: https://developer.mozilla.org/en-US/docs/Web/API/FormData
$http->post($url, [], new FormData(['name' => 'Alice', 'avatar' => $file]));

On top of this, file uploads need some kind of API to reference files from the filesystem (or elsewhere/external). See note above, as non-blocking filesystem access remains challenging (fstat + fopen + fread). Here are some possible options what this could look like in the future (brainstorming here, any input is welcome!):

// suggestion 1: use native stream resources
// challenge: error handling via PHP warnings, low-level access, blocking operations
// challenge: filename can be derived from resource, but what about file content-type?
$file = fopen('avater.png', 'r');

// suggestion 2: reuse SplFileObject
// challenge: slightly better error handling, but still blocking operations
// challenge: filename can be derived from object, but what about file content-type?
$file = new \SplFileObject('avatar.png', 'r');

// suggestion 3: reuse PSR-7 `UploadedFileInterface`, add public method or constructor?
// challenge: designed for server side, so slightly different semantics on client side
// bonus: already supports explicit file content-type, allows wrapping low-level filesystem access
$file = UploadedFile::open('avatar.png');

Any input is welcome!

[Fneufneu]

that would be great!
I use React\Http in many project and i'm not sure how to properly stream a file in a POST request,
so today i use reactphp/child-process to do it with curl .
And amphp support this but i don't want to switch.

[shanginn]

so I wrote this wrapper to simplify this task and send multipart form data requests like this:

$client = (new Browser())->withTimeout(false);

return $client->postFormData(
    $url,
    new MultipartFormData([
        'chat_id' => $chatId,
        'photo' => __DIR__ . '/../assets/images/backside.jpeg',
    ])
);

It's dumb and not async, but it works.

<?php

declare(strict_types=1);

namespace Mystaro\Bot;

use Psr\Http\Message\ResponseInterface;
use React\Http\Browser as ReactBrowser;
use React\Promise\PromiseInterface;

class Browser extends ReactBrowser
{
    /**
     * Sends an HTTP Multipart Form Data POST request
     *
     * @param string $url
     * @param MultipartFormData $formData
     * @param array<string, string> $headers
     *
     * @return PromiseInterface<ResponseInterface>
     */
    public function postFormData(
        string $url,
        MultipartFormData $formData,
        array $headers = []
    ) {
        return $this->post(
            url: $url,
            headers: array_merge(
                $headers,
                $formData->getHeaders(),
            ),
            body: $formData->getBody()
        );
    }
}

<?php

declare(strict_types=1);

namespace Mystaro\Bot;

readonly class FormField
{
    public string $bodyContent;

    public function __construct(
        string $content,
        ?int $contentLength = null,
        ?string $contentType = null,
        ?string $filename = null,
    )
    {
        $data = '';

        if ($filename !== null) {
            $data .= "; filename=\"{$filename}\"";
        }

        $data .= "\r\n";

        if ($contentType !== null) {
            $data .= "Content-Type: {$contentType}\r\n";
        }

        if ($contentLength !== null) {
            $data .= "Content-Length: {$contentLength}\r\n";
        }

        $data .= "\r\n";
        $data .= $content;

        $this->bodyContent = $data;
    }
}

<?php

declare(strict_types=1);

namespace Mystaro\Bot;

use HttpException;

class MultipartFormData
{
    /** @var FormField[] */
    private array $fields = [];

    private string $boundary;

    private string $body;

    private int $size;

    public function __construct(
        array $fields = [],
        ?string $boundary = null
    )
    {
        try {
            $this->boundary = $boundary ?? \bin2hex(\random_bytes(16));
        } catch (\Exception $exception) {
            throw new HttpException('Failed to obtain random boundary', 0, $exception);
        }

        foreach ($fields as $name => $field) {
            if ($field instanceof FormField) {
                $this->fields[$name] = $field;
            } elseif (is_file($field)) {
                $this->addFile($name, $field);
            } else {
                $this->addField($name, $field);
            }
        }
    }

    public function addField(
        string $name,
        string $content,
        ?int $contentLength = null,
        ?string $contentType = null,
        ?string $filename = null,
    ): void
    {
        if (isset($this->body)) {
            unset($this->body);
        }

        if (isset($this->size)) {
            unset($this->size);
        }

        $this->fields[$name] = new FormField(
            content: $content,
            contentLength: $contentLength,
            contentType: $contentType,
            filename: $filename,
        );
    }

    public function addFile(string $name, string $path, ?string $contentType = null): void
    {
        if (!\is_file($path)) {
            throw new HttpException("File not found: {$path}");
        }

        $file = \fopen($path, 'r');
        if ($file === false) {
            throw new HttpException("Failed to open file: {$path}");
        }

        try {
            $info = \fstat($file);
            $content = \fread($file, $info['size']);
            $contentType = $contentType ?? (\mime_content_type($file) ?: null) ?? 'application/octet-stream';
        } catch (\Throwable $exception) {
            throw new HttpException("Failed to read file: {$path}", 0, $exception);
        } finally {
            \fclose($file);
        }

        $this->addField(
            name: $name,
            content: $content,
            contentLength: $info['size'],
            contentType: $contentType,
            filename: \basename($path),
        );
    }

    public function getBody(): string
    {
        if (!isset($this->body)) {
            $body = '';

            foreach ($this->fields as $name => $field) {
                $body .= <<<BODY
                    --{$this->boundary}\r
                    Content-Disposition: form-data; name="{$name}"{$field->bodyContent}\r

                    BODY;
            }

            $body .= "--{$this->boundary}--\r\n";

            $this->body = $body;
        }

        return $this->body;
    }

    private function getSize(): int
    {
        if (!isset($this->size)) {
            $this->size = \strlen($this->getBody());
        }

        return $this->size;
    }

    public function getHeaders(): array
    {
        return [
            'Content-Type'   => "multipart/form-data; boundary={$this->boundary}",
            'Content-Length' => $this->getSize(),
        ];
    }
}

[wpjscc]

hi , Based on the example above, a composer package was created that supports array and asynchronous file stream uploads

https://github.com/reactphp-x/multipart-form-data

[Fneufneu]

Looks really cool, thank you @wpjscc