[LazyImage] data_uri_thumbnail is not production ready

[phtmgt]

Even at smaller dimensions (100 pixels) data_uri_thumbnail is bringing down my server to its knees with very low traffic. I have pages that need to process 15-20 images and apparently this is not sustainable. I get 5 page views and the server is bogged down tremendously. I don't know about others' experience, but this does not work for me at all.

An alternative would be to create a custom liip_imagine filter that implements Blurhash and use it instead of data_uri_thumbnail. This way we will have image caching.

Not complaining here, just suggesting a note should be added to docs, so people know data_uri_thumbnail is not suitable for production use. I spent more than a few hours trying to figure out what's going on.

[weaverryan]

I agree - this is not good enough. The BlurHash implementation needs to have caching built-into it: https://github.com/symfony/ux/blob/1.x/src/LazyImage/BlurHash/BlurHash.php

In other words, not a docs issue - I think it's a bug that should be fixed.

[ToshY]

Hey @plamenh and @weaverryan 👋

I just want to add my two cents to this topic. I also have a lot of images shown on several pages, thus I came to the same conclusion that data_uri_thumbnail is really not that suitable, because like stated in the docs, making blurhashes is CPU-intensive.
BUT what you can do is pre-generating the blurhash and/or dataUri instead and cache them.

Here's a "simplification" on how I tackled the problem:

1. Firstly, create a console command, let's say BlurHashGenerateCommand.
2. In BlurHashGenerateCommand, I'll dispatch a job/message called CreateBlurHashMessage, which requires a file/url and cache key (specific for the image) as arguments.
3. In the corresponding handler CreateBlurHashMessageHandler, you'll need at least 3 things in your constructor:
   1. BlurHashInterface $blurHash. This interface contains the underlying methods to create the dataUri createDataUriThumbnail and blurhash encode.
   2. CacheInterface $appBlurhashCache. As defined in the cache.yml, I'm using a cache pool app.blurhash.cache, which uses a Redis provider.
   3. BlurHashService $blurHashService, a custom service which contains the actual getting/creating of blurhash.

As an example I'll use dimensions of 50x71¹, which roughly correspond to the aspect ratio of my original images 300x427.

• Option 1 | Blurhash only

// BlurHashService
public function getBlurHash(string $url, string $cacheKey): void
{
    return $this->appBlurhashCache->get(
        $cacheKey,
        function () use ($url) {
            return $this->blurHash->encode(
                $url,
                50,
                71,
            );
        }
    );
}

• Option 2 (preferred) | Serialized (custom) model BlurHashModel having properties $blurHash and $dataUri.

// BlurHashService
public function getBlurHashModel(string $url, string $cacheKey): void
{
    return unserialize(
        $this->appBlurhashCache->get(
            $cacheKey,
            function () use ($url) {
                $blurHash = $this->blurHash->encode(
                    $url,
                    50,
                    71,
                );
                $dataUri = $this->getDataUriThumbnailByBlurHash($blurHash, $cacheKey);
                return serialize(
                    new BlurHashModel(
                        $blurHash,
                        $dataUri
                    )
                );
            }
        )
    );
}

/**
* This is almost the same implementation as the 'createDataUriThumbnail' method from the BlurHashInterface, 
* except it directly uses the blurhash as argument instead.
*/
public function getDataUriThumbnailByBlurHash(string $blurHash, int $width, int $height): string
{
  $pixels = BlurhashEncoder::decode(
      $blurHash,
      $width,
      $height
  );

  $thumbnail = $this->imageManager->canvas(
      $width,
      $height
  );
  for ($y = 0; $y < $height; ++$y) {
      for ($x = 0; $x < $width; ++$x) {
          $thumbnail->pixel($pixels[$y][$x], $x, $y);
      }
  }

  return sprintf(
      'data:image/jpeg;base64,%s',
      base64_encode((string)$thumbnail->encode('jpg', 80))
  );
}

7. Now in the __invoke method (still in CreateBlurHashMessageHandler), you simply call the service.

public function __invoke(CreateBlurHashMessage $message): void
{
    $url = $message->getUrl();
    $cacheKey = $message->getCacheKey();

    // Option 2
    $this->blurHashService->getBlurHashModel($url, $cacheKey);
}

8. Kick of the BlurHashGenerateCommand to fill your cache with blurhashes/dataUris.
9. Now in your controller/twig extension you can simply use the BlurHashService again and call the getBlurHashModel to retrieve the blurhash/dataUri.
10. Instead of using data_uri_thumbnail in your template, you use your own dataUri in the src.

Bonus:

• If you don't want to use "Option 2" or do not want to use the dataUri in cases like using it in a JSON response, and want to decode the blurhash on the frontend, you can do that by using the decode JS implementation (in combination with array-to-image to create a dataUri).
• CSS aspect-ratio is a nice alternative if you don't want to use fixed width/height.

---

Summary

Generation of blurhash/dataUri on-the-fly is a no-go. Therefore pre-generating seems a good way to overcome this problem. The blurhash/dataUri creation still takes the same time per image in a job, but with the usage of multiple workers² jobs can be processed a lot faster.

To put the results in perspective. Before it took +/- 5-6 seconds for 28 images (+/- 200ms per image) to get the dataUri to show on-the-fly. Now to get a single serialized model (+/- 1.8Kb) from Redis takes only +/- 0.2-0.3 ms. Think it's safe to say this will definitely improve the render time.

Hope this will help others struggling with the same problem 😄

Footnotes

1. "For decoding, you can use small placeholder images of 20-32 pixels width and let the UI layer scale them up" ↩

2. If using for example docker-compose and have a separate service for handling messages, e.g. workers, you can significantly speed things up by scaling it to have more containers: docker compose up -d workers --scale workers=10. ↩

[carsonbot]

Thank you for this issue.
There has not been a lot of activity here for a while. Has this been resolved?

[phtmgt]

@ToshY, thanks for the guide. Unfortunately, this seems like too much of a hassle to implement at this point. I decided on serving 1x1 pixel white SVG instead.

Some suggestion on my side: I have user-uploadable images and a command will not cut it. If I were to do it, I would just create a MessageHandler that generates the BlurHash for the image via a Service and dispatch the Message via an EntityListener (postPersist).

[kbond]

Does #1755 help with this at all?

[carsonbot]

Thank you for this issue.
There has not been a lot of activity here for a while. Has this been resolved?