Utilise `compute()`

[nodinosaur]

With Chopper is there a way that we can utilise Isolates / compute() for loading and parsing large documents?

https://flutter.dev/docs/cookbook/networking/background-parsing

https://api.flutter.dev/flutter/dart-isolate/Isolate-class.html

I have seen that some Devs are moving to Dio as it can already do this 👉 README.md, but I am quite invested in Chopper and don't really want to rework my network layer.

Is it possible, or could such a feature be added?

Thanks!

[nodinosaur]

I forgot to mention:
https://github.com/flutterchina/dio_flutter_transformer

[stewemetal]

We'll look into implementing this feature ASAP.

@JEuler @lejard-h we should look into how Dio does it (thanks @nodinosaur for the readme link!) and implement our own solution.

[lejard-h]

It should be possible via Converter but never tested

class FlutterConverter implements Converter {
    Future<Response> decodeJson<BodyType, InnerType>(Response response) {
      var contentType = response.headers[contentTypeKey];
      var body = response.body;
      if (contentType != null && contentType.contains(jsonHeaders)) {
        body = utf8.decode(response.bodyBytes);
      }

      body = _tryDecodeJson(body);
      if (isTypeOf<BodyType, Iterable<InnerType>>()) {
        body = body.cast<InnerType>();
      } else if (isTypeOf<BodyType, Map<String, InnerType>>()) {
        body = body.cast<String, InnerType>();
      }

      return response.copyWith<BodyType>(body: body);
    }
    
    ///
    /// Compute HERE
    ///
    Future<dynamic> _tryDecodeJson(String data) {
      try {
        return compute(json.decode, data);
      } catch (e) {
        chopperLogger.warning(e);
        return data;
      }
    }

    Future<Response<BodyType>> convertResponse<BodyType, InnerType>(
      Response response,
    ) async {
      return decodeJson<BodyType, InnerType>(response);
    }
}

I partially copied JsonConverter, it would be easier to implements if we had FutureOr on every JsonConverter functions and if _tryDecodeJson was public

class FlutterConverter extends JsonConverter {
  Future<dynamic> tryDecodeJson(String data) {
    return compute(tryDecodeJson, data);
  }
}

[DevNico]

Did anything ever get merged / what is the way to go if usage of compute is desired?

[nodinosaur]

I am still hoping that there will be some official documentation with a recommended example on this, I could not get the above example to work 🤔

[JanKn]

I got it working, I can see isolates being spawned and the data is loaded in our app.
Let me know if that works for you too @nodinosaur .

Two steps:

1. Creating ComputeConverter:

class ComputeConverter extends Converter {
  ComputeConverter(this.converter);

  final JsonSerializableConverter converter;

  @override
  FutureOr<Request> convertRequest(Request request) {
    return converter.convertRequest(request);
  }

  @override
  FutureOr<Response<BodyType>> convertResponse<BodyType, InnerType>(Response response) {
    return compute((Response response) => converter.convertResponse<BodyType, InnerType>(response), response);
  }
}

2. And then wrapping the JsonSerializableConverter with ComputeConverter in the Chopper Client setup:

final client = ChopperClient(
      baseUrl: baseUrl,
      converter: ComputeConverter(const JsonSerializableConverter(
        {
          XData: XData.fromJson,
          ...
        },
      )),
      interceptors: [
        HttpRequestLoggingInterceptor(),
      ],
    );

[vaetas]

Hey everybody,
I want to note one thing: compute starts a new Isolate for each call. Starting new Isolate may take between 100-200 milliseconds. Each Isolate also copies some memory therefore having many Isolates might increase memory load of your app.

Example from @JanKn works well if you have a demanding requests or response once in a while. If your responses are small, using compute might actually increase the real time required to process it (at least it will not block the UI isolate). And if you have many concurrent requests then compute will create many Isolates at the same time (therefore increasing memory footprint).

The best solution I found seems to be spawning one Isolate at the app startup and then using it only for converting requests & responses. This can be done using isolate package (that is for unknown reason discontinued, but I don't think it significantly matters).

You spawn the IsolateRunner in your main and then use it inside the converter. Spawning takes about 200ms (debug mode, iOS simulator).

late final IsolateRunner isolateRunner;

Future<void> main() async {
  isolateRunner = await IsolateRunner.spawn();
  runApp(const ExampleApp());
}

import 'package:isolate/isolate_runner.dart';

class _IsolateConverter extends Converter {
  _IsolateConverter(
    this.converter, {
    this.isolateRunner,
  });

  final JsonSerializableConverter converter;
  final IsolateRunner? isolateRunner;

  bool get runIsolate => isolateRunner != null;

  @override
  FutureOr<Request> convertRequest(Request request) {
    if (runIsolate) {
      return isolateRunner!.run(converter.convertRequest, request);
    } else {
      return converter.convertRequest(request);
    }
  }

  @override
  FutureOr<Response<BodyType>> convertResponse<BodyType, InnerType>(
    Response response,
  ) {
    if (runIsolate) {
      return isolateRunner!.run(
        (argument) {
          return converter.convertResponse<BodyType, InnerType>(
            argument as Response<dynamic>,
          );
        },
        response,
      );
    } else {
      return converter.convertResponse<BodyType, InnerType>(response);
    }
  }
}

My code example falls back to main Isolate when you do not provide custom IsolateRunner. I want to run some real-life tests with and without IsolateRunner in the future.

This IsolateRunner can probably be used for different computations so it is not idle when no JSON parsing is used. Again, this would probably depend on your use case.

There are also few packages for isolate (thread) pooling which allow you to spawn multiple Isolates at startup (say 2 or 4) and then split the work load between them. This seems overkill for my JSON parsing.

[thaihuynhxyz]

I think we should apply something like ThreadPool. Spam spawn Isolates can lead to Out of memory crashes. Take a look at squadron.

[techouse]

Any news on this one?

As @lejard-h pointed out above, it would be very helpful if all the JsonConverter methods had a FutureOr return type and the _tryDecodeJson were made public.

EDIT: I've started adding these types to a fork here https://github.com/techouse/chopper/tree/futureor-json-converter and will play around with compute on them tomorrow. This is however quite the breaking change as it will force people to change the return types from Response to FutureOr<Response> and migrate their code to async/await.

EDIT 2: My preliminary tests using the above fork seem to work fine with and without compute the only thing the user has to do is edit their JsonConverter's convertResponse and convertError to return FutureOr<Response<ResultType>> and FutureOr<Response> respectively.

I'll make a draft PR.

[JEuler]

Thank you so much for your contribution. I will have a look soon!

[techouse]

Thank you so much for your contribution. I will have a look soon!

Sure, no problem 😊 All the breaking changes are limited to classes extending JsonConverter so it's not that bad, unless one has many JsonConverters. 🙈