Understanding ReArch Fundamentals

[rubenferreira97]

Firstly, I wanted to acknowledge the excellent work on Rearch! It's impressive how Rearch approaches state management differently.
Keep up the fantastic work, and thanks for encouraging developers to think in a different way about this topic. Looking forward to a macro-based implementation to simplify the development.

---

As a flutter developer, I normally want the following scenarios solved by a state management library:

a) Create local state for each new widget instance (Ephemeral/Local state).

1. State should be created and bounded to each new widget instance.
2. State should be disposed when the bounded widget is removed from the widget tree.
3. State should not be shared across multiple widgets in the widget tree.
4. Each widget should have its own state, even if they are of the same type. For example, a widget instance named FooWidget should have its own state that is not shared with another FooWidget instance in the widget tree.
5. Changes to this state should trigger a rebuild of the bounded widget.

b) Share state globally across multiple widgets in the widget tree (App/Global State).

1. State should be created once and shared across multiple widgets in the widget tree.
2. State should not be disposed when the dependent widgets are removed from the widget tree.
3. State should be shared across multiple widgets in the widget tree. For example, a widget named FooWidget should have its state shared with another BarWidget instance in the widget tree.
4. Changes to this state should trigger a rebuild to the app (or if optimized, only to the dependent widgets).

c) Share state scoped to a "zone".

1. State should be created once by zone.
2. State should be disposed when the zone is removed from the widget tree.
3. State can be shared across any widget present in the zone.
4. Changes to this state should trigger a rebuild to the zone (or if optimized, only to the dependent widgets).

d) Share state scoped to a specific set of widgets in the widget tree.

1. State should be created once by the first widget in the scope.
2. State should be disposed when all widgets in the scope are removed from the widget tree.
3. State should be shared across multiple widgets in the scope. For example, a widget named FooWidget could have this state shared with another BarWidget instance.
4. Changes to this state should trigger a rebuild of the dependent widgets.

Given these scenarios I created a simple example project to try to solve these using Rearch.

Repository: Rearch Example

Disclaimer: This example project was written without ever touching Rearch. It's for learning purposes, and there are definitely some bad implementations here. I am seeking feedback and answers to some questions I have about it. These are mainly written in comments.

---

I understand that reviewing a project can be time-consuming, so I've prepared a set of TLDR (Too Long; Didn't Read) questions while writing this. I'd appreciate it if somebody could spare a moment to address any of them.

Understanding Rearch architecture by asking practical questions:

1. How are capsules created on Flutter? Are they created on the first use in a RearchConsumer? Are they stored on RearchBootstrapper container?
2. If a RearchConsumer is removed from the widget tree and added again, is the capsule disposed and recreated (if is the only useing it)? Or is the capsule cached and reused? In other words, how is managed a capsule lifecycle?
3. If multiple RearchConsumer use the same capsule, are they sharing the same instance or is a new instance created for each widget?
4. Would be reasonable to have a RearchZone/RearchScope widget that replaces the RearchBootstrapper? Check scoped_zone_state.dart.

[GregoryConrad]

Hi, and thanks for such a thoughtful discussion post! I might pin it, because I think it covers a very large amount of ground pretty concisely...

I'll try to address what I can here quickly, but can go into more depth if you'd like clarification/have any questions.

a) Create local state for each new widget instance (Ephemeral/Local state).

No worries, all your bullet points mentioned here are effectively addressed via RearchConsumer (or RearchBuilder, assuming you want a Builder style widget). Eventually, you will be able to write widgets like:

@rearchWidget
Widget myWidget(WidgetHandle use) { // you can also have a BuildContext or some other widget constructor params here
  final count = use.data(0); // or the older use.state, if you prefer
  return TextButton(
    onPressed: () => count.value++,
    child: Text(count.value.toString()),
  ):
}

But that relies upon Dart macros to generate the underlying RearchConsumer for you. See #89, which implements this. Until then, you need to subclass RearchConsumer, which is just a tad bit of boilerplate (but nothing more than a normal widget).

b) Share state globally across multiple widgets in the widget tree (App/Global State).

Also no worries here, this is exactly how traditional capsules operate. (Except capsules have some optimizations that allow them to be disposed when idempotent, to free up unused memory; but rest assured any stateful capsules will not be disposed on you!)

The cool part is that capsules and widgets both use the same "side effects" API, so the use.data I showed above in the Widget also works the same way in capsules to manage global state.

c) Share state scoped to a "zone".

d) Share state scoped to a specific set of widgets in the widget tree.

Both of these use cases seem about the same, so I'm grouping them together.

That is done via scoping state. Again, the macros I have in #89 will make this a lot less tedious boilerplate-wise.

Your Example Application

I'll gladly take a look at your example app at some point, but it may take awhile for me to get around to it. Currently working on some other things at the moment...see the /packages/ramen folder :)

1. How are capsules created on Flutter? Are they created on the first use in a RearchConsumer? Are they stored on RearchBootstrapper container?

They are created via /package/rearch's CapsuleContainer. This means that capsules are built lazily, on the first container.read or a capsule's use(). The RearchBootstrapper internally stores a CapsuleContainer and provides it to the rest of the widget tree via an InheritedWidget.

2. If a RearchConsumer is removed from the widget tree and added again, is the capsule disposed and recreated (if is the only useing it)? Or is the capsule cached and reused? In other words, how is managed a capsule lifecycle?

The easiest way to think of this is that no state is disposed. I.e., if you read a capsule, the widget is removed, and you read the capsule again when the widget is added back in (assuming the capsule's state wasn't update via some external means), you'll get the same value by reading that capsule.

This gets a little complicated to answer more thoroughly because the true technical answer relies on capsule idempotence (a capsule is idempotent if it does not register any side effects). While what I said above is true, the ReArch implementation makes some smart optimizations that will remove idempotent capsules from memory if it can safely do so. Thus, you get the benefits of auto disposal/preventing memory leaks, but with the mental model that a capsule's state will always be intact and correct. If you want to learn more about this process, I'd recommend taking a look at my Master's Thesis. Specifically, Sections 3.3-3.3.4 for some background info, followed by Section 3.3.6. There is a bit of technical jargon there so feel free to add a comment here on anything that doesn't click at first.

3. If multiple RearchConsumer use the same capsule, are they sharing the same instance or is a new instance created for each widget?

Capsules are "global," so they share the same instance of the capsule. Accordingly, they will both get state updates of that same capsule.

For local/ephemeral state, instead use the side effects provided on the WidgetHandle (e.g., use.data or use.future).

4. Would be reasonable to have a RearchZone/RearchScope widget that replaces the RearchBootstrapper? Check scoped_zone_state.dart.

I'll take a look and get back to you. RearchBootstrapper is meant to bootstrap the app with a global CapsuleContainer accessible via dependency inversion, so I'd doubt anything should "replace" it. But perhaps something could supplement it.

[rubenferreira97]

Thanks for the detailed answer.

The easiest way to think of this is that no state is disposed. I.e., if you read a capsule, the widget is removed, and you read the capsule again when the widget is added back in (assuming the capsule's state wasn't update via some external means), you'll get the same value by reading that capsule.

If I understood this correctly. Capsules are always cached after the first read, even if the widgets are removed from the widget tree. I do think this have disadvantages. A Flutter App would start with zero memory used (if everything is lazy loaded), when I navigate to multiple parts of the app the memory would grow from the capsules that are read. If I don't ever navigate to that page again, the data will be cached, wasting memory resources. In a ideally scenario I would expect to have the possibility to configure this behavior; cache or auto dispose the capsule (by disposing it when no more widgets are listening/useing it).

Example:
App
├── PageA──More nested pages
└── PageB──More nested pages

PageAState pageACapsule(CapsuleHandle use) {
  final (state, setState) = use.data(/*variable using 100MB*/);
  return (state: state, setState: (newState) => setState(newState));
}

PageBState pageBCapsule(CapsuleHandle use) {
  final (state, setState) = use.data(/*variable using 100MB*/);
  return (state: state, setState: (newState) => setState(newState));
}

What happens If I accidently open PageA, pop it and go to PageB. Does Rearch has no concept of active listeners on capsules (read/use)?

I assume developers in the majority of cases want to dispose memory when data is not being used anymore. I don' like to compare solutions but Riverpod does this, and IMO for a good reason.

A common use case is to destroy the state of a provider when it is no-longer used.

There are multiple reasons for doing so, such as:

When using Firebase, to close the connection and avoid unnecessary cost.
To reset the state when the user leaves a screen and re-enters it.
Providers come with built-in support for this use case, through the .autoDispose modifier.

Are there alternatives that I am not aware in Rearch for this use case?

[GregoryConrad]

Capsules are always cached after the first read, even if the widgets are removed from the widget tree.

Well, stateful capsules (ones that use.something) will always be cached, because they are stateful. Idempotent capsules with no downstream non-idempotent capsules will be automatically disposed and be recreated later when needed, giving the illusion that they were always cached when read, even though they aren't.

What happens If I accidentally open PageA, pop it and go to PageB. Does Rearch has no concept of active listeners on capsules (read/use)?

First issue here is that these are capsules--if these capsules correspond to data shown on only one page, then they should be data stored on that page, and not in a capsule (achievable perhaps via scoped data). Capsules are designed to store app/global state.

The second thing to note is that 100 MB of data is quite a substantial amount, which most applications will never have sitting around in global/app state. Since such a situation is an odd one, your capsule can also return a free()/reset() method to free the capsule's data if the device memory gets too constrained. (But that being said--if this is global/app state, can you actually free it and have other parts of your app still function correctly? I'd argue in most cases, that answer is no, unless this state is actually just a cache of some other data.)

Does Rearch has no concept of active listeners on capsules (read/use)?

It does, but listeners (like from container.listen) are also treated as capsules internally, just with a different external API.

Are there alternatives that I am not aware in Rearch for this use case?

Sure, just create idempotent capsules, and ReArch's optimizations will automatically dispose everything that it can safely.

For the firebase example cited in that docs excerpt, simply create an idempotent capsule like so:

// This is a piece of app-level logic. Thus, it is natural to keep it in a capsule:
Stream<List<MyDocument>> myDocumentStreamCapsule(CapsuleHandle use) {
  final user = use(signedInUserCapsule);
  throw UnimplementedError('call firebase api functions here');
}

// And in your widget:
@rearchWidget
Widget myDocumentListView(WidgetHandle use) {
  final myDocumentStream = use(myDocumentStreamCapsule);
  final AsyncValue<List<MyDocument>> myDocumentsState = use.stream(myDocumentStream);
  // myDocumentStreamCapsule will be disposed once this widget is disposed
  throw UnimplementedError('return a ListView here');
}

[GregoryConrad]

Also for the firebase code example I just gave, you can ensure the capsule is idempotent at compile time by swapping out the parameter CapsuleHandle with a CapsuleReader; i.e.:

Stream<List<MyDocument>> myDocumentStreamCapsule(CapsuleReader use) {
  final user = use(signedInUserCapsule);
  throw UnimplementedError('call firebase api functions here');
}

That prevents you from accidentally registering a side effect.

[GregoryConrad]

Taking a look at your example now.

 return _ScopedCount(
      // How to fix `_TypeError (type '_WidgetHandleImpl' is not a subtype of
      // type 'CapsuleHandle' in type cast)` here, `use` is `WidgetHandle`
      // but `CapsuleHandle` is expected.
      // Placing a cast here for compilation
      scopedZoneCountManager(use as CapsuleHandle),
      child: child,
    );

scopedZoneCountManager should consume a WidgetHandle, not a CapsuleHandle. This is because it is designed to be part of the widget tree, and not a part of the global capsule data flow graph.

//    `app.dart`
//    RearchZone(
//      child: Column(
//        children:[
//          RearchZone(
//            capsules: [myCapsule],
//            child: Counter(), // This one will use the capsule in the inner `RearchZone`.
//          ),
//          Counter(), // This one will create a new capsule on the outer `RearchZone`
//        ],
//      )
//    ),

You could likely just use a RearchBootstrapper instead of your RearchZone here and it should likely work fine. However, you shouldn't need to do that. Think of a RearchBootstrapper/CapsuleContainer as a global service locater (like GetIt); you shouldn't be scoping multiple different service locaters throughout your app.

 final (count, setCount) = use.data(0);
  return (
    count: count,
    incrementCount: () => setCount(count() + 1),
  );

While you can use use.data here, I'd probably just recommend sticking to use.state based on how you have it written. Unless you wanted to write as:

final count = use.data(0);
return (count.value, () => count.value++);

class LocalState extends RearchConsumer {
  const LocalState({super.key});

  @override
  Widget build(BuildContext context, WidgetHandle use) {
    // Can we replace the following code with a capsule (or factory) that
    // creates a new state for each new widget instance?
    final (:count, :incrementCount) = (() {
      // Using a IIFE to encapsulate the state and actions.
      // If possible use a capsule to allow reuse and encapsulation across
      // multiple widgets.
      final (count, setCount) = use.data(0);
      return (
        count: count,
        incrementCount: () => setCount(count() + 1),
      );
    })();

Each instance of LocalState above will have its own independent state. If you mean you want to re-use that exact same counter logic across different RearchConsumers, then you can do that via:

extension Counter on SideEffectRegistrar {
  (int, void Function()) count() {
    final count = use.data(0);
    return (count.value, () => count.value++);
  }
}

// In your widgets/capsules you are now free to do a final (count, incrementCount) = use.count();

Alas, just looking at your example above, I really wish dart had proper expression support. Having to make a closure just to immediately invoke it just to avoid scope creep is really annoying.

// Feedback & Questions:
//
// 1. This capsule is not being disposed when `ShareMultipleStatePage` widget is
// removed from the widget tree. Is this a bug or by design? If by design, how
// can we autoDispose the capsule when the widget is removed from the widget tree?

CountManager scopedWidgetsCountManager(CapsuleHandle use) {

That's intentional; I know you wrote this example before I left another comment yesterday, but hopefully that comment should clear this up.

I didn't look into every line because that'd take a bit longer but I hope I addressed all your concerns.

[aford22]

I think a video providing a tutorial or tutorial series showing how to use Rearch in a real app, maybe with clean architecture and domain driven design, would really help drive understanding and adoption of Rearch. For example, I am very keen to use it but there are no tutorials on it that a dummy like me can follow, as contrasted with what I can find for Riverpod.

[GregoryConrad]

@aford22 ReArch is pretty indifferent to how you lay out your project (clean, DDD, what have you); ReArch just allows you to build out your pieces of app state (with capsules) and local state (via scoping or widget-local state).

To see how I would write certain things like network I/O, take a look at the weather_app example. The other example apps are also good for showcasing different techniques (some more advanced than others).

What ReArch really boils down to at the end of the day is:

• Decide if what you're implementing is app state or ephemeral state
  • If ephemeral, do multiple nested widgets in the tree need to access it?
    • Yes, then use scoped state via InheritedWidget + RearchConsumer (macro written in feat: add flutter and widget related macros #89)
    • No, then use a regular RearchConsumer and use.someEffect
  • If app state, make a new capsule
• Next, decide what API you need for the feature you're building.
  • This allows you to pick the right side effect (if one is even needed) for the job.
  • For example, a user-triggerable request is often a good candidate for use.mutation
    • Or maybe you just need to watch a future, perhaps with some extra needs (use.future, use.invalidatableFuture, use.refreshableFuture)
  • A really complicated event-dispatch based state might be good with use.reducer
  • Or, you can just build something custom with use.data (the route I typically go down, since you can make any API you like with less boilerplate).
  • Perhaps your feature just relies on the combination of a few other features--then just use(otherCapsule) to handle that.

That's a minimized flow chart, but hope it helps.

As a heads up, you can easily build your own side effects using an extension on SideEffectRegistrar. You normally can build whatever you like on top of use.data fairly easily (in fact, use.future and use.stream are largely implemented via just use.data!).