lazy O(1) commit

[YuMingLiao]

I am interested in O(1) commit. I've read some codes and docs about it.

For now, uncommitted RelationVariables loads the whole GraphRefRelationalExpr from last commit.
If we initialize uncommitted RelationVariables with relvarname@last-commit-uuid instead, then we can have lazy commit as you said in doc, right?

it seems just one step away from O(1) commit.

[agentm]

Hi @YuMingLiao, the current claim that commits happen in constant time is accurate relative to tuple count. Indeed, we can commit the expression regardless of how many tuples the expression may change. Furthermore, we don't have to write n tuples to disk, just the update expression.

Just like lazy Haskell functions, the expressions compose through the graph, so we only load the tuples per se if the expression itself includes raw tuples.

In short, I believe Project:M36 already implements maximum laziness in the graph that you allude to above. Do you have a counterexample where we could increase laziness?

[YuMingLiao]

Hi @agentm

Here is what I found:

If I createarbitraryrelation a {a Int} 10000000-10000000 and commit. It's about 85.9MB in my db.
And keyboard input becomes slow.

If I close tutd and reopen it, it opens slowly.

If I commit again without any changes, the second transaction is about 85.9MB too.

So I believe the real data is saved twice. (Or is any change that I am not aware of recently?)

If I save the second transaction as relvarname@last-commit-uuid instead of the big data actually, then the real data wouldn't be saved twice. The database will grow slower. The loading of tutd will be lighter.
And if there is no :showexpr a in a session, then maybe the loading of tutd can be omitted entirely in some way. It helps in many writes, little reads situations.