Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

a StableHashMap module. #300

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

a StableHashMap module. #300

wants to merge 5 commits into from

Conversation

matthewhammer
Copy link
Contributor

@matthewhammer matthewhammer commented Nov 3, 2021

An outgrowth the the discussion in #299

A non-OO, stable version of our (currently OO) HashMap implementation.

@matthewhammer matthewhammer changed the title a stable hash map a StableHashMap Nov 3, 2021
@matthewhammer matthewhammer changed the title a StableHashMap a StableHashMap module. Nov 3, 2021
@matthewhammer matthewhammer marked this pull request as ready for review November 3, 2021 20:18
@matthewhammer
Copy link
Contributor Author

BTW, I wanted to add @nomeata as I reviewer on this one and #299 but the GH UI does not permit it. I guess he needs to be added to a group of potential reviewers somewhere in the admin settings for the repo?

@nomeata
Copy link
Contributor

nomeata commented Nov 3, 2021

Yeah, you can only assign to people having push permission. I guess not having to review is a benefit of not being able to commit :-)

@nomeata
Copy link
Contributor

nomeata commented Nov 3, 2021

Shouldn't we finish the discussion first whether the idiom of “class interface backed by stable data” is good enough? That would not need a new module and could even be backwards compatible with the existing interface. Maybe I need to create a PR to explain.

nomeata added a commit to nomeata/motoko-base that referenced this pull request Nov 3, 2021
this is to demonstrate what I meant in
dfinity#300 (comment)
and
dfinity#299 (comment),
and how to introduce this without breaking changes (although it’s kinda
ugly)

What I would _want_ here is to introduce a second, more general,
constructor for the given class, but Motoko does not allow me to do that
easily. But I can hack around that by

 * Creating a new class, not public `HashMap_` with the constructor I
   want
 * In `HashMap`’s constructor, call the `HashMap_` constructor to create
   an inner object (`i`)
 * In `HashMap`, simply copy all the fields from the inner objects to
   the outer object.
 * A public module-level function (here `wrapS`) exposes the new
   constructor.

With this (generic, ugly) trick I can suppor the idiom
```
stable var userS : HashMap.S <UserId,UserData> = newS();
let user : HashMap.HashMap<UserId,UserData> = HashMap.wrapS(10, Nat.eq, Nat.hash, userS)
```
without changing the API.

But it is ugly, and the effect on documentation generation is probably
bad as well. So maybe a better course of action would be to have a midly
breaking change where we only have the “new” constructor, and people
will have to fix their code by passing `HashMap.newS()` as a new fourth
argument if they want their old behavior. Probably better than piling up
hacks like this.

In that case, simply rename `class HashMap_` to `HashMap`, remove `wrapS` and
the old `class HashMap`.
Copy link
Contributor

@nomeata nomeata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I’d prefer the idiom described in #299 (comment) over adding new modules or classes.

Copy link
Contributor

@rossberg rossberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'm not sure I understand how I would use this API in practice. The bifurcation between types HashMap and HashMap_ would seem to make it impossible to perform relevant operations on an actual stable hash-map, which I can't even use get on? What would I actually do with a plain HashMap?

src/StableHashMap.mo Outdated Show resolved Hide resolved
@matthewhammer
Copy link
Contributor Author

matthewhammer commented Nov 5, 2021

I think I’d prefer the idiom described in #299 (comment) over adding new modules or classes.

I also see the benefits of not adding these to master. I favor @nomeata's alternative approach, al things considered, assuming we can reach consensus about it.

If so, then this PR is here because

  • Potentially unblocking anyone "out there" in the greater Motoko/IC land who wants this abstraction now for whatever reason. I wanted to demonstrate these self-contained versions of the Stable versions for them.
  • Support the larger stable vs OO discussion, by showing another point in the design space.

Of course, each Stable version duplicates a lot of code, which is not very compelling or satisfying, and ultimately why we should not use them.

@matthewhammer
Copy link
Contributor Author

matthewhammer commented Nov 5, 2021

impossible to perform relevant operations on an actual stable hash-map ... What would I actually do with a plain HashMap?

I admit that the API for that type is limited.

One could fill a Buffer with the contents efficiently, via

  • size (to pre-allocate that Buffer of the right size) and
  • entries, to iterate over the elements.

But really, my full intention is to refactor this to be more layered, like @nomeata's suggestion, where the stable part can go into a stable variable. I need to refactor this code for that to work, I realize.

... which I can't even use get on?

Well, get requires key equality and hashing, so you need to have the non-stable HashMap_ version to project specific keys, and call those operations.

@rossberg
Copy link
Contributor

rossberg commented Nov 8, 2021

Well, get requires key equality and hashing, so you need to have the non-stable HashMap_ version to project specific keys, and call those operations.

Sure, but that makes HashMap a useless type, doesn't it? Yes, I can now define a stable variable of this type, but I can neither look up anything nor modify it ever again. I can convert it to a list of entries, but if that's all I wanted then I could just as well have stored it as a plain list or array right away.

@matthewhammer
Copy link
Contributor Author

matthewhammer commented Nov 9, 2021

@rossberg Yes, the type definitions you saw here are not as useful as the one where the types are "layered", rather than flattened, as I mentioned above:

But really, my full intention is to refactor this to be more layered, like @nomeata's suggestion, where the stable part can go into a stable variable. I need to refactor this code for that to work, I realize.

It was my original intention to do this layering, but I first tried this variation, which I agree is not as helpful.

As for the "original idea", see this commit: e7a2787

Now, a stable variable may hold the mutable state, and its partner, a flexible variable that shares the stable state, wraps it with the necessary higher-order functions. The entire API is available to the code that defines a pair of variables this way, just like in @nomeata's more OO version.

Compared with that version, the module-based version here is more cumbersome, as it uses no class at all, just a module and functions, forcing users to be very aware of which API calls require which kind of argument. However, if the two variables are initialized with sharing, there is no inherent lack of functionality here.

@ggreif ggreif force-pushed the master branch 2 times, most recently from d52aecd to 08507fc Compare October 21, 2022 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants