State Management -- Some Personal Thoughts and Notes

[IronCore864]

0. What's Needed in the State

• plugin
  • name
  • version (name and version decides if it needs to be updated or not if a new version shows up in the config)
• last operation
  • parameters (if params change, maybe a reinstall is required)
  • status: failed or succeeded
  • install or uninstall or reinstall
  • operation date

What else?

1. Code Structure

See here.

According to the second paragraph, the following is reasonable:

internal/
  pkg/
    backend/
      local/
      s3/
    state/
      state.go  # structs and stuff
      statemanager.go  # GetState(), UpsertState(), DeleteState(), ...
      statemanager_test.go
    other_modules_1/
    other_modules_2/
    ...

2. The Relationship of config/backend/state/state manager/plugin engine

I had a quick look at the source code of Terraform, which didn't do a clean job here. It mixes config/state/backend into a context, then executes, and the backend calls state manager, and vice versa.

I think we can improve the idea by standing on the shoulder of Terraform.

Our original design is here.

NOTE: in the current MVP version, the configuration loader isn't strictly called by the plugin engine.

The general idea is like the builder pattern, where the plugin engine acts as a director and coordinates the config, state, and backend.

The relationship between backend and state also baffles me, which leads to the next section:

3. The Relationship Between State Manager and Backend

There are a few structural ambiguities:

• State manager doesn't really "belong to" a backend. The backend operates a file (local) or an S3 obj (remote) directly, and it doesn't have a strong relationship with the state.
• If the first point is agreed upon, we need to be a bit more clear on the naming: there shouldn't be anything called "manager" or "state" under internal/pkg/local/, because otherwise, it would be misleading.

If we think about how the state manager is going to be used from the standpoint of the code inside the plugin engine:

plugin := loadPlugIn(config)

state := statemanager.getState(plugin)

action := getAction(state, config)

if action == "Install" {
    res := plugin.Install(params)
    statemanager.update(res, action)	
}

Summary

Hence, the following:

• state manager doesn't "belong to" a backend; backend should be invisible to the plugin engine;
• state manager only exposes get/set interfaces;
• state manager loads the backend according to the config (or, plugin engine loads the config, gets the backend, then passes it to the state manager to use)
• state manager does all the calculations according to the previous state, and the result

internal/
  pkg/
    state/
      state.go  # structs and stuff
      statemanager.go  # GetState(), UpsertState(), DeleteState(), ...
    backend/
      backend.go # the interface
      local/
        local.go # readComponentFromFile(file), updateComponentInFile(file), updateComponentLastUpdatedTime(file) ...
        helper.go # some funcs only used by local, for example, file operations
      s3/
        s3.go # readComponentFromS3(obj), updateComponentFromS3(obj), ...
        helper.go # some funcs only used by s3, for example, S3 API related things

The state manager decides what to do and which interface of the backend to call, based on the result, action, and the previous state. Examples:

• if res is true，action is install，the previous state is nil, then statemanager.update calls backend.insert(res)
• if res is false, action is reinstall, the previous state is not nil and failed, then statemanager.update does nothing (maybe updates the last operation time, if that's recorded in the state)

[daniel-hutao]

I agree with most of the points you mentioned above. Today I will write a demo based on the suggestions mentioned above, and then we discuss it further.