Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model recovery from an event log (recovery 3/3) #3284

Merged
merged 23 commits into from
Jun 1, 2024
Merged

Conversation

amolenaar
Copy link
Member

@amolenaar amolenaar commented Apr 28, 2024

This PR builds on #3319 and #3322

PR Type

What kind of change does this PR introduce?

  • Bug fix
  • Feature
  • Chore (refactoring, formatting, local variables, other cleanup)
  • Documentation content changes

What is the current behavior?

Models are not automatically saved.

Issue Number: #2831

What is the new behavior?

In this PR, I want to pivot the idea of an event log. The event log contains all (model related) events. This log can be stored on disk and replayed. The goal is to provide a feature in Gaphor that prevents users from losing their work in case Gaphor crashes and the user has not saved his work in time.

The idea is that we log all transactions to a file. This file is based on the hash of the absolute filename of the model , similar to the per model user properties.
If such a log exists after a model is loaded, the log is applied and a notification is shown to the user.

The log will replay all user transactions, so the user should be able to continue his work and undo any step he's done previously. The model will show as modified.

When a model is saved, the event log (on disk) is truncated/reset.
If we have an event log for an unsaved model, we display that as first entry in the "Recent Models" list on the greeter window.

New models should get a unique event log file, so that you can have multiple new models open at any time.
The Greeter window should show any new, unsaved model (with a date?) so a user can re-open it and continue working.

Extra

  • Move diagram update to sanitizer service, so they're also performed if no diagrams UI is opened yet
  • move get_config_dir() and get_cache_dir() to gaphor.settings
  • better cache events in main window, so notifications are displayed as soon as the window shows.
  • Clean undo/redo stacks on model flush, instead of model-ready, so it does not interfere with recovery updates.

To do

  • Load saved model if recovery fails -> undo changes and remove event log?
  • What to do if the model changed in the meantime (e.g. due to git pull)? -> Add a checksum/hash at the beginning of the event log file.
  • Clear event log when a model is reloaded (model changed banner) -> checksum/hash in event log preamble does not match.
  • Do not replay events when the same model is opened twice. -> Second time, open model is "forced". In that case, we do not store the model file name in the Recovery service and no event log can be created.
  • How to deal with recording if the same model is opened twice? -> Same as above.
  • How does a user know if a recovery log is created?

Future ideas

  • Compact recovery records: reduce matrix updates and handle position changes.
  • If the user exits the application, we can just quit, any changes not saved can be re-applied when Gaphor is started again. Gaphor can then always start with the models that were open the last time Gaphor was quit.
  • Instead of basing the event log file on the model loaded (filename), it could be a unique identifier (a session id if you like). In the Greeter we can show all progress sessions with unsaved data (at the top, before the recent files). This gives users the option to either continue where they left off, or just open a model. This will also work for for new models that have not been saved to file yet.

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

An alternative is to actually save the model every so many minutes. However, saving can take a bit of time, especially for big models. With the event log approach we can restore the session to the point where Gaphor was shut down (or crashed).

@amolenaar amolenaar marked this pull request as draft April 28, 2024 20:46
@github-actions github-actions bot added the python Pull requests that update Python code label Apr 28, 2024
@amolenaar amolenaar changed the title Model recovery Model recovery from an event log Apr 28, 2024
@amolenaar amolenaar force-pushed the model-recovery branch 6 times, most recently from 9c53eb7 to 8e2e6c1 Compare May 5, 2024 21:44
@amolenaar amolenaar changed the base branch from main to model-ready May 22, 2024 20:15
@amolenaar amolenaar force-pushed the model-recovery branch 2 times, most recently from 5a63582 to 5af83fd Compare May 22, 2024 21:23
This covers the simple ones.

The challenge will be RevertibleEvents: hey come in many forms and each
one is applied differently.
Only reconnect is needed, we do not need the disconnect.
This service automatically runs after a model is loaded and adds an
extra transaction for replayed state.
So users get a hint that there are unsaved changes.
This avoids a range of issues that can happen if we
apply changes to another model (-version) than what we
worked with.
@amolenaar amolenaar changed the base branch from model-ready to model-recovery-prep May 25, 2024 11:53
@amolenaar amolenaar changed the title Model recovery from an event log Model recovery from an event log (recovery 3/3) May 25, 2024
@amolenaar amolenaar marked this pull request as ready for review May 25, 2024 12:12
@amolenaar amolenaar requested a review from danyeaw May 26, 2024 09:14
@amolenaar amolenaar mentioned this pull request May 26, 2024
6 tasks
Base automatically changed from model-recovery-prep to main May 27, 2024 16:46
gaphor/storage/recovery.py Outdated Show resolved Hide resolved
Copy link
Member

@danyeaw danyeaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amolenaar These changes look really great!

One thing I noticed is if I create a model, save it, edit it, and then force close Gaphor, when it comes back up it recovers the changes when I open the model which is great. However, I am not then able to save the model during that session from the menu or by hitting Ctrl+s. For example, I can hit Ctrl+s, the unsaved changes shows in the titlebar, and it will ask if I want to save when I go to close the model even though I just tried to save them.

tests/test_session_recovery.py Outdated Show resolved Hide resolved
@danyeaw danyeaw added feature A new feature and removed python Pull requests that update Python code labels May 31, 2024
@amolenaar
Copy link
Member Author

I see that the "edited" dot remains after saving. Saving itself works with me.

@github-actions github-actions bot added the python Pull requests that update Python code label May 31, 2024
@amolenaar
Copy link
Member Author

@danyeaw The modified indicator works again.

I also changed how the modified indicator is set when changes are done, so it will always show modified after modification. Now, if you do edit, save, undo, it will show as modified.

Windows doesn't like that.
@danyeaw danyeaw removed the python Pull requests that update Python code label Jun 1, 2024
@danyeaw danyeaw merged commit 36ac82b into main Jun 1, 2024
18 checks passed
@danyeaw danyeaw deleted the model-recovery branch June 1, 2024 01:10
@danyeaw
Copy link
Member

danyeaw commented Jun 1, 2024

Nice one @amolenaar!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants