Skip unnecessary steps when package list hasn't changed #1429
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
anttimaki
force-pushed
the
chunky-pt2
branch
from
29309fb
to
d4c5311
Compare
The hash identifies the last seen package list index, allowing us to skip updating the database from file cache if there's no changes to the package list. The new table contains 0..1 rows for each game (because community identifiers aren't available on the mod manager currently). Internal game identifier acts as the primary key to prevent duplicates. Compound index of game+hash is added for performance, and to silence "warnings" that Dexie would otherwise log into the console. Dexie's put method, i.e. upsert, is used to update the hash to the database.
The hashes will be used to identify previously seen package list indices. The Thunderstore API uses the same hash to identify blobs. The hash is recalculated on the mod manager because: - The API returns a redirect response, and we can't access the headers of the original response, only the result of redirect from the CDN - The redirect can't be prevented and handled manually in the browser environment - The response from the CDN does contain the hash as its used in the file name, but the only place Axios stores it in the response is a field that looks like it could easily change in the future versions Since hashing the amount of data we're handling should be in a scale of single digit milliseconds per chunk, this shouldn't be a problem.
While this makes some parts of the code (updating the progress text and percentage shown on the UI) harder to follow, I feel this improves the readability of the code overall. Especially when the upcoming additions to the flow are taken into account.
In SplashMixin, skip downloading the chunks and updating them into the IndexedDB if the index file indicates the packages haven't changed. If downloading the index file itself fails, try to load earlier mod list from the IndexedDB like it did earlier. In UtilityMixin, skip the whole process if the index file indicates the packages haven't changed, as at this point we already have the mod list loaded in to Vuex. If loading the index file itself fails, the error is caught by the caller, which handles retries as it did earlier.
Old approach used to read the timestamp from the package list table since it wasn't available elsewhere. This approach no longer works when we don't update the package list into database if it didn't change. Instead, store the information in the same table as the community's last seen index chunk hash. This timestamp is updated whenever the same index hash is seen again, or when the hash changes AND the new package list is successfully written to DB. Old approach loaded the timestamp into Vuex when the package list was read to Vuex state. With the new approach we need to take into account that UtilityMixin skips updating the package list in memory if the list didn't change. Instead, we need to update the timestamp manually.
anttimaki
force-pushed
the
chunky-pt2
branch
from
d4c5311
to
92e3210
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.