forked from Slimefun/Slimefun4
-
-
Notifications
You must be signed in to change notification settings - Fork 60
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
42 changed files
with
963 additions
and
154 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| docs/adr |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,19 @@ | ||
| #!/usr/bin/env bash | ||
| #!/bin/sh | ||
|
|
||
| echo "[pre-commit check]" | ||
|
|
||
| if ! [ -x "$(command -v mvn)" ]; then | ||
| echo -e 'Maven are not installed!' | ||
| echo | ||
| echo 'Please visit following site to download latest maven:' | ||
| echo | ||
| echo " https://maven.apache.org/download.cgi" | ||
| echo | ||
| echo 'If you are sure that you have installed maven,' | ||
| echo 'Please check whether your environment variables is being set properly.' | ||
| exit 1 | ||
| STAGED_FILES=$(git --no-pager diff --name-only --staged --line-prefix=$(git rev-parse --show-toplevel)/ | paste -sd ',') | ||
|
|
||
| if [ ! -z "$STAGED_FILES" ] | ||
| then | ||
| echo -n "Checking code format with spotless: " | ||
| mvn spotless:check -DspotlessFiles=$STAGED_FILES > /tmp/spotless.out 2>&1 | ||
| RETURN_VALUE=$? | ||
| if [ $RETURN_VALUE -gt 0 ] | ||
| then | ||
| echo "Please run 'mvn spotless:check' for more details or 'mvn spotless:apply' to automatically fix the violations." | ||
| fi | ||
| exit $RETURN_VALUE | ||
| fi | ||
|
|
||
| mvn spotless:apply | ||
| exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,7 @@ | |
| /target/ | ||
| /.idea/ | ||
| /.vscode/ | ||
| /data-storage/ | ||
|
|
||
| .classpath | ||
| .factorypath | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| # 1. Storage layer | ||
|
|
||
| Date: 2023-11-15 | ||
| Last update: 2023-12-27 | ||
|
|
||
| **DO NOT rely on any APIs introduced until we finish the work completely!** | ||
|
|
||
| ## Status | ||
|
|
||
| Work in progress | ||
|
|
||
| ## Context | ||
|
|
||
| Slimefun has been around for a very long time and due to that, the way we | ||
| wrote persistence of data has also been around for a very long time. | ||
| While Slimefun has grown, the storage layer has never been adapted. | ||
| This means that even all these years later, it's using the same old saving/loading. | ||
| This isn't necessarily always bad, however, as Slimefun has grown both in terms of content | ||
| and the servers using it - we've seen some issues. | ||
|
|
||
| Today, files are saved as YAML files (sometimes with just a JSON object per line), | ||
| which is good for a config format but not good for a data store. It can create very large files | ||
| that can get corrupted, the way we've been saving data often means loading it all at once as well | ||
| rather than lazy-loading and generally isn't very performant. | ||
|
|
||
| For a long time we've been talking about rewriting our data storage in multiple forms | ||
| (you may have seen this referenced for "BlockStorage rewrite" or "SQL for PlayerProfiles", etc.). | ||
| Now is the time we start to do this, this will be a very large change and will not be done quickly or rushed. | ||
|
|
||
| This ADR talks about the future of our data persistence. | ||
|
|
||
| ## Decision | ||
|
|
||
| We want to create a new storage layer abstraction and implementations | ||
| which will be backwards-compatible but open up new ways of storing data | ||
| within Slimefun. The end end goal is we can quickly and easily support | ||
| new storage backends (such as binary storage, SQL, etc.) for things like | ||
| [PlayerProfile](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java), [BlockStorage](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java), etc. | ||
|
|
||
| We also want to be generally more efficient in the way we save and load data. | ||
| Today, we load way more than is required. | ||
| We can improve memory usage by only loading what we need, when we need it. | ||
|
|
||
| We will do this incrementally and at first, in an experimental context. | ||
| In that regard, we should aim to minimise the blast radius and lift as much | ||
| as possible. | ||
|
|
||
| ### Quick changes overview | ||
|
|
||
| * New abstraction over storage to easily support multiple backends. | ||
| * Work towards moving away from the legacy YAML based storage. | ||
| * Lazy load and save data to more efficiently handle the data life cycle. | ||
|
|
||
| ### Implementation details | ||
|
|
||
| There is a new interface called [`Storage`](TBD) which is what all storage | ||
| backends will implement. | ||
| This will have methods for loading and saving things like | ||
| [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java) and [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java). | ||
|
|
||
| Then, backends will implement these | ||
| (e.g. [`LegacyStorageBackend`](TBD) (today's YAML situation)) | ||
| in order to support these functions. | ||
| Not all storage backends are required support each data type. | ||
| e.g. SQL may not support [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java). | ||
|
|
||
|
|
||
| ## Addons | ||
|
|
||
| The goal is that Addons will be able to use and implement new storage backends | ||
| if they wish and also be extended so they can load/save things as they wish. | ||
|
|
||
| The first few iterations will not focus on Addon support. We want to ensure | ||
| this new storage layer will work and supports what we need it to today. | ||
|
|
||
| This ADR will be updated when we get to supporting Addons properly. | ||
|
|
||
| ## Considerations | ||
|
|
||
| This will be a big change therefore we will be doing it as incrementally as | ||
| possible. | ||
| Changes will be tested while in the PR stage and merged into the Dev releases when possible. | ||
| We may do an experimental release if required. | ||
|
|
||
| Phases do not (and very likely will not) be done within a single PR. They will also not have any timeframe attached to them. | ||
|
|
||
| The current plan looks like this: | ||
|
|
||
| * Phase 1 - Implement legacy data backend for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java). | ||
| * We want to load player data using the new storage layer with the current | ||
| data system. | ||
| * We'll want to monitor for any possible issues and generally refine | ||
| how this system should look | ||
| * Phase 2 - Implement new experimental binary backend for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java). | ||
| * Create a new backend for binary storage | ||
| * Implement in an experimental capacity and allow users to opt-in | ||
| * Provide a warning that this is **experimental** and there will be bugs. | ||
| * Implement new metric for storage backend being used | ||
| * Phase 3 - Mark the new backend as stable for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java). | ||
| * Mark it as stable and remove the warnings once we're sure things are | ||
| working correctly | ||
| * Create a migration path for users currently using "legacy". | ||
| * Enable by default for new servers | ||
| * Phase 4 - Move [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java) to new storage layer. | ||
| * The big one! We're gonna tackle adding this to BlockStorage. | ||
| This will probably be a large change and we'll want to be as | ||
| careful as possible here. | ||
| * Implement `legacy` and `binary` as experimental storage backends | ||
| for BlockStorage and allow users to opt-in | ||
| * Provide a warning that this is **experimental** and there will be bugs. | ||
| * Phase 5 - Mark the new storage layer as stable for [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java). | ||
| * Mark it as stable and remove the warnings once we're sure things are | ||
| working correctly | ||
| * Ensure migration path works here too. | ||
| * Enable by default for new servers | ||
| * Phase 6 - Finish up and move anything else we want over | ||
| * Move over any other data stores we have to the new layer | ||
| * We should probably still do experimental -> stable but it should have | ||
| less of a lead time. | ||
|
|
||
| ## State of work | ||
|
|
||
| * Phase 1: In progress | ||
| * https://github.com/Slimefun/Slimefun4/pull/4065 | ||
| * Phase 2: Not started | ||
| * Phase 3: Not started | ||
| * Phase 4: Not started | ||
| * Phase 5: Not started | ||
| * Phase 6: Not started |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # ADR | ||
|
|
||
| An ADR (Architecture Decision Record) is a document describing large changes, why we made them, etc. | ||
|
|
||
| ## Making a new ADR | ||
|
|
||
| If you're making a large change to Slimefun, we recommend creating an ADR | ||
| in order to document why this is being made and how it works for future contributors. | ||
|
|
||
| Please follow the general format of the former ADRs or use a tool | ||
| such as [`adr-tools`](https://github.com/npryce/adr-tools) to generate a new document. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # Update Procedure | ||
|
|
||
| Date: 2024-01-15 | ||
| Last updated: 2024-01-15 | ||
|
|
||
| ## Goal | ||
|
|
||
| This SOP will go over updating Slimefun to the newest Minecraft version, most of this will only apply to major versions, but we have also seen minor versions break things. So please read through the whole SOP and make sure you do everything applicable. | ||
|
|
||
| ## Updating | ||
|
|
||
| ### Updating Bukkit/Spigot | ||
|
|
||
| The first step is just updating Spigot in the pom.xml. This should only be done in 2 cases: | ||
| * There's a new major version (well, MC major - 1.19 -> 1.20 is a major) | ||
| * There was a change within MC or Bukkit/Spigot that broke the API | ||
|
|
||
| To update the Spigot version, you will need to go to the `pom.xml` and find the `spigot.version` property, this will be within the `properties` property. Simply make this the MC version (e.g. `1.20` or in the case of minor `1.20.4`). | ||
|
|
||
| Once updated, **make sure to run a build** to check for compilation failures with `mvn clean package -DskipTests=true`. We will go over the tests next. | ||
|
|
||
| ### Updating tests | ||
|
|
||
| The next step is making sure our tests are still working correctly as is. This can be done by running `mvn test` and verifying that all tests pass correctly without any failures or errors. | ||
|
|
||
| If there are any failures you will need to investigate these, it's best to run them one at a time, so you don't have the potential for cross-test contamination. If you find any issues with the tests, please fix them and make sure to add a comment to the PR explaining why the test was changed. | ||
|
|
||
| If you need any help fixing tests feel free to join the [Discord](https://discord.gg/slimefun). | ||
|
|
||
| Once all the tests are passed, check to see if there's a new version of [MockBukkit](https://github.com/MockBukkit/MockBukkit), this is the framework handling the Bukkit side of our tests. There very well may not be a new version, they usually lag updates a bit. If not, that's perfectly ok, just make sure to note it on the PR. | ||
|
|
||
| ### Testing in game | ||
|
|
||
| The final and most important step is testing this in game. While I'd love for our tests to be perfect, they are not (especially if MockBukkit hasn't had an update yet). We need to ensure that everything is working in-game before we can ship a new version release. | ||
|
|
||
| To do this, you will need to build the plugin with `mvn clean package` and then copy the jar from `target/` to your server's `plugins/` folder. Once you've done this, start the server. You will want to test various things but the things we always want covered are: | ||
| * Commands, verify running a few commands work | ||
| * `/sf versions` | ||
| * `/sf cheat` | ||
| * `/sf search` | ||
| * Items, verify you can use a few items (you can grab these from `/sf cheat`) | ||
| * Wind staff | ||
| * One of the talismans | ||
| * One of the backpacks | ||
| * Blocks, verify you can place, break and ensure they all work | ||
| * Ancient altar | ||
| * Ore washer | ||
| * Coal generator | ||
|
|
||
| It is important to verify heads are still working (part of the energy network and the coal generator). If head skins are not loading, consider it as a bug: try figuring out what the issue is, and ask in the [Discord](https://discord.gg/slimefun) if you are not sure what the cause may be. | ||
|
|
||
| Also make sure to verify that there are no errors in the console, any errors here should be investigated and fixed. | ||
|
|
||
| If you find any issues, please fix them and make sure to add a comment to the PR explaining why the fix was needed. | ||
|
|
||
| > **Note** | ||
| > An issue here usually means that we need to update Dough. If this is the case, please open a PR to Dough and then update the Dough version in the `pom.xml` to the new version. Once you've done this, make sure to run a build to verify everything is working correctly. | ||
| ### Final steps | ||
|
|
||
| Once you've verified everything is working, you can go ahead and open the PR. We will get to this as soon as we can :) | ||
|
|
||
| While the PR is open, make sure to verify the E2E tests are passing, and you should also verify the output of these. If the E2E tests look good then finally we will update these. | ||
|
|
||
| #### Updating E2E tests | ||
|
|
||
| **This is only needed in a major version** | ||
|
|
||
| In the `e2e-testing.yml` file you will need to update the matrix strategy, please add the latest version of the old major (e.g. if 1.21 came out, add 1.20.x where x is the latest released version). If MC is requiring a new Java version make sure that is updated too in the `latest` version. | ||
|
|
||
| Once updated, push and re-verify that the E2E tests are still passing. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.