Skip to content

Improve database resilience, load handling, cleanup, and Folia reloads - #4772

Open
Minecraft0122 wants to merge 3 commits into
plan-player-analytics:masterfrom
Minecraft0122:master
Open

Improve database resilience, load handling, cleanup, and Folia reloads#4772
Minecraft0122 wants to merge 3 commits into
plan-player-analytics:masterfrom
Minecraft0122:master

Conversation

@Minecraft0122

@Minecraft0122 Minecraft0122 commented Jul 30, 2026

Copy link
Copy Markdown

Summary

  • retain queued database transactions during transient/recoverable connection outages and retry them in order with bounded exponential backoff
  • reduce extension database lock contention by caching provider and table metadata and only writing it when the metadata changes
  • handle MySQL 1205 statement failures with bounded iterative retries, return every connection exactly once, and report the actual 2-minute throttling window
  • add /plan db remove_registered <after-date> <before-date> for confirmed, inclusive bulk deletion by first-join date
  • make Folia reloads lifecycle-safe: cancelled tasks from the old lifecycle can no longer run after reload, and shutdown waits for already-running tasks before closing the database
  • correct Folia asynchronous tick-to-millisecond conversion, which previously scheduled periodic work roughly 1000 times too frequently
  • add regression coverage for merging a SQLite database into a non-empty MySQL-compatible destination while preserving existing server data
  • fix transaction queue accounting when throwaway transactions are dropped

Command details

New command: /plan db remove_registered <after-date> <before-date>

  • Aliases: /plan db removeregistered and /plan db remove_between
  • Permission: plan.data.clear
  • Accepts ISO dates in yyyy-MM-dd format and uses Plan's configured time zone.
  • Selects players by their first-join/registration timestamp. Both dates are inclusive, from the start of after-date through the end of before-date.
  • Rejects malformed dates and a range where after-date is later than before-date.
  • Shows a confirmation prompt before making changes. After confirmation, it removes all Plan data linked to every matching player from the currently active database, invalidates their cached query data, and reports the number removed.
  • Example: /plan db remove_registered 2026-01-01 2026-01-31

Existing command covered by this PR: /plan db merge <source> <destination>

  • Permission: plan.data.merge
  • The first database argument is the source and the second is the destination. For SQLite to MySQL, use /plan db merge SQLite MySQL and confirm the prompt.
  • This command already existed; the PR does not add or rename it. The new regression test starts with data in both databases and verifies that the destination's original server and session data remain while the SQLite server and sessions are added.
  • Existing server-UUID conflict options remain available: --on-conflict-delete deletes the conflicting destination server, while --on-conflict-swap assigns the imported conflicting server a different UUID. They are not needed when server UUIDs do not conflict and were not introduced by this PR.

Verification

  • GitHub Actions full CI passed on the exact head commit (51ee784f6): https://github.com/Minecraft0122/Plan/actions/runs/30634140383
  • full Gradle build and test suite, including MariaDB/MySQL, SQLite, Selenium/frontend tests, release JAR assembly, Javadocs generation, and Javadocs deployment
  • new regression tests cover unchanged and changed extension metadata, dynamic table rows vs. table schema, failed/skipped metadata writes, bounded MySQL lock retries, connection return accounting, recovery, and non-critical transaction dropping
  • MySQLTest > mergeSQLiteIntoNonEmptyDatabase() and SQLiteTest > mergeSQLiteIntoNonEmptyDatabase() both passed on the current head
  • local full :common:check, Checkstyle, and targeted regression tests passed

The first CI attempt passed all code and test steps but could not deploy Javadocs because the fork's workflow token was read-only. After enabling read/write workflow permissions on the fork (without PR approval permission), the complete second attempt passed.

The SQLite-to-MySQL merge command already exists as /plan db merge SQLite MySQL; this PR adds the non-empty destination regression test requested for that workflow.

Closes #4727
Closes #4719
Closes #4599
Closes #2755

Keep queued transactions during temporary connection failures, add registration-date player removal, and verify SQLite merges into non-empty databases.
@Minecraft0122 Minecraft0122 changed the title Improve database outage recovery and player data cleanup Improve database resilience, cleanup, and Folia reloads Jul 31, 2026
Cache unchanged extension metadata and replace recursive lock-timeout retries with bounded iterative attempts that return each connection exactly once.\n\nAffects issues:\n- Fix plan-player-analytics#2755
@Minecraft0122 Minecraft0122 changed the title Improve database resilience, cleanup, and Folia reloads Improve database resilience, load handling, cleanup, and Folia reloads Jul 31, 2026
@AuroraLS3

Copy link
Copy Markdown
Collaborator

If you make further changes please keep each change to its own PR so that it's easier to review - It's difficult if one part of a PR is good and another isn't

@AuroraLS3 AuroraLS3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't unfortunately merge this like this because it has 3 different completely unrelated things in it and it seems to be written by AI.

If it was 3 different PRs I could merge one and cleanup, then merge the next one and cleanup - but with it all in big lump I can't possibly hope of cleaning up the AI code to be up to standard.

Please separate the three things into three different branches and 3 PRs so that it is possible to review properly.

I've left some comments that may help when doing the changes.


I did not review properly because I got the feeling that my comments would be fed to AI to generate the fixes, and the larger the PR, the smaller the chances of success for AI to fix them are.
If you wrote this yourself and did not use AI, I can take another look.

/**
* Folia runnable factory that prevents tasks from an old Plan lifecycle from running after reload.
*/
public class PlanFoliaRunnableFactory implements RunnableFactory {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in this file should be done in https://github.com/plan-player-analytics/Platform-abstraction-layer instead.

/**
* Plan-specific Folia platform layer with reload-safe task scheduling.
*/
public class PlanFoliaPlatformLayer extends FoliaPlatformLayer {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra PlatformLayer specificly for Plan is not required in this case, changes to it should be done in https://github.com/plan-player-analytics/Platform-abstraction-layer instead

Comment thread Plan/folia/build.gradle
runtimeOnly "net.playeranalytics:platform-abstraction-layer-folia:$palVersion"
implementation "net.playeranalytics:platform-abstraction-layer-folia:$palVersion"

compileOnly "dev.folia:folia-api:1.20.1-R0.1-SNAPSHOT"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in this file are not required after moving the other modifications in folia module to pal library.

try {
// Attempt to load and use the Folia library for Java 17+
Class<?> foliaPlatformLayer = Class.forName("net.playeranalytics.plugin.FoliaPlatformLayer");
Class<?> foliaPlatformLayer = Class.forName("net.playeranalytics.plugin.PlanFoliaPlatformLayer");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Class<?> foliaPlatformLayer = Class.forName("net.playeranalytics.plugin.PlanFoliaPlatformLayer");
Class<?> foliaPlatformLayer = Class.forName("net.playeranalytics.plugin.FoliaPlatformLayer");

}

private static final class MetadataKey {
private final Database database;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Database should not be stored in persistent variables, it is also not required here since DataValueGatherer is recreated at the same time if Database instance changes.

}

private static void addIcon(List<Object> values, Icon icon) {
if (icon == null) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the icon is null it is not stored, so this if-block is unnecessary

@Minecraft0122

Minecraft0122 commented Aug 2, 2026 via email

Copy link
Copy Markdown
Author

@Minecraft0122

Minecraft0122 commented Aug 2, 2026 via email

Copy link
Copy Markdown
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants