Conversation
Adds a way to adopt a pre-existing Lagoon project as a PolydockAppInstance so it benefits from scheduled auto-redeploys without going through the create/deploy pipeline. - ClaimExistingProjectService: verifies the project exists on Lagoon, grants the store app's deploy group access to it, and lands a new instance directly at RUNNING_HEALTHY_CLAIMED with data['adopted']=true. - Filament 'Claim existing Lagoon project' header action on the instance list. - Safety: adopted instances detach on REMOVE (Lagoon environment left intact) and are skipped by project purge, so Polydock never deletes a project it did not create. - FakeLagoonClient gains getProjectByName/addGroupToProject; feature test covers claim, rejection, double-claim and the purge guard.
Remove the deprecated Private GPT app and its Private-AI-only config, and
replace the stringly-typed AmazeeClaw key mode with a typed AmazeeAiKeyMode
enum that supports injected, anonymous, and (new) user-generated keys.
- Delete app/Polydock/Apps/PrivateGpt and its test; drop the $aisettings
block, getDataForPrivateGPTSettings(), and the per-instance merge hack.
- Remove redundant empty form-schema stubs from PolydockAiApp (already
provided by PolydockAppBase).
- Introduce AmazeeAiKeyMode {injected, anonymous, user}; user-scoped keys
are generated at claim time against the claiming user's email. Expose the
mode selector in the AmazeeClaw store-app settings form. Legacy auto/manual
config values remain supported.
MAIL_CC_ALL was read via env() in three trial-email jobs and the status-change listener, but registered in no config file — so it silently returned null once `config:cache` ran in production, dropping the CC. Register it as `mail.cc_all` and read it via config() everywhere. Document it in .env.example. No behaviour change when config isn't cached.
- PreRemoveAppInstanceTrait now has the same adopted-project guard as Remove/PostRemove. Removal enters the pipeline at PENDING_PRE_REMOVE, so without it the Lagoon ping/validation ran before any guard and a detach could not succeed while Lagoon was unreachable. The detach test now enters at PENDING_PRE_REMOVE to cover the real pipeline entry. - ClaimExistingProjectService rejects store apps with scheduled redeploys disabled (an adopted instance would silently never redeploy — the stated point of adopting) and rejects store apps with no Lagoon deploy group (the grant is what lets the global token trigger redeploys). The Filament select only lists redeploy-capable store apps and the action is gated by the same authorization as CreateAction. - Drop the never-read data['adopted-added-group'] key; document that PurgeResult::AlreadyGone also covers the adopted-detach path; add tests for the group-grant rejection and redeploy-disabled rejection.
…ost-create warning - User mode now fails the claim when no user-email is set on the instance, instead of silently generating keys under the anonymous @autogen.null identity — a claim that "succeeds" with keys tied to the wrong identity is the worse failure mode. - Post-create skips credential injection entirely for user mode: those credentials don't exist until claim, so the unconditional call only produced a spurious "no auto-generated credentials" warning.
Grep-verified deletions only — no behavior change: - AmazeeAi backend client: drop 22 public methods with zero callers (team/region/user CRUD, audit logs, token listing) plus the orphaned put/delete helpers; it's an in-repo client, not a published SDK. - Delete unreferenced blades (old* email templates, mjmlbutton component) and the unreferenced PollHealthJob TODO stub. - Drop dead model/interface methods: isTrialActive, deleteKeyValue, verifyLagoonProjectAndIdAreAvailable, getGraphqlClient, setRequiresAiInfrastructure, and a default-only attachments() override. - Remove per-enum getValues() overrides that duplicate HasEnumOptions. - Remove unused composer deps (cuyz/valinor, spatie/laravel-slack-alerts, laravel/slack-notification-channel, hotmeteor/spectator) and the redundant spatie/mjml-php root declaration (transitive via lara-mjml). - Delete config/spectator.php and config/pennant.php (skeleton default; Pennant merges its own) and the never-set NOCAPTCHA_* env fallbacks.
Same behavior, less code: - Three ~90%-identical trial-email jobs collapse into an abstract TrialEmailJob (guard -> send-to-owners -> mark sent); subclasses only declare their store-app toggle, schedule field, sent flag, Mailable and label. The matching three Mailables collapse into TrialEmailMail the same way. Dispatch sites and class names unchanged. - ProgressToNextStageJob's 13-case switch becomes a status=>next match (the POST_DEPLOY branch and end-of-line cases stay explicit). - The status-change listener's 14 identical dispatch cases become a status=>[job, queue] match; the two real behaviors (ready-email flow, force purge) move to named private methods. - The verbatim addGroupToProject error-unwrap block shared by the Generic and AmazeeClaw post-create traits becomes PolydockApp::addDeployGroupToLagoonProject(). - The 4-level config cascade written three times in the AmazeeClaw traits becomes one resolveInstanceOrAppConfig(); getDataFromPath() is replaced by Laravel's data_get(). - The duplicated app-resolution block in PolydockEngineFunctionCallerTrait becomes one resolveAppFunction(). - queue.failer binding keeps only the database-uuids branch actually configured (ponytail comment notes how to restore other drivers). - Drop queue-trait redeclarations in Trial jobs (BaseJob provides them) and a manual count loop; type UserGroup::users()/owners() generics and stop passing an enum where the exception ctor expects int.
Same admin UI, less repetition: - PolydockStoreAppResource: the 7 hand-copied "Lagoon Scripts" form sections and the matching 21 infolist entries are generated from one stage-prefix map; the 3 trial-email form sections and 3 infolist sections are generated from one prefix map the same way. - The four dashboard charts (new users, remote registrations, instances by store, instances by type) shared the same 7-bucket weekly window, MySQL week bucketing, fill loop, and bar type — extracted into an abstract WeeklyBarChartWidget; each chart keeps only its query and series definition. - ListPolydockAppInstances tabs: each tab's where-clause was written twice (query + badge count) with the in-progress status list duplicated wholesale — scopes are now defined once and reused. - Relation-manager status columns hand-wired color/icon/label closures that ->badge() already resolves via the enum's HasColor/HasIcon/ HasLabel — dropped.
Restore cuyz/valinor, spatie/laravel-slack-alerts, laravel/slack-notification-channel, spatie/mjml-php, and hotmeteor/spectator (with config/spectator.php) — they are planned for strict object typing, Slack alerting, MJML email styling, and OpenAPI contract tests against both polydock-engine and the consumed api.amazee.ai endpoints. Also drop the stale PollHealthJob PHPStan baseline entry and the deleteKeyValue leftover on the AmazeeClaw test double.
…e log wording Address review: the queue.failer override is now only installed when queue.failed.driver is database-uuids — any other driver keeps Laravel's stock resolution instead of silently binding a null provider and losing failed-job records. Also restore the POST_UPGRADE_COMPLETED log line to 'Polling should start now.' (it is the only status mapping to no next stage, so the generic end-of-line wording was misleading).
- Memoize startDate() so the query window and label window come from the same instant (a render straddling a week boundary could previously fetch and label from different weeks). - weekBucketSql() now rejects anything but a plain [table.]column identifier before interpolating into DB::raw(). - Document why the fill loop uses ->count ?? 0 rather than ?->: the null coalesce already has isset() semantics and PHPStan's nullsafe.neverNull rule rejects ?-> on the left of ??.
- .env.example documents the Slack bot-token vars (notification channel), SLACK_ALERT_WEBHOOK (spatie/slack-alerts — its jobs run on the 'default' queue, which Horizon supervisor-1 already consumes), and SPEC_PATH. - Spectator: default the local source to tests/fixtures/openapi (the engine's own spec is generated there via the new 'composer spec:export' script wrapping scramble:export; consumed-API specs like api.amazee.ai get vendored alongside), and set path_prefix to 'api' since Scramble exports paths relative to its /api server URL. The generated engine spec is gitignored — it is rebuilt from code so it can never go stale. - Add an OpenAPI contract smoke test proving the export+validate pipeline end to end against GET /api/store-apps. It also surfaced that endpoints documented with Scribe-style @response docblocks (e.g. /regions) produce broken Scramble schemas — noted in the test for the follow-up migration. - Valinor and mjml need nothing: mjml's node binary ships in the cli image (worker inherits it) and valinor is config-free until first use.
Scribe-style '@response <status> {...}' docblocks broke Scramble's schema inference: the status number was parsed as a literal body type, so six operations (/regions, /register, /register/{uuid}, /groups, /instance, /enums) published 'integer const <status>' as their entire 200-response schema in /api/openapi.json. Remove the @response blocks (Scramble infers responses from the code — verified all 2xx schemas are now real objects), convert @group to Scramble's @tags, and drop @subgroup (no equivalent). @Unauthenticated is Scramble-native and stays; the inert @bodyParam/@queryParam/@urlParam lines stay as human-readable docs. Add a /regions contract test locking in the fix (this endpoint is how the breakage was found) and hoist spec export into the test setUp.
Adds focused tests for the five areas a qualitative gap analysis ranked highest by silent-breakage risk: - Status-change listener: pins the full pending-status -> stage-job/queue dispatch table (a mis-mapped case strands instances mid-lifecycle), the completed -> ProgressToNextStage path, the ready-email + registration SUCCESS flow, the manual-rerun skip marker, and both force-purge branches. - UserGroup allocation: pool grab via allocation_lock, no double allocation of a locked instance, fresh-create fallback (born NEW until ProcessNewJob advances it), and custom-name pool bypass. - LagoonProjectPurgeService: every attemptPurge branch — adopted guard, missing/unknown project, lingering-environment deletion + wait, empty-project purge, Lagoon refusal, API exception, and project_name precedence. FakeLagoonClient gains delete/env stubs. - EmailBlockerService: invalid format, exact + wildcard banned patterns, disposable domains incl. the subdomain-hierarchy walk, clean pass. - getPrivateAICredentialsFromBackend via Http::fake: existing-user reuse, user creation, @autogen.null fallback, missing region, incomplete credential response, non-admin token rejection. Also promotes DoublePolydockAppInstance from AmazeeClawConfigTest into tests/Doubles (third consumer), and removes the placeholder ExampleTest.
Implements plans 003 and 004 from the advisory audit: - LagoonHelper read FTLAGOON_ENDPOINT via env() at request time; under `php artisan config:cache` (standard in production) env() returns null, so region lookups silently fell back to the hard-coded endpoint and admin region labels rendered blank. Read the already-existing config key instead, drop the now-fixed larastan baseline entry, and pin the behavior with a unit test. - BaseJob issued ~8 SELECTs per lifecycle job before any real work: getPolydockJobId() did find()+refresh() and was called from middleware(), polydockJobStart() (which also did its own find()+refresh()), and polydockJobDone(); storeApp then lazy-loaded twice for log context. getPolydockJobId() is now DB-free (the unique-id format is unchanged and needs only the constructor id), and polydockJobStart() performs the single fetch with storeApp eager-loaded — find() is already fresh, so the refresh() calls were pure waste. Existence is still enforced once per job in polydockJobStart().
…tention - Schedule horizon:snapshot every five minutes — the Horizon metrics dashboard has been permanently empty because snapshots were configured for retention but never taken. - Register DatabaseCheck and RedisCheck alongside the existing HorizonCheck; a Redis or DB degradation now surfaces on the health endpoint instead of only as cascading job failures. - BaseJob::failed() now pages Slack via the installed spatie/laravel-slack-alerts, deduplicated to one message per job class per five minutes so a systemic outage cannot flood the channel; guarded by webhook config and the existing never-throw block. - New polydock:prune-instance-logs command (scheduled daily, --dry-run, POLYDOCK_INSTANCE_LOG_RETENTION_DAYS default 7): operational instance logs (health pings, stage chatter) stop mattering after about a week, while the durable who-did-what audit trail stays in activity_log under its own retention. Deletes are chunked via plucked primary keys — LIMIT on DELETE is MySQL-only grammar and silently unbounded elsewhere. - Add generics to PolydockAppInstance::logs() while touching it.
Implements plans 005 and 006 from the advisory audit: - The instance list fired six separate COUNT queries per render for its tab badges; all badges now derive from one grouped status-count query, pinned by a new badge-correctness test with mixed-status fixtures. - Dashboard StatsOverview ran three uncached full-table COUNT(*) per render; the totals are now cached for 60 seconds. - The store-app infolist re-counted allocatedInstances although getEloquentQuery() already eager-counts it; reuse the withCount value. - CI loaded xdebug but never collected coverage — paying the slowdown for nothing. Switch to pcov and gate with --min=35: baseline measured at 37.15% lines (via a php:8.4-cli + pcov container run), floor set two points under per the maintainer decision. Ratchet upward deliberately; never lower silently.
Implements plans 007 and 008 from the advisory audit:
- LagoonClientService minted a token over SSH (process spawn, 30s
timeout) on every call unless a fetcher was bound — the deployment
service pays that per redeploy trigger and per poll, so rollout bursts
serialized workers behind repeated SSH auth. Successful tokens are now
cached for 110 seconds (under the FTLagoon provider stack's 2-minute
max token age), keyed on user/server/port/keyfile via a public static
tokenCacheKey() so tests and implementation cannot drift. Failures ('')
are never cached — one SSH blip must not poison callers for the TTL —
and a bound token_fetcher still short-circuits everything.
- The pre-warm poller evaluated needs_unallocated_maintenance per store
app every 5 seconds — up to ~3N COUNT/EXISTS queries per tick, scaling
with catalog size regardless of change. The tick now eager-loads the
pool count in one grouped withCount (the accessor short-circuits on the
preloaded attribute, unchanged) and checks the query-free deficit
condition first, so idle ticks skip the EXISTS probes entirely. The
per-tick body moved to checkOnce() so tests don't fight the sleep loop;
the truth table (deficit dispatches / satisfied pool doesn't) is pinned.
PollDeployProgressAppInstanceTrait is left unmigrated: it is a polling trait with no running/completed transition of its own — it never sets a *_RUNNING status, derives the instance status from a Lagoon deployment state mapping, and early-returns without a terminal status, which does not fit the validate -> running -> body -> completed template.
UpgradeAppInstanceTrait is left unmigrated: it sets UPGRADE_RUNNING without persisting it (no ->save()), while the template always saves the running status. Saving would fire the UPGRADE_RUNNING status-changed event and its listener dispatch, changing behavior.
Implements plan 010 from the advisory audit: - Both Polydock service providers hand-rolled the six logger methods that PolydockAppLoggerTrait already supplies — and their setLogger copies lacked the trait's cache-logger flush, silently dropping startup log lines buffered before a real logger attached. Both now use the trait (~100 lines removed); the trait's over-specific @return docblocks (hardcoding its two previous consumers) become @return $this. - LagoonProjectPurgeService::makeWithDefaults() duplicated the exact client resolution its lazy client() method performs — dropped the eager copy. - The authenticated API implemented the mutually-exclusive group_id/group_slug validation + lookup three times (getInstances, assignInstanceToGroup, resolveTargetGroup). One shared resolveExistingGroupFromRequest() helper now serves all three, with byte-identical validation messages; resolveTargetGroup keeps its group_name/personal-group creation paths.
Incident response: bulk deployment triggers overloaded Lagoon with concurrent builds. Two sources, both now staged: - Scheduled redeploys ran up to 50 instances every 10 minutes, and the NULLs-first next_redeploy_at ordering front-loaded entire cohorts (adopted projects, newly enabled cadence) in one burst. Now: max 10 per run (POLYDOCK_DEPLOY_MAX_PER_RUN default lowered), hourly schedule, and most-outdated-first ordering (never-redeployed counts as oldest, then by oldest last deployment) so backlogs drain smoothly. - Pre-warm refresh queued EVERY stale instance for removal at once and recreated the full deficit in a single pass — each recreation is a full Lagoon create+deploy. Now: one refresh batch per hour globally (cache gate), batch size capped (POLYDOCK_PREWARM_BATCH default 10, oldest instances first via the removal query's existing ordering), and the creation loop is capped per app per pass — in-progress instances count toward the pool, so large deficits fill over successive passes. Both behaviors are test-pinned: oldest-first trigger order, per-run caps, the hourly refresh gate (including the next batch after the hour), and capped deficit fill.
Implements plan 011 (promoted from the audit's parked DX-04 by maintainer decision): "how long does each stage take — pre-warm to claimed, fresh to claimed, and time sitting unclaimed" was previously unanswerable because the activity log deliberately does not record status and no transition history existed. - New polydock_app_instance_status_transitions table: one immutable row per status change (from/to/created_at), cascade-deleted with the instance, indexed on (instance, created_at). - New RecordPolydockAppInstanceStatusTransition listener on the existing PolydockAppInstanceStatusChanged event (which already carries previousStatus). Synchronous by design — a single INSERT whose ordering makes duration math trustworthy. - PolydockAppInstance gains statusTransitions() plus three helpers: secondsBetweenStatuses(), secondsUnclaimedBeforeClaim() (pool wait), and secondsFromCreationToClaimed(). All return null for instances that predate recording. - The instance view page gains a collapsed "Stage Timings" section: headline durations (New → Claimed, pool wait) plus every transition with its timestamp and delta; hidden when no transitions exist. - 5 tests: recording, pool-wait math under time travel, creation→claimed, null for unobserved statuses, cascade delete.
Review feedback: before the template-method refactor the ': starting' info line fired unconditionally ahead of the adopted guard, so adopted detaches always produced it. The refactor deferred it into runLifecyclePhase, which only the non-adopted path reaches — restore the line inside the adopted guard in all three Remove traits so log correlation is unchanged.
…erty Regression from #232: PolydockAppLoggerTrait::setLogger() reads $this->logger for the cache-logger flush check, but the FTLagoon and AmazeeAiBackend service providers call setLogger() from their constructors — before the typed property has ever been assigned. Every lifecycle job on dev fataled with "Typed property PolydockServiceProviderFTLagoon::$logger must not be accessed before initialization" (13k+ failed jobs, ~3k instances stuck in pending-pre-remove). One-line fix: isset() guard ahead of the instanceof check. Two regression tests: the exact production failure path (constructing the FTLagoon provider with a fresh logger) and the cache-flush behavior the guard must not break. After deploy, retry the accumulated failures: php artisan queue:retry all
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes in this release:
31de863 chore: update deps
2d1d860 chore: add per-app project naming prefix to store apps
2b9696a fix: guard the logger cache-flush check against an uninitialized property
17bfe7c chore: restore the starting log line for adopted-instance removals
8667216 chore: record status transitions and surface per-stage durations
52f70c9 chore: throttle deployment bursts that overloaded the Lagoon core
f1e2e57 chore: reuse the shared logger trait and dedupe group/client resolution
9d1ec70 chore: add unit test pinning runLifecyclePhase's three exit paths
877c9d2 chore: migrate AmazeeClaw and DependencyTrack traits to runLifecyclePhase
95a9b11 chore: migrate Generic Claim trait to runLifecyclePhase
264428f chore: migrate Generic Upgrade traits to runLifecyclePhase
b2a06df chore: migrate Generic Deploy traits to runLifecyclePhase
80fb2be chore: migrate Generic Create traits to runLifecyclePhase
330c8b0 chore: migrate Generic Remove traits to runLifecyclePhase
0776284 chore: add runLifecyclePhase template method to Generic PolydockApp
a5f8982 chore: cache the Lagoon token and aggregate the pre-warm pool check
2017ef2 chore: slim Filament per-render queries and gate CI coverage via pcov
efadfe2 chore: wire up queue metrics, health checks, Slack alerts, and log retention
0ad4da0 chore: fix LagoonHelper under config:cache and collapse BaseJob queries
a8bc6cf chore: cover the highest-risk untested paths
45333b4 chore: migrate API controller docblocks from Scribe to Scramble syntax
82a2859 chore: wire up the kept dependencies for their planned use
1c1d459 chore: harden WeeklyBarChartWidget per review
f287691 chore: scope the failed-job override to its driver and restore upgrade log wording
cf8f055 chore: keep planned dependencies and tidy review leftovers
18c62d0 chore: dedupe Filament schemas and dashboard chart widgets
352e27c chore: collapse duplicated job, mail, and trait logic
9334674 chore: prune dead code, unused dependencies, and redundant config
6705cb9 chore: fail user-mode claims without an email and silence user-mode post-create warning
25adcb6 chore: guard pre-remove for adopted projects and fail claims loudly
a210dd6 chore: read MAIL_CC_ALL via config instead of env()
349eacf chore: deprecate Private AI app and add amazee.ai key-mode selection
2d02cda chore: claim existing Lagoon projects as adopted Polydock instances
68cb08b chore: exempt public hosted forms from CSRF for cross-origin iframe embedding
05772ad chore: update deps, update scramble