Never garbage-collect data on an isCRR location

[francoisferrand]

Data living on an isCRR location is production data owned by a remote site. We may read it (that is the whole point of localizing a clean room object), but we must never delete it. That is a general rule keyed on the location type, not a clean room conditional and not behind a config flag.

It needs to hold at two independent points, one commit each:

BB-813 - do not trigger the GC. The copy engine reuses the lifecycle transition pipeline: CopyLocationTask copies the data locally, then LifecycleUpdateTransitionTask merges the new location into the metadata and publishes a deleteData action for the old one. Against an isCRR source that would wipe the remote production copy, so unlike a transition, the localization merge leaves the from-location alone and publishes nothing when there is nothing left to collect. This also gives us replay safety for duplicate copy actions: the second merge supersedes the first and collects its copy, which is only safe because the first never touched the remote source.

BB-818 - guard the GC service. The same rule applied where data actually gets deleted, per location part, whatever published the action. It matters on its own: restored-object expiration goes through a different publisher, so expiring a clean room object that was never localized would otherwise delete the production copy. Reaching the GC service with an isCRR location means something upstream is wrong, so it warns, but deleting is never the right answer - the entry is still completed and the offset committed, and no completion metric is emitted for a delete that did not happen.

The two share a small lib/util/locations.js helper, which is why they are in one PR rather than two.

This is a prerequisite for BB-814 (#2826), which is what starts publishing copyLocation actions for clean room objects; it should not be enabled in a real clean room without this. There is no code dependency on that branch though - the two are independent apart from the shared location-crr-source test fixture, which is byte-identical on both.

One test-only side effect worth flagging: adding a location to the conf/locationConfig.json fixture breaks a CircuitBreakerGroup case that deep-compares a probe array built from every location, so its expectations are updated here too.

Issue: BB-813
Issue: BB-818

[bert-e]

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request.
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Incorrect fix version

The Fix Version/s in issue BB-813 contains:

• None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 9.6.0

Please check the Fix Version/s of BB-813, or the target
branch of this pull request.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.54%. Comparing base (929145d) to head (fc65001).

Additional details and impacted files

Files with missing lines	Coverage Δ
extensions/gc/tasks/GarbageCollectorTask.js	88.37% <100.00%> (+0.46%)	⬆️
...s/lifecycle/tasks/LifecycleUpdateTransitionTask.js	94.23% <100.00%> (+0.23%)	⬆️
lib/util/locations.js	100.00% <100.00%> (ø)

... and 2 files with indirect coverage changes

Components	Coverage Δ
Bucket Notification	80.27% <ø> (ø)
Core Library	81.35% <100.00%> (-0.61%)	⬇️
Ingestion	70.09% <ø> (ø)
Lifecycle	80.52% <100.00%> (+0.05%)	⬆️
Oplog Populator	85.83% <ø> (ø)
Replication	62.01% <ø> (ø)
Bucket Scanner	85.76% <ø> (ø)

@@                 Coverage Diff                 @@
##           development/9.6    #2828      +/-   ##
===================================================
- Coverage            75.77%   75.54%   -0.24%     
===================================================
  Files                  200      201       +1     
  Lines                13922    13934      +12     
===================================================
- Hits                 10549    10526      -23     
- Misses                3363     3398      +35     
  Partials                10       10              

Flag	Coverage Δ
api:retry	9.10% <0.00%> (-0.01%)	⬇️
api:routes	8.86% <0.00%> (-0.01%)	⬇️
bucket-scanner	85.76% <ø> (ø)
ft_test:queuepopulator	9.13% <0.00%> (-1.90%)	⬇️
ingestion	12.25% <0.00%> (-0.02%)	⬇️
lifecycle	19.28% <0.00%> (-0.04%)	⬇️
notification	1.02% <0.00%> (-0.01%)	⬇️
oplogPopulator	0.13% <0.00%> (-0.01%)	⬇️
replication	18.84% <0.00%> (-0.02%)	⬇️
unit	54.92% <100.00%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[francoisferrand]

Note on the red checks, neither is caused by this branch:

• lib tests: all failures are in tests/functional/lib/BackbeatConsumer.js ("ledger drain on rebalance", then a cascade of Local: Erroneous state poisoning the following suites). PR Skip non-localized entries in replication and lifecycle #2827 and PR Trigger localization of clean room objects #2826 fail with the exact same assertion ("expected committed offset to advance from N to N+1 after deferred onEntryCommittable") and neither touches BackbeatConsumer either, so it looks like a flaky/broken test on development/9.6.
• review / claude-review: infra, the job dies at checkout with git exit 128 and warns that ACTIONS_APP_PRIVATE_KEY is not set.

Everything else is green.

[delthas]

lib tests: all failures are in tests/functional/lib/BackbeatConsumer.js ("ledger drain on rebalance", then a cascade of Local: Erroneous state poisoning the following suites). PR Skip non-localized entries in replication and lifecycle #2827 and PR Trigger localization of clean room objects #2826 fail with the exact same assertion ("expected committed offset to advance from N to N+1 after deferred onEntryCommittable") and neither touches BackbeatConsumer either, so it looks like a flaky/broken test on development/9.6.

Now fixed, rebase and enjoy :)