test(smoke): drop order-dependent ListBuckets cases that flake under -shuffle

[Peeja]

I asked Claude to fix these flaky tests, and it first tried to just delete them, and then did a bunch of complicated work. I have no idea if anything in here is helpful; I haven't looked much yet myself. But maybe there's at least a starting point here?

---

What

Removes three cases from the known-passing TestSmoke_ListBuckets:
success, truncated, and with_prefix. Keeps empty_success and
invalid_max_buckets.

Why — the smoke-test flake

The go-test smoke suite failed non-deterministically:

• commit 127c080 — failed on two OSes
• PR fee/ecdhkw: ECDH-ES+A256KW tenant recipient wrap/unwrap over X25519 (FIL-471) #3 — failed on one OS (ubuntu; macos + windows passed)

In every case the failing assertion was the same:

--- FAIL: TestSmoke_ListBuckets/success
bucket names are not equal: test-bucket-10 != test-bucket-6
expected list buckets result to be test-bucket-6,7,8,9,10,11,
            instead got            test-bucket-10,11,6,7,8,9

TestSmokeXFail_HeadObject (the test the logs end on) actually passes — the
package FAIL comes from TestSmoke_ListBuckets earlier in the run.

Root cause

The upstream versitygw cases build their expected slice in bucket creation
order and compare it positionally (compareBuckets, no sort) against the
ListBuckets response. ingot returns buckets in lexicographic order
(s3frontend/bucket.go — sort.Slice by name), which is the correct S3 contract:

• it matches versitygw's own posix backend (os.ReadDir returns entries sorted by name), and
• it matches AWS's paginated ListBuckets, which the truncated case drives via MaxBuckets/ContinuationToken (ingot paginates by name: st.Name <= ContinuationToken).

Creation order equals lexicographic order only while the bucket names stay the
same width. Upstream names buckets test-bucket-N from a process-global
counter (bcktCount), and the unified Go-test workflow runs go test -shuffle.
Shuffle randomizes how many getBucketName() calls run before these cases, so
whether a case's window of sequential names straddles a digit-width boundary
(…-9 → …-10, which sorts as 10,11,6,7,8,9) is random per OS/seed. Hence the
flake — and it's not an ingot bug.

with_prefix has the identical defect (prefixed buckets at counter offsets
v0, v0+2, v0+4; misorders for v0 ∈ {6,7,8,9}), so it's removed too even
though it happened to pass on these particular seeds.

Why removal (not XFail)

The framework is binary: TestSmoke_* must always pass; TestSmokeXFail_* must
always fail (an unexpected pass t.Errorfs). These cases flake — they pass
whenever no boundary is straddled — so they fit neither bucket. They can be
restored once the upstream comparison is made order-independent.

Verification

• Reproduced the exact failure locally by running the cases until the counter
  crossed 9 → 10: success failed with test-bucket-7..12 → 10,11,12,7,8,9
  and with_prefix failed at the same boundary; other windows passed —
  confirming the non-determinism.
• After the change, go test ./testing/ is green across -shuffle=1/42/99999
  and under -race -shuffle=on.
• gofmt / go vet clean.

Note: the go-check / All failure on PR #3 is unrelated and is being handled
separately.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C9Aqx9yLR5im6Qrm9y7aqz

---

Generated by Claude Code

[Peeja]

Can we keep these and make them non-order-dependent to make them non-flaky?

[Peeja]

Good call — done in 63ee5e3, kept all three rather than dropping them.

success/truncated/with_prefix are now local, order-independent cases (testing/listbuckets_smoke_test.go) wired into the same table. Each creates its buckets under a unique Prefix (isolating them from the process-global test-bucket-N counter and from any other buckets) and asserts ListBuckets returns the lexicographically-sorted set it created — independent of creation order.

The flake came from the upstream cases comparing the response positionally against creation order, while ingot (correctly) returns buckets sorted lexicographically; under -shuffle the shared counter intermittently pushed a case's names across a digit-width boundary (…-9 → …-10), so creation order ≠ sorted order only on some seeds. The local versions deliberately straddle that boundary so a regression that stopped sorting is still caught.

Coverage preserved: full list + ordering, MaxBuckets/ContinuationToken pagination, and prefix filtering. Verified green across -shuffle=1/7/42/1000/999999 and under -race -shuffle.

---

Generated by Claude Code

[frrist]

I think the issue here is that state is being preserved accross test runs, my hunch at least. If it's alright with you, I'd like to leave this in a holding pattern until #9 lands - which re-writes and completes much of the currently incomplete implementation. When that lands and if we are still getting flakes here we can circle back?

[Peeja]

Yeah, that sounds great. I'll just close this for now, and we can reopen it if we need to.