feat(router): parallel candidate route-group setup (steady-connection fix)

[0pcom]

Problem (root-caused live)

Route-group setup walked candidate routes sequentially. In DialRoutes, the policy picked ONE route, saveRouteGroupRules set it up with one handshake + the #4057 retransmit over handshakeAwaitTimeout (~10s); only on failure did a bounded retry loop query a fresh route and try the next candidate — one at a time. Each dead / non-forwarding candidate cost up to the full ~10s handshake await, so a run of bad candidates burned the whole ~90s dial ceiling → Route group setup canceled … context deadline exceeded and the app wedged in "starting". Observed live: skysocks-client → Frankfurt at min_hops>=2 sitting in "starting" for minutes.

Fix

DialRoutes now sets up the top-K candidates concurrently; the first candidate to complete its reciprocal handshake wins and becomes the working base (the dial returns on it immediately — mux legs grow on top later and never unseat it). Losing / failed candidates are canceled + torn down. This collapses establishment from "up to N×handshakeAwaitTimeout" to ~one RTT and makes a dead-hop candidate lose the race instead of blocking — so the app converges to a working route (single multihop at min_hops>=2, or direct at min_hops==1) instead of wedging. min-hops-agnostic.

Where the race goes

Top of the DialRoutes retry loop, gated on K>1 && hookDone==nil && !forceLocal && !RouteSelectingHook. It fetches the top-K candidates in ONE route-finder query (fetchCandidateRoutes) and runs raceCandidateSetup. On total race failure it excludes the raced intermediates and falls through to the existing sequential path (with its local-calc fallback) in the same attempt. The old sequential path is fully preserved for K==1, cold-start (hookDone), local-route mode, and route-selecting policies. Post-setup wiring (mux/self-heal/hooks) was extracted into finishDial, shared by both paths.

Data-model constraint (why two phases)

Incoming handshake/data packets demux to a route group by RouteDescriptor, and every candidate of one dial shares one descriptor, so two reciprocal handshakes cannot run concurrently on it. The race is therefore two-phase:

• phase 1 (concurrent): reserve route IDs across every candidate's hops — where an unreachable intermediate fails id_reservation and loses cheaply (the dominant live failure mode);
• phase 2 (serial, fastest-reservation-first): complete the handshake on reserved candidates; first to complete wins. Because candidates are pre-reserved, moving to the next on a handshake failure costs no re-fetch and no re-reservation, unlike the old loop.

No leaked goroutines/route-IDs on cancel (child ctx + buffered result channel); winner's rules stay installed, losers' local rules removed (orphaned remote reservations expire via remote rule GC, same as failed candidates before).

Config knob

routing.parallel_route_setup → router.Config.ParallelRouteSetup, with a per-dial DialOptions.ParallelRouteSetup override. Default a small N (3); 1 restores strictly-sequential setup (clean rollback). Concurrency is bounded by maxParallelRouteSetup (5) and by the number of candidates the finder returns.

Loser penalty + composition with #4063

A candidate that loses or fails setup has its intermediate hops marked suspect for a short TTL (suspectHopCache), so the NEXT dial front-loads known-good hops (soft deprioritize in candidate ranking, not a hard exclude). Armed on every failure path — reservation failure, handshake timeout, and ctx-deadline / setup-canceled — which is the arming gap #4063 alone did not fire on live.

Composition with #4063 (fix/router-suspect-hop-failover, pkg/router/suspect_hops.go, per-destination TTL suspect cache): the router holds a single suspects field, consulted only in candidate ranking and armed only at the race's onLoser / DialRoutes failure paths. To reconcile, point those call sites at #4063's per-destination cache (add the destination PK) and drop this global one — the two are structured to be swapped without touching the race logic.

Tests

pkg/router/parallel_route_setup_test.go: winner selection (first handshake to complete wins), handshake-failure fall-through to the next reserved candidate, all-fail, single-candidate sequential-equivalent, ctx-cancel, suspect cache arm/TTL/count, K resolution + clamping, and candidate ranking (suspect deprioritization, hard-exclude, disjoint preference). Race-detector clean.

Validation done

go build ./... (root + router + visorcore), go vet ./pkg/router/, targeted go test ./pkg/router/ -race. Not merged — core routing; leaving for live validation.

Suggested live validation (coordinator)

On the native visor: with routing.parallel_route_setup: 3 (default), start skysocks-client to a Frankfurt exit at min_hops=2 and measure re-establishment time after a route drop vs. parallel_route_setup: 1 (sequential). Expect: sequential wedges/takes minutes on dead candidates; parallel reaches "running" in ~one RTT and never wedges in "starting". Also confirm a min_hops=1 direct dial still behaves (single-candidate → sequential path, no race).