Surfaced when watching PR #39 CI in real time — the e2e job appears to be doing nothing for the first 2+ minutes of every run, even on healthy runs. Not a misconfiguration; default test-framework behavior. Worth fixing because the silence makes diagnosis hard.
What's happening today
.github/workflows/e2e.yml invokes make test-e2e, which is:
test-e2e:
go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m ./test/e2e/...
Three things compound to produce the dead air:
- Tests aren't Ginkgo specs. Only
e2e_suite_test.go calls RunSpecs. Everything else is plain func Test...(t *testing.T). So the Ginkgo runner is effectively just delegating to go test and isn't applying its own per-spec streaming.
go test default verbosity is silent. Without -v, Go's test framework prints nothing between tests. --- FAIL: TestX appears only on failure; PASS only at the very end. A passing run prints nothing for the whole duration.
TestMain takes 30-120 seconds before any test starts. test/e2e/setup_test.go does (silently) go build ./cmd/a7, then waitForHealthy(adminURL, 120s), then resolveGatewayGroupID(...). None of it prints anything.
Combined: a healthy CI run shows zero output for 2+ minutes, then either a sudden ok github.com/api7/a7/test/e2e ... or a single --- FAIL: line.
Proposed fix
Small, two-file PR:
Makefile — add -v to the ginkgo invocation:
test-e2e:
- go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m ./test/e2e/...
+ go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m -v ./test/e2e/...
Same for test-e2e-full.
test/e2e/setup_test.go — three progress markers in TestMain (to stderr, so they're not swallowed by test framework stdout buffering):
fmt.Fprintln(os.Stderr, "Building a7 binary...")
// go build
fmt.Fprintln(os.Stderr, "Waiting for API7 EE dashboard at "+adminURL+" ...")
// waitForHealthy
fmt.Fprintln(os.Stderr, "Resolving gateway group ...")
// resolveGatewayGroupID
(The "Resolved gateway group ..." line at the end already exists; these add visibility to the steps before it.)
Trade-offs
- Log size: bigger logs (one or two lines per test). GitHub Actions log retention is generous; this is almost always the right trade.
- No behavioral change. Tests run the same way, with the same coverage.
Future option (post-GA)
The tests aren't actual Ginkgo specs, so the Ginkgo runner is overhead. Could simplify to go test -tags=e2e -v -count=1 -timeout=45m ./test/e2e/.... Defer to post-GA; not blocking.
Done when
Part of #22.
Surfaced when watching PR #39 CI in real time — the
e2ejob appears to be doing nothing for the first 2+ minutes of every run, even on healthy runs. Not a misconfiguration; default test-framework behavior. Worth fixing because the silence makes diagnosis hard.What's happening today
.github/workflows/e2e.ymlinvokesmake test-e2e, which is:test-e2e: go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m ./test/e2e/...Three things compound to produce the dead air:
e2e_suite_test.gocallsRunSpecs. Everything else is plainfunc Test...(t *testing.T). So the Ginkgo runner is effectively just delegating togo testand isn't applying its own per-spec streaming.go testdefault verbosity is silent. Without-v, Go's test framework prints nothing between tests.--- FAIL: TestXappears only on failure;PASSonly at the very end. A passing run prints nothing for the whole duration.TestMaintakes 30-120 seconds before any test starts.test/e2e/setup_test.godoes (silently)go build ./cmd/a7, thenwaitForHealthy(adminURL, 120s), thenresolveGatewayGroupID(...). None of it prints anything.Combined: a healthy CI run shows zero output for 2+ minutes, then either a sudden
ok github.com/api7/a7/test/e2e ...or a single--- FAIL:line.Proposed fix
Small, two-file PR:
Makefile— add-vto the ginkgo invocation:Same for
test-e2e-full.test/e2e/setup_test.go— three progress markers inTestMain(to stderr, so they're not swallowed by test framework stdout buffering):(The "Resolved gateway group ..." line at the end already exists; these add visibility to the steps before it.)
Trade-offs
Future option (post-GA)
The tests aren't actual Ginkgo specs, so the Ginkgo runner is overhead. Could simplify to
go test -tags=e2e -v -count=1 -timeout=45m ./test/e2e/.... Defer to post-GA; not blocking.Done when
make test-e2eprints test names as they run.TestMainsetup phase has visible progress markers.=== RUN TestX/--- PASS: TestXlines for each test.Part of #22.