Skip to content

ci(e2e): improve CI verbosity so the suite isn't dark for minutes #41

@shreemaan-abhishek

Description

@shreemaan-abhishek

Surfaced when watching PR #39 CI in real time — the e2e job appears to be doing nothing for the first 2+ minutes of every run, even on healthy runs. Not a misconfiguration; default test-framework behavior. Worth fixing because the silence makes diagnosis hard.

What's happening today

.github/workflows/e2e.yml invokes make test-e2e, which is:

test-e2e:
	go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m ./test/e2e/...

Three things compound to produce the dead air:

  1. Tests aren't Ginkgo specs. Only e2e_suite_test.go calls RunSpecs. Everything else is plain func Test...(t *testing.T). So the Ginkgo runner is effectively just delegating to go test and isn't applying its own per-spec streaming.
  2. go test default verbosity is silent. Without -v, Go's test framework prints nothing between tests. --- FAIL: TestX appears only on failure; PASS only at the very end. A passing run prints nothing for the whole duration.
  3. TestMain takes 30-120 seconds before any test starts. test/e2e/setup_test.go does (silently) go build ./cmd/a7, then waitForHealthy(adminURL, 120s), then resolveGatewayGroupID(...). None of it prints anything.

Combined: a healthy CI run shows zero output for 2+ minutes, then either a sudden ok github.com/api7/a7/test/e2e ... or a single --- FAIL: line.

Proposed fix

Small, two-file PR:

Makefile — add -v to the ginkgo invocation:

 test-e2e:
-	go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m ./test/e2e/...
+	go run github.com/onsi/ginkgo/v2/ginkgo -r --procs=1 --tags=e2e --timeout=45m -v ./test/e2e/...

Same for test-e2e-full.

test/e2e/setup_test.go — three progress markers in TestMain (to stderr, so they're not swallowed by test framework stdout buffering):

fmt.Fprintln(os.Stderr, "Building a7 binary...")
// go build
fmt.Fprintln(os.Stderr, "Waiting for API7 EE dashboard at "+adminURL+" ...")
// waitForHealthy
fmt.Fprintln(os.Stderr, "Resolving gateway group ...")
// resolveGatewayGroupID

(The "Resolved gateway group ..." line at the end already exists; these add visibility to the steps before it.)

Trade-offs

  • Log size: bigger logs (one or two lines per test). GitHub Actions log retention is generous; this is almost always the right trade.
  • No behavioral change. Tests run the same way, with the same coverage.

Future option (post-GA)

The tests aren't actual Ginkgo specs, so the Ginkgo runner is overhead. Could simplify to go test -tags=e2e -v -count=1 -timeout=45m ./test/e2e/.... Defer to post-GA; not blocking.

Done when

  • make test-e2e prints test names as they run.
  • The dark TestMain setup phase has visible progress markers.
  • CI log on a passing run shows === RUN TestX / --- PASS: TestX lines for each test.

Part of #22.

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions