Exclude test_rolling_upgrade from integrationTests; add testSmokeBats task

[dsmiley]

See dsmiley#27

test_rolling_upgrade.bats requires Docker and published images, making it unsuitable for the standard integrationTests run. Since bats does not recurse into subdirectories, moving the file is sufficient to exclude it.

Changes

• test/smoke/test_rolling_upgrade.bats — moved from test/ into a new test/smoke/ subdirectory; integrationTests (which runs bats test/) will no longer discover it
• build.gradle — adds a testSmokeBats task (type BatsTask, testDir = 'test/smoke') with the same env-var setup as integrationTests, intended for standalone invocation (e.g. from smokeTestRelease.py when Docker is present)

# Run smoke tests explicitly
./gradlew :solr:packaging:testSmokeBats
# or directly
bats solr/packaging/test/smoke/test_rolling_upgrade.bats

[dsmiley]

Developed with Copilot Agent.

TODO: update the smoketester python script to call this gradle task. Or maybe simply update that Jenkins job to call it.

[epugh]

I am somewhat ambivalent about this... I guess I am worried that by carving this into it's own test infrastructure, that we end up never running it. I worry it will become like some of the Nightly tests, that never get run by us locally and therefore when they do get run they break, which is frustrating, and or no one pays attention to the errors.

Thinking out loud...

1. How could we make the test faster? It might be interesting to try and understand why it feels slow... Maybe identify some things we need to work on?

2. What about moving it to run as part of the Docker tests? That way we could test rolling upgrade against a freshly built Docker image which might be useful.

3. We are moving towards more Docker based testing infrastructure, like for Tika. Maybe we just declare that Docker is required for the iTest?

What do you think the criteria SHOULD be for moving items into testSmokeBats? Maybe having clearer expectations and having more tests moving would help me wrap my head around it. Right now it just feels like we are sending this one test off to bit rot!

[epugh]

how much of the below lines of code could be refactored out so we don't list this code twice, once for iTest and a second time for testSmokeBats?

[dsmiley]

Yeah I don't like repetition; I suspected it but didn't look. I may try to reduce it.

What I want to see happen (separate issue/PR) is what I said in my dev list email -- this tech becoming a kind of gradle build infra that can be used wherever we want -- not just specifically where it lives now. If we do that, then I would imagine a new gradle task type for BATS for Solr that would have such vars built-in and we wouldn't need to say much to add a new gradle target using it.

[dsmiley]

A failure here only needs to be caught at release time, so I'm okay with the risk of us collectively not noticing till then.

Even in saying that, I'm implying that absolutely nobody ever reviews the state of our jenkins jobs. While somewhat true, and a freaking crying shame, I think that should be solved independently out of scope of this conversation. @iamsanjay and I are discussing ideas to address that. I don't think we should expect GHA/PR to be our total coverage of tests, albeit certainly the vast majority -- yes.

How could we make the test faster? It might be interesting to try and understand why it feels slow... Maybe identify some things we need to work on?

We'll always have some tests that are not fast so we need to deal with it. Personally I'm not interested in optimizing this specific test further but go for it if you are so inclined.

What about moving it to run as part of the Docker tests? That way we could test rolling upgrade against a freshly built Docker image which might be useful.

Did you see my dev-list thread?: For this rolling upgrade test, Docker isn't the point but is a practical implementation choice.

We are moving towards more Docker based testing infrastructure, like for Tika. Maybe we just declare that Docker is required for the iTest?

Shrug; I'm not moving this out because it uses Docker. I'm moving it out both because it's expensive and because a failure here is very unlikely and it's acceptible to be caught at release time. But it's very important for an eventual failure here to be noticed by us before a release is made. I think "smoke" fits the bill.

I was looking at our smoke tester, and I wonder if much of it could be converted to bats tests. So that also was a factor for me choosing to create a new "testSmokeBats" target & directory, even though it only has one test there now.

[dsmiley]

Admittedly I did a faux pax. The description is entirely LLM generated; I should have reviewed it; sorry! It over-emphasizes the Docker nature of the test. That isn't the point.