enable merge queues

[cgwalters]

Basically merge queues are the next logical step after going to automerge; they can help us solve two things:

• PR 1 and PR 2 pass tests but break when merged together (rare but happens)
• Reduce impact of flaky tests if we set "Only merge non-failing pull requests" to No (effectively the inverse of the above - PR 1 and PR 2 will merge if they pass tests together)

Plan:

• Change all of our actions to enable merge_group
• Test it out on one repo (maybe this one? ci-sandbox?)
• Investigate packit: https://packit.dev/docs/configuration mentions this so hopefully it can be made to work
• Try it out on bootc

[cgwalters]

OK I set this up on https://github.com/containers/composefs-rs/ and it seems to be working OK. Next step: all the repos ehre

[hone]

I was pointed to merge queue is susceptible to GitHub incidents as a potential thing to watch out for. I think it's still low risk.

[cgwalters]

Ouch, thanks for pointing that out!

It is indeed a risk, but...a while back when I was working on OpenShift core dev we had an infra issue which temporarily made our Prow instance (that effectively implemented a merge queue strategy before Github supported it natively) merge PRs that weren't passing tests. It was a huge mess.

There's always a tension here in basically how much one outsources infrastructure.

Note that in not using merge queues, if one is merging PRs one by one "manually" then it's absolutely possible to hit the "A xor B" problem, that just happened recently in bootc-dev/bcvk#238 (review) to us.

I hope Github ensures after this incident they have really good tests for their merge queues at least. Running a merge queue externally just shifts risk around I think.

[cgwalters]

Now enabled in bootc!

[cgwalters]

Note from discussion: We can generally shift CI focus/weight to the merge_group target, and e.g. there's generally much less need for postsubmit "push to main" jobs for example, and pull_request can be lighter weight

[cgwalters]

Investigating this now,

Note from discussion: We can generally shift CI focus/weight to the merge_group target, and e.g. there's generally much less need for postsubmit "push to main" jobs for example, and pull_request can be lighter weight

We actually need to do this with our required_checks now to really gain benefit.