Skip to content

Revert "[multiple] Use Python 3.12 and ansible-core 2.16.19 in CI" - #4302

Merged
Valkyrie00 merged 1 commit into
openstack-k8s-operators:mainfrom
michburk:bump-revert
Oct 27, 2026
Merged

Valkyrie00 merged 1 commit into
openstack-k8s-operators:mainfrom
michburk:bump-revert

Conversation

@michburk

Copy link
Copy Markdown
Contributor

This reverts commit f9a3410.

This reverts commit f9a3410.

Signed-off-by: Michael Burke <michburk@redhat.com>
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Revert Python 3.12 and ansible-core 2.16 CI migration

⚙️ Configuration changes 🐞 Bug fix 🕐 20-40 Minutes

Grey Divider

AI Description

• Restore ansible-core 2.15.13 and the system Python workflow across CI, documentation, and tests.
• Remove Python 3.12 provisioning and controller-specific dependency installation from CI and
 reproducer setup.
• Restore earlier repository setup and networking behavior as part of the migration rollback.
Diagram

graph TD
  Pins["Ansible 2.15 pins"] --> Bootstrap["CI bootstrap"] --> Controller["Reproducer controller"] --> Playbooks["Ansible playbooks"]
  Image["CI container"] --> Bootstrap
  Packages["CI setup packages"] --> Controller
  Bootstrap --> Tests["Test runners"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Repair the Python 3.12 migration
  • ➕ Retains the newer Python and Ansible versions.
  • ➕ Avoids returning to the older CI dependency baseline.
  • ➖ Requires coordinated validation of system-Python modules, controller tools, and both test modes.
  • ➖ Is a larger change if the immediate goal is to restore CI.

Recommendation: A coordinated rollback is reasonable for restoring the previous CI baseline; revisit the upgrade separately with end-to-end coverage. Before merging, verify the restored shell case patterns: y|yes|true): appears to match values ending in a colon, so ordinary USE_VENV=yes may skip virtualenv setup and test-runner selection.

Files changed (21) +31 / -148

Tests (3) +6 / -26
converge.ymlRemove controller interpreter import checks +0/-20

Remove controller interpreter import checks

• Deletes Molecule checks for Python 3.12 controller imports and system-Python Kubernetes imports.

roles/reproducer/molecule/crc_layout/converge.yml

test_approve_csr.pyRestore prior CSR assertion calls +3/-3

Restore prior CSR assertion calls

• Changes three CSR test assertions from 'assertEqual' back to 'assertEquals' without changing their expected values.

tests/unit/modules/test_approve_csr.py

test_bridge_vlan.pyRestore prior VLAN assertion calls +3/-3

Restore prior VLAN assertion calls

• Changes three bridge VLAN error assertions from 'assertEqual' back to 'assertEquals' without changing their expected values.

tests/unit/modules/test_bridge_vlan.py

Other (18) +25 / -122
MakefileRestore direct Molecule generator invocation +1/-3

Restore direct Molecule generator invocation

• Removes the Python 3.12-oriented interpreter variable and invokes the role Molecule generator through its executable script.

Makefile

bindep.txtDrop Python 3.12 development headers +0/-2

Drop Python 3.12 development headers

• Removes the RPM dependency added for building extensions against Python 3.12.

bindep.txt

common-requirements.txtRestore shared Ansible dependency pin +1/-4

Restore shared Ansible dependency pin

• Pins ansible-core back to 2.15.13 and removes the explicit netaddr and dnspython runtime pins introduced with the migration.

common-requirements.txt

Containerfile.ciRemove Python 3.12 from the CI image +1/-1

Remove Python 3.12 from the CI image

• Drops Python 3.12, its pip and development packages, and gcc from the CI image installation command.

containerfiles/Containerfile.ci

doc-requirements.txtAlign documentation with ansible-core 2.15 +1/-1

Align documentation with ansible-core 2.15

• Restores the documentation dependency pin to ansible-core 2.15.13.

docs/doc-requirements.txt

packages.ymlRemove local-controller pip Ansible installation +0/-6

Remove local-controller pip Ansible installation

• Deletes the task that installed the running Ansible version through pip3.12 after OS package setup.

roles/ci_setup/tasks/packages.yml

main.ymlRestore default CI package list +0/-4

Restore default CI package list

• Removes Python 3.12, its pip package, cryptography, and sshpass from the default CI setup package list.

roles/ci_setup/vars/main.yml

redhat.ymlRestore RPM-provided Ansible on Red Hat +1/-10

Restore RPM-provided Ansible on Red Hat

• Adds ansible-core back to the Red Hat package list and removes packages previously added to support the pip-installed Python 3.12 controller.

roles/ci_setup/vars/redhat.yml

generate_networking_data.ymlRestore inventory conversion command behavior +2/-1

Restore inventory conversion command behavior

• Removes the user-local executable PATH prefix from ansible-inventory and restores ignored errors for inventory conversion.

roles/libvirt_manager/tasks/generate_networking_data.yml

install.ymlRestore system-site access for repo-setup +1/-1

Restore system-site access for repo-setup

• Creates the repo-setup virtual environment with '--system-site-packages' again.

roles/repo_setup/tasks/install.yml

configure_bm_ocp_controller.ymlReturn bare-metal controller setup to default pip +1/-15

Return bare-metal controller setup to default pip

• Removes Python 3.12 build packages and interpreter-specific pip installs, including the separate system-Python Kubernetes dependencies. Restores non-failing behavior for the Ansible dependency installation task.

roles/reproducer/tasks/configure_bm_ocp_controller.yml

configure_controller.ymlRemove Python 3.12 controller provisioning +0/-14

Remove Python 3.12 controller provisioning

• Drops Python 3.12 and gcc packages, removes the explicit pip3.12 executable, and deletes the separate system-Python Kubernetes dependency installation.

roles/reproducer/tasks/configure_controller.yml

main.ymlRestore default controller execution environment +0/-3

Restore default controller execution environment

• Removes the user-local PATH override for dev-tool installation and the Python 3.12 interpreter override for CRC OpenShift setup.

roles/reproducer/tasks/main.yml

script.sh.j2Remove user-local executable PATH export +0/-3

Remove user-local executable PATH export

• Stops prepending '$HOME/.local/bin' to PATH in generated reproducer scripts.

roles/reproducer/templates/script.sh.j2

run_ansible_testRestore system-command Ansible test workflow +3/-12

Restore system-command Ansible test workflow

• Removes explicit Python and ansible-galaxy executable selection, uses system python3 to determine the Ansible version, and removes the hard failure after collection-install retries. The restored virtualenv case pattern includes a trailing colon.

scripts/run_ansible_test

setup_envRestore system-Python CI environment setup +9/-35

Restore system-Python CI environment setup

• Replaces Python 3.12 installation and interpreter selection with the discovered system python3 and pip3. Removes split-interpreter Kubernetes installation while restoring the prior system-pip requirements path; the virtualenv case pattern includes a trailing colon.

scripts/setup_env

setup_moleculeRestore Molecule pip selection +3/-6

Restore Molecule pip selection

• Removes Python 3.12 interpreter selection and returns to direct pip3 or virtualenv-pip installation of test requirements.

scripts/setup_molecule

test-requirements.txtRestore test Ansible dependency pin +1/-1

Restore test Ansible dependency pin

• Pins the test environment to ansible-core 2.15.13 instead of 2.16.19.

test-requirements.txt

@qodo-code-review

qodo-code-review Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (6) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. CentOS controller bootstrap loses Ansible 🐞 Bug ≡ Correctness
Description
ci_setup no longer installs Ansible through pip, and its generic package list does not include the
RPM added only for Red Hat. On a supported CentOS controller, bootstrap can reach the bare
ansible-playbook invocation without an executable supplied by that setup path; removing the
user-local PATH entry also prevents that invocation from finding a pip-installed copy.
Code

roles/ci_setup/tasks/packages.yml[L71-75]

-- name: Install Ansible tools for local controller operations
-  ansible.builtin.pip:
-    name: "ansible-core=={{ ansible_version.full }}"
-    executable: /usr/bin/pip3.12
-    extra_args: --user
Evidence
CentOS is supported, but the generic package list has no Ansible package; the reproducer later runs
ansible-playbook by name.

roles/reproducer/defaults/main.yml[37-42]
roles/ci_setup/vars/main.yml[21-27]
roles/ci_setup/vars/redhat.yml[1-10]
roles/reproducer/tasks/main.yml[425-435]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The generic controller package path no longer provisions a discoverable Ansible executable.
## Fix Focus Areas
- roles/ci_setup/tasks/packages.yml[66-70]
- roles/ci_setup/vars/main.yml[21-27]
- roles/reproducer/tasks/main.yml[425-435]
## Recommended Fix
Install Ansible on generic controller hosts as well as Red Hat hosts, and ensure bootstrap commands can resolve its executable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


2. Failed conversion can erase the inventory 🐞 Bug ≡ Correctness
Description
generate_networking_data.yml now ignores a failed ansible-inventory conversion, while the
following task still removes the source INI inventory. If conversion fails, the role continues with
a missing or incomplete YAML inventory and no source file to retry from.
Code

roles/libvirt_manager/tasks/generate_networking_data.yml[148]

+  ignore_errors: true  # noqa: ignore-errors
Evidence
The conversion writes the destination through shell redirection, and the next task deletes the same
source whenever it existed, regardless of the conversion result.

roles/libvirt_manager/tasks/generate_networking_data.yml[138-153]
roles/libvirt_manager/defaults/main.yml[93-94]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed inventory conversion is ignored, but the source inventory is still deleted.
## Fix Focus Areas
- roles/libvirt_manager/tasks/generate_networking_data.yml[142-153]
## Recommended Fix
Let conversion failures stop the task, or gate source deletion on a successful conversion and retain the failure for diagnosis.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗



Remediation recommended

3. CentOS secret generation can lack Python 🐞 Bug ≡ Correctness
Description
ci_setup removes python3-cryptography from its generic package list, while kustomize_deploy
explicitly runs system python3 to generate a Barbican Fernet key. On a CentOS host without that
module already installed, deployments that need a generated key fail at the import rather than
producing the secret.
Code

roles/ci_setup/vars/main.yml[26]

-  - python3-cryptography
Evidence
The generic package list no longer supplies cryptography, and the key-generation command imports it
using python3 when no key was found.

roles/ci_setup/vars/main.yml[21-27]
roles/kustomize_deploy/tasks/inject_osp_secret_keys.yml[122-133]
roles/kustomize_deploy/tasks/execute_step.yml[288-296]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Generic CI setup no longer guarantees the system-Python module used for Fernet key generation.
## Fix Focus Areas
- roles/ci_setup/vars/main.yml[21-27]
- roles/kustomize_deploy/tasks/inject_osp_secret_keys.yml[122-133]
## Recommended Fix
Keep `python3-cryptography` in the generic package list, or install it explicitly before the system-Python key-generation task.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


4. Controller dependency failures are hidden 🐞 Bug ☼ Reliability
Description
configure_bm_ocp_controller.yml sets failed_when: false on the task that installs the
controller's Ansible and Kubernetes requirements. If pip cannot install those requirements,
bare-metal controller setup continues until a later task needs a missing dependency, obscuring the
installation failure.
Code

roles/reproducer/tasks/configure_bm_ocp_controller.yml[147]

+      failed_when: false
Evidence
The suppressed pip task installs the requirements containing Kubernetes libraries; subsequent
controller setup invokes Kubernetes modules.

roles/reproducer/tasks/configure_bm_ocp_controller.yml[127-147]
common-requirements.txt[1-4]
roles/openshift_setup/tasks/patch_network_operator.yml[5-36]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The bare-metal controller suppresses pip installation failures despite relying on the installed requirements.
## Fix Focus Areas
- roles/reproducer/tasks/configure_bm_ocp_controller.yml[127-147]
## Recommended Fix
Remove the unconditional `failed_when: false`, or handle only a specifically expected failure while reporting other installation errors.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


5. Test runs can use a stale collection 🐞 Bug ≡ Correctness
Description
run_ansible_test no longer exits when all ansible-galaxy collection install attempts fail. If an
older collection already exists at the expected path, the script can append sanity exceptions there
and run tests against that old installation instead of the current checkout.
Code

scripts/run_ansible_test[L53-55]

-if [[ "$n" -ge "$ANSIBLE_GALAXY_RETIRES" ]]; then
-    echo "ERROR: collection installation failed after ${ANSIBLE_GALAXY_RETIRES} attempts" >&2
-    exit 1
Evidence
The retry loop has no post-loop success check, and every subsequent test command operates from the
collection installation directory.

scripts/run_ansible_test[42-62]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The test runner continues after every collection-install attempt fails and may test an existing stale collection.
## Fix Focus Areas
- scripts/run_ansible_test[42-60]
## Recommended Fix
Restore an explicit nonzero exit after exhausted retries, before reading collection files or launching tests.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


View medium (1)
6. CentOS iDRAC cleanup can skip a reset 🐞 Bug ≡ Correctness
Description
ci_setup removes sshpass from its generic package list, but bm_eject_vmedia.yml uses that
executable to check whether an iDRAC remote file share needs resetting. On a CentOS bare-metal
reproducer without a separately installed sshpass, the status command's failure is suppressed and
its empty output prevents the reset branch from running.
Code

roles/ci_setup/vars/main.yml[30]

-  - sshpass
Evidence
The reproducer prepare play runs CI setup before the bare-metal role; that role invokes sshpass,
suppresses status-command failures, and conditions its reset on status output.

reproducer.yml[67-81]
roles/ci_setup/vars/main.yml[21-27]
roles/reproducer/tasks/main.yml[209-230]
roles/bm_sno/tasks/bm_eject_vmedia.yml[29-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Generic CI setup drops the executable required to inspect and clear lingering iDRAC remote file shares.
## Fix Focus Areas
- roles/ci_setup/vars/main.yml[21-27]
- roles/bm_sno/tasks/bm_eject_vmedia.yml[29-56]
## Recommended Fix
Restore `sshpass` for generic hosts that run bare-metal SNO, and do not treat a missing status command as evidence that no reset is needed.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


Grey Divider

Context sources
Review mode: Auto: ⚖️ Balanced: Downgraded extended -> standard: change is below the extended eligibility bar (hunks 34/18, lines 179/200; both must reach the floor). Router rationale: Broad CI and Ansible changes span many independent, defect-prone paths.

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

ansible-inventory
-i {{ cifmw_libvirt_manager_tmp_inv_file }}
-y --list > {{ cifmw_libvirt_manager_dst_inv_file }}
ignore_errors: true # noqa: ignore-errors

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Failed conversion can erase the inventory 🐞 Bug ≡ Correctness

generate_networking_data.yml now ignores a failed ansible-inventory conversion, while the
following task still removes the source INI inventory. If conversion fails, the role continues with
a missing or incomplete YAML inventory and no source file to retry from.
Agent Prompt
## Issue description
A failed inventory conversion is ignored, but the source inventory is still deleted.
## Fix Focus Areas
- roles/libvirt_manager/tasks/generate_networking_data.yml[142-153]
## Recommended Fix
Let conversion failures stop the task, or gate source deletion on a successful conversion and retain the failure for diagnosis.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

- oauthlib
executable: /usr/bin/pip3
extra_args: "--user --constraint {{ _controller_ansible_dependencies.requirements | quote }}"
failed_when: false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

3. Controller dependency failures are hidden 🐞 Bug ☼ Reliability

configure_bm_ocp_controller.yml sets failed_when: false on the task that installs the
controller's Ansible and Kubernetes requirements. If pip cannot install those requirements,
bare-metal controller setup continues until a later task needs a missing dependency, obscuring the
installation failure.
Agent Prompt
## Issue description
The bare-metal controller suppresses pip installation failures despite relying on the installed requirements.
## Fix Focus Areas
- roles/reproducer/tasks/configure_bm_ocp_controller.yml[127-147]
## Recommended Fix
Remove the unconditional `failed_when: false`, or handle only a specifically expected failure while reporting other installation errors.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

Comment on lines -71 to -75
- name: Install Ansible tools for local controller operations
ansible.builtin.pip:
name: "ansible-core=={{ ansible_version.full }}"
executable: /usr/bin/pip3.12
extra_args: --user

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Centos controller bootstrap loses ansible 🐞 Bug ≡ Correctness

ci_setup no longer installs Ansible through pip, and its generic package list does not include the
RPM added only for Red Hat. On a supported CentOS controller, bootstrap can reach the bare
ansible-playbook invocation without an executable supplied by that setup path; removing the
user-local PATH entry also prevents that invocation from finding a pip-installed copy.
Agent Prompt
## Issue description
The generic controller package path no longer provisions a discoverable Ansible executable.
## Fix Focus Areas
- roles/ci_setup/tasks/packages.yml[66-70]
- roles/ci_setup/vars/main.yml[21-27]
- roles/reproducer/tasks/main.yml[425-435]
## Recommended Fix
Install Ansible on generic controller hosts as well as Red Hat hosts, and ensure bootstrap commands can resolve its executable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

- bash-completion
- make
- tmux
- python3-cryptography

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

4. Centos secret generation can lack python 🐞 Bug ≡ Correctness

ci_setup removes python3-cryptography from its generic package list, while kustomize_deploy
explicitly runs system python3 to generate a Barbican Fernet key. On a CentOS host without that
module already installed, deployments that need a generated key fail at the import rather than
producing the secret.
Agent Prompt
## Issue description
Generic CI setup no longer guarantees the system-Python module used for Fernet key generation.
## Fix Focus Areas
- roles/ci_setup/vars/main.yml[21-27]
- roles/kustomize_deploy/tasks/inject_osp_secret_keys.yml[122-133]
## Recommended Fix
Keep `python3-cryptography` in the generic package list, or install it explicitly before the system-Python key-generation task.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

- python3-pip
- python3.12
- python3.12-pip
- sshpass

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

5. Centos idrac cleanup can skip a reset 🐞 Bug ≡ Correctness

ci_setup removes sshpass from its generic package list, but bm_eject_vmedia.yml uses that
executable to check whether an iDRAC remote file share needs resetting. On a CentOS bare-metal
reproducer without a separately installed sshpass, the status command's failure is suppressed and
its empty output prevents the reset branch from running.
Agent Prompt
## Issue description
Generic CI setup drops the executable required to inspect and clear lingering iDRAC remote file shares.
## Fix Focus Areas
- roles/ci_setup/vars/main.yml[21-27]
- roles/bm_sno/tasks/bm_eject_vmedia.yml[29-56]
## Recommended Fix
Restore `sshpass` for generic hosts that run bare-metal SNO, and do not treat a missing status command as evidence that no reset is needed.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

Comment thread scripts/run_ansible_test
Comment on lines -53 to -55
if [[ "$n" -ge "$ANSIBLE_GALAXY_RETIRES" ]]; then
echo "ERROR: collection installation failed after ${ANSIBLE_GALAXY_RETIRES} attempts" >&2
exit 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

6. Test runs can use a stale collection 🐞 Bug ≡ Correctness

run_ansible_test no longer exits when all ansible-galaxy collection install attempts fail. If an
older collection already exists at the expected path, the script can append sanity exceptions there
and run tests against that old installation instead of the current checkout.
Agent Prompt
## Issue description
The test runner continues after every collection-install attempt fails and may test an existing stale collection.
## Fix Focus Areas
- scripts/run_ansible_test[42-60]
## Recommended Fix
Restore an explicit nonzero exit after exhausted retries, before reading collection files or launching tests.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

@Valkyrie00 Valkyrie00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@openshift-ci

openshift-ci Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Valkyrie00

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Valkyrie00
Valkyrie00 merged commit afe31d0 into openstack-k8s-operators:main Oct 27, 2026
6 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants