From bda997ef93b14ad195f9347713567a004bb588cb Mon Sep 17 00:00:00 2001 From: Markus Frei Date: Fri, 28 Aug 2026 14:58:24 +0200 Subject: [PATCH 1/3] feat(roles/bootloader): add role Manages the kernel command line, for parameters that only take effect at boot time such as psi=1. On the Red Hat family the boot entries are written with grubby, on Debian and Ubuntu through a drop-in in /etc/default/grub.d/ followed by update-grub, so the packaged configuration files stay untouched. An option counts as present only when every boot entry carries it, and it is escaped before it goes into the comparison. A changed command line requests a reboot through schedule_reboot instead of rebooting right away, and a --check run reports what it would change without touching the host. The role deliberately does not use fedora.linux_system_roles.bootloader: that role rewrites GRUB_TIMEOUT in /etc/default/grub and every `set timeout=` in grub.cfg on each run, including the ones of the menu_auto_hide and menu_show_once blocks, and its bootloader_settings module declares supports_check_mode without ever reading module.check_mode, so a --check run rewrites the boot entries for real. Verified on Rocky 8, Rocky 9, Rocky 10, Fedora 44 and Debian 13: the check run leaves every file and boot entry byte-identical, the first run applies the options to all boot entries, the second reports no change, and removing them restores the original state. --- CHANGELOG.md | 3 +- COMPATIBILITY.md | 1 + extensions/molecule/bootloader/converge.yml | 2 + .../group_vars/systems_under_test.yml | 3 + .../molecule/bootloader/inventory/hosts.yml | 15 +++ extensions/molecule/bootloader/molecule.yml | 1 + extensions/molecule/bootloader/verify.yml | 88 ++++++++++++++ playbooks/README.md | 10 ++ playbooks/all.yml | 1 + playbooks/bootloader.yml | 49 ++++++++ roles/bootloader/README.md | 113 ++++++++++++++++++ roles/bootloader/defaults/main.yml | 12 ++ roles/bootloader/meta/argument_specs.yml | 30 +++++ roles/bootloader/tasks/Debian.yml | 44 +++++++ roles/bootloader/tasks/RedHat.yml | 63 ++++++++++ roles/bootloader/tasks/main.yml | 55 +++++++++ .../etc/default/grub.d/z00-lfops.cfg.j2 | 7 ++ roles/bootloader/vars/Debian.yml | 4 + roles/bootloader/vars/Ubuntu.yml | 4 + roles/bootloader/vars/main.yml | 31 +++++ 20 files changed, 535 insertions(+), 1 deletion(-) create mode 100644 extensions/molecule/bootloader/converge.yml create mode 100644 extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml create mode 100644 extensions/molecule/bootloader/inventory/hosts.yml create mode 100644 extensions/molecule/bootloader/molecule.yml create mode 100644 extensions/molecule/bootloader/verify.yml create mode 100644 playbooks/bootloader.yml create mode 100644 roles/bootloader/README.md create mode 100644 roles/bootloader/defaults/main.yml create mode 100644 roles/bootloader/meta/argument_specs.yml create mode 100644 roles/bootloader/tasks/Debian.yml create mode 100644 roles/bootloader/tasks/RedHat.yml create mode 100644 roles/bootloader/tasks/main.yml create mode 100644 roles/bootloader/templates/etc/default/grub.d/z00-lfops.cfg.j2 create mode 100644 roles/bootloader/vars/Debian.yml create mode 100644 roles/bootloader/vars/Ubuntu.yml create mode 100644 roles/bootloader/vars/main.yml diff --git a/CHANGELOG.md b/CHANGELOG.md index 341c1f3c0..68790511c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -**Highlights:** On RHEL 8, a MariaDB package upgrade no longer cuts applications on the same host off from their database. Apache no longer loads `mod_info`, which served the complete configuration including other modules' credentials. A broken PHP-FPM configuration aborts the run instead of taking the service down on the restart. Sudo rules deployed by `freeipa_server` can carry their commands again. The Bitwarden lookup can be told to abort instead of silently generating a new password, for runs against hosts whose credentials must already exist. The Grafana graph configuration for the Monitoring Plugins is no longer deployed on every ordinary run and has to be requested explicitly by its tag. +**Highlights:** Kernel parameters that only take effect at boot time, `psi=1` among them, can be deployed from the inventory with the new `bootloader` role. On RHEL 8, a MariaDB package upgrade no longer cuts applications on the same host off from their database. Apache no longer loads `mod_info`, which served the complete configuration including other modules' credentials. A broken PHP-FPM configuration aborts the run instead of taking the service down on the restart. Sudo rules deployed by `freeipa_server` can carry their commands again. The Bitwarden lookup can be told to abort instead of silently generating a new password, for runs against hosts whose credentials must already exist. The Grafana graph configuration for the Monitoring Plugins is no longer deployed on every ordinary run and has to be requested explicitly by its tag. ### Breaking Changes @@ -20,6 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +* **role:bootloader**: New role that manages the kernel command line, for parameters that only take effect at boot time such as `psi=1`. Options are applied to every boot entry of the host, on the Red Hat family with `grubby` and on Debian and Ubuntu through a GRUB drop-in of its own. A changed command line requests a reboot at the maintenance window instead of rebooting right away, and a `--check` run reports what it would change without touching the host. * **role:files**: A file can opt out of the backup copy that is written before it is overwritten, via the `backup` subkey of `files__files__*_var`. * **role:collabora**: The `collabora:configure` tag deploys `coolwsd.xml` and the logrotate configuration without touching the packages. * **role:docker**: The address pools docker assigns container network subnets from (`default-address-pools`) can be configured. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 7db4f2a19..9de965a6d 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -15,6 +15,7 @@ Which Ansible role is proven to run on which OS? | audit | | | x | x | (x) | | | | | | bind | | | x | x | x | | | | | | blocky | | | x | x | (x) | | | | | +| bootloader | (x) | x | x | x | x | (x) | (x) | (x) | Fedora 44 | | borg_local | | | x | (x) | (x) | | | | | | chromium_headless | x | (x) | x | x | x | | | | | | chrony | | | x | x | x | | | | | diff --git a/extensions/molecule/bootloader/converge.yml b/extensions/molecule/bootloader/converge.yml new file mode 100644 index 000000000..320eba099 --- /dev/null +++ b/extensions/molecule/bootloader/converge.yml @@ -0,0 +1,2 @@ +- name: 'Converge bootloader playbook' + ansible.builtin.import_playbook: 'linuxfabrik.lfops.bootloader' diff --git a/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml b/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml new file mode 100644 index 000000000..a6a3471c4 --- /dev/null +++ b/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml @@ -0,0 +1,3 @@ +bootloader__cmdline_options__group_var: + - name: 'psi' + value: 1 diff --git a/extensions/molecule/bootloader/inventory/hosts.yml b/extensions/molecule/bootloader/inventory/hosts.yml new file mode 100644 index 000000000..ea8d641b0 --- /dev/null +++ b/extensions/molecule/bootloader/inventory/hosts.yml @@ -0,0 +1,15 @@ +# yamllint disable rule:empty-values +lfops_bootloader: + children: + systems_under_test: + +systems_under_test: + hosts: + debian12-vm: + debian13-vm: + rocky8-vm: + rocky9-vm: + rocky10-vm: + ubuntu2204-vm: + ubuntu2404-vm: + ubuntu2604-vm: diff --git a/extensions/molecule/bootloader/molecule.yml b/extensions/molecule/bootloader/molecule.yml new file mode 100644 index 000000000..1e47cbff8 --- /dev/null +++ b/extensions/molecule/bootloader/molecule.yml @@ -0,0 +1 @@ +# Molecule scenario marker diff --git a/extensions/molecule/bootloader/verify.yml b/extensions/molecule/bootloader/verify.yml new file mode 100644 index 000000000..e357a57fa --- /dev/null +++ b/extensions/molecule/bootloader/verify.yml @@ -0,0 +1,88 @@ +# The kernel command line can only be observed on the running kernel, and it only changes on a +# reboot. The scenario therefore reboots the host once, on the run where the option is still +# missing from /proc/cmdline, and asserts the running kernel came up with it. verify runs twice +# (after converge and after idempotence, see the test_sequence in config.yml); on the second run +# the host already carries the option, so it neither expects a reboot request nor reboots again. +- name: 'Verify the kernel command line carries the configured option' + hosts: 'systems_under_test' + tasks: + + # Persistent state, visible before the reboot: on the Red Hat family the role writes the + # boot entries with grubby, on the Debian family update-grub regenerates grub.cfg. + - name: 'grubby --info=ALL' + ansible.builtin.command: 'grubby --info=ALL' + register: '__molecule__grubby_info_result' + changed_when: false + when: + - 'ansible_facts["os_family"] == "RedHat"' + + - name: 'Assert that every boot entry has psi=1 on its command line' + ansible.builtin.assert: + that: '__molecule__grubby_info_result["stdout_lines"] + | select("match", "^args=") + | reject("search", "psi=1") + | list | length == 0' + when: + - 'ansible_facts["os_family"] == "RedHat"' + + - name: 'grep "^[[:space:]]*linux" /boot/grub/grub.cfg' + ansible.builtin.command: 'grep --extended-regexp "^[[:space:]]*linux" /boot/grub/grub.cfg' + register: '__molecule__grub_cfg_linux_result' + changed_when: false + when: + - 'ansible_facts["os_family"] == "Debian"' + + - name: 'Assert that every menu entry has psi=1 on its command line' + ansible.builtin.assert: + that: '__molecule__grub_cfg_linux_result["stdout_lines"] + | reject("search", "psi=1") + | list | length == 0' + when: + - 'ansible_facts["os_family"] == "Debian"' + + - name: 'slurp /proc/cmdline' + ansible.builtin.slurp: + src: '/proc/cmdline' + register: '__molecule__proc_cmdline_result' + + # Only true right after converge changed the boot loader configuration. The role requests the + # reboot instead of performing it, so the request has to be waiting in the schedule_reboot spool. + - name: 'Verify the pending reboot and apply it' + when: '"psi=1" not in (__molecule__proc_cmdline_result["content"] | ansible.builtin.b64decode)' + block: + + - name: 'stat /run/schedule-reboot/bootloader' + ansible.builtin.stat: + path: '/run/schedule-reboot/bootloader' + register: '__molecule__reboot_request_stat_result' + + - name: 'Assert that the role requested a reboot' + ansible.builtin.assert: + that: '__molecule__reboot_request_stat_result["stat"]["exists"] | bool' + + - name: 'systemctl reboot' + ansible.builtin.reboot: # yamllint disable-line rule:empty-values + + - name: 'slurp /proc/cmdline' + ansible.builtin.slurp: + src: '/proc/cmdline' + register: '__molecule__proc_cmdline_result' + + - name: 'Assert that the running kernel booted with psi=1' + ansible.builtin.assert: + that: '"psi=1" in (__molecule__proc_cmdline_result["content"] | ansible.builtin.b64decode)' + + # The option is only worth setting if the kernel acts on it. RHEL 8 ships kernels without + # pressure stall information, so this is checked everywhere else. + - name: 'slurp /proc/pressure/cpu' + ansible.builtin.slurp: + src: '/proc/pressure/cpu' + register: '__molecule__proc_pressure_cpu_result' + when: + - 'not (ansible_facts["os_family"] == "RedHat" and ansible_facts["distribution_major_version"] | int < 9)' + + - name: 'Assert that the kernel reports pressure stall information' + ansible.builtin.assert: + that: '(__molecule__proc_pressure_cpu_result["content"] | ansible.builtin.b64decode) is search("^some ")' + when: + - 'not (ansible_facts["os_family"] == "RedHat" and ansible_facts["distribution_major_version"] | int < 9)' diff --git a/playbooks/README.md b/playbooks/README.md index a17fbc301..b0e73d4b8 100644 --- a/playbooks/README.md +++ b/playbooks/README.md @@ -99,6 +99,16 @@ Calls the following roles (in order): * [blocky](https://github.com/Linuxfabrik/lfops/tree/main/roles/blocky) +## bootloader.yml + +Calls the following roles (in order): + +* [postfix](https://github.com/Linuxfabrik/lfops/tree/main/roles/postfix): `bootloader__skip_postfix` +* [mailto_root](https://github.com/Linuxfabrik/lfops/tree/main/roles/mailto_root): `bootloader__skip_mailto_root` +* [schedule_reboot](https://github.com/Linuxfabrik/lfops/tree/main/roles/schedule_reboot): `bootloader__skip_schedule_reboot` +* [bootloader](https://github.com/Linuxfabrik/lfops/tree/main/roles/bootloader) + + ## borg_local.yml Calls the following roles (in order): diff --git a/playbooks/all.yml b/playbooks/all.yml index 5e80cda73..d08c39865 100644 --- a/playbooks/all.yml +++ b/playbooks/all.yml @@ -9,6 +9,7 @@ - import_playbook: 'audit.yml' - import_playbook: 'bind.yml' - import_playbook: 'blocky.yml' +- import_playbook: 'bootloader.yml' - import_playbook: 'borg_local.yml' - import_playbook: 'chromium_headless.yml' - import_playbook: 'chrony.yml' diff --git a/playbooks/bootloader.yml b/playbooks/bootloader.yml new file mode 100644 index 000000000..b0a4ac5f8 --- /dev/null +++ b/playbooks/bootloader.yml @@ -0,0 +1,49 @@ +- name: 'Playbook linuxfabrik.lfops.bootloader' + hosts: + - 'lfops_bootloader' + + pre_tasks: + - ansible.builtin.import_role: + name: 'shared' + tasks_from: 'log-start.yml' + tags: + - 'always' + + - ansible.builtin.import_role: + name: 'shared' + tasks_from: 'global-variables.yml' + tags: + - 'always' + + + roles: + + - role: 'linuxfabrik.lfops.postfix' + postfix__aliases__dependent_var: '{{ + mailto_root__postfix__aliases__dependent_var + }}' + postfix__sender_canonicals__dependent_var: '{{ + mailto_root__postfix__sender_canonicals__dependent_var + }}' + when: + - 'not bootloader__skip_postfix | d(false)' + + - role: 'linuxfabrik.lfops.mailto_root' + when: + - 'not bootloader__skip_mailto_root | d(false)' + + # deployed before the bootloader role, so the schedule-reboot command is in place when a + # changed kernel command line requests a reboot + - role: 'linuxfabrik.lfops.schedule_reboot' + when: + - 'not bootloader__skip_schedule_reboot | d(false)' + + - role: 'linuxfabrik.lfops.bootloader' + + + post_tasks: + - ansible.builtin.import_role: + name: 'shared' + tasks_from: 'log-end.yml' + tags: + - 'always' diff --git a/roles/bootloader/README.md b/roles/bootloader/README.md new file mode 100644 index 000000000..4bd56557a --- /dev/null +++ b/roles/bootloader/README.md @@ -0,0 +1,113 @@ +# Ansible Role linuxfabrik.lfops.bootloader + +This role manages the kernel command line of a host, for parameters that only take effect at boot time. + +On the Red Hat family the boot entries are written with `grubby`. Debian and Ubuntu do not package `grubby`, so there the role deploys a GRUB drop-in of its own and regenerates the boot loader configuration. + + +*Available in the next LFOps release.* + + +## How the Role Behaves + +* Red Hat family: options are applied to every boot entry of the host (`grubby --update-kernel=ALL`), so the running kernel and every kernel still installed alongside it carry the same command line. `grubby` keeps `GRUB_CMDLINE_LINUX` in `/etc/default/grub` in sync while doing so, appending only the managed options and leaving the rest of the file alone. Nothing else in `/etc/default/grub` and nothing in `grub.cfg` is touched. +* Debian family: the options are written to `/etc/default/grub.d/z00-lfops.cfg` and `update-grub` regenerates `/boot/grub/grub.cfg` from it. `grub-mkconfig` sources `/etc/default/grub` first and every `/etc/default/grub.d/*.cfg` after it, so the drop-in wins without the packaged configuration file ever being edited, and it appends to whatever `GRUB_CMDLINE_LINUX` already holds instead of replacing it (the sourcing order was read from the `grub-mkconfig` of grub-common 2.12-9+deb13u2 on Debian 13, 2.12-1ubuntu7.3 on Ubuntu 24.04 and 2.14-2ubuntu2.1 on Ubuntu 26.04). Once no option is left to set, the drop-in is removed instead of being left behind empty. +* A configured option counts as present only when **every** boot entry carries it, and it is compared as a whole word with the option escaped, so an option containing a dot matches a dot. +* A run against a host that already carries the configured command line changes nothing and reports no change, and it neither requests a reboot nor touches any file. Changing the value of an option that is already set replaces it rather than adding a second one. +* `--check` changes nothing. The dry run reads the current boot entries and reports what it would add or remove. +* The change only takes effect on the next boot. When the [schedule_reboot](https://github.com/Linuxfabrik/lfops/tree/main/roles/schedule_reboot) mechanism is deployed, a changed command line requests a reboot at the next maintenance window (spool entry `bootloader`). Without it, the role only prints a message and leaves the reboot to the operator. +* On the Red Hat family a kernel installed later inherits the command line from the running kernel. `kernel-install` builds the boot entry of a new kernel from `/etc/kernel/cmdline`, from `/usr/lib/kernel/cmdline`, or, when neither exists, from `/proc/cmdline` of the running kernel (verified against `/usr/lib/kernel/install.d/20-grub.install` on Rocky 9). A kernel installed between the change and the reboot therefore still comes up without the new options; run the role again afterwards. On the Debian family this cannot happen, because installing a kernel regenerates `/boot/grub/grub.cfg` from the drop-in. +* The role manages the kernel command line only. It does not add, remove or reorder boot entries, does not change the boot loader timeout, and does not manage the GRUB password. + + +## Known Limitations + +* GRUB 2 only. Hosts booted by zipl or systemd-boot are not supported. +* Debian family: `state: 'absent'` only drops an option from the command line this role writes. An option that comes from `/etc/default/grub` or from another drop-in stays, because the role never edits files it does not own. On the Red Hat family the same option is removed with `grubby --remove-args`. + + +## Dependent Roles + +Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/README.md) that installs this role runs these for you. Optional ones can be disabled via the playbook's skip variables. + +* Optional: the reboot mechanism should be in place (role: [linuxfabrik.lfops.schedule_reboot](https://github.com/Linuxfabrik/lfops/tree/main/roles/schedule_reboot)), so a changed kernel command line reboots the host at the maintenance window instead of waiting for a manual reboot. + + +## Requirements + +* The host is booted by GRUB 2. +* Red Hat family: `grubby` is installed. It is part of every GRUB installation there, since `kernel-install` relies on it. +* Debian family: `grub2-common` is installed. It provides `update-grub`, which the role calls. + + +## Tags + +`bootloader` + +* Configures the kernel command line. +* Requests a reboot when the kernel command line changed. +* Triggers: none. + + +## Optional Role Variables + +These variables are intended to be used in a host / group variable file in the Ansible inventory. Note that the group variable can only be used in one group at a time. + +`bootloader__cmdline_options__host_var` / `bootloader__cmdline_options__group_var` + +* Kernel command line options. An option that is already present with a different value is overwritten. On the Debian family the options end up in `GRUB_CMDLINE_LINUX`, so they apply to the recovery entries as well. +* Type: List of dictionaries. +* Default: `[]` +* Subkeys: + + * `name`: + + * Mandatory. Name of the option, for example `psi`. + * Type: String. + + * `value`: + + * Optional. Value of the option. Omit it for options that stand on their own, for example `quiet`. Quote a value YAML reads as a boolean, `'on'` and `'off'` among them, otherwise it reaches the command line as `True` or `False`. + * Type: String or Number. + + * `state`: + + * Optional. Whether the option is added to or removed from the kernel command line. One of `present` or `absent`. On the Debian family see "Known Limitations". + * Type: String. + * Default: `'present'` + +Example: +```yaml +# optional +bootloader__cmdline_options__group_var: + - name: 'psi' + value: 1 + - name: 'quiet' + - name: 'nosmt' + state: 'absent' +``` + + +## Troubleshooting + +**The option is configured, but `/proc/cmdline` does not contain it** + +* The host has not been rebooted since the change. Check the boot entries with `grubby --info=ALL` respectively `grep linux /boot/grub/grub.cfg`; they carry the new command line right away, `/proc/cmdline` only after the reboot. + +**On a Red Hat-family host a newly installed kernel boots without the configured options** + +* The kernel was installed while the change was still pending a reboot, so it inherited the command line of the running kernel. Run the role again to update the entry of the new kernel. + +**On a Debian-family host an option is still on the command line although it is set to `state: 'absent'`** + +* The option comes from `/etc/default/grub` or from another drop-in in `/etc/default/grub.d/`, which this role does not touch. Remove it there. + + +## License + +[The Unlicense](https://unlicense.org/) + + +## Author Information + +[Linuxfabrik GmbH, Zurich](https://www.linuxfabrik.ch) diff --git a/roles/bootloader/defaults/main.yml b/roles/bootloader/defaults/main.yml new file mode 100644 index 000000000..7f779279f --- /dev/null +++ b/roles/bootloader/defaults/main.yml @@ -0,0 +1,12 @@ +bootloader__cmdline_options__combined_var: '{{ ( + bootloader__cmdline_options__role_var + + bootloader__cmdline_options__dependent_var + + bootloader__cmdline_options__group_var + + bootloader__cmdline_options__host_var + ) | linuxfabrik.lfops.combine_lod + }}' + +bootloader__cmdline_options__dependent_var: [] +bootloader__cmdline_options__group_var: [] +bootloader__cmdline_options__host_var: [] +bootloader__cmdline_options__role_var: [] diff --git a/roles/bootloader/meta/argument_specs.yml b/roles/bootloader/meta/argument_specs.yml new file mode 100644 index 000000000..efcfafb6c --- /dev/null +++ b/roles/bootloader/meta/argument_specs.yml @@ -0,0 +1,30 @@ +argument_specs: + main: + options: + + bootloader__cmdline_options__dependent_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: >- + Kernel command line options to apply (`name`/`value`/`state`). + Dependent-role injection. + + bootloader__cmdline_options__group_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: >- + Kernel command line options to apply (`name`/`value`/`state`). + Group-level override. + + bootloader__cmdline_options__host_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: >- + Kernel command line options to apply (`name`/`value`/`state`). + Host-level override. diff --git a/roles/bootloader/tasks/Debian.yml b/roles/bootloader/tasks/Debian.yml new file mode 100644 index 000000000..9b2de0795 --- /dev/null +++ b/roles/bootloader/tasks/Debian.yml @@ -0,0 +1,44 @@ +# grub-mkconfig sources /etc/default/grub first and every /etc/default/grub.d/*.cfg after it, so +# a drop-in overrides the packaged configuration without editing a file the package owns. +# Verified against grub-common 2.12-9+deb13u2 on Debian 13 and 2.12-1ubuntu7.3 on Ubuntu 24.04. +- name: 'Deploy {{ __bootloader__grub_dropin_path }}' + ansible.builtin.template: + backup: true + src: 'etc/default/grub.d/z00-lfops.cfg.j2' + dest: '{{ __bootloader__grub_dropin_path }}' + owner: 'root' + group: 'root' + mode: 0o644 + register: '__bootloader__grub_dropin_result' + when: + - '__bootloader__wanted_options | length > 0' + +# nothing left to set means the drop-in has nothing to say, so it goes away instead of staying +# behind as an empty file +- name: 'rm {{ __bootloader__grub_dropin_path }}' + ansible.builtin.file: + path: '{{ __bootloader__grub_dropin_path }}' + state: 'absent' + register: '__bootloader__grub_dropin_removed_result' + when: + - '__bootloader__wanted_options | length == 0' + +# update-grub is a wrapper that execs grub-mkconfig, which in turn calls grub-probe and friends, +# all of them in /usr/sbin. An unprivileged login has no /usr/sbin in its PATH, and both `su` and +# a sudoers file without secure_path hand that PATH to the task, so the command needs one of its +# own. Verified on Debian 13, where the run otherwise ends in +# "/usr/sbin/update-grub: 3: exec: grub-mkconfig: not found". +- name: '{{ __bootloader__update_grub_command }}' + ansible.builtin.command: + cmd: '{{ __bootloader__update_grub_command }}' + environment: + PATH: '/usr/sbin:/usr/bin:/sbin:/bin' + changed_when: true + when: + - '(__bootloader__grub_dropin_result is changed) or (__bootloader__grub_dropin_removed_result is changed)' + +- name: 'Remember that the kernel command line changed' + ansible.builtin.set_fact: + __bootloader__reboot_needed: true + when: + - '(__bootloader__grub_dropin_result is changed) or (__bootloader__grub_dropin_removed_result is changed)' diff --git a/roles/bootloader/tasks/RedHat.yml b/roles/bootloader/tasks/RedHat.yml new file mode 100644 index 000000000..d3fcf5531 --- /dev/null +++ b/roles/bootloader/tasks/RedHat.yml @@ -0,0 +1,63 @@ +# grubby writes the boot entries on the Red Hat family, so the role calls it directly instead of +# going through fedora.linux_system_roles.bootloader. That role rewrites GRUB_TIMEOUT in +# /etc/default/grub and every `set timeout=` in grub.cfg on each run, which also replaces the +# timeouts of the menu_auto_hide and menu_show_once blocks (`set timeout="${menu_show_once_timeout}"` +# becomes a literal), and its bootloader_settings module declares supports_check_mode without ever +# reading module.check_mode, so a --check run rewrites the boot entries for real. +# Verified against grubby 8.40 on Rocky 9. +- name: 'grubby --info=ALL' + ansible.builtin.command: + cmd: 'grubby --info=ALL' + register: '__bootloader__grubby_info_result' + changed_when: false + check_mode: false # read-only, and it has to run in check mode too to report what would change + +# The comparison covers every boot entry, not just the default one, and the option is escaped +# before it goes into the pattern, so a dot in an option name matches a dot. +- name: 'Determine the options missing from at least one boot entry' + ansible.builtin.set_fact: + __bootloader__missing_options: '{{ __bootloader__missing_options + [item] }}' + loop: '{{ __bootloader__wanted_options }}' + when: + - '__bootloader__current_args | reject("search", "(^| )" ~ (item | regex_escape) ~ "( |$)") + | list | length > 0' + +- name: 'Determine the options still set on at least one boot entry' + ansible.builtin.set_fact: + __bootloader__superfluous_options: '{{ __bootloader__superfluous_options + [item] }}' + loop: '{{ __bootloader__unwanted_option_names }}' + when: + - '__bootloader__current_args | select("search", "(^| )" ~ (item | regex_escape) ~ "(=| |$)") + | list | length > 0' + +# a `command` task is skipped in check mode, so without this a dry run would report nothing to do +# although there is. This reports what would happen and marks the run as changed. +- name: 'Report the pending kernel command line change' + ansible.builtin.debug: + msg: 'grubby --update-kernel=ALL would add [{{ __bootloader__missing_options | join(" ") }}] + and remove [{{ __bootloader__superfluous_options | join(" ") }}]' + changed_when: true + when: + - 'ansible_check_mode | bool' + - '(__bootloader__missing_options | length > 0) or (__bootloader__superfluous_options | length > 0)' + +- name: 'grubby --update-kernel=ALL --remove-args' + ansible.builtin.command: + cmd: 'grubby --update-kernel=ALL --remove-args={{ __bootloader__superfluous_options + | join(" ") | quote }}' + changed_when: true + when: + - '__bootloader__superfluous_options | length > 0' + +- name: 'grubby --update-kernel=ALL --args' + ansible.builtin.command: + cmd: 'grubby --update-kernel=ALL --args={{ __bootloader__missing_options | join(" ") | quote }}' + changed_when: true + when: + - '__bootloader__missing_options | length > 0' + +- name: 'Remember that the kernel command line changed' + ansible.builtin.set_fact: + __bootloader__reboot_needed: true + when: + - '(__bootloader__missing_options | length > 0) or (__bootloader__superfluous_options | length > 0)' diff --git a/roles/bootloader/tasks/main.yml b/roles/bootloader/tasks/main.yml new file mode 100644 index 000000000..ebafd84ca --- /dev/null +++ b/roles/bootloader/tasks/main.yml @@ -0,0 +1,55 @@ +- block: + + - name: 'Set platform/version specific variables' + ansible.builtin.import_role: + name: 'shared' + tasks_from: 'platform-variables.yml' + + tags: + - 'always' + + +- block: + + - ansible.builtin.debug: + msg: + - 'Combined cmdline_options:' + - '{{ bootloader__cmdline_options__combined_var }}' + + # reset on every run, so a second run of the role in the same play does not inherit the + # findings of the first one + - name: 'Reset the state of a previous run' + ansible.builtin.set_fact: + __bootloader__missing_options: [] + __bootloader__reboot_needed: false + __bootloader__superfluous_options: [] + + # include_tasks is required here because import_tasks cannot handle dynamic variables. + - name: 'Perform platform specific tasks' + ansible.builtin.include_tasks: '{{ ansible_facts["os_family"] }}.yml' + + - name: 'stat /usr/local/sbin/schedule-reboot' + ansible.builtin.stat: + path: '/usr/local/sbin/schedule-reboot' + register: '__bootloader__schedule_reboot_stat_result' + + - name: 'schedule-reboot bootloader' + ansible.builtin.command: + argv: + - 'schedule-reboot' + - 'bootloader' + - 'kernel command line changed' + register: '__bootloader__schedule_reboot_result' + changed_when: '__bootloader__schedule_reboot_result["rc"] == 0' + when: + - '__bootloader__reboot_needed | bool' + - '__bootloader__schedule_reboot_stat_result["stat"]["exists"] | bool' + + - ansible.builtin.debug: + msg: 'The kernel command line has changed. Please reboot the server manually to apply it.' + when: + - '__bootloader__reboot_needed | bool' + - 'not __bootloader__schedule_reboot_stat_result["stat"]["exists"] | bool' + + tags: + - 'bootloader' diff --git a/roles/bootloader/templates/etc/default/grub.d/z00-lfops.cfg.j2 b/roles/bootloader/templates/etc/default/grub.d/z00-lfops.cfg.j2 new file mode 100644 index 000000000..12de000ad --- /dev/null +++ b/roles/bootloader/templates/etc/default/grub.d/z00-lfops.cfg.j2 @@ -0,0 +1,7 @@ +# {{ ansible_managed }} +# 20260828 + +# Sourced by grub-mkconfig after /etc/default/grub, so the setting below wins. Run +# `update-grub` after changing this file to regenerate /boot/grub/grub.cfg. +# Appended to whatever /etc/default/grub set, so the packaged parameters are kept. +GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX {{ __bootloader__wanted_options | join(' ') }}" diff --git a/roles/bootloader/vars/Debian.yml b/roles/bootloader/vars/Debian.yml new file mode 100644 index 000000000..52bb50e9e --- /dev/null +++ b/roles/bootloader/vars/Debian.yml @@ -0,0 +1,4 @@ +__bootloader__grub_dropin_path: '/etc/default/grub.d/z00-lfops.cfg' +# absolute path on purpose: /usr/sbin is not in the PATH of an unprivileged login, and +# `become_method: su` keeps that PATH, so the bare command name is not found there +__bootloader__update_grub_command: '/usr/sbin/update-grub' diff --git a/roles/bootloader/vars/Ubuntu.yml b/roles/bootloader/vars/Ubuntu.yml new file mode 100644 index 000000000..52bb50e9e --- /dev/null +++ b/roles/bootloader/vars/Ubuntu.yml @@ -0,0 +1,4 @@ +__bootloader__grub_dropin_path: '/etc/default/grub.d/z00-lfops.cfg' +# absolute path on purpose: /usr/sbin is not in the PATH of an unprivileged login, and +# `become_method: su` keeps that PATH, so the bare command name is not found there +__bootloader__update_grub_command: '/usr/sbin/update-grub' diff --git a/roles/bootloader/vars/main.yml b/roles/bootloader/vars/main.yml new file mode 100644 index 000000000..048404137 --- /dev/null +++ b/roles/bootloader/vars/main.yml @@ -0,0 +1,31 @@ +# The options that have to be on the kernel command line, rendered the way they appear there: +# 'name=value', or a bare 'name' for options that stand on their own. Options with and without a +# value are assembled differently, then concatenated. +__bootloader__present_options: '{{ + (bootloader__cmdline_options__combined_var | selectattr("state", "undefined") | list) + + (bootloader__cmdline_options__combined_var | selectattr("state", "defined") + | selectattr("state", "ne", "absent") | list) + }}' +__bootloader__wanted_options: '{{ + (__bootloader__present_options | selectattr("value", "defined") | map(attribute="name") | list + | zip(__bootloader__present_options | selectattr("value", "defined") + | map(attribute="value") | list) + | map("join", "=") | list) + + (__bootloader__present_options | selectattr("value", "undefined") | map(attribute="name") | list) + }}' + +# The names of the options that have to be off the kernel command line, whatever their value is. +__bootloader__unwanted_option_names: '{{ + bootloader__cmdline_options__combined_var | selectattr("state", "defined") + | selectattr("state", "eq", "absent") | map(attribute="name") | list + }}' + +# The args="..." line of every boot entry, as reported by `grubby --info=ALL`. The quotes are +# turned into spaces so that a search for a whole option matches at the ends of the line as well. +# Only meaningful once tasks/RedHat.yml has registered that command. +__bootloader__current_args: "{{ + __bootloader__grubby_info_result['stdout_lines'] | d([]) + | select('match', '^args=') + | map('regex_replace', '\"', ' ') + | list + }}" From 70569d91cc87aa377f888e750d6d186f12ae8413 Mon Sep 17 00:00:00 2001 From: Navid Sassan Date: Tue, 1 Sep 2026 14:56:06 +0200 Subject: [PATCH 2/3] fix(roles/bootloader): reject a GRUB environment block the boot loader cannot read On Red Hat family 8 the kernel command line lives in the GRUB environment block, which grub2-efi-x64 ships as a symlink onto the EFI System Partition. A host that boots BIOS from an image carrying both boot paths cannot follow that symlink into the ESP, so grubby reports the option as applied while the running kernel never receives it. Detect the combination and abort, instead of reporting a converged run that changed nothing. Further findings from the review of the role: * Name the platform tasks file directly instead of using the first_found lookup, so an unsupported OS family aborts the run rather than reporting ok and never touching the kernel command line. * Reject `root`, which grubby reports outside the kernel command line and which the role would therefore set again on every run. * Split the molecule scenario into bootloader/install and bootloader/remove, and supply the postfix and mailto_root variables the playbook's dependent roles require. * Read /proc/cmdline outside the reboot block in install/verify.yml. A register on a skipped task overwrites the variable with the skip result, which broke the second verify run. * Promote Debian 12 and Ubuntu 22.04, 24.04 and 26.04 in COMPATIBILITY.md, now that both scenarios pass on all eight targets. --- .ansible-lint-ignore | 1 + COMPATIBILITY.md | 2 +- .../bootloader/{ => install}/converge.yml | 0 .../group_vars/systems_under_test.yml | 19 ++++ .../{ => install}/inventory/hosts.yml | 0 .../molecule/bootloader/install/molecule.yml | 10 +++ .../molecule/bootloader/install/prepare.yml | 37 ++++++++ .../bootloader/{ => install}/verify.yml | 12 ++- .../group_vars/systems_under_test.yml | 3 - extensions/molecule/bootloader/molecule.yml | 1 - .../molecule/bootloader/remove/converge.yml | 2 + .../group_vars/systems_under_test.yml | 17 ++++ .../bootloader/remove/inventory/hosts.yml | 15 ++++ .../molecule/bootloader/remove/molecule.yml | 11 +++ .../molecule/bootloader/remove/prepare.yml | 53 +++++++++++ .../molecule/bootloader/remove/verify.yml | 87 +++++++++++++++++++ roles/bootloader/README.md | 20 +++++ roles/bootloader/tasks/RedHat.yml | 41 ++++++++- roles/bootloader/tasks/main.yml | 44 ++++++++-- roles/bootloader/vars/main.yml | 12 +++ 20 files changed, 369 insertions(+), 18 deletions(-) rename extensions/molecule/bootloader/{ => install}/converge.yml (100%) create mode 100644 extensions/molecule/bootloader/install/inventory/group_vars/systems_under_test.yml rename extensions/molecule/bootloader/{ => install}/inventory/hosts.yml (100%) create mode 100644 extensions/molecule/bootloader/install/molecule.yml create mode 100644 extensions/molecule/bootloader/install/prepare.yml rename extensions/molecule/bootloader/{ => install}/verify.yml (88%) delete mode 100644 extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml delete mode 100644 extensions/molecule/bootloader/molecule.yml create mode 100644 extensions/molecule/bootloader/remove/converge.yml create mode 100644 extensions/molecule/bootloader/remove/inventory/group_vars/systems_under_test.yml create mode 100644 extensions/molecule/bootloader/remove/inventory/hosts.yml create mode 100644 extensions/molecule/bootloader/remove/molecule.yml create mode 100644 extensions/molecule/bootloader/remove/prepare.yml create mode 100644 extensions/molecule/bootloader/remove/verify.yml diff --git a/.ansible-lint-ignore b/.ansible-lint-ignore index 61efab572..51baa4e8e 100644 --- a/.ansible-lint-ignore +++ b/.ansible-lint-ignore @@ -8,6 +8,7 @@ roles/at/defaults/main.yml jinja[spacing] roles/audit/defaults/main.yml jinja[spacing] roles/bind/defaults/main.yml jinja[spacing] roles/blocky/defaults/main.yml jinja[spacing] +roles/bootloader/defaults/main.yml jinja[spacing] roles/borg_local/defaults/main.yml jinja[spacing] roles/chrony/defaults/main.yml jinja[spacing] roles/clamav/defaults/main.yml jinja[spacing] diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 9de965a6d..6466abca1 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -15,7 +15,7 @@ Which Ansible role is proven to run on which OS? | audit | | | x | x | (x) | | | | | | bind | | | x | x | x | | | | | | blocky | | | x | x | (x) | | | | | -| bootloader | (x) | x | x | x | x | (x) | (x) | (x) | Fedora 44 | +| bootloader | x | x | x | x | x | x | x | x | Fedora 44 | | borg_local | | | x | (x) | (x) | | | | | | chromium_headless | x | (x) | x | x | x | | | | | | chrony | | | x | x | x | | | | | diff --git a/extensions/molecule/bootloader/converge.yml b/extensions/molecule/bootloader/install/converge.yml similarity index 100% rename from extensions/molecule/bootloader/converge.yml rename to extensions/molecule/bootloader/install/converge.yml diff --git a/extensions/molecule/bootloader/install/inventory/group_vars/systems_under_test.yml b/extensions/molecule/bootloader/install/inventory/group_vars/systems_under_test.yml new file mode 100644 index 000000000..39bd70a1f --- /dev/null +++ b/extensions/molecule/bootloader/install/inventory/group_vars/systems_under_test.yml @@ -0,0 +1,19 @@ +# The option under test. psi is a good candidate: it takes effect at boot time only, and the +# kernel reports whether it acted on it (/proc/pressure/cpu), so verify.yml can check the effect +# instead of just the string. +bootloader__cmdline_options__group_var: + - name: 'psi' + value: 1 + +# Inputs for the roles the bootloader playbook pulls in. mailto_root is mandatory: it needs a +# from and a list of recipients. A local address is enough, the test never sends real mail. +mailto_root__from: 'root@localhost' +mailto_root__to: + - 'root@localhost' + +# postfix needs a relayhost; an unreachable example host is fine, the test does not deliver mail. +postfix__relayhost: 'mail.example.com' + +# Reboot window far from the test run, so the schedule_reboot timer never fires mid-test. The +# reboot this scenario needs is performed by verify.yml itself. +schedule_reboot__reboot_time__group_var: '03:00' diff --git a/extensions/molecule/bootloader/inventory/hosts.yml b/extensions/molecule/bootloader/install/inventory/hosts.yml similarity index 100% rename from extensions/molecule/bootloader/inventory/hosts.yml rename to extensions/molecule/bootloader/install/inventory/hosts.yml diff --git a/extensions/molecule/bootloader/install/molecule.yml b/extensions/molecule/bootloader/install/molecule.yml new file mode 100644 index 000000000..592546ffe --- /dev/null +++ b/extensions/molecule/bootloader/install/molecule.yml @@ -0,0 +1,10 @@ +# Molecule scenario marker for the 'install' sub-scenario, run with +# `molecule test --scenario-name bootloader/install`. +# +# The prepare playbook is overridden to work around a property of the Rocky 8 cloud image that an +# installed host does not have; see the scenario-local prepare.yml. It imports the shared +# vm-prepare.yml first, so the systems under test are still waited for and their facts still +# gathered. +provisioner: + playbooks: + prepare: '${MOLECULE_SCENARIO_DIRECTORY}/prepare.yml' diff --git a/extensions/molecule/bootloader/install/prepare.yml b/extensions/molecule/bootloader/install/prepare.yml new file mode 100644 index 000000000..3485dbe5f --- /dev/null +++ b/extensions/molecule/bootloader/install/prepare.yml @@ -0,0 +1,37 @@ +# Wait for the systems under test and gather their facts, exactly as every other scenario does. +- name: 'Prepare VMs for Ansible' + ansible.builtin.import_playbook: '../../playbooks/vm-prepare.yml' + + +# Bring the EL8 test image into the shape an installed EL8 host has, so this scenario tests the +# role rather than a property of the cloud image. +# +# The GenericCloud image carries both boot paths, so grub2-efi-x64 is installed and ships +# /boot/grub2/grubenv as a symlink onto the EFI System Partition. These VMs boot BIOS, where GRUB +# cannot read across into the ESP, which is the combination the role rejects (see the role's +# "Known Limitations"). An installed BIOS host has a regular file there, so the scenario creates +# one. EL9 and later neither ship the symlink nor keep the command line in that file. +- name: 'Make the EL8 grub environment block readable from a BIOS boot' + hosts: 'systems_under_test' + + tasks: + + - name: 'stat /boot/grub2/grubenv' + ansible.builtin.stat: + path: '/boot/grub2/grubenv' + follow: false + register: '__molecule__grubenv_stat_result' + when: + - 'ansible_facts["os_family"] == "RedHat"' + - 'ansible_facts["distribution_major_version"] | int == 8' + + - name: 'Replace the grubenv symlink with a regular file' + ansible.builtin.command: + argv: + - 'cp' + - '--remove-destination' + - '{{ __molecule__grubenv_stat_result["stat"]["lnk_source"] | d("") }}' + - '/boot/grub2/grubenv' + changed_when: true + when: + - '__molecule__grubenv_stat_result["stat"]["islnk"] | d(false) | bool' diff --git a/extensions/molecule/bootloader/verify.yml b/extensions/molecule/bootloader/install/verify.yml similarity index 88% rename from extensions/molecule/bootloader/verify.yml rename to extensions/molecule/bootloader/install/verify.yml index e357a57fa..b79e53404 100644 --- a/extensions/molecule/bootloader/verify.yml +++ b/extensions/molecule/bootloader/install/verify.yml @@ -63,10 +63,14 @@ - name: 'systemctl reboot' ansible.builtin.reboot: # yamllint disable-line rule:empty-values - - name: 'slurp /proc/cmdline' - ansible.builtin.slurp: - src: '/proc/cmdline' - register: '__molecule__proc_cmdline_result' + # Read again, after the reboot above if there was one. This has to sit outside the block: a + # `register` on a skipped task overwrites the variable with the skip result, so re-registering + # inside the block would leave the assertion below without content on the second verify run, + # where the host already carries the option and the block does not run. + - name: 'slurp /proc/cmdline' + ansible.builtin.slurp: + src: '/proc/cmdline' + register: '__molecule__proc_cmdline_result' - name: 'Assert that the running kernel booted with psi=1' ansible.builtin.assert: diff --git a/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml b/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml deleted file mode 100644 index a6a3471c4..000000000 --- a/extensions/molecule/bootloader/inventory/group_vars/systems_under_test.yml +++ /dev/null @@ -1,3 +0,0 @@ -bootloader__cmdline_options__group_var: - - name: 'psi' - value: 1 diff --git a/extensions/molecule/bootloader/molecule.yml b/extensions/molecule/bootloader/molecule.yml deleted file mode 100644 index 1e47cbff8..000000000 --- a/extensions/molecule/bootloader/molecule.yml +++ /dev/null @@ -1 +0,0 @@ -# Molecule scenario marker diff --git a/extensions/molecule/bootloader/remove/converge.yml b/extensions/molecule/bootloader/remove/converge.yml new file mode 100644 index 000000000..320eba099 --- /dev/null +++ b/extensions/molecule/bootloader/remove/converge.yml @@ -0,0 +1,2 @@ +- name: 'Converge bootloader playbook' + ansible.builtin.import_playbook: 'linuxfabrik.lfops.bootloader' diff --git a/extensions/molecule/bootloader/remove/inventory/group_vars/systems_under_test.yml b/extensions/molecule/bootloader/remove/inventory/group_vars/systems_under_test.yml new file mode 100644 index 000000000..a13d243db --- /dev/null +++ b/extensions/molecule/bootloader/remove/inventory/group_vars/systems_under_test.yml @@ -0,0 +1,17 @@ +# prepare.yml has put the option on the kernel command line, this run removes it again. +bootloader__cmdline_options__group_var: + - name: 'psi' + state: 'absent' + +# Inputs for the roles the bootloader playbook pulls in. mailto_root is mandatory: it needs a +# from and a list of recipients. A local address is enough, the test never sends real mail. +mailto_root__from: 'root@localhost' +mailto_root__to: + - 'root@localhost' + +# postfix needs a relayhost; an unreachable example host is fine, the test does not deliver mail. +postfix__relayhost: 'mail.example.com' + +# Reboot window far from the test run, so the schedule_reboot timer never fires mid-test. This +# scenario never reboots: it asserts on the boot entries, which change right away. +schedule_reboot__reboot_time__group_var: '03:00' diff --git a/extensions/molecule/bootloader/remove/inventory/hosts.yml b/extensions/molecule/bootloader/remove/inventory/hosts.yml new file mode 100644 index 000000000..ea8d641b0 --- /dev/null +++ b/extensions/molecule/bootloader/remove/inventory/hosts.yml @@ -0,0 +1,15 @@ +# yamllint disable rule:empty-values +lfops_bootloader: + children: + systems_under_test: + +systems_under_test: + hosts: + debian12-vm: + debian13-vm: + rocky8-vm: + rocky9-vm: + rocky10-vm: + ubuntu2204-vm: + ubuntu2404-vm: + ubuntu2604-vm: diff --git a/extensions/molecule/bootloader/remove/molecule.yml b/extensions/molecule/bootloader/remove/molecule.yml new file mode 100644 index 000000000..9998c8011 --- /dev/null +++ b/extensions/molecule/bootloader/remove/molecule.yml @@ -0,0 +1,11 @@ +# Molecule scenario marker for the 'remove' sub-scenario, run with +# `molecule test --scenario-name bootloader/remove`. +# +# The prepare playbook is overridden so the option is on the kernel command line before converge +# runs. Without it, converge would remove an option that was never there and every assertion in +# verify.yml would pass on a host the role never touched. The scenario-local prepare.yml imports +# the shared vm-prepare.yml first, so the systems under test are still waited for and their facts +# still gathered. +provisioner: + playbooks: + prepare: '${MOLECULE_SCENARIO_DIRECTORY}/prepare.yml' diff --git a/extensions/molecule/bootloader/remove/prepare.yml b/extensions/molecule/bootloader/remove/prepare.yml new file mode 100644 index 000000000..5ff30f096 --- /dev/null +++ b/extensions/molecule/bootloader/remove/prepare.yml @@ -0,0 +1,53 @@ +# Wait for the systems under test and gather their facts, exactly as every other scenario does. +- name: 'Prepare VMs for Ansible' + ansible.builtin.import_playbook: '../../playbooks/vm-prepare.yml' + + +# Bring the EL8 test image into the shape an installed EL8 host has, so this scenario tests the +# role rather than a property of the cloud image. +# +# The GenericCloud image carries both boot paths, so grub2-efi-x64 is installed and ships +# /boot/grub2/grubenv as a symlink onto the EFI System Partition. These VMs boot BIOS, where GRUB +# cannot read across into the ESP, which is the combination the role rejects (see the role's +# "Known Limitations"). An installed BIOS host has a regular file there, so the scenario creates +# one. EL9 and later neither ship the symlink nor keep the command line in that file. +- name: 'Make the EL8 grub environment block readable from a BIOS boot' + hosts: 'systems_under_test' + + tasks: + + - name: 'stat /boot/grub2/grubenv' + ansible.builtin.stat: + path: '/boot/grub2/grubenv' + follow: false + register: '__molecule__grubenv_stat_result' + when: + - 'ansible_facts["os_family"] == "RedHat"' + - 'ansible_facts["distribution_major_version"] | int == 8' + + - name: 'Replace the grubenv symlink with a regular file' + ansible.builtin.command: + argv: + - 'cp' + - '--remove-destination' + - '{{ __molecule__grubenv_stat_result["stat"]["lnk_source"] | d("") }}' + - '/boot/grub2/grubenv' + changed_when: true + when: + - '__molecule__grubenv_stat_result["stat"]["islnk"] | d(false) | bool' + + +# Put the option on the kernel command line, so the converge run has something to remove. The +# role is called directly rather than through the playbook: schedule_reboot is deliberately not +# deployed here, so the role only reports that a reboot would be needed and the boot entries stay +# the single thing this play changes. The role parameter beats the scenario group_vars, which set +# the same variable to state: 'absent' for the converge run. +- name: 'Put psi=1 on the kernel command line' + hosts: 'systems_under_test' + + roles: + + - role: 'linuxfabrik.lfops.bootloader' + bootloader__cmdline_options__group_var: + - name: 'psi' + value: 1 diff --git a/extensions/molecule/bootloader/remove/verify.yml b/extensions/molecule/bootloader/remove/verify.yml new file mode 100644 index 000000000..a6057aad1 --- /dev/null +++ b/extensions/molecule/bootloader/remove/verify.yml @@ -0,0 +1,87 @@ +# Verify the removal. prepare.yml put psi=1 on the kernel command line, converge removed it +# again. Nothing rebooted in between, so /proc/cmdline never carried the option and the check is +# on the persistent state: the boot entries, and on the Debian family the drop-in that produced +# them. The counterpart to install/verify.yml, which asserts the same state is present. +- name: 'Verify the kernel command line no longer carries the removed option' + hosts: 'systems_under_test' + tasks: + + - name: 'grubby --info=ALL' + ansible.builtin.command: 'grubby --info=ALL' + register: '__molecule__grubby_info_result' + changed_when: false + when: + - 'ansible_facts["os_family"] == "RedHat"' + + # The quotes are turned into spaces first, the way the role does it, so that an option sitting + # right behind args=" is still matched as a whole word. + - name: 'Assert that no boot entry has psi on its command line' + ansible.builtin.assert: + that: "__molecule__grubby_info_result['stdout_lines'] + | select('match', '^args=') + | map('regex_replace', '\"', ' ') + | select('search', '(^| )psi(=| |$)') + | list | length == 0" + when: + - 'ansible_facts["os_family"] == "RedHat"' + + # --remove-args must not have taken the boot entries with it. + - name: 'Assert that the host still has boot entries' + ansible.builtin.assert: + that: '__molecule__grubby_info_result["stdout_lines"] + | select("match", "^args=") + | list | length > 0' + when: + - 'ansible_facts["os_family"] == "RedHat"' + + - name: 'grep "^[[:space:]]*linux" /boot/grub/grub.cfg' + ansible.builtin.command: 'grep --extended-regexp "^[[:space:]]*linux" /boot/grub/grub.cfg' + register: '__molecule__grub_cfg_linux_result' + changed_when: false + when: + - 'ansible_facts["os_family"] == "Debian"' + + - name: 'Assert that no menu entry has psi on its command line' + ansible.builtin.assert: + that: '__molecule__grub_cfg_linux_result["stdout_lines"] + | select("search", "(^| )psi(=| |$)") + | list | length == 0' + when: + - 'ansible_facts["os_family"] == "Debian"' + + # update-grub has to have produced a usable configuration, not an empty one: every menu entry + # still names its root device. + - name: 'Assert that every menu entry still names its root device' + ansible.builtin.assert: + that: '__molecule__grub_cfg_linux_result["stdout_lines"] + | reject("search", "(^| )root=") + | list | length == 0' + when: + - 'ansible_facts["os_family"] == "Debian"' + + # With nothing left to set the drop-in has nothing to say, so the role removes it instead of + # leaving an empty file behind. + - name: 'stat /etc/default/grub.d/z00-lfops.cfg' + ansible.builtin.stat: + path: '/etc/default/grub.d/z00-lfops.cfg' + register: '__molecule__grub_dropin_stat_result' + when: + - 'ansible_facts["os_family"] == "Debian"' + + - name: 'Assert that the GRUB drop-in is gone' + ansible.builtin.assert: + that: 'not __molecule__grub_dropin_stat_result["stat"]["exists"] | bool' + when: + - 'ansible_facts["os_family"] == "Debian"' + + # The role requests the reboot rather than performing it, so the request has to be waiting in + # the schedule_reboot spool. Nothing rebooted, so the request is still there on the second + # verify run as well. + - name: 'stat /run/schedule-reboot/bootloader' + ansible.builtin.stat: + path: '/run/schedule-reboot/bootloader' + register: '__molecule__reboot_request_stat_result' + + - name: 'Assert that the role requested a reboot' + ansible.builtin.assert: + that: '__molecule__reboot_request_stat_result["stat"]["exists"] | bool' diff --git a/roles/bootloader/README.md b/roles/bootloader/README.md index 4bd56557a..9549401f7 100644 --- a/roles/bootloader/README.md +++ b/roles/bootloader/README.md @@ -24,6 +24,8 @@ On the Red Hat family the boot entries are written with `grubby`. Debian and Ubu * GRUB 2 only. Hosts booted by zipl or systemd-boot are not supported. * Debian family: `state: 'absent'` only drops an option from the command line this role writes. An option that comes from `/etc/default/grub` or from another drop-in stays, because the role never edits files it does not own. On the Red Hat family the same option is removed with `grubby --remove-args`. +* the `root` option cannot be managed. `grubby` reports it on a line of its own rather than as part of the kernel command line, so the role would never see it as applied and would set it again on every run. It is rejected with an error instead. This does not apply to `initrd`, which stays on the command line. +* Red Hat family 8: a host that boots in BIOS mode and has `/boot/grub2/grubenv` as a symlink into the EFI System Partition is rejected with an error. Boot entries there reference the command line as `options $kernelopts` and keep the value in that environment block; from a BIOS boot GRUB reads it on `/boot` and cannot follow the symlink into the ESP, so it falls back to the command line compiled into `grub.cfg` and the option never reaches the kernel. `grubby` reports the option as applied either way, so the role would otherwise report a converged run that does nothing. The combination comes from images built to boot both ways. An installed host does not have it: booted BIOS it has a regular file there, booted UEFI it reads the copy in the ESP directly. Red Hat family 9 and later put the options in the boot entry itself and are unaffected. ## Dependent Roles @@ -90,6 +92,24 @@ bootloader__cmdline_options__group_var: ## Troubleshooting +**The run aborts with `Could not find or access '.yml'`** + +* The host runs an operating system family this role ships no tasks for. It manages the kernel command line through `grubby` on the Red Hat family and through a GRUB drop-in on the Debian family; there is no third path. The role aborts rather than skipping the host, so a kernel parameter never goes silently unapplied. + +**The run aborts with `/boot/grub2/grubenv is a symlink onto the EFI System Partition`** + +* The host is a Red Hat family 8 machine that boots in BIOS mode from an image that also carries a UEFI boot path (see "Known Limitations"). The boot loader cannot read the file the kernel command line is stored in, so the option would be written and never applied. Replace the symlink with a regular copy of its target, which is what an installed BIOS host has: + + ```bash + cp --remove-destination "$(readlink --canonicalize /boot/grub2/grubenv)" /boot/grub2/grubenv + ``` + + The next run then applies the options normally. Hosts booting in UEFI mode are not affected and are not checked. + +**The run aborts with `grubby reports root outside the kernel command line`** + +* the `root` option is configured in `bootloader__cmdline_options__*_var`. It cannot be managed here (see "Known Limitations"); remove the entry. The root device belongs in the partitioning or in `/etc/default/grub`. + **The option is configured, but `/proc/cmdline` does not contain it** * The host has not been rebooted since the change. Check the boot entries with `grubby --info=ALL` respectively `grep linux /boot/grub/grub.cfg`; they carry the new command line right away, `/proc/cmdline` only after the reboot. diff --git a/roles/bootloader/tasks/RedHat.yml b/roles/bootloader/tasks/RedHat.yml index d3fcf5531..cbabf37ab 100644 --- a/roles/bootloader/tasks/RedHat.yml +++ b/roles/bootloader/tasks/RedHat.yml @@ -1,3 +1,40 @@ +# Red Hat family 8 keeps the kernel command line in the GRUB environment block and lets the boot +# entries reference it as `options $kernelopts`. grubby writes it there and reads it back expanded, +# so it reports success whether or not the boot loader can read the file: only the next boot shows +# the difference. The grub2-efi-x64 package ships /boot/grub2/grubenv as a symlink to the copy on +# the EFI System Partition. Booted UEFI that is where GRUB looks anyway, but booted BIOS GRUB reads +# $prefix/grubenv on /boot and cannot follow the link into the vfat ESP: load_env fails, $kernelopts +# stays empty and the command line compiled into grub.cfg is used instead. The role would report a +# converged, idempotent run while the option never reaches the kernel, so the combination is +# rejected rather than reported as done. An installed host does not have it, only an image carrying +# both boot paths. Red Hat family 9 and later put the options in the boot entry itself. +# Verified against grubby 8.40-51.el8_10 and grub2-efi-x64 2.02-170 on Rocky 8. +- name: 'stat /sys/firmware/efi' + ansible.builtin.stat: + path: '/sys/firmware/efi' + register: '__bootloader__efi_stat_result' + when: + - 'ansible_facts["distribution_major_version"] | int == 8' + +- name: 'stat /boot/grub2/grubenv' + ansible.builtin.stat: + path: '/boot/grub2/grubenv' + follow: false + register: '__bootloader__grubenv_stat_result' + when: + - 'ansible_facts["distribution_major_version"] | int == 8' + +- name: 'Assert that the boot loader can read the GRUB environment block' + ansible.builtin.assert: + that: + - 'not (__bootloader__grubenv_stat_result["stat"]["islnk"] | d(false) | bool)' + quiet: true + fail_msg: '/boot/grub2/grubenv is a symlink onto the EFI System Partition, but this host boots in BIOS mode, so the boot loader cannot read it and the kernel command line would never take effect. Replace the symlink with a regular file: cp --remove-destination "$(readlink --canonicalize /boot/grub2/grubenv)" /boot/grub2/grubenv' + when: + - 'ansible_facts["distribution_major_version"] | int == 8' + - 'not __bootloader__efi_stat_result["stat"]["exists"] | bool' + + # grubby writes the boot entries on the Red Hat family, so the role calls it directly instead of # going through fedora.linux_system_roles.bootloader. That role rewrites GRUB_TIMEOUT in # /etc/default/grub and every `set timeout=` in grub.cfg on each run, which also replaces the @@ -41,7 +78,7 @@ - 'ansible_check_mode | bool' - '(__bootloader__missing_options | length > 0) or (__bootloader__superfluous_options | length > 0)' -- name: 'grubby --update-kernel=ALL --remove-args' +- name: 'grubby --update-kernel=ALL --remove-args={{ __bootloader__superfluous_options | join(" ") }}' ansible.builtin.command: cmd: 'grubby --update-kernel=ALL --remove-args={{ __bootloader__superfluous_options | join(" ") | quote }}' @@ -49,7 +86,7 @@ when: - '__bootloader__superfluous_options | length > 0' -- name: 'grubby --update-kernel=ALL --args' +- name: 'grubby --update-kernel=ALL --args={{ __bootloader__missing_options | join(" ") }}' ansible.builtin.command: cmd: 'grubby --update-kernel=ALL --args={{ __bootloader__missing_options | join(" ") | quote }}' changed_when: true diff --git a/roles/bootloader/tasks/main.yml b/roles/bootloader/tasks/main.yml index ebafd84ca..73b365330 100644 --- a/roles/bootloader/tasks/main.yml +++ b/roles/bootloader/tasks/main.yml @@ -11,10 +11,27 @@ - block: - - ansible.builtin.debug: - msg: - - 'Combined cmdline_options:' - - '{{ bootloader__cmdline_options__combined_var }}' + # grubby reports these outside args="...", where the role cannot see them, so it would add them + # again on every run. Rejected up front instead of silently reporting a change forever. + - name: 'Assert that no option grubby reports separately is managed' + ansible.builtin.assert: + that: + - '__bootloader__configured_option_names | intersect(__bootloader__grubby_reserved_option_names) | length == 0' + quiet: true + fail_msg: 'grubby reports {{ __bootloader__configured_option_names | intersect(__bootloader__grubby_reserved_option_names) | join(", ") }} outside the kernel command line, so this role cannot manage it. Remove the entry from bootloader__cmdline_options__*_var.' + when: + - 'ansible_facts["os_family"] == "RedHat"' + + tags: + # use 'always' so the validation runs even when other roles reference these variables. + - 'always' + + +- block: + + - name: 'Combined cmdline options' + ansible.builtin.debug: + var: 'bootloader__cmdline_options__combined_var' # reset on every run, so a second run of the role in the same play does not inherit the # findings of the first one @@ -25,6 +42,15 @@ __bootloader__superfluous_options: [] # include_tasks is required here because import_tasks cannot handle dynamic variables. + # + # Deliberately not the first_found lookup from CONTRIBUTING.md ("OS-specific Tasks"). That + # pattern ends in `skip: true`, which is right for a role whose platform file holds a few + # extras on top of a common main.yml: a platform without one loses those extras and the rest + # of the role still runs. Here the platform file IS the role, so skipping it would leave a run + # that reports ok, changes nothing and never touches the kernel command line, on a host the + # admin put in lfops_bootloader precisely to have it changed. Naming the file directly fails + # the run instead, with "Could not find or access 'Suse.yml'" naming the family the role would + # need a file for. - name: 'Perform platform specific tasks' ansible.builtin.include_tasks: '{{ ansible_facts["os_family"] }}.yml' @@ -33,10 +59,13 @@ path: '/usr/local/sbin/schedule-reboot' register: '__bootloader__schedule_reboot_stat_result' - - name: 'schedule-reboot bootloader' + # absolute path on purpose: the default sudoers of the Red Hat family sets + # `secure_path = /sbin:/bin:/usr/sbin:/usr/bin`, which leaves /usr/local/sbin out, so the bare + # command name is not found under `become: true`. Verified against sudo on Rocky 9. + - name: '/usr/local/sbin/schedule-reboot bootloader "kernel command line changed"' ansible.builtin.command: argv: - - 'schedule-reboot' + - '/usr/local/sbin/schedule-reboot' - 'bootloader' - 'kernel command line changed' register: '__bootloader__schedule_reboot_result' @@ -45,7 +74,8 @@ - '__bootloader__reboot_needed | bool' - '__bootloader__schedule_reboot_stat_result["stat"]["exists"] | bool' - - ansible.builtin.debug: + - name: 'Report that a manual reboot is required' + ansible.builtin.debug: msg: 'The kernel command line has changed. Please reboot the server manually to apply it.' when: - '__bootloader__reboot_needed | bool' diff --git a/roles/bootloader/vars/main.yml b/roles/bootloader/vars/main.yml index 048404137..c5dc51ed1 100644 --- a/roles/bootloader/vars/main.yml +++ b/roles/bootloader/vars/main.yml @@ -1,3 +1,10 @@ +# Option names grubby reports on a line of their own instead of inside args="...", which puts +# them out of reach of the comparison below: the role would add them again on every run. `initrd` +# is not one of them, an initrd= in --args stays in args="...", and the initrd= grubby reports +# comes from the boot entry's own initrd line. Verified against grubby 8.40 on Rocky 9. +__bootloader__grubby_reserved_option_names: + - 'root' + # The options that have to be on the kernel command line, rendered the way they appear there: # 'name=value', or a bare 'name' for options that stand on their own. Options with and without a # value are assembled differently, then concatenated. @@ -20,6 +27,11 @@ __bootloader__unwanted_option_names: '{{ | selectattr("state", "eq", "absent") | map(attribute="name") | list }}' +# Every configured option name, whatever its state. Used by the reserved-name assert. +__bootloader__configured_option_names: '{{ + bootloader__cmdline_options__combined_var | map(attribute="name") | list + }}' + # The args="..." line of every boot entry, as reported by `grubby --info=ALL`. The quotes are # turned into spaces so that a search for a whole option matches at the ends of the line as well. # Only meaningful once tasks/RedHat.yml has registered that command. From ea7eb3b2f5e4edceaca9cc7216fb0180ec3bc75c Mon Sep 17 00:00:00 2001 From: Navid Sassan Date: Tue, 1 Sep 2026 15:07:08 +0200 Subject: [PATCH 3/3] docs(CONTRIBUTING): list .ansible-lint-ignore in the deliverables for a new role The multi-line __combined_var expression in defaults/main.yml is reported as jinja[spacing]. We keep the readable form, so every role that has one needs a line in .ansible-lint-ignore. --- CONTRIBUTING.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4fca56c0d..216b8b65b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -219,6 +219,7 @@ When creating a new role, make sure to deliver: * Update `playbooks/all.yml`. * Update `COMPATIBILITY.md`. * Update `CHANGELOG.md`. +* Update `.ansible-lint-ignore` if the role defines a `__combined_var`. ### OS Coverage