| Message ID | 20250305-fix-debian-apt-v1-1-09acd002b2f5@samsung.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Revert "bringup: devconfig: fix python3-apt debian issue" | expand |
On Wed, Mar 05, 2025 at 10:13:19PM +0100, da.gomez@kernel.org wrote: > From: Daniel Gomez <da.gomez@samsung.com> > > This reverts commit bcea53d0710cc0722520430277c80279e90e0863 > > Commit bcea53d0710cc0722520430277c80279e90e0863 ("bringup: devconfig: > fix python3-apt debian issue") was introduced as workaround to fix a > Debian python3-apt issue. We now know this was the actual problem [2] > [3]: > > "APT has a fluctuating ABI that > depends on which std:: symbols it calls, and which of those calls > the compiler inlines and omits the symbols for, so it can change > with every update to APT or the compiler." > > Ensuring distro and especially apt package is uptodate we align always > packages versions to their latest available. Do this at bringup when > image is firstboot. In addition, add a new firstconfig target, to make > sure we have the system upgraded even if the image has already been > booted and we have an outdated system. > > [1] > https://github.com/linux-kdevops/linux-mm-kpd/actions/runs/13677761684/job/38242628757 > > [2] > https://lore.kernel.org/kdevops/20250208134034.GA4079710@debian.org/T/#u > > [3] > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1095111 > > Reported-by: Luis Chamberlain <mcgrof@kernel.org> > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> Acked-by: Luis Chamberlain <mcgrof@kernel.org> Luis > --- > Revert python3-apt workaround and fix issue by upgrading apt and distro > packages. More details in Debian bug report [1]. > > [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1095111 > > This also fixes: > > https://github.com/linux-kdevops/linux-mm-kpd/actions/runs/13677761684/job/38242628757 > --- > Makefile | 1 + > .../devconfig/tasks/install-deps/debian/main.yml | 51 +++------------------- > scripts/bringup_guestfs.sh | 1 + > scripts/firstconfig.Makefile | 15 +++++++ > 4 files changed, 22 insertions(+), 46 deletions(-) > > diff --git a/Makefile b/Makefile > index e0e5d8f962d2f33be1c91a66aeaf4f5829f044d5..5ee7db7d988b9271fed7cb470f807bb9e9ce3cb6 100644 > --- a/Makefile > +++ b/Makefile > @@ -95,6 +95,7 @@ LOCALHOST_SETUP_WORK := > ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS) > > include scripts/provision.Makefile > +include scripts/firstconfig.Makefile > include scripts/systemd-timesync.Makefile > include scripts/journal-server.Makefile > include scripts/update_etc_hosts.Makefile > diff --git a/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml b/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml > index 36ff92c5719f0753dd209d10e3b4a2fe7db42a08..22fe396b3c3a531571cfc6dea5a3250564fa853d 100644 > --- a/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml > +++ b/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml > @@ -1,51 +1,10 @@ > --- > -- name: Update package lists > - become: true > - become_method: sudo > - ansible.builtin.shell: > - cmd: > > - DEBIAN_FRONTEND=noninteractive apt > - -o Acquire::Check-Valid-Until=false update > - when: > - - ansible_os_family == "Debian" > - - ansible_distribution_release == "trixie" > - tags: vars_simple > - register: apt_update_result > - retries: 5 > - delay: 10 > - until: apt_update_result.rc == 0 > - > -- name: Install required packages > - become: true > - become_method: sudo > - ansible.builtin.shell: > - cmd: > > - DEBIAN_FRONTEND=noninteractive apt install -y > - python3-setuptools libapt-pkg-dev python3-pip git > - when: > - - ansible_os_family == "Debian" > - - ansible_distribution_release == "trixie" > - tags: vars_simple > - > -- name: Install python-apt from Git with pip3 > - become: true > - become_method: sudo > - ansible.builtin.shell: > - cmd: > > - pip3 install > - git+https://salsa.debian.org/apt-team/python-apt.git@2.9.6 > - --break-system-packages > - when: > - - ansible_os_family == "Debian" > - - ansible_distribution_release == "trixie" > - tags: vars_simple > - > -- name: Hold python3-apt package version > +- name: Upgrade Packages > become: yes > - ansible.builtin.command: apt-mark hold python3-apt > - when: > - - ansible_os_family == "Debian" > - tags: vars_simple > + become_method: sudo > + ansible.builtin.apt: > + upgrade: full > + tags: firstconfig > > - name: Remove unattended-upgrades package > become: yes > diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh > index 08c868c467077f118668a18028ba6cbefa74665b..32ffe3f037de16a724aa66741a99efa4e5eedcf0 100755 > --- a/scripts/bringup_guestfs.sh > +++ b/scripts/bringup_guestfs.sh > @@ -222,6 +222,7 @@ append-line /etc/network/interfaces.d/enp1s0:iface enp1s0 inet dhcp > firstboot-command systemctl stop ssh > firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force openssh-server > firstboot-command systemctl start ssh > +firstboot-command apt update && apt upgrade --yes > _EOT > # CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST will not work > # if /etc/nsswitch.conf has a line like this: > diff --git a/scripts/firstconfig.Makefile b/scripts/firstconfig.Makefile > new file mode 100644 > index 0000000000000000000000000000000000000000..808975027ef475f07cb34fbf270dd404b7cbacf3 > --- /dev/null > +++ b/scripts/firstconfig.Makefile > @@ -0,0 +1,15 @@ > +# SPDX-License-Identifier: copyleft-next-0.3.1 > + > +firstconfig: > + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \ > + -f 30 -i hosts \ > + --extra-vars '{ kdevops_cli_install: True }' \ > + --tags vars_simple,firstconfig \ > + $(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml > + > +KDEVOPS_BRING_UP_DEPS_EARLY += firstconfig > + > +firstconfig-help: > + @echo "firstconfig - Setup firstconfig" > + > +HELP_TARGETS += firstconfig-help > > --- > base-commit: b22f5e838593b7c4577a308a5d0de13db1eae220 > change-id: 20250305-fix-debian-apt-38c9bcb95ab6 > > Best regards, > -- > Daniel Gomez <da.gomez@samsung.com> > >
On Wed, Mar 05, 2025 at 04:05:01PM +0100, Luis Chamberlain wrote: > On Wed, Mar 05, 2025 at 10:13:19PM +0100, da.gomez@kernel.org wrote: > > From: Daniel Gomez <da.gomez@samsung.com> > > > > This reverts commit bcea53d0710cc0722520430277c80279e90e0863 > > > > Commit bcea53d0710cc0722520430277c80279e90e0863 ("bringup: devconfig: > > fix python3-apt debian issue") was introduced as workaround to fix a > > Debian python3-apt issue. We now know this was the actual problem [2] > > [3]: > > > > "APT has a fluctuating ABI that > > depends on which std:: symbols it calls, and which of those calls > > the compiler inlines and omits the symbols for, so it can change > > with every update to APT or the compiler." > > > > Ensuring distro and especially apt package is uptodate we align always > > packages versions to their latest available. Do this at bringup when > > image is firstboot. In addition, add a new firstconfig target, to make > > sure we have the system upgraded even if the image has already been > > booted and we have an outdated system. > > > > [1] > > https://github.com/linux-kdevops/linux-mm-kpd/actions/runs/13677761684/job/38242628757 > > > > [2] > > https://lore.kernel.org/kdevops/20250208134034.GA4079710@debian.org/T/#u > > > > [3] > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1095111 > > > > Reported-by: Luis Chamberlain <mcgrof@kernel.org> > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com> > > Acked-by: Luis Chamberlain <mcgrof@kernel.org> > > Luis > Pushed and added a fix to prevent apt lock [1]. This issue was not reproducible in my kdevops machine but only in CI. So, if anyone like me needs to test CI before merging patches a reminder that we can push branches to kdevops for CI testing/validation and/or use kdevops-kpd [2] for the same purpose. [1] 734f0afad33cb8d10ee41480f574791a2ee6b682 devconfig: fix firstconfig debian upgrade [2] https://github.com/linux-kdevops/kdevops-kpd Daniel
diff --git a/Makefile b/Makefile index e0e5d8f962d2f33be1c91a66aeaf4f5829f044d5..5ee7db7d988b9271fed7cb470f807bb9e9ce3cb6 100644 --- a/Makefile +++ b/Makefile @@ -95,6 +95,7 @@ LOCALHOST_SETUP_WORK := ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS) include scripts/provision.Makefile +include scripts/firstconfig.Makefile include scripts/systemd-timesync.Makefile include scripts/journal-server.Makefile include scripts/update_etc_hosts.Makefile diff --git a/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml b/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml index 36ff92c5719f0753dd209d10e3b4a2fe7db42a08..22fe396b3c3a531571cfc6dea5a3250564fa853d 100644 --- a/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml +++ b/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml @@ -1,51 +1,10 @@ --- -- name: Update package lists - become: true - become_method: sudo - ansible.builtin.shell: - cmd: > - DEBIAN_FRONTEND=noninteractive apt - -o Acquire::Check-Valid-Until=false update - when: - - ansible_os_family == "Debian" - - ansible_distribution_release == "trixie" - tags: vars_simple - register: apt_update_result - retries: 5 - delay: 10 - until: apt_update_result.rc == 0 - -- name: Install required packages - become: true - become_method: sudo - ansible.builtin.shell: - cmd: > - DEBIAN_FRONTEND=noninteractive apt install -y - python3-setuptools libapt-pkg-dev python3-pip git - when: - - ansible_os_family == "Debian" - - ansible_distribution_release == "trixie" - tags: vars_simple - -- name: Install python-apt from Git with pip3 - become: true - become_method: sudo - ansible.builtin.shell: - cmd: > - pip3 install - git+https://salsa.debian.org/apt-team/python-apt.git@2.9.6 - --break-system-packages - when: - - ansible_os_family == "Debian" - - ansible_distribution_release == "trixie" - tags: vars_simple - -- name: Hold python3-apt package version +- name: Upgrade Packages become: yes - ansible.builtin.command: apt-mark hold python3-apt - when: - - ansible_os_family == "Debian" - tags: vars_simple + become_method: sudo + ansible.builtin.apt: + upgrade: full + tags: firstconfig - name: Remove unattended-upgrades package become: yes diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh index 08c868c467077f118668a18028ba6cbefa74665b..32ffe3f037de16a724aa66741a99efa4e5eedcf0 100755 --- a/scripts/bringup_guestfs.sh +++ b/scripts/bringup_guestfs.sh @@ -222,6 +222,7 @@ append-line /etc/network/interfaces.d/enp1s0:iface enp1s0 inet dhcp firstboot-command systemctl stop ssh firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force openssh-server firstboot-command systemctl start ssh +firstboot-command apt update && apt upgrade --yes _EOT # CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST will not work # if /etc/nsswitch.conf has a line like this: diff --git a/scripts/firstconfig.Makefile b/scripts/firstconfig.Makefile new file mode 100644 index 0000000000000000000000000000000000000000..808975027ef475f07cb34fbf270dd404b7cbacf3 --- /dev/null +++ b/scripts/firstconfig.Makefile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +firstconfig: + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \ + -f 30 -i hosts \ + --extra-vars '{ kdevops_cli_install: True }' \ + --tags vars_simple,firstconfig \ + $(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml + +KDEVOPS_BRING_UP_DEPS_EARLY += firstconfig + +firstconfig-help: + @echo "firstconfig - Setup firstconfig" + +HELP_TARGETS += firstconfig-help