From patchwork Thu Mar 6 12:38:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Gomez X-Patchwork-Id: 14004414 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E157202C5C for ; Thu, 6 Mar 2025 12:39:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741264745; cv=none; b=sU2Y+XHLggM6A/eEu1F4iLr8SgFexVO3eskdr5mmj3tGTfYlKIy8q+v2qy/sNkTZUK4RnEy+T/OpcxJfgJLngdCqAhMFUu/zI6fHxqiXad69JQOIiThjG2/PHklGMQ5JsAmiMbT3PMJADdbVoWYGqvGcVvsS9cYv+38cblrt9Qk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741264745; c=relaxed/simple; bh=/o7u4lQfCNlhw0saTFLWKxY/eOAxuRX9eLPaJ99bpVw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=uE4ENw1hPqYVdxIBYkidNL5GDGsUJ5d3zx11GSuhHjEd6D9tBZVy/wy72+9PrSr/UgUhlm9W087dGa+xtuzVqsBa9Wz+IWq+KVSNlDCQikR0OpdTbAMhk6WHa8z1zha/TzYDLyhkqsuYSazpX8e5HuOAu/xid2lUTBg2QWeqPCg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BI9ZhBAp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BI9ZhBAp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A86E1C4CEE2; Thu, 6 Mar 2025 12:39:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741264744; bh=/o7u4lQfCNlhw0saTFLWKxY/eOAxuRX9eLPaJ99bpVw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BI9ZhBApV/P4TqgBGvyOxjz5s7h+RyVfF9W+QKiHeyUZPyiJyk+7kEz0X9g+LWeyd B3bM5Oy3i9rMCDQoFh0Q7Eu+/3gGd544aNxVIeuBBJ/0B5lhSoSNi0s04g2UKJy503 6K6CM8E13fZeXjKuOwzj5AAvTCogXWz/OmcWgb2Rx+mE3QtUJe/RRPJaJdQR9adDTF K9OqLhXg52QUsjX1t8PvEqiLMJO/1IiHBirlH4I57rwtxBY5aWHx/JKXMt+3ZR5fZS UILk4FvX+lETkMIQTuex8Q33yf+sez8CHnIlJzhvRk/w5c2AohLGbN2ubnJ/oLw4Vh hItBO0Ug9HZQg== From: Daniel Gomez Date: Thu, 06 Mar 2025 12:38:43 +0000 Subject: [PATCH v2 1/2] ansible: libvirt_user: Install libvirt-daemon-config-network for debian Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250306-fix-bringup-network-v2-1-85298d7a0ba5@samsung.com> References: <20250306-fix-bringup-network-v2-0-85298d7a0ba5@samsung.com> In-Reply-To: <20250306-fix-bringup-network-v2-0-85298d7a0ba5@samsung.com> To: Luis Chamberlain , Chuck Lever , Joel Granados Cc: kdevops@lists.linux.dev, Daniel Gomez X-Mailer: b4 0.14.2 From: Joel Granados Iinstall libvirt-daemon-config-network: so we have networks/defaults.xml on the system and are sure we are able to generate the virtual network if needed. Signed-off-by: Joel Granados Signed-off-by: Daniel Gomez --- playbooks/roles/libvirt_user/tasks/install-deps/debian/main.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/playbooks/roles/libvirt_user/tasks/install-deps/debian/main.yml b/playbooks/roles/libvirt_user/tasks/install-deps/debian/main.yml index 792597f930271bae53dfce1314d0aba7e5d618f1..463c1517b1d8b9999fac05e7f0970f278ccfb813 100644 --- a/playbooks/roles/libvirt_user/tasks/install-deps/debian/main.yml +++ b/playbooks/roles/libvirt_user/tasks/install-deps/debian/main.yml @@ -13,6 +13,7 @@ name: - libvirt-daemon - libvirt-daemon-system + - libvirt-daemon-config-network - libvirt-clients - qemu-kvm - qemu-system-x86 From patchwork Thu Mar 6 12:38:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Gomez X-Patchwork-Id: 14004415 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9569C1F3FC2 for ; Thu, 6 Mar 2025 12:39:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741264746; cv=none; b=X3RBuNK+NgFSOIITTFDlHLx/ovR0AvVuTXSSFbkCbizuq3mA0uBjpn+xra3qoinRi8nal4R/9hO4TV+0EVtEqdc6vANY/4M18YpVm/n3POyrNIJgcn5YmqUxtBs3WQ8h3hxZUROwBgC3Wip7y0BwX9piI/bxZEkKvYLn60oyLvg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741264746; c=relaxed/simple; bh=tRNsHw+Z43QHO/DnHay+sMsxvjex9IAbG67ObZHlP4c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=amFTsXD+W6sDadmsrMR7DpAe+EGBRq94rNAqg+NoPPBYwD46dMAyOlC+cBOZbGfH0cGShOG07getBrSpLIfgnrgVtakqchl5PHAfEMkcMXkCwW3CVSZbbMwloaFBPzcs5HM9iQEE/woUtx9aJC26/aJ4UnNiYJLtc6J4DJEicc0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pE7fj31h; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pE7fj31h" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18533C4CEE4; Thu, 6 Mar 2025 12:39:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741264746; bh=tRNsHw+Z43QHO/DnHay+sMsxvjex9IAbG67ObZHlP4c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pE7fj31hg/v9950Xdw0Qjegr6PZusJWae95Um2nujplxkah/yLTkY8rgaNCDT82Zx qg3EztUpqLlQ326P0OvGt0bIpB5J59BYMzHzsUVFIaIhnQnqRk8EZFwh2SNcDKrQG/ 7S008+R0kNAeYGZrZqlOmJLL5jfPeHkSqFKepVaMAt6t+lVsMNE/MCxQ4Igs+lWuII +9VokVdKdNWI7NYTu3k6MEuHYTP3Vj+4Z6tXSKeRwxe0d3z3vGPejknPSkcucCsqgT +YytljBhhLE/PQgTOVETFuEdmTSUvq3zILOtiYNF80fFzVgaELry3+9sEELAR1dEhh awE2HFVV65OEA== From: Daniel Gomez Date: Thu, 06 Mar 2025 12:38:44 +0000 Subject: [PATCH v2 2/2] guestfs: bringup: fix debian networking issue Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250306-fix-bringup-network-v2-2-85298d7a0ba5@samsung.com> References: <20250306-fix-bringup-network-v2-0-85298d7a0ba5@samsung.com> In-Reply-To: <20250306-fix-bringup-network-v2-0-85298d7a0ba5@samsung.com> To: Luis Chamberlain , Chuck Lever , Joel Granados Cc: kdevops@lists.linux.dev, Daniel Gomez X-Mailer: b4 0.14.2 From: Daniel Gomez When doing make bringup, virt-builder boots an image with network to be able to install packages for image baseline creation. In order to have network in the booted image, virt-builder assumes isc-dhcp-client is the default Debian package for this task. However, this is not the case as Debian trixie has replaced the default dhcp client isc-dhcp-client with dhcpcd. The proper way to fix this is to address the problem at libguestfs and ensure both packages can be used. This was supported already in libguestfs for Fedora/Red Hat but not for Debian. So, package support has been added to libguestfs for Debian hosts [1][2][3]. The fix is now merged in libguestfs v1.54.1. [1] https://lists.libguestfs.org/archives/list/guestfs@lists.libguestfs.org/ thread/ANINW4N4D2VYRD5GTAUQWDZDECAN6XFM/#ANINW4N4D2VYRD5GTAUQWDZDECAN6XF M [2] 91fab3f498b6847454656acd2f3d5788c533033b ("appliance: add dhcpcd support on Debian") [3] https://github.com/libguestfs/libguestfs/commit/91fab3f498b6847454656acd 2f3d5788c533033b While we wait Debian to upgrade the package to a newer libguestfs release that includes the fix, we need to ensure isc-dhcp-client is part of the host tools. Note that we need to be explicit as isc-dhcp-client is listed as optional runtime dependency for libguestfs [4]. [4] https://packages.debian.org/trixie/libguestfs0t64 This was reported already in a previous Debian release: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775514 In order to fix this, introduce a new target under the bringup_guestfs playbook to install libguestfs and it's dependencies. Add this for Debian, Suse and Fedora/RedHat. This new target also allows to remove the dependency check at min_deps introduced by commit 0111187b7ac23a12b6c71e99ff39092e6e752f2d ("Makefile.min_deps: ensure you have virt-builder when using guestfs"). Min deps is intended for minimal kdevops dependencies to run kdevops. And it's kdevops job with Ansible/Makefile dedicated playbooks-roles/targets to ensure libguestfs (including virt-builder) is installed in the host. To replicate the issue in Debian testing (without isc-dhcp-client installed and a libguestfs version < 1.54.1 ), run any of the following virt-builder commands: virt-builder fedora-27 --install qemu-guest-agent [ 3.2] Downloading: http://archive.libguestfs.org/builder/fedora-27.xz [ 3.7] Planning how to build this image [ 3.7] Uncompressing [ 5.3] Opening the new disk libguestfs: warning: current user is not a member of the KVM group (group ID 994). This user cannot access /dev/kvm, so libguestfs may run very slowly. It is recommended that you 'chmod 0666 /dev/kvm' or add the current user to the KVM group (you might need to log out and log in again). [ 27.3] Setting a random seed [ 27.4] Installing packages: qemu-guest-agent Error: Failed to synchronize cache for repo 'updates' virt-builder: error: dnf -y install 'qemu-guest-agent': command exited with an error If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] sudo virt-builder debian-12 --install sudo [ 0.7] Downloading: http://builder.libguestfs.org/debian-12.xz [ 1.0] Planning how to build this image [ 1.0] Uncompressing [ 2.9] Opening the new disk [ 6.3] Setting a random seed virt-builder: warning: random seed could not be set for this type of guest [ 6.4] Installing packages: sudo Ign:1 http://security.debian.org/debian-security bookworm-security InRelease Ign:2 http://deb.debian.org/debian bookworm InRelease Ign:3 http://deb.debian.org/debian bookworm-updates InRelease Ign:1 http://security.debian.org/debian-security bookworm-security InRelease Ign:2 http://deb.debian.org/debian bookworm InRelease Ign:3 http://deb.debian.org/debian bookworm-updates InRelease Ign:2 http://deb.debian.org/debian bookworm InRelease Ign:1 http://security.debian.org/debian-security bookworm-security InRelease Ign:3 http://deb.debian.org/debian bookworm-updates InRelease Err:2 http://deb.debian.org/debian bookworm InRelease Temporary failure resolving 'deb.debian.org' Err:1 http://security.debian.org/debian-security bookworm-security InRelease Temporary failure resolving 'security.debian.org' Err:3 http://deb.debian.org/debian bookworm-updates InRelease Temporary failure resolving 'deb.debian.org' Reading package lists... W: Failed to fetch http://deb.debian.org/debian/dists/bookworm/InRelease Temporary failure resolving 'deb.debian.org' W: Failed to fetch http://security.debian.org/debian-security/dists/bookworm-security/InRel ease Temporary failure resolving 'security.debian.org' W: Failed to fetch http://deb.debian.org/debian/dists/bookworm-updates/InRelease Temporary failure resolving 'deb.debian.org' W: Some index files failed to download. They have been ignored, or old ones used instead. Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: sudo 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 1889 kB of archives. After this operation, 6199 kB of additional disk space will be used. Ign:1 http://deb.debian.org/debian bookworm/main amd64 sudo amd64 1.9.13p3-1 Ign:1 http://deb.debian.org/debian bookworm/main amd64 sudo amd64 1.9.13p3-1 Ign:1 http://deb.debian.org/debian bookworm/main amd64 sudo amd64 1.9.13p3-1 Err:1 http://deb.debian.org/debian bookworm/main amd64 sudo amd64 1.9.13p3-1 Temporary failure resolving 'deb.debian.org' E: Failed to fetch http://deb.debian.org/debian/pool/main/s/sudo/sudo_1.9.13p3-1_amd64.deb Temporary failure resolving 'deb.debian.org' E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? virt-builder: error: export DEBIAN_FRONTEND=noninteractive apt_opts='-q -y -o Dpkg::Options::=--force-confnew' apt-get $apt_opts update apt-get $apt_opts install 'sudo' : command exited with an error If reporting bugs, run virt-builder with debugging enabled and include the complete output: virt-builder -v -x [...] Signed-off-by: Daniel Gomez Reviewed-by: Chuck Lever --- Makefile.min_deps | 5 ----- .../bringup_guestfs/tasks/install-deps/debian/main.yml | 11 +++++++++++ .../roles/bringup_guestfs/tasks/install-deps/main.yml | 17 +++++++++++++++++ .../bringup_guestfs/tasks/install-deps/redhat/main.yml | 11 +++++++++++ .../bringup_guestfs/tasks/install-deps/suse/main.yml | 10 ++++++++++ playbooks/roles/bringup_guestfs/tasks/main.yml | 4 ++++ scripts/guestfs.Makefile | 9 +++++++++ 7 files changed, 62 insertions(+), 5 deletions(-) diff --git a/Makefile.min_deps b/Makefile.min_deps index a1cb99a500575807042fdd7c47b1b192df5db1bd..9d8f205799c463ca85dd80cff7a813ded5b11eea 100644 --- a/Makefile.min_deps +++ b/Makefile.min_deps @@ -10,11 +10,6 @@ BINARY_DEPS += sudo BINARY_DEPS += make BINARY_DEPS += nc BINARY_DEPS += ansible-playbook -ifneq (y,$(CONFIG_KDEVOPS_FIRST_RUN)) -ifeq (y,$(CONFIG_GUESTFS)) -BINARY_DEPS += virt-builder -endif -endif BINARY_DEPS_VCHECKS := $(subst $(TOPDIR)/,,$(wildcard $(TOPDIR)/scripts/version_check/*)) diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml new file mode 100644 index 0000000000000000000000000000000000000000..9fbfb4703fd268500f643b4522b872a70b0c1808 --- /dev/null +++ b/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml @@ -0,0 +1,11 @@ +--- +- name: Install libguestfs + become: true + become_method: ansible.builtin.sudo + ansible.builtin.apt: + update_cache: true + name: + - libguestfs-tools + - isc-dhcp-client + state: present + tags: ['install-deps'] diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml new file mode 100644 index 0000000000000000000000000000000000000000..af3be6dea3c3445ecca4f3b3302e4640272882c6 --- /dev/null +++ b/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml @@ -0,0 +1,17 @@ +--- +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.yml + when: ansible_facts['os_family']|lower == 'debian' + tags: ['install-deps'] + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml + when: ansible_facts['os_family']|lower == 'suse' + tags: ['install-deps'] + +- name: Fedora/Red Hat Enterprise-specific setup + ansible.builtin.include_tasks: redhat/main.yml + when: + - ansible_facts['os_family']|lower == 'redhat' + - ansible_facts['distribution']|lower != "fedora" + tags: ['install-deps'] diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml new file mode 100644 index 0000000000000000000000000000000000000000..029b76a2f34927718b36c50080d330f80ee283ba --- /dev/null +++ b/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml @@ -0,0 +1,11 @@ +--- +- name: Install libguestfs on Fedora/RedHat + become: true + become_method: ansible.builtin.sudo + ansible.builtin.yum: + update_cache: true + name: + - libguestfs-tools + - dhclient + state: present + tags: ['install-deps'] diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml new file mode 100644 index 0000000000000000000000000000000000000000..9aa5178fd972b90d008f01e9dc127539608feb15 --- /dev/null +++ b/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml @@ -0,0 +1,10 @@ +--- +- name: Install libguestfs on SUSE + become: true + become_method: ansible.builtin.sudo + ansible.builtin.zypper: + name: + - libguestfs-tools + - dhclient + state: present + tags: ['install-deps'] diff --git a/playbooks/roles/bringup_guestfs/tasks/main.yml b/playbooks/roles/bringup_guestfs/tasks/main.yml index 8e0f82de5d0b9fa958fd1107ca75827f1b394b09..dcbbaef02522ef561b9d990b74a3e7573d912afb 100644 --- a/playbooks/roles/bringup_guestfs/tasks/main.yml +++ b/playbooks/roles/bringup_guestfs/tasks/main.yml @@ -1,3 +1,7 @@ +- name: Install dependencies + ansible.builtin.include_tasks: install-deps/main.yml + tags: ['install-deps'] + - name: Verify we're configured {{ topdir_path }}/.config directory exists stat: path: "{{ topdir_path }}/.config" diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile index 52397d54b89b584b675cf6182113d5a7e22da3c3..a8aa82fa96fa5bd9c181f10ef347bff8e3f78f44 100644 --- a/scripts/guestfs.Makefile +++ b/scripts/guestfs.Makefile @@ -47,6 +47,7 @@ ANSIBLE_EXTRA_ARGS += $(GUESTFS_ARGS) GUESTFS_BRINGUP_DEPS := GUESTFS_BRINGUP_DEPS += $(9P_HOST_CLONE) GUESTFS_BRINGUP_DEPS += $(LIBVIRT_PCIE_PASSTHROUGH) +GUESTFS_BRINGUP_DEPS += install_libguestfs KDEVOPS_PROVISION_METHOD := bringup_guestfs KDEVOPS_PROVISION_DESTROY_METHOD := destroy_guestfs @@ -72,6 +73,14 @@ $(KDEVOPS_PROVISIONED_SSH): $(Q)ansible $(ANSIBLE_VERBOSE) -i hosts all -e 'ansible_python_interpreter=/usr/bin/python3' -m wait_for_connection $(Q)touch $(KDEVOPS_PROVISIONED_SSH) +install_libguestfs: + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ + --inventory localhost, \ + playbooks/bringup_guestfs.yml \ + -e 'ansible_python_interpreter=/usr/bin/python3' \ + --extra-vars=@./extra_vars.yaml \ + --tags install-deps + bringup_guestfs: $(GUESTFS_BRINGUP_DEPS) $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ --inventory localhost, \