From patchwork Wed Feb 5 21:29:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 639A1924 for ; Wed, 5 Feb 2020 21:34:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A2F82082E for ; Wed, 5 Feb 2020 21:34:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="zUyrQKX0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A2F82082E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57144 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJf-0006gF-8V for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:34:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47205) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSIu-00054x-NT for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSIt-0005zw-N7 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:48 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:38988) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSIt-0005xL-H8 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:47 -0500 Received: by mail-pj1-x102a.google.com with SMTP id e9so1552459pjr.4 for ; Wed, 05 Feb 2020 13:33:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ylAb0fonLdpVNdNrr/Msmq6GA3Xtf0QpCchhiYOBib0=; b=zUyrQKX0TNovhHu15n5osFOM21VK9Jf+BHEL7dv1idJolSTqjsqpcFb4tGAnBBus5/ EVqCcKLef9KUqhfuBbG3qFKh+hM8T5GXtK5uHwpSChnBIIuVfhAsSdTIVCfmA8nz6ODe RsQtcPVQ7npmb7a0MDg+6iFykOkYAuQYOa5zVHj2LBahMrFE5eTo3UWUY8XEeuzWjSwc 4Qd4ciCY/NNEfuyxUYaXzNYGwSyS0yEHR/0DKmZToA6CgvsqTIwm9EV4ixJtyVNcN8Oz U64DjJYD92m5E9EuN6zjtlJMCbxJqC7zdeperh+NV2ch5/cV8Adt6OQsrI9f1pseEPBU Kn+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ylAb0fonLdpVNdNrr/Msmq6GA3Xtf0QpCchhiYOBib0=; b=Y9NI47OcQNxiNRJpUcu1Zq4Ex3nUSIofHggA9zVkQhH6tj65GVuph7QHFB20tGIU/S e3+ovK9AygbKj1IfRj2h2+3t90HTe8awdgT6BojmtAZ1OaMocPpgWBeyyy0WiYvw+yeF Cgf+s5O7ataAAKeeLmw5elwI9Fnbb5U6OUJ8EHcex6mu4cA8QSCgQLP8vMH/GEPCX4cp GUQNCoXsFz/P7B/GZ0YktWi1U8izSt4ZXZg6jS6n1lyUXhTzrZ4h+gKnj9jQRwQh73kA qzMbXvG++O+XNgwZaX8eF6U8tvJtymAH7un5vPnb8wmprvqf2d8OUpKQUL+OIZ8+htql eiiQ== X-Gm-Message-State: APjAAAUA/HYYe9+hHxhASNjEMujG5iuvWvcRsULTbiN62GUQLP+HKpVC c78o3FfRgmLvQG5MhFdA0n/uu1OoOQU= X-Google-Smtp-Source: APXvYqyPxGlz9bDHhTzkcqStWLMp7XeObhNrLwwm0v//lgEaI0/xhLsYXuYWitSlmeb7Ll8fQs7YvQ== X-Received: by 2002:a17:902:9f88:: with SMTP id g8mr27573plq.100.1580938426260; Wed, 05 Feb 2020 13:33:46 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:45 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 01/14] tests/vm: use $(PYTHON) consistently Date: Wed, 5 Feb 2020 16:29:07 -0500 Message-Id: <20200205212920.467-2-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::102a X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Change Makefile.include to use $(PYTHON) so for vm-boot-ssh to be consistent with other cases like vm-build. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée --- tests/vm/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 9e7c46a473..778e506755 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -80,7 +80,7 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(call quiet-command, \ - $(SRC_PATH)/tests/vm/$* \ + $(PYTHON) $(SRC_PATH)/tests/vm/$* \ $(if $(J),--jobs $(J)) \ --image "$<" \ --interactive \ From patchwork Wed Feb 5 21:29:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC350924 for ; Wed, 5 Feb 2020 21:34:40 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A32E42072B for ; Wed, 5 Feb 2020 21:34:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="pi4a91E7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A32E42072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJj-0006s0-Qq for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:34:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47246) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSIx-000583-33 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSIw-00064U-42 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:50 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]:38104) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSIv-00063a-TP for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:50 -0500 Received: by mail-pg1-x531.google.com with SMTP id a33so1595165pgm.5 for ; Wed, 05 Feb 2020 13:33:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OV9/GYr8QMb2MpdxO4ew9/UTUBDETqsBEY+AcIapd70=; b=pi4a91E7mprUaGo5YBnkhpkYXUAhZvdH3YKAiTCImDjqq3mtCeUpKe7m4I1xGIsMXh SNxNSAZJOLG9qN0Re5bLVG3Q5qCNZrYgCZAFibzmp3AZFXeuPtk77HaR9Wtu9xCETwaB EnGtCowSl1jfBGXY5LMi/F8zehzNGYyeNIBkhsgNK2jDQqPUFezDnDHZaU0CiLkcxqTY ZhcaW3lH1Rf8nsorw6x+ZFklSVptgIIvs6Qcy5Mk5AnZh0f0pwJ7b5cj4OG3POZKtY4j CScZPYq0HhF4SDpw/a1QSDSBHBCsSIl798sChPOS5AmIaxO+Y/M0cKc6WApsBqV3xuD9 ug2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OV9/GYr8QMb2MpdxO4ew9/UTUBDETqsBEY+AcIapd70=; b=og32aSRD74okhiLhRJeHC4v0QzRPxWjDTDpj1Hiupw8TzIlVVIfnr4idGvQve5E2dd 3vXpi8l7wFwLoZ/uDj4/1gLUAE/nADcDqoForcMgLqZ9gyikaMXSo+fVxriLfYLGWBTQ DvPD78ml7kEQkAWVo+Uey/vPj7g8B3jYWOnE4lfhJJ7IIk9mDi6sARWz1jGcB+RMENzb NE3vj5zwTt72tIfUsycfxLPSt5GY8jwUybKbilTj+lcAlACaaTOHXUiYqCHbz3Y7dTp6 c9GnIjvsAxDT7/4LjSmeTL1H/xHacnerffE/NwG1fLAmtgyUvq4BnERrmUh/lXymiSd5 i5PA== X-Gm-Message-State: APjAAAUk70xTzy99UMxxiEq58SZjLjCY4SNX+xkjt5tySy5Y5UDLz9MH 7Iky7mrLT290fbIM+lvqJZX1vszVHLg= X-Google-Smtp-Source: APXvYqxJF4Hs/ejNWnPdUBSIZb8wSiee7uMKnwsBNgKOpnzMH1NzSsuOc8puh3j8L+ILUIxiMQwQVg== X-Received: by 2002:aa7:8096:: with SMTP id v22mr38435217pff.240.1580938428236; Wed, 05 Feb 2020 13:33:48 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:47 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 02/14] tests/vm: Debug mode shows ssh output. Date: Wed, 5 Feb 2020 16:29:08 -0500 Message-Id: <20200205212920.467-3-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::531 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Add changes to tests/vm/basevm.py so that during debug mode we show ssh output. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index ed5dd4f3d0..991115e44b 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -122,11 +122,16 @@ class BaseVM(object): return fname def _ssh_do(self, user, cmd, check): - ssh_cmd = ["ssh", "-q", "-t", + ssh_cmd = ["ssh", + "-t", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=" + os.devnull, "-o", "ConnectTimeout=1", "-p", self.ssh_port, "-i", self._ssh_key_file] + # If not in debug mode, set ssh to quiet mode to + # avoid printing the results of commands. + if not self.debug: + ssh_cmd.append("-q") for var in self.envvars: ssh_cmd += ['-o', "SendEnv=%s" % var ] assert not isinstance(cmd, str) From patchwork Wed Feb 5 21:29:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19F8E112B for ; Wed, 5 Feb 2020 21:34:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E4CF02072B for ; Wed, 5 Feb 2020 21:34:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Cf8tA09l" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E4CF02072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJm-0006ug-3u for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:34:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47272) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSIy-0005BD-Qr for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSIx-00066K-Jo for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:52 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:34654) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSIx-00065w-EO for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:51 -0500 Received: by mail-pf1-x436.google.com with SMTP id i6so1890292pfc.1 for ; Wed, 05 Feb 2020 13:33:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CXBScSagi44Obo0j+z38gP3z1ZronJ0NsG9MHnQZm+M=; b=Cf8tA09l2c451f/RF9zFVviarRZy1Sm+Vpp+d26keIGoXlKIjdI3eXq0liF7xw/8vz nCr50EuhqB4sOqGDMrg8RitWfuYwWRIVKHNqht++R78vHSe6q/9m92MuHHQ7Mop3RVV/ PLDoI0ZJiIvqLD2/QhozPrxt+5rBLibRqv3iK8nV4I6yoJOEbcJyom4UjE+bc0tMTzOD q++IWOKy0NFrxjDBTDBdpNjFrmcrQ6by3eMHreKCxN1UxcjrdWqKvHOUgFfK4xIKGFmK CUpwagAuJ30S5jEy9vk7DUIf6DYjtpEEWo5aSSrVsq2rVb/Di2ANZY/Aer8qDJ6C0f/C KEmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CXBScSagi44Obo0j+z38gP3z1ZronJ0NsG9MHnQZm+M=; b=b1R9OKO0VJzfy4CZpkkJd0DQikOSsQTayOEOWNZ7BZgCTydLDNpNsMMrWQc4JlwmV9 R0pOfx6kmFnnSbPvxqtC2S7iEeTplD+uHq1/vDUqno96jlgRwvtzqcK/XOS9Z0aQUWGc a/976V3j2iQcWZxHdj3ovpBeu9Fj55J+0aTmXTIosNcMPqunk6tYu1tvHOpR5yyvAnE2 oMtmCzIXRhXk/JhpcYlhRuVz6IJ66spLwzbVwYVsql3S8uuXys3r3foE6zpdW+8mBY+Q 8SoQ20gzYP0VI7nYRDdJUWaYVcP7iZONfaEgDwbosK+715tcxKffp+FvmLIVbi1n1n+N Z9xA== X-Gm-Message-State: APjAAAUTAAgnWPWEVPxdCzm8J9Agwfs0+1VIp9xGdEEVAxxSWtQs5AeY PVJff4QCg997Wa0UwaPzYsLTELcA5NM= X-Google-Smtp-Source: APXvYqwWqA7oNOr0UvQG1YTvs56mKt/a256oIhq4gU9453z6TpIbxEqpKvpYeknUO1ZLD2FZ1GATdw== X-Received: by 2002:a62:14e:: with SMTP id 75mr13743pfb.54.1580938430139; Wed, 05 Feb 2020 13:33:50 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:49 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 03/14] tests/vm: increased max timeout for vm boot. Date: Wed, 5 Feb 2020 16:29:09 -0500 Message-Id: <20200205212920.467-4-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::436 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Add change to increase timeout waiting for VM to boot. Needed for some emulation cases where it can take longer than 5 minutes to boot. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 991115e44b..4de358ae22 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -59,6 +59,10 @@ class BaseVM(object): poweroff = "poweroff" # enable IPv6 networking ipv6 = True + # Scale up some timeouts under TCG. + # 4 is arbitrary, but greater than 2, + # since we found we need to wait more than twice as long. + tcg_ssh_timeout_multiplier = 4 def __init__(self, debug=False, vcpus=None): self._guest = None self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", @@ -311,6 +315,9 @@ class BaseVM(object): sys.stderr.write("### %s ...\n" % text) def wait_ssh(self, seconds=300): + # Allow more time for VM to boot under TCG. + if not kvm_available(self.arch): + seconds *= self.tcg_ssh_timeout_multiplier starttime = datetime.datetime.now() endtime = starttime + datetime.timedelta(seconds=seconds) guest_up = False From patchwork Wed Feb 5 21:29:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367223 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8A71112B for ; Wed, 5 Feb 2020 21:36:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8CF4A2072B for ; Wed, 5 Feb 2020 21:36:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="qGqdd/0x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8CF4A2072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57220 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSLA-0001bW-Md for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:36:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47297) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJ0-0005F3-O5 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSIz-000695-KB for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:54 -0500 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:36880) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSIz-000684-Dw for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:53 -0500 Received: by mail-pf1-x430.google.com with SMTP id p14so1883309pfn.4 for ; Wed, 05 Feb 2020 13:33:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u1zXQRTCt65TSYwJmTXjVmOnrC3M0DDokMmWoCnk7AI=; b=qGqdd/0xD0gvMFcvR5BE7RJBtSb6tNz9tRwXs3pt0ZzRayk8YDKDXhsWRDUh3pxUPx q4iHrqbPCXvNVD9uTrsFPkdJO0TsJl9v+JGMMmwF5Sx9EjhwE+NkXiETsRjGn4ZRxN8Q BHp3EoEgc0dwhWH15BkZiZH0AsFpJchzuhMomzzA62F4ofSxLUU0nvzj7z3HB44n48XX PEj0UELwiFrFFE3DTp7LzbM4N/OJDxDDrp0iNRRiOPczR3V2Ig6RJV1cP+0QP5WNh0+T cMqklLqkSp/mCL09Ek5RFLu06Q5Wfq0a842gtm9bxJwWnbdyFuJNp3fGPT4qtENVvOVb QHJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=u1zXQRTCt65TSYwJmTXjVmOnrC3M0DDokMmWoCnk7AI=; b=kCVdMU1FtJTbN2v+oHqm0YBDRx4AT5ucCmBLz4NKdkkSpaoTHQKEfEf9xV0IWkwVtA OPhjpyNGdVLiimdCOoAs1ET+EOqzcIJTGfNFoYKwDh10kriXToLf7eZBzX6iW1++IuUM Swi5qzhS7yaQgigOc9M+fbrp9XOWODuKyKQJTJWYxT3h1vEdqBBhM16t1W6VrqEGQuhH v2CI39pCVrRHtKVL0GL+N3QOPk3zlScNqDfuHqliECrGz/1/Cck0w5jTDm6sRY5lo/52 R5KtUHAlOHfDF2RbElIcIBrrvNm2Oca+2T176Zl5ajpRq3qEnew/2ZYojERIQuUWZEj5 lHug== X-Gm-Message-State: APjAAAVGrlKha66t81isXQHE0kbCA3mzINfyQFB2hSk7SmUCN/wlE3X0 OU8Wj89jc/Kg7yS/4BAxIj2B9Fa0K6o= X-Google-Smtp-Source: APXvYqxxgwOgnKxKvjUX9TKTprlfKqyONxsB2Y/7F4qzSRT2ZWErP98bxDLyOsQV8l1FdH50FXcH+w== X-Received: by 2002:a63:d60a:: with SMTP id q10mr11548460pgg.244.1580938432093; Wed, 05 Feb 2020 13:33:52 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:51 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 04/14] tests/vm: give wait_ssh() option to wait for root Date: Wed, 5 Feb 2020 16:29:10 -0500 Message-Id: <20200205212920.467-5-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::430 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Allow wait_ssh to wait for root user to be ready. This solves the issue where we perform a wait_ssh() successfully, but the root user is not yet ready to be logged in. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 4de358ae22..a29099f6f1 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -314,7 +314,7 @@ class BaseVM(object): def print_step(self, text): sys.stderr.write("### %s ...\n" % text) - def wait_ssh(self, seconds=300): + def wait_ssh(self, wait_root=False, seconds=300): # Allow more time for VM to boot under TCG. if not kvm_available(self.arch): seconds *= self.tcg_ssh_timeout_multiplier @@ -322,7 +322,10 @@ class BaseVM(object): endtime = starttime + datetime.timedelta(seconds=seconds) guest_up = False while datetime.datetime.now() < endtime: - if self.ssh("exit 0") == 0: + if wait_root and self.ssh_root("exit 0") == 0: + guest_up = True + break + elif self.ssh("exit 0") == 0: guest_up = True break seconds = (endtime - datetime.datetime.now()).total_seconds() @@ -333,7 +336,6 @@ class BaseVM(object): def shutdown(self): self._guest.shutdown() - def wait(self): self._guest.wait() From patchwork Wed Feb 5 21:29:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 692BB924 for ; Wed, 5 Feb 2020 21:35:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3FC442082E for ; Wed, 5 Feb 2020 21:35:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="oHAfiQMS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3FC442082E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57171 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSKS-0008QM-Ch for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:35:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47345) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJ3-0005ME-RE for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJ1-0006DQ-T4 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:57 -0500 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]:33431) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJ1-0006Bf-Kc for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:55 -0500 Received: by mail-pg1-x52c.google.com with SMTP id 6so1612549pgk.0 for ; Wed, 05 Feb 2020 13:33:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7GIcztAphvqkD0Pp3nbluYcOKPaeZeGBEwgC+UA1pto=; b=oHAfiQMS/aUfXYO1/tfz0x82ixNOz1AjJG7n44/gruVebwTzleAc75R+5ij6dU5N4y TMAS0dPvSoWlEc+e7v5TcxbMeWEgCnaDUQhX9b5wwDgt7J4zqk8fOMB9S/DklPgDxs6S XbrFTFOk3E38v5syiVbaPfyRWrrP2Ji9V1/ejGw+sRstZghGE2YQObMt8UBFnhPRrpNl htZPwtIo2a0KtjJbsnFPYEc8gT6TM4c7wkrj5Ut/ruFEk2OqnmARqUZv0HTEgAzAeoTH WGHd+U9GuBz6DP/nuG4MRLZXIxh+ArfONG8c/WG/8fRWo/9M2zmRPzydE01/ZQ0xnZLF fiAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7GIcztAphvqkD0Pp3nbluYcOKPaeZeGBEwgC+UA1pto=; b=D+8nTx50viYGT8+xyhSFo29tjr8w4ANdteNA5tkYqGTzLQunxT4n97N79eSwaCvvLd sVn2aAkbtIaVjl/f59pmz5GdUJzXDHPsFeHn2sWlgDwRVqQaa2PYd1+JCkvNxo/Yh2at V5xQndUnVRTVOZ3rdxlvFf/X526m5JngX9XCezLjUybSfhOrutE8oHQ/TuYhfgjSigpa uCKm5F6C5vNMbtxZJHjFAR3nRBgR6AYXyKhdt9Fo33ZyZXWSB0rI9r1CDtNpu1zlX0Ax XWe8NDQ7d/rBSjZtWl8x6UhYlZqiH98IzHNogg5twninWsyQCOZ55Krqw4TTTiLPLxo1 aSPA== X-Gm-Message-State: APjAAAUv97uwLt3bGhAE/iGsgzo95yReViFeAB03qWc40teJoe2Yz3wD WuZ3PbUZEMFGiDwfITlC3KVBxoY725M= X-Google-Smtp-Source: APXvYqxEitoFpDNd8Z+AVaagrmH1hy47EGUYSRCMRI80u8BA9jaqOeaui/T8Dsh0tEQSfbIRnrdERQ== X-Received: by 2002:a63:dc4f:: with SMTP id f15mr35580980pgj.300.1580938434121; Wed, 05 Feb 2020 13:33:54 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:53 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 05/14] tests/vm: Added gen_cloud_init_iso() to basevm.py Date: Wed, 5 Feb 2020 16:29:11 -0500 Message-Id: <20200205212920.467-6-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::52c X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This method was located in both centos and ubuntu.i386. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 40 ++++++++++++++++++++++++++++++++++++++++ tests/vm/centos | 33 +-------------------------------- tests/vm/ubuntu.i386 | 37 +------------------------------------ 3 files changed, 42 insertions(+), 68 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index a29099f6f1..a926211da8 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -346,6 +346,46 @@ class BaseVM(object): def qmp(self, *args, **kwargs): return self._guest.qmp(*args, **kwargs) + def gen_cloud_init_iso(self): + cidir = self._tmpdir + mdata = open(os.path.join(cidir, "meta-data"), "w") + name = self.name.replace(".","-") + mdata.writelines(["instance-id: {}-vm-0\n".format(name), + "local-hostname: {}-guest\n".format(name)]) + mdata.close() + udata = open(os.path.join(cidir, "user-data"), "w") + print("guest user:pw {}:{}".format(self._config['guest_user'], + self._config['guest_pass'])) + udata.writelines(["#cloud-config\n", + "chpasswd:\n", + " list: |\n", + " root:%s\n" % self._config['root_pass'], + " %s:%s\n" % (self._config['guest_user'], + self._config['guest_pass']), + " expire: False\n", + "users:\n", + " - name: %s\n" % self._config['guest_user'], + " sudo: ALL=(ALL) NOPASSWD:ALL\n", + " ssh-authorized-keys:\n", + " - %s\n" % self._config['ssh_pub_key'], + " - name: root\n", + " ssh-authorized-keys:\n", + " - %s\n" % self._config['ssh_pub_key'], + "locale: en_US.UTF-8\n"]) + proxy = os.environ.get("http_proxy") + if not proxy is None: + udata.writelines(["apt:\n", + " proxy: %s" % proxy]) + udata.close() + subprocess.check_call(["genisoimage", "-output", "cloud-init.iso", + "-volid", "cidata", "-joliet", "-rock", + "user-data", "meta-data"], + cwd=cidir, + stdin=self._devnull, stdout=self._stdout, + stderr=self._stdout) + + return os.path.join(cidir, "cloud-init.iso") + def parse_args(vmcls): def get_default_jobs(): diff --git a/tests/vm/centos b/tests/vm/centos index f2f0befd84..c108bd6799 100755 --- a/tests/vm/centos +++ b/tests/vm/centos @@ -31,37 +31,6 @@ class CentosVM(basevm.BaseVM): make docker-test-mingw@fedora {verbose} J={jobs} NETWORK=1; """ - def _gen_cloud_init_iso(self): - cidir = self._tmpdir - mdata = open(os.path.join(cidir, "meta-data"), "w") - mdata.writelines(["instance-id: centos-vm-0\n", - "local-hostname: centos-guest\n"]) - mdata.close() - udata = open(os.path.join(cidir, "user-data"), "w") - udata.writelines(["#cloud-config\n", - "chpasswd:\n", - " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS), - " expire: False\n", - "users:\n", - " - name: %s\n" % self.GUEST_USER, - " sudo: ALL=(ALL) NOPASSWD:ALL\n", - " ssh-authorized-keys:\n", - " - %s\n" % basevm.SSH_PUB_KEY, - " - name: root\n", - " ssh-authorized-keys:\n", - " - %s\n" % basevm.SSH_PUB_KEY, - "locale: en_US.UTF-8\n"]) - udata.close() - subprocess.check_call(["genisoimage", "-output", "cloud-init.iso", - "-volid", "cidata", "-joliet", "-rock", - "user-data", "meta-data"], - cwd=cidir, - stdin=self._devnull, stdout=self._stdout, - stderr=self._stdout) - return os.path.join(cidir, "cloud-init.iso") - def build_image(self, img): cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz") img_tmp = img + ".tmp" @@ -69,7 +38,7 @@ class CentosVM(basevm.BaseVM): subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"]) subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"]) self.exec_qemu_img("resize", img_tmp, "50G") - self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()]) + self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()]) self.wait_ssh() self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") self.ssh_root_check("yum update -y") diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 3834cd7a8d..7df54ae094 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -29,41 +29,6 @@ class UbuntuX86VM(basevm.BaseVM): make --output-sync {target} -j{jobs} {verbose}; """ - def _gen_cloud_init_iso(self): - cidir = self._tmpdir - mdata = open(os.path.join(cidir, "meta-data"), "w") - mdata.writelines(["instance-id: ubuntu-vm-0\n", - "local-hostname: ubuntu-guest\n"]) - mdata.close() - udata = open(os.path.join(cidir, "user-data"), "w") - udata.writelines(["#cloud-config\n", - "chpasswd:\n", - " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS), - " expire: False\n", - "users:\n", - " - name: %s\n" % self.GUEST_USER, - " sudo: ALL=(ALL) NOPASSWD:ALL\n", - " ssh-authorized-keys:\n", - " - %s\n" % basevm.SSH_PUB_KEY, - " - name: root\n", - " ssh-authorized-keys:\n", - " - %s\n" % basevm.SSH_PUB_KEY, - "locale: en_US.UTF-8\n"]) - proxy = os.environ.get("http_proxy") - if not proxy is None: - udata.writelines(["apt:\n", - " proxy: %s" % proxy]) - udata.close() - subprocess.check_call(["genisoimage", "-output", "cloud-init.iso", - "-volid", "cidata", "-joliet", "-rock", - "user-data", "meta-data"], - cwd=cidir, - stdin=self._devnull, stdout=self._stdout, - stderr=self._stdout) - return os.path.join(cidir, "cloud-init.iso") - def build_image(self, img): cimg = self._download_with_cache( "https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img", @@ -71,7 +36,7 @@ class UbuntuX86VM(basevm.BaseVM): img_tmp = img + ".tmp" subprocess.check_call(["cp", "-f", cimg, img_tmp]) self.exec_qemu_img("resize", img_tmp, "50G") - self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()]) + self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()]) self.wait_ssh() self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") self.ssh_root_check("apt-get update") From patchwork Wed Feb 5 21:29:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0317924 for ; Wed, 5 Feb 2020 21:37:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A692220720 for ; Wed, 5 Feb 2020 21:37:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="SR43YNNM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A692220720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57258 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSMb-0004FG-RG for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:37:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47367) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJ5-0005SE-FA for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJ3-0006FP-OU for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:59 -0500 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]:43806) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJ3-0006Eu-GP for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:57 -0500 Received: by mail-pf1-x42e.google.com with SMTP id s1so1868564pfh.10 for ; Wed, 05 Feb 2020 13:33:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=He5LBREQA24cBQ1DXRllAwt0MoWNHv1kGRWxO6ZPqnI=; b=SR43YNNMGbxY1vTaR4RDW89Uh8L6D+IpXlVAFCbJDS/1nXWa8NLCrXjZbGbeSc080R j+6f6tnCf3Dv519lByoIYvmlVa8YcY/SLAtnQiNd/HSyheGt6s5WrAU4DTsnau4k378a VmD8e01CAHBBrkzhFWixpMT1nOU679Xhess5UtY99ZXvH33+rRikkSD+qW/WIICaRg6r wumVk5KDMMXUq8FKFFIOP0ZGXk9ciKTrxIST++Rkvi3HYtROSkPTA9KZgixusfTsI4tG CyMQd8Sp6+I97WwgkRPusKTzdrN9g/WLhDpQXLdLoxwFh8ehQGjJYz0Na5gGp1eG0hCP +vtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=He5LBREQA24cBQ1DXRllAwt0MoWNHv1kGRWxO6ZPqnI=; b=QUwM9fdJ/Z+rgldTEHNGE/gHRsVcGviGsKUYpbNcDK9MmJcxj7pq9Wu3iOzdOfQewF l0Gj8s7q2LiWEBLeV4/HDQNM4O6xEXTvZ2pvQVd4zn4fwSn31WlSMZHEPf6kW3j3PlE9 0bceeM6j/z372F0BqvekaM0FXq8/L52+sI7zxGSFfHNzVIedx1/wnlgbe33J/Df3vRfv +VbRxeuD+d+BBDNDrEpRux7e2MBgtEmPKbVLSj3PfmBdwlGT8qNqfkZGB0j+L0NMvUQB kxszTzaatiVRY5IgGdNkLvsOjs7GXY75OrivyDkSvF8fKhcjm1uOQP4mF+vinQYSpvz+ UC8g== X-Gm-Message-State: APjAAAW/ArtQ5NgHQ/fUXxmmtdxvZ7zqe+4DpWUKGNBvRxuErrlmrdbF epYmRYdakeySjcwkDx2Y0LxAf2kqOjA= X-Google-Smtp-Source: APXvYqymcO9f3yF2kK+paOlUyHAIM++uhZEqN6HEHZsejLmupi5EsbRMZmZdu9ATpxdFDGfvGrlaZA== X-Received: by 2002:a62:342:: with SMTP id 63mr38753663pfd.19.1580938436042; Wed, 05 Feb 2020 13:33:56 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:55 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 06/14] tests/vm: Add logging of console to file. Date: Wed, 5 Feb 2020 16:29:12 -0500 Message-Id: <20200205212920.467-7-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42e X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This adds logging of the char device used by the console to a file. The basevm.py then uses this file to read chars from the console. One reason to add this is to aid with debugging. But another is because there is an issue where the QEMU might hang if the characters from the character device are not consumed by the script. Signed-off-by: Robert Foley --- tests/vm/basevm.py | 48 ++++++++++++++++++++++--- tests/vm/socket_thread.py | 73 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 116 insertions(+), 5 deletions(-) create mode 100644 tests/vm/socket_thread.py diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index a926211da8..87a484c55c 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -31,12 +31,17 @@ import tempfile import shutil import multiprocessing import traceback +from socket_thread import SocketThread SSH_KEY = open(os.path.join(os.path.dirname(__file__), "..", "keys", "id_rsa")).read() SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__), "..", "keys", "id_rsa.pub")).read() +class ConsoleTimeoutException(Exception): + """Raise this exception when read from console times out.""" + pass + class BaseVM(object): GUEST_USER = "qemu" GUEST_PASS = "qemupass" @@ -59,12 +64,18 @@ class BaseVM(object): poweroff = "poweroff" # enable IPv6 networking ipv6 = True + # This is the timeout on the wait for console bytes. + socket_timeout = 120 # Scale up some timeouts under TCG. # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 + console_logfile = "console.log" def __init__(self, debug=False, vcpus=None): self._guest = None + self._console_fd = None + self._socket_thread = None + self._console_timeout_sec = self.socket_timeout self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) @@ -179,6 +190,15 @@ class BaseVM(object): "-device", "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)] + def init_console(self, socket): + """Init the thread to dump console to a file. + Also open the file descriptor we will use to + read from the console.""" + self._socket_thread = SocketThread(socket, self.console_logfile) + self._console_fd = open(self.console_logfile, "r") + self._socket_thread.start() + print("console logfile is: {}".format(self.console_logfile)) + def boot(self, img, extra_args=[]): args = self._args + [ "-device", "VGA", @@ -201,6 +221,7 @@ class BaseVM(object): raise atexit.register(self.shutdown) self._guest = guest + self.init_console(guest.console_socket) usernet_info = guest.qmp("human-monitor-command", command_line="info usernet") self.ssh_port = None @@ -212,9 +233,10 @@ class BaseVM(object): raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \ usernet_info) - def console_init(self, timeout = 120): - vm = self._guest - vm.console_socket.settimeout(timeout) + def console_init(self, timeout = None): + if timeout == None: + timeout = self.socket_timeout + self._console_timeout_sec = timeout def console_log(self, text): for line in re.split("[\r\n]", text): @@ -230,13 +252,27 @@ class BaseVM(object): # log console line sys.stderr.write("con recv: %s\n" % line) + def console_recv(self, n): + """Read n chars from the console_logfile being dumped to + by the socket thread we created earlier.""" + start_time = time.time() + while True: + data = self._console_fd.read(1) + if data != "": + break + time.sleep(0.1) + elapsed_sec = time.time() - start_time + if elapsed_sec > self._console_timeout_sec: + raise ConsoleTimeoutException + return data.encode('latin1') + def console_wait(self, expect, expectalt = None): vm = self._guest output = "" while True: try: - chars = vm.console_socket.recv(1) - except socket.timeout: + chars = self.console_recv(1) + except ConsoleTimeoutException: sys.stderr.write("console: *** read timeout ***\n") sys.stderr.write("console: waiting for: '%s'\n" % expect) if not expectalt is None: @@ -335,6 +371,8 @@ class BaseVM(object): raise Exception("Timeout while waiting for guest ssh") def shutdown(self): + self._socket_thread.join() + self._console_fd.close() self._guest.shutdown() def wait(self): self._guest.wait() diff --git a/tests/vm/socket_thread.py b/tests/vm/socket_thread.py new file mode 100644 index 0000000000..6160e9163d --- /dev/null +++ b/tests/vm/socket_thread.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python +# +# This python module defines a thread object which +# reads from a socket and dumps it to a file. +# +# The main use case is for reading QEMU console char dev and +# dumping them to a file either for debugging or for +# parsing by QEMU itself. +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import sys +import re +import threading +import time +import traceback +import gettext + +class SocketThread(threading.Thread): + """ Implements the standard threading.Thread API.(start, join, etc.). + dumps all characters received on socket into a file. + """ + def __init__(self, socket, filename): + super(SocketThread, self).__init__() + self.alive = threading.Event() + self.alive.set() + self.socket = socket + self.log_file = open(filename, "w") + self.debug = True + + def receive(self): + """Until the user calls join, we will read chars from + the socket and dump them as is to the file.""" + self.socket.setblocking(0) + self.socket.settimeout(1.0) + while self.alive.isSet(): + try: + chars = self.socket.recv(1) + except: + continue + output = chars.decode("latin1") + self.log_file.write("{}".format(output)) + # Flush the file since we need the characters to be + # always up to date in case someone is reading the file + # waiting for some characters to show up. + self.log_file.flush() + self.socket.setblocking(1) + + def run(self): + """This is the main loop of the socket thread. + Simply receive from the file until the user + calls join.""" + while self.alive.isSet(): + try: + self.receive() + except Exception as e: + sys.stderr.write("Exception encountered\n") + traceback.print_exc() + continue + + def join(self, timeout=None): + """Time to destroy the thread. + Clear the event to stop the thread, and wait for + it to complete.""" + self.alive.clear() + threading.Thread.join(self, timeout) + self.log_file.close() From patchwork Wed Feb 5 21:29:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367225 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2EFFE924 for ; Wed, 5 Feb 2020 21:36:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E904E20720 for ; Wed, 5 Feb 2020 21:36:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="HIN7QMTq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E904E20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57234 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSLr-0002nS-5U for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:36:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47392) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJ7-0005Y7-Q9 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJ5-0006Jf-Ra for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:01 -0500 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]:36874) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJ5-0006IS-Iu for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:33:59 -0500 Received: by mail-pf1-x429.google.com with SMTP id p14so1883491pfn.4 for ; Wed, 05 Feb 2020 13:33:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BTo2IbS/Iw9aPjF8JvV5QAW0iHf49XaNmUCUIWa/t68=; b=HIN7QMTqZa7eGiwKX3QPSoPxRoepzlORT+gIPul3AiDFR70qjIiW87Dq1EofUPEWKs YYQqpEkxtty/3GG8canLgENp2Pq+KZ1mZyAWtjo0Tz8mLla911ibHbDTpYkt9w1WMPK6 huuWRTOloc9UcqlIcmxlv3MDncZAv2Y/9kPQD0j/kruhT1fkUDc988WgsDX7JGLTmZK8 qDkzcyJSCPWso1+ymAf11LcWJyzLgqrTd6lhOGq7WVuAAYFk/ILGVopkyYEddBYrPKPi 1zMX63iRAc5QzleGMLt+BFG93Xw24HdtB8a4ETqkzVBBTqlDZV/4dubLHi184UIQiBDu NIWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BTo2IbS/Iw9aPjF8JvV5QAW0iHf49XaNmUCUIWa/t68=; b=NAik4MP9HO+Pcag5wbDfH7mkgUrY9+PylULKYn2F31VMsgs7ZfA+v+rtPJWUMnNdoz zBar8R0sTdwiOT3ZjAt+joVA3EHkHSTHW4qoWmAwZVFzBqL/+J+rMPIBi3hLjBiLoed9 E2X5XZQyL7KpmYXTm2t3Zv5j2WJ2IMg2pUekJboXzdG3n/o8r7Tap4InQrxpthbXW8oN zzigR02iB8FeaYuZNYZBekub6QeobnuRpQngrpkX/QdHoR0UfpprfOMQCsgCYlbNliDx n6ud/psKrPPa9yKIVI3KJBaPb9iDbT6l2zVHJ8tSh65Nt8DV0T/o72uFvT83z8bqJCFS FhoQ== X-Gm-Message-State: APjAAAViFB+33hsAtp4n/pK4Don7z/UhUzsVU/pcsxXyvihlxQ5q3FlV 8+94VuchsXCL8X2bv2KJlyuHkDkKdlM= X-Google-Smtp-Source: APXvYqxCuoBzfx9OCphWMSwmr9t1QLKXcqpcBRXha984RIFCxRoKq678QlsfisuYTJcGeb0Qfl95xg== X-Received: by 2002:aa7:82d5:: with SMTP id f21mr39708118pfn.245.1580938438017; Wed, 05 Feb 2020 13:33:58 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:57 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 07/14] tests/vm: Add configuration to basevm.py Date: Wed, 5 Feb 2020 16:29:13 -0500 Message-Id: <20200205212920.467-8-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::429 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Added use of a configuration to tests/vm/basevm.py. The configuration provides parameters used to configure a VM. This allows for providing alternate configurations to the VM being created/launched. cpu, machine, memory, and NUMA configuration are all examples of configuration which we might want to vary on the VM being created or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley --- tests/vm/basevm.py | 141 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 111 insertions(+), 30 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 87a484c55c..33004934af 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -33,19 +33,43 @@ import multiprocessing import traceback from socket_thread import SocketThread -SSH_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa")).read() -SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa.pub")).read() +SSH_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa") +SSH_PUB_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa.pub") class ConsoleTimeoutException(Exception): """Raise this exception when read from console times out.""" pass +# This is the standard configuration. +# Any or all of these can be overridden by +# passing in a config argument to the VM constructor. +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : 'pc', + 'guest_user' : "qemu", + 'guest_pass' : "qemupass", + 'root_pass' : "qemupass", + 'ssh_key_file' : SSH_KEY_FILE, + 'ssh_pub_key_file': SSH_PUB_KEY_FILE, + 'memory' : "4G", + 'extra_args' : [], + 'qemu_args' : "", + 'dns' : "", + 'ssh_port' : 0, + 'install_cmds' : "", + 'boot_dev_type' : "block", + 'ssh_timeout' : 1, +} +BOOT_DEVICE = { + 'block' : "-drive file={},if=none,id=drive0,cache=writeback "\ + "-device virtio-blk,drive=drive0,bootindex=0", + 'scsi' : "-device virtio-scsi-device,id=scsi "\ + "-drive file={},format=raw,if=none,id=hd0 "\ + "-device scsi-hd,drive=hd0,bootindex=0", +} class BaseVM(object): - GUEST_USER = "qemu" - GUEST_PASS = "qemupass" - ROOT_PASS = "qemupass" envvars = [ "https_proxy", @@ -71,22 +95,33 @@ class BaseVM(object): # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 console_logfile = "console.log" - def __init__(self, debug=False, vcpus=None): + def __init__(self, debug=False, vcpus=None, config=None): self._guest = None self._console_fd = None self._socket_thread = None self._console_timeout_sec = self.socket_timeout + # Allow input config to override defaults. + self._config = DEFAULT_CONFIG.copy() + if config != None: + self._config.update(config) + self.validate_ssh_keys() self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) atexit.register(shutil.rmtree, self._tmpdir) - - self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa") - open(self._ssh_key_file, "w").write(SSH_KEY) - subprocess.check_call(["chmod", "600", self._ssh_key_file]) - - self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") - open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) + # Copy the key files to a temporary directory. + # Also chmod the key file to agree with ssh requirements. + self._config['ssh_key'] = \ + open(self._config['ssh_key_file']).read().rstrip() + self._config['ssh_pub_key'] = \ + open(self._config['ssh_pub_key_file']).read().rstrip() + self._ssh_tmp_key_file = os.path.join(self._tmpdir, "id_rsa") + open(self._ssh_tmp_key_file, "w").write(self._config['ssh_key']) + subprocess.check_call(["chmod", "600", self._ssh_tmp_key_file]) + + self._ssh_tmp_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") + open(self._ssh_tmp_pub_key_file, + "w").write(self._config['ssh_pub_key']) self.debug = debug self._stderr = sys.stderr @@ -95,11 +130,14 @@ class BaseVM(object): self._stdout = sys.stdout else: self._stdout = self._devnull + netdev = "user,id=vnet,hostfwd=:127.0.0.1:{}-:22" self._args = [ \ - "-nodefaults", "-m", "4G", - "-cpu", "max", - "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + - (",ipv6=no" if not self.ipv6 else ""), + "-nodefaults", "-m", self._config['memory'], + "-cpu", self._config['cpu'], + "-netdev", + netdev.format(self._config['ssh_port']) + + (",ipv6=no" if not self.ipv6 else "") + + (",dns=" + self._config['dns'] if self._config['dns'] else ""), "-device", "virtio-net-pci,netdev=vnet", "-vnc", "127.0.0.1:0,to=20"] if vcpus and vcpus > 1: @@ -110,6 +148,45 @@ class BaseVM(object): logging.info("KVM not available, not using -enable-kvm") self._data_args = [] + if self._config['qemu_args'] != None: + qemu_args = self._config['qemu_args'] + qemu_args = qemu_args.replace('\n',' ').replace('\r','') + # Remove any empty strings from list. + self._config['extra_args'] = [x for x in qemu_args.split(' ') if x] + + def validate_ssh_keys(self): + """Check to see if the ssh key files exist.""" + if 'ssh_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + + def wait_boot(self, wait_string=None): + """Wait for the standard string we expect + on completion of a normal boot. + The user can also choose to override with an + alternate string to wait for.""" + if wait_string is None: + if self.login_prompt is None: + raise Exception("self.login_prompt not defined") + wait_string = self.login_prompt + # Intentionally bump up the default timeout under TCG, + # since the console wait below takes longer. + timeout = self.socket_timeout + if not kvm_available(self.arch): + timeout *= 8 + self.console_init(timeout=timeout) + self.console_wait(wait_string) + + def __getattr__(self, name): + # Support direct access to config by key. + # for example, access self._config['cpu'] by self.cpu + if name.lower() in self._config.keys(): + return self._config[name.lower()] + return object.__getattribute__(self, name) + def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -141,8 +218,9 @@ class BaseVM(object): "-t", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=" + os.devnull, - "-o", "ConnectTimeout=1", - "-p", self.ssh_port, "-i", self._ssh_key_file] + "-o", + "ConnectTimeout={}".format(self._config["ssh_timeout"]), + "-p", self.ssh_port, "-i", self._ssh_tmp_key_file] # If not in debug mode, set ssh to quiet mode to # avoid printing the results of commands. if not self.debug: @@ -200,15 +278,15 @@ class BaseVM(object): print("console logfile is: {}".format(self.console_logfile)) def boot(self, img, extra_args=[]): - args = self._args + [ - "-device", "VGA", - "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img, - "-device", "virtio-blk,drive=drive0,bootindex=0"] - args += self._data_args + extra_args + boot_dev = BOOT_DEVICE[self._config['boot_dev_type']] + boot_params = boot_dev.format(img) + args = self._args + boot_params.split(' ') + args += self._data_args + extra_args + self._config['extra_args'] + args += ["-device", "VGA"] logging.debug("QEMU args: %s", " ".join(args)) qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch) guest = QEMUMachine(binary=qemu_bin, args=args) - guest.set_machine('pc') + guest.set_machine(self._config['machine']) guest.set_console() try: guest.launch() @@ -332,7 +410,8 @@ class BaseVM(object): self.console_send(command) def console_ssh_init(self, prompt, user, pw): - sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip() + sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" \ + % self._config['ssh_pub_key'].rstrip() self.console_wait_send("login:", "%s\n" % user) self.console_wait_send("Password:", "%s\n" % pw) self.console_wait_send(prompt, "mkdir .ssh\n") @@ -461,15 +540,17 @@ def parse_args(vmcls): parser.disable_interspersed_args() return parser.parse_args() -def main(vmcls): +def main(vmcls, config=None): try: + if config == None: + config = {} args, argv = parse_args(vmcls) if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(debug=args.debug, vcpus=args.jobs) + vm = vmcls(debug=args.debug, vcpus=args.jobs, config=config) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Wed Feb 5 21:29:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8CD33924 for ; Wed, 5 Feb 2020 21:39:13 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 63AB320720 for ; Wed, 5 Feb 2020 21:39:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="jYLeZz1k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 63AB320720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSO8-0006r5-K0 for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:39:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47428) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJ9-0005c7-Nf for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJ7-0006Pa-Rc for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:03 -0500 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:37972) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJ7-0006N2-IQ for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:01 -0500 Received: by mail-pg1-x544.google.com with SMTP id a33so1595500pgm.5 for ; Wed, 05 Feb 2020 13:34:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IQap2T8JqAz08Y5Tf6O+E0PgwZXgb93paPBw5e7BGIg=; b=jYLeZz1kOa+bKXYWeahYthoPBBEwPBx9dWOz8i1El5IF+7cZMVEiVDA1nrfBl73aR1 GZ3JKMDYMTHejAcNM/To88Bs74XQlJJfBAkA9fN0YWpUKghRhw52KW2cw4LQWdjx97q5 FX/cU58rObL45q8lVPf4qxbrN66M2s7mcESGpX6C+Aen6DTV0CabLVVSKl3zRyigYY8j Z9bd6rVrR3TTcGPIswq2pcip836vuKP9TXRDQlQMlx/6a4uOAyzWnsP8k/u/XRiDFjzz vZ//kB2Krgy8WMy8FxJ6pv1ni/PXkrimqmfm3mJx2ufnhZLWHXeNtGIuyMpCXe8VFyRk m3QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IQap2T8JqAz08Y5Tf6O+E0PgwZXgb93paPBw5e7BGIg=; b=OVbLfa8kChyhZvWYgNg+Vse3TUHNLxabDrnjuNcyq0hDBS9jqreIY7fkZbOIPOBr4G OjRdKfQY7CW299BmVTD6eRoouHX1rrW8xY1Jp6qj1yJqNNAHezzSrb8OVHIXpvmnxTaM zL3EJT/dQWF/mcsoOoQMecdT1isEzJxhjBskf3UTDy4B/3wKUjpKnC0FfyBgD6L5W7UI UBhtuqStUMSx2P9vBdc7yGCT2Ce02vKBYicE4m/nSxqE6feAlzRwyOsOFybzTWkQA/rc v6bo4JEbW7M0eJLUbPY+/S87J0RoztGOmd0C/V6BpGZMN4v6D1/1MOsMEiyfFaNP2V6+ 9XGg== X-Gm-Message-State: APjAAAWhDbDi8Z8UKlUiWu5s5uEX2Zx+yHRBZtaELNfQaA2unNXn+JXM Yvx4CP1AsPUY5NXEgEOFyLvRxApRVbg= X-Google-Smtp-Source: APXvYqyKBihD+AWn3Cj9ol35zNTjRFEKJBkGRhEzZhGCS04GBgulgIz4tEqVBdjQx5UOz4oHTzlSJw== X-Received: by 2002:a62:e30f:: with SMTP id g15mr3461pfh.124.1580938440027; Wed, 05 Feb 2020 13:34:00 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.33.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:33:59 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 08/14] tests/vm: Added configuration file support Date: Wed, 5 Feb 2020 16:29:14 -0500 Message-Id: <20200205212920.467-9-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::544 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Changes to tests/vm/basevm.py to allow accepting a configuration file as a parameter. Allows for specifying VM options such as cpu, machine, memory, and arbitrary qemu arguments for specifying options such as NUMA configuration. Also added an example conf_example_aarch64.yml and conf_example_x86.yml. Signed-off-by: Robert Foley --- tests/vm/Makefile.include | 2 ++ tests/vm/basevm.py | 29 +++++++++++++++++- tests/vm/conf_example_aarch64.yml | 51 +++++++++++++++++++++++++++++++ tests/vm/conf_example_x86.yml | 50 ++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 1 deletion(-) create mode 100644 tests/vm/conf_example_aarch64.yml create mode 100644 tests/vm/conf_example_x86.yml diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 778e506755..e9ed33226d 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -35,6 +35,8 @@ vm-help vm-test: @echo " V=1 - Enable verbose ouput on host and guest commands" @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=/path/to/qemu-img - Change path to qemu-img tool" + @echo " QEMU_CONFIG=/path/conf.yml - Change path to VM configuration .yml file." + @echo " See conf_example_*.yml for file format details." vm-build-all: $(addprefix vm-build-, $(IMAGES)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 33004934af..f488d4103c 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -32,6 +32,7 @@ import shutil import multiprocessing import traceback from socket_thread import SocketThread +import yaml SSH_KEY_FILE = os.path.join(os.path.dirname(__file__), "..", "keys", "id_rsa") @@ -500,9 +501,31 @@ class BaseVM(object): cwd=cidir, stdin=self._devnull, stdout=self._stdout, stderr=self._stdout) - return os.path.join(cidir, "cloud-init.iso") +def parse_config(config, args): + """ Parse yaml config and populate our config structure. + The yaml config allows the user to override the + defaults for VM parameters. In many cases these + defaults can be overridden without rebuilding the VM.""" + if args.config: + config_file = args.config + elif 'QEMU_CONFIG' in os.environ: + config_file = os.environ['QEMU_CONFIG'] + else: + return config + if not os.path.exists(config_file): + raise Exception("config file {} does not exist".format(config_file)) + with open(config_file) as f: + yaml_dict = yaml.safe_load(f) + + if 'qemu-conf' in yaml_dict: + config.update(yaml_dict['qemu-conf']) + else: + raise Exception("config file {} is not valid"\ + " missing qemu-conf".format(config_file)) + return config + def parse_args(vmcls): def get_default_jobs(): @@ -537,6 +560,9 @@ def parse_args(vmcls): help="Interactively run command") parser.add_option("--snapshot", "-s", action="store_true", help="run tests with a snapshot") + parser.add_option("--config", "-c", default=None, + help="Provide config yaml for configuration. "\ + "See config_example.yaml for example.") parser.disable_interspersed_args() return parser.parse_args() @@ -548,6 +574,7 @@ def main(vmcls, config=None): if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 + config = parse_config(config, args) logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) vm = vmcls(debug=args.debug, vcpus=args.jobs, config=config) diff --git a/tests/vm/conf_example_aarch64.yml b/tests/vm/conf_example_aarch64.yml new file mode 100644 index 0000000000..9d44ae356f --- /dev/null +++ b/tests/vm/conf_example_aarch64.yml @@ -0,0 +1,51 @@ +# +# Example yaml for use by any of the scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: qemu + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # If one key is provided, both must be provided. + #ssh_key: /complete/path/of/your/keyfile/id_rsa + #ssh_pub_key: /complete/path/of/your/keyfile/id_rsa.pub + + cpu: max + machine: virt,gic-version=max + memory: 16G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=16,sockets=2,cores=8 + -numa node,cpus=0-3,nodeid=0 -numa node,cpus=4-7,nodeid=1 + -numa node,cpus=8-11,nodeid=2 -numa node,cpus=12-15,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: 1.234.567.89 + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + #ssh_port: 5555 + + # To install a different set of packages, provide a command to issue + #install_cmds: "apt-get update ; apt-get build-dep -y qemu" + + # Or to skip the install entirely, just provide "" + #install_cmds: "" diff --git a/tests/vm/conf_example_x86.yml b/tests/vm/conf_example_x86.yml new file mode 100644 index 0000000000..78d3f5830f --- /dev/null +++ b/tests/vm/conf_example_x86.yml @@ -0,0 +1,50 @@ +# +# Example yaml for use by any of the x86 based scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: "qemu" + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # Provide default ssh keys of current user. + # You need to edit the below for your user. + #ssh_key_file: /home//.ssh/id_rsa + #ssh_pub_key_file: /home//.ssh/id_rsa.pub + + cpu: max + machine: pc + memory: 8G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=8,sockets=2,cores=4 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node0 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node1 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node2 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node3 + -numa node,cpus=0-1,nodeid=0 -numa node,cpus=2-3,nodeid=1 + -numa node,cpus=4-5,nodeid=2 -numa node,cpus=6-7,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: "1.234.567.89" + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + ssh_port: 5555 From patchwork Wed Feb 5 21:29:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367231 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 752B9112B for ; Wed, 5 Feb 2020 21:38:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B8282072B for ; Wed, 5 Feb 2020 21:38:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="wwOr4EJ3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B8282072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57282 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSNL-0005eJ-Fx for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:38:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47454) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJB-0005ff-A1 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJ9-0006TF-Rp for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:05 -0500 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:45580) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJ9-0006RW-L8 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:03 -0500 Received: by mail-pf1-x432.google.com with SMTP id 2so1860290pfg.12 for ; Wed, 05 Feb 2020 13:34:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Q+lXVoFe6aYIoxKIqqudPU6MusUvHE0l2Wqhqs9iBy0=; b=wwOr4EJ30YU5NlmkSpXCuasNmB5P4zuqaL3w9ypCLTaUkVEjJDHkfHRud6KmaBzdsX uNhRXj/KSnY4LejTiKpA/GZexPNrlK+ULu6S0ISeMywIfqhqjIoKuXSebd5iHXUc4Boq MDmsQqfBcb2baGxkN9K3L/DuJndI8jYhVsOg0Eo5g5HduOLJV8VTvT+UY9B4ikd9Jz42 ok+0TANO0LfxGq7KRhi1VAeSIjKWRy279Lvy6lt2yvYcUcWzVtdc/OCKBR26FZGrtqT1 BCrV2ehaU1vYzn0PSwoGIw2W7KFgsFp/rOzImXyN+jcwf4JL8kvMqBtoYfr4vObRtwv6 oqDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Q+lXVoFe6aYIoxKIqqudPU6MusUvHE0l2Wqhqs9iBy0=; b=bPjUhIbG1B6vStg7vSk8Dj5coo/lmIjzxC+3Oou+1KSRa5YBXd0XvV5oeP+x9WeTtX iZNy2YlFpBxIULLsIbrrXohkrEAFUGIUpI+fw0hBqeHOGGFdmdXdLzyS64xBhW5vHDfF SOFtPWcflUIIo6E/Xpfe0AwG2LxfnsuTux6VbV/7OEGDUNt1MDC9J2NMxm31Ywlj3hli AqOuOSw/fIzloZE+Ht9z3llFVl6WdDfkLWihKcxGsSnIdRfV2+bHcso0cK7SaQaOj24C bwdxOiJSCRvhNLX2idGmUHrcj5rlmz9ueUviUwsy1KE6+iG+bGlVkyECiOrf3P9mFUAJ R2pw== X-Gm-Message-State: APjAAAWxmUmYd3D7TMeXmrbcFbQf9P775Awnw7zAq42hEkGWK0kP9c+S SXxM/xCNU8u0m8j2YsWhBWnFzhkH8A8= X-Google-Smtp-Source: APXvYqzObYbyYWFzXBHJ5VqVFpwyHFTEu6ZpN/Hd08+hgLQF62OXvYE5eIyWUQ1OPnCDX5SHoXSMmg== X-Received: by 2002:a62:2cd8:: with SMTP id s207mr38531950pfs.247.1580938442172; Wed, 05 Feb 2020 13:34:02 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:01 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 09/14] tests/vm: add --boot-console switch Date: Wed, 5 Feb 2020 16:29:15 -0500 Message-Id: <20200205212920.467-10-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::432 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Added ability to view console during boot via --boot-console switch to basevm.py. This helps debug issues that occur during the boot sequence. Also added a new special variable to vm-build: BOOT_CONSOLE=1 will cause this new --boot-console switch to be set. Signed-off-by: Robert Foley --- tests/vm/Makefile.include | 4 ++++ tests/vm/basevm.py | 11 +++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index e9ed33226d..d72babd5bf 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -32,6 +32,7 @@ vm-help vm-test: @echo ' EXTRA_CONFIGURE_OPTS="..."' @echo " J=[0..9]* - Override the -jN parameter for make commands" @echo " DEBUG=1 - Enable verbose output on host and interactive debugging" + @echo " BOOT_CONSOLE=1 - Show the console output at boot time. " @echo " V=1 - Enable verbose ouput on host and guest commands" @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=/path/to/qemu-img - Change path to qemu-img tool" @@ -50,6 +51,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(call quiet-command, \ $(PYTHON) $< \ $(if $(V)$(DEBUG), --debug) \ + $(if $(BOOT_CONSOLE),--boot-console) \ --image "$@" \ --force \ --build-image $@, \ @@ -64,6 +66,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(DEBUG), --interactive) \ $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ + $(if $(BOOT_CONSOLE),--boot-console) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -84,6 +87,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(call quiet-command, \ $(PYTHON) $(SRC_PATH)/tests/vm/$* \ $(if $(J),--jobs $(J)) \ + $(if $(BOOT_CONSOLE),--boot-console) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index f488d4103c..b99ab0f20a 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -96,9 +96,11 @@ class BaseVM(object): # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 console_logfile = "console.log" - def __init__(self, debug=False, vcpus=None, config=None): + def __init__(self, debug=False, vcpus=None, config=None, + boot_console=None): self._guest = None self._console_fd = None + self._boot_console = boot_console self._socket_thread = None self._console_timeout_sec = self.socket_timeout # Allow input config to override defaults. @@ -563,6 +565,8 @@ def parse_args(vmcls): parser.add_option("--config", "-c", default=None, help="Provide config yaml for configuration. "\ "See config_example.yaml for example.") + parser.add_option("--boot-console", action="store_true", + help="Show console during boot. ") parser.disable_interspersed_args() return parser.parse_args() @@ -577,7 +581,8 @@ def main(vmcls, config=None): config = parse_config(config, args) logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(debug=args.debug, vcpus=args.jobs, config=config) + vm = vmcls(debug=args.debug, vcpus=args.jobs, config=config, + boot_console=args.boot_console) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, @@ -597,6 +602,8 @@ def main(vmcls, config=None): if args.snapshot: img += ",snapshot=on" vm.boot(img) + if vm._boot_console: + vm.wait_boot() vm.wait_ssh() except Exception as e: if isinstance(e, SystemExit) and e.code == 0: From patchwork Wed Feb 5 21:29:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47A34112B for ; Wed, 5 Feb 2020 21:40:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C4EF20720 for ; Wed, 5 Feb 2020 21:40:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="JuzTDzCQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C4EF20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57344 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSPV-000087-9y for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:40:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47475) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJD-0005k4-2z for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJB-0006Xi-Kg for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:06 -0500 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:42159) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJB-0006Vp-DR for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:05 -0500 Received: by mail-pg1-x535.google.com with SMTP id w21so1590348pgl.9 for ; Wed, 05 Feb 2020 13:34:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=F7wIuakRPzV81cpEyf3HEY4sa2Mhg8c8varfC5JwemE=; b=JuzTDzCQDxlGTiWhwyaJRq5AOwxfKcoppt/lPWxDN7uIKOOIv8II2vJ7HCeGlH3B9g IhgmzFjcoilg/Fkg0nXuJBgNL0Xv45DDX/2v4HRaXKTP/7HUoBLt5PMYeO0J8MseR+OE PHkxF1XAjdeIZ0Obwxbndz0a3eEhMXJh+FGzYr5QhnUkNg8srgz5KeZDdfOx0r4cWJwJ usxQ1HJ10w9+WruxfapT3Cvhv9XVA3G5SL2gi9EyVmiZ9ag1aEaTgg7p2WbFSg+xSXGv bMXrn4fO9kUywpQcUdBVvrCWjiyl5/KaSpnPI57RpzYcxfvY0me1HUZn3X1e+eeqaoOz YZEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=F7wIuakRPzV81cpEyf3HEY4sa2Mhg8c8varfC5JwemE=; b=gUZNpr53f+AkctYX7EqBlK7AoFSgCAFhNVMNpFjnybWY9ycQdbIvSL8top8xvHzCjS 6kgjDRmbXNSXVIlsBwUfrwtxprDyfGFTU57FlxXfawSU8w8dSF/WtaZyPIQQ/cR9tN1R NO1YWJASQoLWLQWggm0ZJC54IDRo41yqwDyL1kyuZYqTT4ZzTVagRwoHiwQdJ/kPy//V 1qyVNEJZRpvrvFeKpRUS94Xy3hp63i+rGMAYtINv4xuJIyTxJrpJodVbeLUgGCnyyOZC 6pffzUxZAeOHGOjTwSUzMIybHSh5cmsqXTst53iC78xkZQCm9HxYfWlXm9P1/ILimJ0M fYXw== X-Gm-Message-State: APjAAAUh4NS8ygmXnvF/kTvcgBz4jByYBYKhYIwBw++S6OH1KJ9YtC/I uyYD0doMsDAIA0pV45W+lAyI6MDjggo= X-Google-Smtp-Source: APXvYqyxGhHLRbvnQUOtDldmCd0tRqAPNUfW5Lhb5KHiPIkR5uQ6qU0J6Hkbd4EFobjKC2gNUZRDdA== X-Received: by 2002:a63:3853:: with SMTP id h19mr35087277pgn.199.1580938444082; Wed, 05 Feb 2020 13:34:04 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:03 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 10/14] tests/vm: Add ability to select QEMU from current build. Date: Wed, 5 Feb 2020 16:29:16 -0500 Message-Id: <20200205212920.467-11-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::535 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Added a new special variable QEMU_LOCAL=1, which will indicate to take the QEMU binary from the current build. Signed-off-by: Robert Foley --- tests/vm/Makefile.include | 4 ++++ tests/vm/basevm.py | 28 ++++++++++++++++++++++++---- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index d72babd5bf..f67438c552 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -34,6 +34,7 @@ vm-help vm-test: @echo " DEBUG=1 - Enable verbose output on host and interactive debugging" @echo " BOOT_CONSOLE=1 - Show the console output at boot time. " @echo " V=1 - Enable verbose ouput on host and guest commands" + @echo " QEMU_LOCAL=1 - Use QEMU binary local to this build." @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=/path/to/qemu-img - Change path to qemu-img tool" @echo " QEMU_CONFIG=/path/conf.yml - Change path to VM configuration .yml file." @@ -52,6 +53,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(PYTHON) $< \ $(if $(V)$(DEBUG), --debug) \ $(if $(BOOT_CONSOLE),--boot-console) \ + --build-path $(BUILD_DIR)\ --image "$@" \ --force \ --build-image $@, \ @@ -67,6 +69,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ $(if $(BOOT_CONSOLE),--boot-console) \ + --build-path $(BUILD_DIR)\ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -88,6 +91,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(PYTHON) $(SRC_PATH)/tests/vm/$* \ $(if $(J),--jobs $(J)) \ $(if $(BOOT_CONSOLE),--boot-console) \ + --build-path $(BUILD_DIR)\ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index b99ab0f20a..97d55f8030 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -97,9 +97,10 @@ class BaseVM(object): tcg_ssh_timeout_multiplier = 4 console_logfile = "console.log" def __init__(self, debug=False, vcpus=None, config=None, - boot_console=None): + boot_console=None, build_path=None): self._guest = None self._console_fd = None + self._build_path = build_path self._boot_console = boot_console self._socket_thread = None self._console_timeout_sec = self.socket_timeout @@ -287,8 +288,8 @@ class BaseVM(object): args += self._data_args + extra_args + self._config['extra_args'] args += ["-device", "VGA"] logging.debug("QEMU args: %s", " ".join(args)) - qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch) - guest = QEMUMachine(binary=qemu_bin, args=args) + qemu_path = get_qemu_path(self.arch, self._build_path) + guest = QEMUMachine(binary=qemu_path, args=args) guest.set_machine(self._config['machine']) guest.set_console() try: @@ -505,6 +506,22 @@ class BaseVM(object): stderr=self._stdout) return os.path.join(cidir, "cloud-init.iso") +def get_qemu_path(arch, build_path=None): + """Fetch the path to the qemu binary.""" + qemu_local = os.environ.get("QEMU_LOCAL", 0) + # If QEMU environment variable set, it takes precedence + if "QEMU" in os.environ: + qemu_path = os.environ["QEMU"] + elif qemu_local: + if not build_path: + raise Exception("--build-path option required with QEMU_LOCAL") + qemu_path = os.path.join(build_path, arch + "-softmmu") + qemu_path = os.path.join(qemu_path, "qemu-system-" + arch) + else: + # Default is to use system path for qemu. + qemu_path = "qemu-system-" + arch + return qemu_path + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the @@ -567,6 +584,8 @@ def parse_args(vmcls): "See config_example.yaml for example.") parser.add_option("--boot-console", action="store_true", help="Show console during boot. ") + parser.add_option("--build-path", default=None, + help="Path of build directory. ") parser.disable_interspersed_args() return parser.parse_args() @@ -582,7 +601,8 @@ def main(vmcls, config=None): logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) vm = vmcls(debug=args.debug, vcpus=args.jobs, config=config, - boot_console=args.boot_console) + boot_console=args.boot_console, + build_path=args.build_path) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Wed Feb 5 21:29:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367229 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F0B5112B for ; Wed, 5 Feb 2020 21:38:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D9E7E20720 for ; Wed, 5 Feb 2020 21:37:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="TGgdMdSV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9E7E20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSMx-00050Z-2o for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:37:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47498) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJE-0005nr-KI for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJD-0006bn-G8 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:08 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:36878) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJD-0006Zn-9w for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:07 -0500 Received: by mail-pf1-x42d.google.com with SMTP id p14so1883704pfn.4 for ; Wed, 05 Feb 2020 13:34:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=97x70tsGL4/Lrbv+jMvABeOjt9tNQeXwhmwDJh1ToSI=; b=TGgdMdSVRErU0JTnEs8tp/DTpBM2tVd59eEt7YOXUZSGolj+VaAX8RYdszz9rfd8d6 YYebLmQWwG4HEebbdYknLOlBsZ6MR/4GmhZv3pWbZJYlvPudOkNyWuzMyZNBtpGBbjfV S9bvp1OHI8bJEpmldDyzdA9Q627xNsVMaJdP8LkIz2+yudepFsJ9rYr95fp8LtyTmGCO Jyi+msyqUwU2vmNaVfIx+OhHm65DE6c5i+Ck1newOTVtcWEc7gCEFtUvaQSt36riV0Ye mz0LDWHGzIn6olGNJ6Bkr6fzUGq1qxuyS0xvBEJOB1yPecFEox7g9W9YxdLfwgmZgeUt JKEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=97x70tsGL4/Lrbv+jMvABeOjt9tNQeXwhmwDJh1ToSI=; b=Dh7yasIbrhmqkdCyL0ENsrjJIUSx49ZhD279bfMY05/8EHfMaByuksaXOwh/NDN0vL P7Iw0azNwQ+C9PV4+zRwFQPbUkooKusWfOm72GBQTwWmRCL4aC2FBuO+LHpOCrv0GTWJ wVwb9L46l4K5Jw0Ur6CXk7tvq8vjWgMBR88OEKgY76vZ/nwKGEoW4ahNrbBWCz+66JZi gHuM7pCrwKpzvTVBwpAdTHoi32Qx9+SXCxb+xEzwpJfm63sHSV6QuKEOOfXFXejkrs+d /sA8wDTQB3DH7gM9nNQ2KffH1AUFY9+eLznDg2R2LpmDt+tIa4+VDxHgmIANfGfBxJsp 3X8w== X-Gm-Message-State: APjAAAUu10ts1NCZ+JPJcMh20BlrRWwsDPmUkjj+Cv98olvPRy53ve27 Xf2shi921IgiejTiH+EWOD5fAy/Es5g= X-Google-Smtp-Source: APXvYqwueasSDkdHtJlEZwg7UkNgypODmlcxi7Daz/GtXw40/0q13UMtQto41R+ujbRFDePoe4MG2w== X-Received: by 2002:a62:ac03:: with SMTP id v3mr38338256pfe.17.1580938446017; Wed, 05 Feb 2020 13:34:06 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:05 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 11/14] tests/vm: allow wait_ssh() to specify command Date: Wed, 5 Feb 2020 16:29:17 -0500 Message-Id: <20200205212920.467-12-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42d X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This allows for waiting for completion of arbitrary commands. Signed-off-by: Robert Foley --- tests/vm/basevm.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 97d55f8030..ebedbce4ae 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -433,24 +433,24 @@ class BaseVM(object): def print_step(self, text): sys.stderr.write("### %s ...\n" % text) - def wait_ssh(self, wait_root=False, seconds=300): + def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"): # Allow more time for VM to boot under TCG. if not kvm_available(self.arch): seconds *= self.tcg_ssh_timeout_multiplier starttime = datetime.datetime.now() endtime = starttime + datetime.timedelta(seconds=seconds) - guest_up = False + cmd_success = False while datetime.datetime.now() < endtime: - if wait_root and self.ssh_root("exit 0") == 0: - guest_up = True + if wait_root and self.ssh_root(cmd) == 0: + cmd_success = True break - elif self.ssh("exit 0") == 0: - guest_up = True + elif self.ssh(cmd) == 0: + cmd_success = True break seconds = (endtime - datetime.datetime.now()).total_seconds() logging.debug("%ds before timeout", seconds) time.sleep(1) - if not guest_up: + if not cmd_success: raise Exception("Timeout while waiting for guest ssh") def shutdown(self): From patchwork Wed Feb 5 21:29:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 67424924 for ; Wed, 5 Feb 2020 21:41:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D2122072B for ; Wed, 5 Feb 2020 21:41:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="s4CbNe83" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D2122072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57378 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSQS-0001Ge-Bx for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:41:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47545) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJI-0005xY-8N for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJG-0006iv-6d for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:12 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:35209) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJF-0006gz-Ta for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:10 -0500 Received: by mail-pj1-x102a.google.com with SMTP id q39so1559881pjc.0 for ; Wed, 05 Feb 2020 13:34:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LVZrl3soVZQb/vpFWgqHxtFbe1UxGUQSeeKoTjHKLoM=; b=s4CbNe838f5B3oemW8werT/h8SUMn7MemNp9t9aLRHXf03XXsY1AsLWJPPFgnu5avK l1f64ZFeW1+7LHHE51pGqHvpp0tmmkpqj36oKMniphinp+BuyyWGLviidZbxGMOYb1ML Ykf/DXPNcNzrtidpYad+Vn4ymJZpO+vTn910F6GSPBbm3NwnJ/17g1JwE61ivO7jPQBz tPQXLR2fT7WLtraWcCx5poUlMMoKGRkSDRJ56xL2lheQuNJDUhxXuAIxvRVf30PUG1m0 3nb/MBjMgC48RRD1MkXUeWGBjvBT99264zyT0oB0WuSbFKrhi3U5u45ICMiaxJUNIxvd 0ScQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LVZrl3soVZQb/vpFWgqHxtFbe1UxGUQSeeKoTjHKLoM=; b=ZBMBKXl3k+RKdPertGHpZXY5lRwPznpFPPpc6s6iyrPuVMh9HOLwXqELsEJTJJE47k EK4SSFuKk72mpoYsvp7XEOUsYa3jzH/fupPqeWXstBB4i6GVAEQF5wplJSvYXtcaFBw8 B8ohOd/+dIQXcYt7Icfoq7oZ9lmHwPjFeqQJtCuvDXH86WlmbcleqKfaE2PgLERQAe7e ikBFGnv/CDGxARjr1W9xFQR4sEM+sHkvtlqgVmMznt+X9Fa8lpTiEaXHAarrakG5SnDh QBsmUY6ZdZ7AK2Si/13ZyQ/gIVvI3kjERV4JucRfOz5IctXIOT1I5jTViHI7KBfFW7Gy 7V3A== X-Gm-Message-State: APjAAAU7JRzNLcaMSwjJLaSIvyotzST9aXqHPOHFw5gClzosv1e+rzYU GM7e4RnZ4IIEmLyXibP8n7yHnH1Oqx0= X-Google-Smtp-Source: APXvYqzDxBov80fZENwHO5Q3I7C0iZBfJ8hv+BCjbdgjvr08i8Vz2Y+cMpg4nU2F6xFy3arZUGiBlA== X-Received: by 2002:a17:90a:cf08:: with SMTP id h8mr194080pju.81.1580938448181; Wed, 05 Feb 2020 13:34:08 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:07 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 12/14] tests/vm: Added a new script for ubuntu.aarch64. Date: Wed, 5 Feb 2020 16:29:18 -0500 Message-Id: <20200205212920.467-13-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::102a X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley --- tests/vm/Makefile.include | 3 +- tests/vm/aarch64vm.py | 100 +++++++++++++++++++++++++++++++++ tests/vm/basevm.py | 8 +++ tests/vm/ubuntu.aarch64 | 113 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 223 insertions(+), 1 deletion(-) create mode 100644 tests/vm/aarch64vm.py create mode 100755 tests/vm/ubuntu.aarch64 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index f67438c552..ccafe966cd 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -2,7 +2,7 @@ .PHONY: vm-build-all vm-clean-all -IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora +IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora ubuntu.aarch64 IMAGES_DIR := $(HOME)/.cache/qemu-vm/images IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES)) @@ -18,6 +18,7 @@ vm-help vm-test: @echo " vm-build-openbsd - Build QEMU in OpenBSD VM" @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker" @echo " vm-build-fedora - Build QEMU in Fedora VM" + @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" @echo "" @echo " vm-build-all - Build QEMU in all VMs" @echo " vm-clean-all - Clean up VM images" diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py new file mode 100644 index 0000000000..a4c9fea925 --- /dev/null +++ b/tests/vm/aarch64vm.py @@ -0,0 +1,100 @@ +#!/usr/bin/env python +# +# VM testing aarch64 library +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available + +# This is the config needed for current version of QEMU. +# This works for both kvm and tcg. +CURRENT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", +} + +# The minimum minor version of QEMU we will support with aarch64 VMs is 3. +# QEMU versions less than 3 have various issues running these VMs. +QEMU_AARCH64_MIN_VERSION = 3 + +# The DEFAULT_CONFIG will default to a version of +# parameters that works for backwards compatibility. +DEFAULT_CONFIG = {'kvm' : {'cpu' : "host", + 'machine' : "virt,gic-version=host"}, + 'tcg' : {'cpu' : "cortex-a57", + 'machine' : "virt"}, +} + +def get_config_defaults(vmcls, default_config): + """Fetch the configuration defaults for this VM, + taking into consideration the defaults for + aarch64 first, followed by the defaults for this VM.""" + config = default_config + config.update(aarch_get_config_defaults(vmcls)) + return config + +def aarch_get_config_defaults(vmcls): + # Set the defaults for current version of QEMU. + config = CURRENT_CONFIG + args, argv = basevm.parse_args(vmcls) + qemu_path = basevm.get_qemu_path(vmcls.arch, args.build_path) + qemu_version = basevm.get_qemu_version(qemu_path) + if qemu_version < QEMU_AARCH64_MIN_VERSION: + error = "\nThis major version of QEMU {} is to old for aarch64 VMs.\n"\ + "The major version must be at least {}.\n"\ + "To continue with the current build of QEMU, "\ + "please restart with QEMU_LOCAL=1 .\n" + print(error.format(qemu_version, QEMU_AARCH64_MIN_VERSION)) + exit(1) + if qemu_version == QEMU_AARCH64_MIN_VERSION: + # We have an older version of QEMU, + # set the config values for backwards compatibility. + if kvm_available('aarch64'): + config.update(DEFAULT_CONFIG['kvm']) + else: + config.update(DEFAULT_CONFIG['tcg']) + return config + +def create_flash_images(flash_dir="./"): + """Creates the appropriate pflash files + for an aarch64 VM.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + subprocess.check_call(["dd", "if=/dev/zero", "of={}".format(flash0_path), + "bs=1M", "count=64"]) + # A reliable way to get the QEMU EFI image is via an installed package. + efi_img = "/usr/share/qemu-efi-aarch64/QEMU_EFI.fd" + if not os.path.exists(efi_img): + sys.stderr.write("*** {} is missing\n".format(efi_img)) + sys.stderr.write("*** please install qemu-efi-aarch64 package\n") + exit(3) + subprocess.check_call(["dd", "if={}".format(efi_img), + "of={}".format(flash0_path), + "conv=notrunc"]) + subprocess.check_call(["dd", "if=/dev/zero", + "of={}".format(flash1_path), + "bs=1M", "count=64"]) + +def get_pflash_args(flash_dir="./"): + """Returns a string that can be used to + boot qemu using the appropriate pflash files + for aarch64.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + pflash_args_str = "-drive file={},format=raw,if=pflash "\ + "-drive file={},format=raw,if=pflash" + pflash_args = pflash_args_str.format(flash0_path, flash1_path) + return pflash_args.split(" ") + +def get_flash_path(flash_dir, name): + return os.path.join(flash_dir, "{}.img".format(name)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index ebedbce4ae..dc975d92c7 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -522,6 +522,14 @@ def get_qemu_path(arch, build_path=None): qemu_path = "qemu-system-" + arch return qemu_path +def get_qemu_version(qemu_path): + """Get the version number from the current QEMU, + and return the major number.""" + output = subprocess.check_output([qemu_path, '--version']) + version_line = output.decode("latin1") + version_num = re.split(' |\(', version_line)[3].split('.')[0] + return int(version_num) + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 new file mode 100755 index 0000000000..7f204d3879 --- /dev/null +++ b/tests/vm/ubuntu.aarch64 @@ -0,0 +1,113 @@ +#!/usr/bin/env python +# +# Ubuntu aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available +import time +import aarch64vm + +DEFAULT_CONFIG = { + 'cpu' : "cortex-a57", + 'machine' : "virt,gic-version=3", + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y qemu,"\ + "apt-get install -y libfdt-dev flex bison", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM + # especially using softmmu. + 'ssh_timeout' : 60, +} + +class UbuntuAarch64VM(basevm.BaseVM): + name = "ubuntu.aarch64" + arch = "aarch64" + image_name = "ubuntu-18.04-server-cloudimg-arm64.img" + image_link = "https://cloud-images.ubuntu.com/releases/18.04/release/" + image_name + login_prompt = "ubuntu-aarch64-guest login:" + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(UbuntuAarch64VM, self).boot(img, extra_args=extra_args) + + def build_image(self, img): + os_img = self._download_with_cache(self.image_link) + img_tmp = img + ".tmp" + subprocess.check_call(["cp", "-f", os_img, img_tmp]) + subprocess.check_call(["qemu-img", "resize", img_tmp, "+50G"]) + ci_img = self.gen_cloud_init_iso() + + self.boot(img_tmp, extra_args = ["-cdrom", ci_img]) + if self._boot_console: + self.wait_boot() + # First command we issue is fix for slow ssh login. + self.wait_ssh(wait_root=True, + cmd="chmod -x /etc/update-motd.d/*") + # Wait for cloud init to finish + self.wait_ssh(wait_root=True, + cmd="ls /var/lib/cloud/instance/boot-finished") + self.ssh_root("touch /etc/cloud/cloud-init.disabled") + # Disable auto upgrades. + # We want to keep the VM system state stable. + self.ssh_root('sed -ie \'s/"1"/"0"/g\' /etc/apt/apt.conf.d/20auto-upgrades') + # If the user chooses *not* to do the second phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + # Don't check the status in case the guest hang up too quickly + self.ssh_root("sync && reboot") + if self._boot_console: + self.wait_boot() + self.wait_ssh(wait_root=True) + self.ssh_root("locale") + # The previous update sometimes doesn't survive a reboot, so do it again + self.ssh_root("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list") + + # Issue the install commands. + # This can be overriden by the user in the config .yml. + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.graceful_shutdown() + self.wait() + os.rename(img_tmp, img) + return 0 + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(UbuntuAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(UbuntuAarch64VM, defaults)) From patchwork Wed Feb 5 21:29:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 102E8924 for ; Wed, 5 Feb 2020 21:42:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA8BC2072B for ; Wed, 5 Feb 2020 21:42:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="B8vL4use" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA8BC2072B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57390 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSRM-0002Ce-Vm for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:42:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47576) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJK-00063M-IL for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJI-0006mq-7n for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:14 -0500 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:38992) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJH-0006kC-UN for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:12 -0500 Received: by mail-pj1-x102e.google.com with SMTP id e9so1552937pjr.4 for ; Wed, 05 Feb 2020 13:34:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=US/OS0QDb0JN6Dy5QXERY45IKwMWxviMx4JGwpBzqS8=; b=B8vL4usewfUmRSokI4x6uXEr/FXK0vosUj9CYjzGu32urJGyvGEWBGN3VS1Rj7nz07 P8NCoVNJlUJy+7vlol1gE1mgs4yvCWS35UV2NHf3HB72kuxBcKE//Z+VRIGpBcu9KsLy cZfZq4tZU1pqrk1Cet5/GsutSZjhgFRW9ZS5Uh37Iuel7YNfLqUY3D+vKVmpk2EHDAud zx81GKz8r08tilgxftFQVlqM5PXAs2kfTEqvfGwJUfxjLI6YMESvHmztkEak2t/XQP9p fUe/NNauDvA8OcAvNJ+d9FWkXsgogaVxtDkAhznXfwO9h4aci0DDAgYyZhVC/P0LrPx1 GugQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=US/OS0QDb0JN6Dy5QXERY45IKwMWxviMx4JGwpBzqS8=; b=A8yuUhfqfvhvzhcsqECUgx2b0yu1WyH49m/xCqFntbkjH/tgifgm5g+dpBr/4/+Rly eyDCWFVfDDkzaPdxzHA5bX9BcszmAsW/8lnFTPpg9Q9oLcZDwnOc4kw08Pi+liAnHry9 P1QHq6ibASuf/GqmzdUuxd0G5rSCUVj88AyM7mEt1r/KWsCvb1DazLgurtgvmZLBuLoS ZlODhkQza8UiggdKH9G5EyWFlhCMDLJJsYeG3unOBzl+ow5hHIdhzMsGwbJC3u7bBKMj 2evGomxJdCfUb4/h5d71ARVduzyNfwG01jSx+gbestfXMUulgGpgeZNdVrj3ujT7tYHV 1TXA== X-Gm-Message-State: APjAAAUeaRKEPxCJn289aKo38KNSMN33yCCUMizJAFzGiBBaw+qSlanu 9yKMzlRGqyTa5NgV9d4Zu99nkhgX6ZM= X-Google-Smtp-Source: APXvYqyhktkGG8BPPVQ16K06hJiaZhPp+4uWW92NRl5wL2qVbk7Xrlc5og6Khr9Q1PJ7TyaTJN8Rmg== X-Received: by 2002:a17:90a:1785:: with SMTP id q5mr182868pja.143.1580938450145; Wed, 05 Feb 2020 13:34:10 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:09 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 13/14] tests/vm: Added a new script for centos.aarch64. Date: Wed, 5 Feb 2020 16:29:19 -0500 Message-Id: <20200205212920.467-14-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::102e X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley --- tests/vm/Makefile.include | 3 +- tests/vm/centos-8-aarch64.ks | 51 ++++++++ tests/vm/centos.aarch64 | 221 +++++++++++++++++++++++++++++++++++ 3 files changed, 274 insertions(+), 1 deletion(-) create mode 100644 tests/vm/centos-8-aarch64.ks create mode 100755 tests/vm/centos.aarch64 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index ccafe966cd..7b65958f4d 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -2,7 +2,7 @@ .PHONY: vm-build-all vm-clean-all -IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora ubuntu.aarch64 +IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora ubuntu.aarch64 centos.aarch64 IMAGES_DIR := $(HOME)/.cache/qemu-vm/images IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES)) @@ -19,6 +19,7 @@ vm-help vm-test: @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker" @echo " vm-build-fedora - Build QEMU in Fedora VM" @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" + @echo " vm-build-centos.aarch64 - Build QEMU in CentOS aarch64 VM" @echo "" @echo " vm-build-all - Build QEMU in all VMs" @echo " vm-clean-all - Clean up VM images" diff --git a/tests/vm/centos-8-aarch64.ks b/tests/vm/centos-8-aarch64.ks new file mode 100644 index 0000000000..fd6ebe4d49 --- /dev/null +++ b/tests/vm/centos-8-aarch64.ks @@ -0,0 +1,51 @@ +# CentOS aarch64 image kickstart file. +# This file is used by the CentOS installer to +# script the generation of the image. +# +# Copyright 2020 Linaro +# +ignoredisk --only-use=vda +# System bootloader configuration +bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda +autopart --type=plain +# Partition clearing information +clearpart --linux --initlabel --drives=vda +# Use text mode install +text +repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream +# Use CDROM installation media +cdrom +# Keyboard layouts +keyboard --vckeymap=us --xlayouts='' +# System language +lang en_US.UTF-8 + +# Network information +network --bootproto=dhcp --device=enp0s1 --onboot=off --ipv6=auto --no-activate +network --hostname=localhost.localdomain +# Run the Setup Agent on first boot +firstboot --enable +# Do not configure the X Window System +skipx +# System services +services --enabled="chronyd" +# System timezone +timezone America/New_York --isUtc + +# Shutdown after installation is complete. +shutdown + +%packages +@^server-product-environment +kexec-tools + +%end + +%addon com_redhat_kdump --enable --reserve-mb='auto' + +%end +%anaconda +pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty +pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok +pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty +%end diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 new file mode 100755 index 0000000000..73bb9c4f86 --- /dev/null +++ b/tests/vm/centos.aarch64 @@ -0,0 +1,221 @@ +#!/usr/bin/env python +# +# Centos aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.aarch64 +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import os +import sys +import subprocess +import basevm +import time +import traceback +import aarch64vm + +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", + 'install_cmds' : "yum install -y docker make git python3 gcc, "\ + "yum install -y glib2-devel pixman-devel zlib-devel, "\ + "yum install -y perl-Test-Harness, "\ + "systemctl enable docker", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM. + 'ssh_timeout' : 60, +} + +class CentosAarch64VM(basevm.BaseVM): + name = "centos.aarch64" + arch = "aarch64" + login_prompt = "localhost login:" + prompt = '[root@localhost ~]#' + image_name = "CentOS-8-aarch64-1905-dvd1.iso" + image_link = "http://mirrors.usc.edu/pub/linux/distributions/centos/8.0.1905/isos/aarch64/" + image_link += image_name + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def set_key_perm(self): + """Set permissions properly on certain files to allow + ssh access.""" + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v /root/.ssh\n") + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v "\ + "/home/{}/.ssh\n".format(self._config["guest_user"])) + + def create_kickstart(self): + """Generate the kickstart file used to generate the centos image.""" + # Start with the template for the kickstart. + ks_file = "../tests/vm/centos-8-aarch64.ks" + subprocess.check_call("cp {} ./ks.cfg".format(ks_file), shell=True) + # Append the ssh keys to the kickstart file + # as the post processing phase of installation. + with open("ks.cfg", "a") as f: + # Add in the root pw and guest user. + rootpw = "rootpw --plaintext {}\n" + f.write(rootpw.format(self._config["root_pass"])) + add_user = "user --groups=wheel --name={} "\ + "--password={} --plaintext\n" + f.write(add_user.format(self._config["guest_user"], + self._config["guest_pass"])) + # Add the ssh keys. + f.write("%post --log=/root/ks-post.log\n") + f.write("mkdir -p /root/.ssh\n") + addkey = 'echo "{}" >> /root/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"]) + f.write(addkey_cmd) + f.write('mkdir -p /home/{}/.ssh\n'.format(self._config["guest_user"])) + addkey = 'echo "{}" >> /home/{}/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"], + self._config["guest_user"]) + f.write(addkey_cmd) + f.write("%end\n") + # Take our kickstart file and create an .iso from it. + # The .iso will be provided to qemu as we boot + # from the install dvd. + # Anaconda will recognize the label "OEMDRV" and will + # start the automated installation. + gen_iso_img = 'genisoimage -output ks.iso -volid "OEMDRV" ks.cfg' + subprocess.check_call(gen_iso_img, shell=True) + + def wait_for_shutdown(self): + """We wait for qemu to shutdown the VM and exit. + While this happens we display the console view + for easier debugging.""" + # The image creation is essentially done, + # so whether or not the wait is successful we want to + # wait for qemu to exit (the self.wait()) before we return. + try: + self.console_wait("reboot: Power down") + except Exception as e: + sys.stderr.write("Exception hit\n") + if isinstance(e, SystemExit) and e.code == 0: + return 0 + traceback.print_exc() + finally: + self.wait() + + def build_base_image(self, dest_img): + """Run through the centos installer to create + a base image with name dest_img.""" + # We create the temp image, and only rename + # to destination when we are done. + img = dest_img + ".tmp" + # Create an empty image. + # We will provide this as the install destination. + qemu_img_create = "qemu-img create {} 50G".format(img) + subprocess.check_call(qemu_img_create, shell=True) + + # Create our kickstart file to be fed to the installer. + self.create_kickstart() + # Boot the install dvd with the params as our ks.iso + os_img = self._download_with_cache(self.image_link) + dvd_iso = "centos-8-dvd.iso" + subprocess.check_call(["cp", "-f", os_img, dvd_iso]) + extra_args = "-cdrom ks.iso" + extra_args += " -drive file={},if=none,id=drive1,cache=writeback" + extra_args += " -device virtio-blk,drive=drive1,bootindex=1" + extra_args = extra_args.format(dvd_iso).split(" ") + self.boot(img, extra_args=extra_args) + self.console_wait_send("change the selection", "\n") + # We seem to need to hit esc (chr(27)) twice to abort the + # media check, which takes a long time. + # Waiting a bit seems to be more reliable before hitting esc. + self.console_wait("Checking") + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + print("Found Checking") + self.wait_for_shutdown() + os.rename(img, dest_img) + print("Done with base image build: {}".format(dest_img)) + + def check_create_base_img(self, img_base, img_dest): + """Create a base image using the installer. + We will use the base image if it exists. + This helps cut down on install time in case we + need to restart image creation, + since the base image creation can take a long time.""" + if not os.path.exists(img_base): + print("Generate new base image: {}".format(img_base)) + self.build_base_image(img_base); + else: + print("Use existing base image: {}".format(img_base)) + # Save a copy of the base image and copy it to dest. + # which we will use going forward. + subprocess.check_call(["cp", img_base, img_dest]) + + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(CentosAarch64VM, self).boot(img, extra_args=extra_args) + + def build_image(self, img): + img_tmp = img + ".tmp" + self.check_create_base_img(img + ".base", img_tmp) + + # Boot the new image for the first time to finish installation. + self.boot(img_tmp) + self.console_init() + self.console_wait_send(self.login_prompt, "root\n") + self.console_wait_send("Password:", + "{}\n".format(self._config["root_pass"])) + + self.set_key_perm() + self.console_wait_send(self.prompt, "rpm -q centos-release\n") + enable_adapter = "sed -i 's/ONBOOT=no/ONBOOT=yes/g'" \ + " /etc/sysconfig/network-scripts/ifcfg-enp0s1\n" + self.console_wait_send(self.prompt, enable_adapter) + self.console_wait_send(self.prompt, "ifup enp0s1\n") + self.console_wait_send(self.prompt, + 'echo "qemu ALL=(ALL) NOPASSWD:ALL" | '\ + 'sudo tee /etc/sudoers.d/qemu\n') + self.console_wait(self.prompt) + + # Rest of the commands we issue through ssh. + self.wait_ssh(wait_root=True) + + # If the user chooses *not* to do the second phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.ssh_root("poweroff") + self.wait_for_shutdown() + os.rename(img_tmp, img) + print("image creation complete: {}".format(img)) + return 0 + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(CentosAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(CentosAarch64VM, defaults)) From patchwork Wed Feb 5 21:29:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 11367235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C093924 for ; Wed, 5 Feb 2020 21:39:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E67FA20720 for ; Wed, 5 Feb 2020 21:39:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="tULfasvG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E67FA20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57314 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSOc-0007Ur-6l for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Feb 2020 16:39:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47594) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1izSJL-00066g-R6 for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1izSJJ-0006p8-Qj for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:15 -0500 Received: from mail-pf1-x441.google.com ([2607:f8b0:4864:20::441]:40772) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1izSJJ-0006oi-Im for qemu-devel@nongnu.org; Wed, 05 Feb 2020 16:34:13 -0500 Received: by mail-pf1-x441.google.com with SMTP id q8so1872518pfh.7 for ; Wed, 05 Feb 2020 13:34:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QXi83h34VPpMPcld87083LVrMKk6MFlcpiSR8RVBVg4=; b=tULfasvGDoFi0UqgOQyCCw58OZ7M9+quY2dpt7bcoLltxQD/XiANcTaSmqO0LTvcbZ v+Pr2o5zzhXuYlJJJ0XtuJK+ZNaHQFaRcGsEM+kGVMcAbdd+rWI4Eks3PMJfQeLhwUhx o3hIf5wre8uedEhwwYRVjWxbntoiqCGzp8H3FmcjHrwHp0xW7PCXLm4NezvSq4WSrWoN W4NXifJSqY//KxErEMVkSNIOQ31ErqWRhyv2ek28vYq/2r3xBuLgGA5tb+7J0wS9Ozvf qfT75fsB1pGtFGsFYz/P+fwiyAxjsYzFw4qcmQ8RaSqaHk8VXqXKyz/dgeNGwYt746iq GuSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QXi83h34VPpMPcld87083LVrMKk6MFlcpiSR8RVBVg4=; b=A1ajajJFRYKXmq5UsaGVjwtEh11FYlzP4fLZt9rb3E80zyhjmlxMhxmWDGHxZ5ua+K LT/ozH91Dr2T0HqYC1nxI8dKowVO8vV9CtH9Bilp8dQF8l8Ppvezz23PRPT6eS+XTbSV SZx0o+MMNzu86Z4H1jccpWcpAJKIrJsX988AxecQyTWwlqTeV/y1pDfLjdbs1O3KluF9 LasnqXCtC8B2FoqUAVSR0lBosJ3khsgADz9fltDNvX1h3IJgCmCaW/HLrPdOmAXPVPsT ErNU7aDYiAoBdrSeRcY6C17ZZKpN5lSgJPjMXTfY7LVD1a50jZrOicBeOXHLl0g5+Pwn qSGQ== X-Gm-Message-State: APjAAAU2XHt//7Y/cemu6/4ofZdHWFIzgv2Hzi7l/iUxr0TzIvZku5qM mqi9nQEYd21XvCDcHL4BV/D4A3esoEg= X-Google-Smtp-Source: APXvYqxhYXi7fqat3wySGt1AZ9ccnwC6uM2tS+2+IsMS0HpgmyBoHG0IqBDPw/8pSxRCfaSGzdXsSQ== X-Received: by 2002:aa7:9633:: with SMTP id r19mr10945pfg.90.1580938452190; Wed, 05 Feb 2020 13:34:12 -0800 (PST) Received: from Rfoley-MA01.usrd.futurewei.com ([12.111.81.71]) by smtp.gmail.com with ESMTPSA id p4sm724576pgh.14.2020.02.05.13.34.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Feb 2020 13:34:11 -0800 (PST) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 14/14] tests/vm: change scripts to use self._config Date: Wed, 5 Feb 2020 16:29:20 -0500 Message-Id: <20200205212920.467-15-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200205212920.467-1-robert.foley@linaro.org> References: <20200205212920.467-1-robert.foley@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::441 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, philmd@redhat.com, alex.bennee@linaro.org, robert.foley@linaro.org, peter.puhov@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley --- tests/vm/basevm.py | 11 ++--------- tests/vm/fedora | 17 +++++++++-------- tests/vm/freebsd | 16 ++++++++-------- tests/vm/netbsd | 19 ++++++++++--------- tests/vm/openbsd | 17 +++++++++-------- 5 files changed, 38 insertions(+), 42 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index dc975d92c7..d2c5d32f34 100755 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -184,13 +184,6 @@ class BaseVM(object): self.console_init(timeout=timeout) self.console_wait(wait_string) - def __getattr__(self, name): - # Support direct access to config by key. - # for example, access self._config['cpu'] by self.cpu - if name.lower() in self._config.keys(): - return self._config[name.lower()] - return object.__getattribute__(self, name) - def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -240,13 +233,13 @@ class BaseVM(object): return r def ssh(self, *cmd): - return self._ssh_do(self.GUEST_USER, cmd, False) + return self._ssh_do(self._config["guest_user"], cmd, False) def ssh_root(self, *cmd): return self._ssh_do("root", cmd, False) def ssh_check(self, *cmd): - self._ssh_do(self.GUEST_USER, cmd, True) + self._ssh_do(self._config["guest_user"], cmd, True) def ssh_root_check(self, *cmd): self._ssh_do("root", cmd, True) diff --git a/tests/vm/fedora b/tests/vm/fedora index 8e270fc0f0..4616d16740 100755 --- a/tests/vm/fedora +++ b/tests/vm/fedora @@ -105,20 +105,20 @@ class FedoraVM(basevm.BaseVM): self.console_wait_send("7) [!] Root password", "7\n") self.console_wait("Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("8) [ ] User creation", "8\n") self.console_wait_send("1) [ ] Create user", "1\n") self.console_wait_send("3) User name", "3\n") - self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER) + self.console_wait_send("ENTER:", "%s\n" % self._config["guest_user"]) self.console_wait_send("4) [ ] Use password", "4\n") self.console_wait_send("5) Password", "5\n") self.console_wait("Password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("7) Groups", "c\n") while True: @@ -136,7 +136,7 @@ class FedoraVM(basevm.BaseVM): if good: break time.sleep(10) - self.console_send("r\n" % self.GUEST_PASS) + self.console_send("r\n" % self._config["guest_pass"]) self.console_wait_send("'b' to begin install", "b\n") @@ -147,12 +147,13 @@ class FedoraVM(basevm.BaseVM): # setup qemu user prompt = " ~]$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = " ~]#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/freebsd b/tests/vm/freebsd index 33a736298a..fd1f595aa9 100755 --- a/tests/vm/freebsd +++ b/tests/vm/freebsd @@ -110,9 +110,9 @@ class FreeBSDVM(basevm.BaseVM): # post-install configuration self.console_wait("New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Network Configuration", "\n") self.console_wait_send("IPv4", "y") @@ -131,9 +131,9 @@ class FreeBSDVM(basevm.BaseVM): # qemu user self.console_wait_send("Add User Accounts", "y") self.console_wait("Username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait_send("Uid", "\n") self.console_wait_send("Login group", "\n") self.console_wait_send("Login group", "\n") @@ -145,9 +145,9 @@ class FreeBSDVM(basevm.BaseVM): self.console_wait_send("Use an empty password", "\n") self.console_wait_send("Use a random password", "\n") self.console_wait("Enter password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Enter password again:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Lock out", "\n") self.console_wait_send("OK", "yes\n") self.console_wait_send("Add another user", "no\n") @@ -161,12 +161,12 @@ class FreeBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "root@freebsd:~ #" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup serial console diff --git a/tests/vm/netbsd b/tests/vm/netbsd index ec6f3563b2..5eaafc27e0 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -128,24 +128,24 @@ class NetBSDVM(basevm.BaseVM): self.console_wait_send("d: Change root password", "d\n") self.console_wait_send("a: Yes", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("o: Add a user", "o\n") self.console_wait("username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("to group wheel") self.console_wait_send("a: Yes", "a\n") self.console_wait_send("a: /bin/sh", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("a: Configure network", "a\n") self.console_wait_send("a: vioif0", "a\n") @@ -178,12 +178,13 @@ class NetBSDVM(basevm.BaseVM): # setup qemu user prompt = "localhost$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "localhost#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/openbsd b/tests/vm/openbsd index d6173506f7..57f1d90bd6 100755 --- a/tests/vm/openbsd +++ b/tests/vm/openbsd @@ -95,9 +95,9 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which network interface", "done\n") self.console_wait_send("DNS domain name", "localnet\n") self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Start sshd(8)", "yes\n") self.console_wait_send("X Window System", "\n") self.console_wait_send("xenodm", "\n") @@ -105,13 +105,13 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which speed", "\n") self.console_wait("Setup a user") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Allow root ssh login", "yes\n") self.console_wait_send("timezone", "UTC\n") @@ -132,12 +132,13 @@ class OpenBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "openbsd#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile)