From patchwork Tue Sep 6 20:05:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sascha Silbe X-Patchwork-Id: 9317915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4CA8B607D3 for ; Tue, 6 Sep 2016 21:11:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46C1B28B70 for ; Tue, 6 Sep 2016 21:11:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3BCBF28EAA; Tue, 6 Sep 2016 21:11:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 426EB28EA8 for ; Tue, 6 Sep 2016 21:11:33 +0000 (UTC) Received: from localhost ([::1]:35854 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhNeT-0005O1-5v for patchwork-qemu-devel@patchwork.kernel.org; Tue, 06 Sep 2016 17:11:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52220) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhNdi-00058X-Ez for qemu-devel@nongnu.org; Tue, 06 Sep 2016 17:10:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bhNd9-0006Bn-Jd for qemu-devel@nongnu.org; Tue, 06 Sep 2016 17:10:42 -0400 Received: from [148.163.156.1] (port=55026 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bhNd9-000657-C4 for qemu-devel@nongnu.org; Tue, 06 Sep 2016 17:10:07 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id u86K39Y5039472 for ; Tue, 6 Sep 2016 16:06:00 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 25a3194h1c-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 06 Sep 2016 16:06:00 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 6 Sep 2016 21:05:58 +0100 Received: from d06dlp01.portsmouth.uk.ibm.com (9.149.20.13) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 6 Sep 2016 21:05:57 +0100 X-IBM-Helo: d06dlp01.portsmouth.uk.ibm.com X-IBM-MailFrom: silbe@linux.vnet.ibm.com X-IBM-RcptTo: qemu-devel@nongnu.org Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id 008FF17D8056 for ; Tue, 6 Sep 2016 21:07:49 +0100 (BST) Received: from d06av07.portsmouth.uk.ibm.com (d06av07.portsmouth.uk.ibm.com [9.149.37.248]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u86K5ud29044470 for ; Tue, 6 Sep 2016 20:05:56 GMT Received: from d06av07.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av07.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u86K5uph017376 for ; Tue, 6 Sep 2016 16:05:56 -0400 Received: from oc4731375738.boeblingen.de.ibm.com (dyn-9-152-222-64.boeblingen.de.ibm.com [9.152.222.64]) by d06av07.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u86K5u8m017354 (version=TLSv1/SSLv3 cipher=AES256-SHA256 bits=256 verify=NO); Tue, 6 Sep 2016 16:05:56 -0400 From: Sascha Silbe To: qemu-devel@nongnu.org, Fam Zheng Date: Tue, 6 Sep 2016 22:05:44 +0200 X-Mailer: git-send-email 1.9.1 In-Reply-To: <1473192351-601-1-git-send-email-silbe@linux.vnet.ibm.com> References: <1473192351-601-1-git-send-email-silbe@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16090620-0016-0000-0000-0000022D21EC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16090620-0017-0000-0000-000022E39707 Message-Id: <1473192351-601-2-git-send-email-silbe@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-09-06_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1609060306 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 1/8] docker.py: don't hang on large docker output X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Unlike Popen.communicate(), subprocess.call() doesn't read from the stdout file descriptor. If the child process produces more output than fits into the pipe buffer, it will block indefinitely. If we don't intend to consume the output, just send it straight to /dev/null to avoid this issue. Signed-off-by: Sascha Silbe Reviewed-by: Janosch Frank --- This fixes a hang for me when building the Ubuntu docker image (empty docker image cache). tests/docker/docker.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 222a105..efb2bf4 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -25,6 +25,10 @@ from tarfile import TarFile, TarInfo from StringIO import StringIO from shutil import copy, rmtree + +DEVNULL = open(os.devnull, 'wb') + + def _text_checksum(text): """Calculate a digest string unique to the text content""" return hashlib.sha1(text).hexdigest() @@ -34,8 +38,7 @@ def _guess_docker_command(): commands = [["docker"], ["sudo", "-n", "docker"]] for cmd in commands: if subprocess.call(cmd + ["images"], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE) == 0: + stdout=DEVNULL, stderr=DEVNULL) == 0: return cmd commands_txt = "\n".join([" " + " ".join(x) for x in commands]) raise Exception("Cannot find working docker command. Tried:\n%s" % \ @@ -98,7 +101,7 @@ class Docker(object): def _do(self, cmd, quiet=True, infile=None, **kwargs): if quiet: - kwargs["stdout"] = subprocess.PIPE + kwargs["stdout"] = DEVNULL if infile: kwargs["stdin"] = infile return subprocess.call(self._command + cmd, **kwargs)