From patchwork Thu May 26 14:27:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 9137033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 11E2E6075A for ; Thu, 26 May 2016 14:29:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0606F2198E for ; Thu, 26 May 2016 14:29:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE64E27BFE; Thu, 26 May 2016 14:29:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 436882198E for ; Thu, 26 May 2016 14:29:40 +0000 (UTC) Received: from localhost ([::1]:38660 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wI7-0002zI-84 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 26 May 2016 10:29:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45222) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wG5-0001q6-HN for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5wFz-0006mh-BV for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:32 -0400 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:36908) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5wFz-0006mV-1o for qemu-devel@nongnu.org; Thu, 26 May 2016 10:27:27 -0400 Received: by mail-wm0-x236.google.com with SMTP id z87so101815156wmh.0 for ; Thu, 26 May 2016 07:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0qrvQi4YrNuQ29pKzLYNufMDokl6kZ9gidIvlGqK8nE=; b=GLrqUjy4FzPqeFJQN0RBc1G/C7IznBRHagVU1TI3a3T2HwSfquWSl/Ndrie7CNUekv Uc13QC/3b52NZqRn1XuM566Musf0s2j3uWF5hflSFm22FERT4UayxfkNSprnyh0QVUoD SyAUHO9+UkzdXNPQujUl7XA+jsSVgPKMKVDLg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0qrvQi4YrNuQ29pKzLYNufMDokl6kZ9gidIvlGqK8nE=; b=CBNHY1IZYb9rsb2Luxt247gATuEOK8ZJZdU5TkDR0mdmfVb8lPUkz4X5YwZ8bxVWt1 Feh76KhSGpJBUJtrnqESKSMWn4jqcJ9GzXsrmoCY+jhMMKP89dyQPp2VSrutS2ycIDut bUqh891sklvNmqb1j/n8agyDJsvkR5PNXWENqHbuOlPjHjCw59z41HQ7WyKxcOpPq7/w AQsAqVeK9723TwXw89Yh4u7qN/xElQlnLPAjp3GlSpoGBVcCAEMLl4oXKwjsxTTdNbG7 v+HfQc9MVUEqrCwZSTmN3t2iNRNf+ImcENWh4r8veKfXoC5jb5FzOK/iyDu3oNleKROH wIkg== X-Gm-Message-State: ALyK8tLouiP7COVcaPutEqfD5A4Gz/6VOcBLgyWAYTKLernl/mlqOdboAqVCZ2UTcJVq+pLl X-Received: by 10.28.60.215 with SMTP id j206mr4133900wma.82.1464272846291; Thu, 26 May 2016 07:27:26 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 131sm3817373wmu.17.2016.05.26.07.27.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 May 2016 07:27:24 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id F40013E0288; Thu, 26 May 2016 15:28:03 +0100 (BST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Thu, 26 May 2016 15:27:43 +0100 Message-Id: <1464272863-2285-3-git-send-email-alex.bennee@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1464272863-2285-1-git-send-email-alex.bennee@linaro.org> References: <1464272863-2285-1-git-send-email-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [RFC PATCH 2/2] add debian-bootstrap.docker target X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, famz@redhat.com, riku.voipio@linaro.org, steve.mcintyre@linaro.org, neil.williams@linaro.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Together with some changes to the docker script you can now build an arbitrary architecture of Debian using debootstrap. To achieve this I introduce the concept of a HOST_CMD in the docker config file. While copying the file into workspace the HOST_CMD is run in the docker build context. This allows debootstrap to set up its first stage before the container is built. To build a container you need a command line like: DEB_ARCH=armhf DEB_TYPE=testing \ ./tests/docker/docker.py build --qemu=qemu-arm debian:armhf \ ./tests/docker/dockerfiles/debian-bootstrap.docker The DEB_ARCH/DEB_TYPE are expanded in the docker file by the HOST_CMD: HOST_CMD fakeroot debootstrap --variant=minbase --foreign \ --arch=$DEB_ARCH $DEB_TYPE . \ http://httpredir.debian.org/debian Signed-off-by: Alex Bennée --- tests/docker/docker.py | 27 +++++++++++++++++------- tests/docker/dockerfiles/debian-bootstrap.docker | 22 +++++++++++++++++++ 2 files changed, 41 insertions(+), 8 deletions(-) create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e9242f3..3ba3d4e 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -21,7 +21,7 @@ import uuid import argparse import tempfile import re -from shutil import copyfile +from shutil import copy def _text_checksum(text): """Calculate a digest string unique to the text content""" @@ -46,7 +46,6 @@ def _find_user_binary(binary_name): for x in linux_user: check_path = "%s/%s/%s" % (top, x, binary_name) if os.path.isfile(check_path): - print ("found %s" % check_path) return check_path return None @@ -58,7 +57,7 @@ def _copy_with_mkdir(src, root_dir, sub_path): except OSError: print "skipping %s" % (full_path) - copyfile(src, "%s/%s" % (full_path, os.path.basename(src))) + copy(src, "%s/%s" % (full_path, os.path.basename(src))) class Docker(object): """ Running Docker commands """ @@ -117,11 +116,23 @@ class Docker(object): tmp_dir = tempfile.mkdtemp(prefix="docker_build") # Copy the dockerfile into our work space - tmp = dockerfile + "\n" + \ - "LABEL com.qemu.dockerfile-checksum=%s" % \ - _text_checksum(dockerfile) + # line by line, stripping and executing HOST_CMDs + # tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker") - tmp_df.write(tmp) + + for l in open(dockerfile).readlines(): + m = re.match("HOST_CMD ", l) + if m: + print "l=%s" % (l) + cmd = l[m.end():] + r = subprocess.check_call(cmd, cwd=tmp_dir, shell=True) + tmp_df.write("# HOST_CMD %s# HOST_RES = %d\n" % (cmd, r)) + else: + tmp_df.write(l) + + tmp_df.write("\n") + tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" % + _text_checksum(dockerfile)) tmp_df.flush() # Do we want to copy QEMU into here? @@ -210,7 +221,7 @@ class BuildCommand(SubCommand): print "Image is up to date." return 0 - dkr.build_image(tag, dockerfile, quiet=args.quiet, qemu=qbin, argv=argv) + dkr.build_image(tag, args.dockerfile, quiet=args.quiet, qemu=qbin, argv=argv) return 0 class CleanCommand(SubCommand): diff --git a/tests/docker/dockerfiles/debian-bootstrap.docker b/tests/docker/dockerfiles/debian-bootstrap.docker new file mode 100644 index 0000000..44d107d --- /dev/null +++ b/tests/docker/dockerfiles/debian-bootstrap.docker @@ -0,0 +1,22 @@ +# Create Debian Bootstrap Image +# +# This is intended to be pre-poluated by: +# - a first stage debootstrap +# - a native qemu-$arch that binfmt_misc will run +FROM scratch + +# HOST_CMD is executed by docker.py while building the context +HOST_CMD fakeroot debootstrap --variant=minbase --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian + +# Add everything from the context into the container +ADD . / + +# Patch all mounts as docker already has stuff set up +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' /debootstrap/functions + +# Run stage 2 +RUN /debootstrap/debootstrap --second-stage + +# At this point we can install additional packages if we want +#RUN apt-get update +#RUN apt-get dist-upgrade