| Message ID | 1464272863-2285-3-git-send-email-alex.bennee@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, 05/26 15:27, Alex Bennée wrote: > Together with some changes to the docker script you can now build an > arbitrary architecture of Debian using debootstrap. To achieve this I > introduce the concept of a HOST_CMD in the docker config file. While > copying the file into workspace the HOST_CMD is run in the docker build > context. This allows debootstrap to set up its first stage before the > container is built. Could you instead introduce the concept of $IMAGE.pre file (in this case debian-bootstrap.pre, aside debian-bootstrap.docker), and exec it in docker.py? It would be much more flexible, and we we wouldn't need to inject a custom directive to filter it out. Fam
Fam Zheng <famz@redhat.com> writes: > On Thu, 05/26 15:27, Alex Bennée wrote: >> Together with some changes to the docker script you can now build an >> arbitrary architecture of Debian using debootstrap. To achieve this I >> introduce the concept of a HOST_CMD in the docker config file. While >> copying the file into workspace the HOST_CMD is run in the docker build >> context. This allows debootstrap to set up its first stage before the >> container is built. > > Could you instead introduce the concept of $IMAGE.pre file (in this case > debian-bootstrap.pre, aside debian-bootstrap.docker), and exec it in > docker.py? It would be much more flexible, and we we wouldn't need to inject a > custom directive to filter it out. I'm ambivalent about that. To be honest this is a bit of a gap in docker's image creation (or if there is a better more docker-y way of doing things I couldn't find it). It does have the benefit of keeping everything in one place. We are copying the dockerfile to the build environment anyway so it seemed natural to do the operation while copying across. How would envison the .pre setup? A #! script we just exec inside the temp directory we create? > > Fam -- Alex Bennée
On Tue, 05/31 16:27, Alex Bennée wrote: > > Fam Zheng <famz@redhat.com> writes: > > > On Thu, 05/26 15:27, Alex Bennée wrote: > >> Together with some changes to the docker script you can now build an > >> arbitrary architecture of Debian using debootstrap. To achieve this I > >> introduce the concept of a HOST_CMD in the docker config file. While > >> copying the file into workspace the HOST_CMD is run in the docker build > >> context. This allows debootstrap to set up its first stage before the > >> container is built. > > > > Could you instead introduce the concept of $IMAGE.pre file (in this case > > debian-bootstrap.pre, aside debian-bootstrap.docker), and exec it in > > docker.py? It would be much more flexible, and we we wouldn't need to inject a > > custom directive to filter it out. > > I'm ambivalent about that. To be honest this is a bit of a gap in > docker's image creation (or if there is a better more docker-y way of > doing things I couldn't find it). It does have the benefit of keeping > everything in one place. I don't like that it pollutes the dockerfile, rendering it invalid for raw "docker build", which is not very good. > > We are copying the dockerfile to the build environment anyway so it > seemed natural to do the operation while copying across. How would > envison the .pre setup? A #! script we just exec inside the temp > directory we create? Yes, since I'm going to send a v7, I'll merge the tmp dir part in to my series, and add a new patch for the "pre" idea to see if we like it. Fam
diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e9242f3..3ba3d4e 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -21,7 +21,7 @@ import uuid import argparse import tempfile import re -from shutil import copyfile +from shutil import copy def _text_checksum(text): """Calculate a digest string unique to the text content""" @@ -46,7 +46,6 @@ def _find_user_binary(binary_name): for x in linux_user: check_path = "%s/%s/%s" % (top, x, binary_name) if os.path.isfile(check_path): - print ("found %s" % check_path) return check_path return None @@ -58,7 +57,7 @@ def _copy_with_mkdir(src, root_dir, sub_path): except OSError: print "skipping %s" % (full_path) - copyfile(src, "%s/%s" % (full_path, os.path.basename(src))) + copy(src, "%s/%s" % (full_path, os.path.basename(src))) class Docker(object): """ Running Docker commands """ @@ -117,11 +116,23 @@ class Docker(object): tmp_dir = tempfile.mkdtemp(prefix="docker_build") # Copy the dockerfile into our work space - tmp = dockerfile + "\n" + \ - "LABEL com.qemu.dockerfile-checksum=%s" % \ - _text_checksum(dockerfile) + # line by line, stripping and executing HOST_CMDs + # tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker") - tmp_df.write(tmp) + + for l in open(dockerfile).readlines(): + m = re.match("HOST_CMD ", l) + if m: + print "l=%s" % (l) + cmd = l[m.end():] + r = subprocess.check_call(cmd, cwd=tmp_dir, shell=True) + tmp_df.write("# HOST_CMD %s# HOST_RES = %d\n" % (cmd, r)) + else: + tmp_df.write(l) + + tmp_df.write("\n") + tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" % + _text_checksum(dockerfile)) tmp_df.flush() # Do we want to copy QEMU into here? @@ -210,7 +221,7 @@ class BuildCommand(SubCommand): print "Image is up to date." return 0 - dkr.build_image(tag, dockerfile, quiet=args.quiet, qemu=qbin, argv=argv) + dkr.build_image(tag, args.dockerfile, quiet=args.quiet, qemu=qbin, argv=argv) return 0 class CleanCommand(SubCommand): diff --git a/tests/docker/dockerfiles/debian-bootstrap.docker b/tests/docker/dockerfiles/debian-bootstrap.docker new file mode 100644 index 0000000..44d107d --- /dev/null +++ b/tests/docker/dockerfiles/debian-bootstrap.docker @@ -0,0 +1,22 @@ +# Create Debian Bootstrap Image +# +# This is intended to be pre-poluated by: +# - a first stage debootstrap +# - a native qemu-$arch that binfmt_misc will run +FROM scratch + +# HOST_CMD is executed by docker.py while building the context +HOST_CMD fakeroot debootstrap --variant=minbase --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian + +# Add everything from the context into the container +ADD . / + +# Patch all mounts as docker already has stuff set up +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' /debootstrap/functions + +# Run stage 2 +RUN /debootstrap/debootstrap --second-stage + +# At this point we can install additional packages if we want +#RUN apt-get update +#RUN apt-get dist-upgrade
Together with some changes to the docker script you can now build an arbitrary architecture of Debian using debootstrap. To achieve this I introduce the concept of a HOST_CMD in the docker config file. While copying the file into workspace the HOST_CMD is run in the docker build context. This allows debootstrap to set up its first stage before the container is built. To build a container you need a command line like: DEB_ARCH=armhf DEB_TYPE=testing \ ./tests/docker/docker.py build --qemu=qemu-arm debian:armhf \ ./tests/docker/dockerfiles/debian-bootstrap.docker The DEB_ARCH/DEB_TYPE are expanded in the docker file by the HOST_CMD: HOST_CMD fakeroot debootstrap --variant=minbase --foreign \ --arch=$DEB_ARCH $DEB_TYPE . \ http://httpredir.debian.org/debian Signed-off-by: Alex Bennée <alex.bennee@linaro.org> --- tests/docker/docker.py | 27 +++++++++++++++++------- tests/docker/dockerfiles/debian-bootstrap.docker | 22 +++++++++++++++++++ 2 files changed, 41 insertions(+), 8 deletions(-) create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker