| Message ID | 1468207242-5015-5-git-send-email-famz@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Fam Zheng <famz@redhat.com> writes: > Signed-off-by: Fam Zheng <famz@redhat.com> > --- > tests/docker/dockerfiles/debootstrap-arm.docker | 35 +++++++++++++++++++++++++ > 1 file changed, 35 insertions(+) > create mode 100644 tests/docker/dockerfiles/debootstrap-arm.docker > > diff --git a/tests/docker/dockerfiles/debootstrap-arm.docker b/tests/docker/dockerfiles/debootstrap-arm.docker > new file mode 100644 > index 0000000..cb15f2f > --- /dev/null > +++ b/tests/docker/dockerfiles/debootstrap-arm.docker > @@ -0,0 +1,35 @@ > +FROM debian:testing > + > +RUN apt-get update > +RUN apt-get install -y fakeroot debootstrap qemu-user-static > + > +RUN mkdir /debootstrap-arm > + > +RUN cd /debootstrap-arm && fakeroot debootstrap --variant=buildd --foreign \ > + --arch=armhf testing . http://httpredir.debian.org/debian > + > +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' \ > + /debootstrap-arm/debootstrap/functions > + > +RUN mkdir -p /debootstrap-arm/usr/local/bin > + > +RUN ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm && \ > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm-static && \ > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm && \ > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm-static > + > +# Run stage 2 > +RUN if ! chroot /debootstrap-arm /debootstrap/debootstrap --second-stage; then \ > + echo "Failed to chroot and do stage 2"; \ > + echo "Please set up binfmt_misc to point arm binary to one of:"; \ > + echo " /usr/bin/qemu-arm"; \ > + echo " /usr/bin/qemu-arm-static"; \ > + echo " /usr/local/bin/qemu-arm"; \ > + echo " /usr/local/bin/qemu-arm-static"; \ > + exit 1; \ > + fi > +RUN chroot /debootstrap-arm sh -c 'cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list' > +RUN chroot /debootstrap-arm apt-get update > +RUN chroot /debootstrap-arm apt-get build-dep -y qemu > +RUN chroot /debootstrap-arm apt-get install -y ccache > +ENV QEMU_CHROOT /debootstrap-arm OK I've done some more experimenting and two things are apparent: debootstrap is widely packaged for various distros And The script it fairly portable so we can always run it directly I hacked up the .pre script to do the following and tested on my Arch VM: #!/bin/sh # # Simple wrapper for debootstrap, run in the docker build context # FAKEROOT=`which fakeroot 2> /dev/null` DEBOOTSTRAP=`which debootstrap 2> /dev/null` DEBOOTSTRAP_SOURCE=https://anonscm.debian.org/git/d-i/debootstrap.git if [ -z $FAKEROOT ]; then echo "Please install fakeroot to enable bootstraping" exit 1 fi if [ -z $DEBOOTSTRAP ]; then echo "No debootstrap installed, attempting to install from SCM" git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git export DEBOOTSTRAP_DIR=./debootstrap.git DEBOOTSTRAP=./debootstrap.git/debootstrap fi echo "Building a rootfs using ${FAKEROOT} and ${DEBOOTSTRAP}" ${FAKEROOT} /bin/sh -x ${DEBOOTSTRAP} --variant=buildd --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian exit 0 So I think it is feasible to but the pre-requisite checking and work around in the pre script and be done with it. It seems neater than the chroot within a container approach. Thoughts? -- Alex Bennée
On Mon, 07/11 20:06, Alex Bennée wrote: > > Fam Zheng <famz@redhat.com> writes: > > > Signed-off-by: Fam Zheng <famz@redhat.com> > > --- > > tests/docker/dockerfiles/debootstrap-arm.docker | 35 +++++++++++++++++++++++++ > > 1 file changed, 35 insertions(+) > > create mode 100644 tests/docker/dockerfiles/debootstrap-arm.docker > > > > diff --git a/tests/docker/dockerfiles/debootstrap-arm.docker b/tests/docker/dockerfiles/debootstrap-arm.docker > > new file mode 100644 > > index 0000000..cb15f2f > > --- /dev/null > > +++ b/tests/docker/dockerfiles/debootstrap-arm.docker > > @@ -0,0 +1,35 @@ > > +FROM debian:testing > > + > > +RUN apt-get update > > +RUN apt-get install -y fakeroot debootstrap qemu-user-static > > + > > +RUN mkdir /debootstrap-arm > > + > > +RUN cd /debootstrap-arm && fakeroot debootstrap --variant=buildd --foreign \ > > + --arch=armhf testing . http://httpredir.debian.org/debian > > + > > +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' \ > > + /debootstrap-arm/debootstrap/functions > > + > > +RUN mkdir -p /debootstrap-arm/usr/local/bin > > + > > +RUN ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm && \ > > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm-static && \ > > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm && \ > > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm-static > > + > > +# Run stage 2 > > +RUN if ! chroot /debootstrap-arm /debootstrap/debootstrap --second-stage; then \ > > + echo "Failed to chroot and do stage 2"; \ > > + echo "Please set up binfmt_misc to point arm binary to one of:"; \ > > + echo " /usr/bin/qemu-arm"; \ > > + echo " /usr/bin/qemu-arm-static"; \ > > + echo " /usr/local/bin/qemu-arm"; \ > > + echo " /usr/local/bin/qemu-arm-static"; \ > > + exit 1; \ > > + fi > > +RUN chroot /debootstrap-arm sh -c 'cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list' > > +RUN chroot /debootstrap-arm apt-get update > > +RUN chroot /debootstrap-arm apt-get build-dep -y qemu > > +RUN chroot /debootstrap-arm apt-get install -y ccache > > +ENV QEMU_CHROOT /debootstrap-arm > > OK I've done some more experimenting and two things are apparent: > > debootstrap is widely packaged for various distros > > And > > The script it fairly portable so we can always run it directly > > I hacked up the .pre script to do the following and tested on my Arch > VM: > > #!/bin/sh > # > # Simple wrapper for debootstrap, run in the docker build context > # > FAKEROOT=`which fakeroot 2> /dev/null` > DEBOOTSTRAP=`which debootstrap 2> /dev/null` > DEBOOTSTRAP_SOURCE=https://anonscm.debian.org/git/d-i/debootstrap.git > > if [ -z $FAKEROOT ]; then > echo "Please install fakeroot to enable bootstraping" > exit 1 > fi > > if [ -z $DEBOOTSTRAP ]; then > echo "No debootstrap installed, attempting to install from SCM" > git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git > export DEBOOTSTRAP_DIR=./debootstrap.git > DEBOOTSTRAP=./debootstrap.git/debootstrap > fi > > echo "Building a rootfs using ${FAKEROOT} and ${DEBOOTSTRAP}" > > ${FAKEROOT} /bin/sh -x ${DEBOOTSTRAP} --variant=buildd --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian > exit 0 > > So I think it is feasible to but the pre-requisite checking and work > around in the pre script and be done with it. It seems neater than the > chroot within a container approach. > > Thoughts? Yes, makes sense to me. Two more questions: Can we have "make docker-images" to update the docker image if pre script has been updated? Can we skip debootstrap (i.e. skip .pre script) if it's only the dockerfile that has changed? Thanks, Fam
Fam Zheng <famz@redhat.com> writes: > On Mon, 07/11 20:06, Alex Bennée wrote: >> >> Fam Zheng <famz@redhat.com> writes: >> >> > Signed-off-by: Fam Zheng <famz@redhat.com> >> > --- >> > tests/docker/dockerfiles/debootstrap-arm.docker | 35 +++++++++++++++++++++++++ >> > 1 file changed, 35 insertions(+) >> > create mode 100644 tests/docker/dockerfiles/debootstrap-arm.docker >> > >> > diff --git a/tests/docker/dockerfiles/debootstrap-arm.docker b/tests/docker/dockerfiles/debootstrap-arm.docker >> > new file mode 100644 >> > index 0000000..cb15f2f >> > --- /dev/null >> > +++ b/tests/docker/dockerfiles/debootstrap-arm.docker >> > @@ -0,0 +1,35 @@ >> > +FROM debian:testing >> > + >> > +RUN apt-get update >> > +RUN apt-get install -y fakeroot debootstrap qemu-user-static >> > + >> > +RUN mkdir /debootstrap-arm >> > + >> > +RUN cd /debootstrap-arm && fakeroot debootstrap --variant=buildd --foreign \ >> > + --arch=armhf testing . http://httpredir.debian.org/debian >> > + >> > +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' \ >> > + /debootstrap-arm/debootstrap/functions >> > + >> > +RUN mkdir -p /debootstrap-arm/usr/local/bin >> > + >> > +RUN ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm && \ >> > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm-static && \ >> > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm && \ >> > + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm-static >> > + >> > +# Run stage 2 >> > +RUN if ! chroot /debootstrap-arm /debootstrap/debootstrap --second-stage; then \ >> > + echo "Failed to chroot and do stage 2"; \ >> > + echo "Please set up binfmt_misc to point arm binary to one of:"; \ >> > + echo " /usr/bin/qemu-arm"; \ >> > + echo " /usr/bin/qemu-arm-static"; \ >> > + echo " /usr/local/bin/qemu-arm"; \ >> > + echo " /usr/local/bin/qemu-arm-static"; \ >> > + exit 1; \ >> > + fi >> > +RUN chroot /debootstrap-arm sh -c 'cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list' >> > +RUN chroot /debootstrap-arm apt-get update >> > +RUN chroot /debootstrap-arm apt-get build-dep -y qemu >> > +RUN chroot /debootstrap-arm apt-get install -y ccache >> > +ENV QEMU_CHROOT /debootstrap-arm >> >> OK I've done some more experimenting and two things are apparent: >> >> debootstrap is widely packaged for various distros >> >> And >> >> The script it fairly portable so we can always run it directly >> >> I hacked up the .pre script to do the following and tested on my Arch >> VM: >> >> #!/bin/sh >> # >> # Simple wrapper for debootstrap, run in the docker build context >> # >> FAKEROOT=`which fakeroot 2> /dev/null` >> DEBOOTSTRAP=`which debootstrap 2> /dev/null` >> DEBOOTSTRAP_SOURCE=https://anonscm.debian.org/git/d-i/debootstrap.git >> >> if [ -z $FAKEROOT ]; then >> echo "Please install fakeroot to enable bootstraping" >> exit 1 >> fi >> >> if [ -z $DEBOOTSTRAP ]; then >> echo "No debootstrap installed, attempting to install from SCM" >> git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git >> export DEBOOTSTRAP_DIR=./debootstrap.git >> DEBOOTSTRAP=./debootstrap.git/debootstrap >> fi >> >> echo "Building a rootfs using ${FAKEROOT} and ${DEBOOTSTRAP}" >> >> ${FAKEROOT} /bin/sh -x ${DEBOOTSTRAP} --variant=buildd --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian >> exit 0 >> >> So I think it is feasible to but the pre-requisite checking and work >> around in the pre script and be done with it. It seems neater than the >> chroot within a container approach. >> >> Thoughts? > > Yes, makes sense to me. Two more questions: > > Can we have "make docker-images" to update the docker image if pre script has > been updated? We could, I'm not sure what the dependency should be for a file that may or may not exist? The larger problem is what to do about the fact that debootstrap is a multi-arch script? Currently I can manually build the qmu:debian-bootstrap image from the command line for any linux-user architecture. Once this is done I can then run all the tests under that single emulated architecture. The question is should the "make docker" expand debian-bootstrap into all the various architecture targets and build them all or just assume what ever one is set-up by the user is the one that gets tested? Currently: make docker-image-debian-bootstrap DEB_ARCH=arm64 DEB_TYPE=testing V=1 J=9 Will fail because the build machinery doesn't know what --include-executable to pass to the docker.py script. However after that everything should work. My current state is at: https://github.com/stsquad/qemu/tree/misc/docker-linux-user-v4 I'll post v5 shortly after I've done some basic testing. > > Can we skip debootstrap (i.e. skip .pre script) if it's only the dockerfile > that has changed? No, because a running a dockerfile implies re-building the container from scratch and the .pre script needs to pre-seed the build context. > > Thanks, > > Fam -- Alex Bennée
diff --git a/tests/docker/dockerfiles/debootstrap-arm.docker b/tests/docker/dockerfiles/debootstrap-arm.docker new file mode 100644 index 0000000..cb15f2f --- /dev/null +++ b/tests/docker/dockerfiles/debootstrap-arm.docker @@ -0,0 +1,35 @@ +FROM debian:testing + +RUN apt-get update +RUN apt-get install -y fakeroot debootstrap qemu-user-static + +RUN mkdir /debootstrap-arm + +RUN cd /debootstrap-arm && fakeroot debootstrap --variant=buildd --foreign \ + --arch=armhf testing . http://httpredir.debian.org/debian + +RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' \ + /debootstrap-arm/debootstrap/functions + +RUN mkdir -p /debootstrap-arm/usr/local/bin + +RUN ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm && \ + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/bin/qemu-arm-static && \ + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm && \ + ln /usr/bin/qemu-arm-static /debootstrap-arm/usr/local/bin/qemu-arm-static + +# Run stage 2 +RUN if ! chroot /debootstrap-arm /debootstrap/debootstrap --second-stage; then \ + echo "Failed to chroot and do stage 2"; \ + echo "Please set up binfmt_misc to point arm binary to one of:"; \ + echo " /usr/bin/qemu-arm"; \ + echo " /usr/bin/qemu-arm-static"; \ + echo " /usr/local/bin/qemu-arm"; \ + echo " /usr/local/bin/qemu-arm-static"; \ + exit 1; \ + fi +RUN chroot /debootstrap-arm sh -c 'cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list' +RUN chroot /debootstrap-arm apt-get update +RUN chroot /debootstrap-arm apt-get build-dep -y qemu +RUN chroot /debootstrap-arm apt-get install -y ccache +ENV QEMU_CHROOT /debootstrap-arm
Signed-off-by: Fam Zheng <famz@redhat.com> --- tests/docker/dockerfiles/debootstrap-arm.docker | 35 +++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 tests/docker/dockerfiles/debootstrap-arm.docker