| Message ID | 20200713145801.29278-1-alex.bennee@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | docker.py: fix fetching of FROM layers | expand |
On Mon, Jul 13, 2020 at 03:58:01PM +0100, Alex Bennée wrote: > This worked on a system that was already bootstrapped because the > stage 2 images already existed even if they wouldn't be used. What we > should have pulled down was the FROM line containers first because > building on gitlab doesn't have the advantage of using our build > system to build the pre-requisite bits. > > We still pull the image we want to build just in case we can use the > cached data. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > tests/docker/docker.py | 16 +++++++++++----- > 1 file changed, 11 insertions(+), 5 deletions(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel
On 7/13/20 4:58 PM, Alex Bennée wrote: > This worked on a system that was already bootstrapped because the > stage 2 images already existed even if they wouldn't be used. What we > should have pulled down was the FROM line containers first because > building on gitlab doesn't have the advantage of using our build > system to build the pre-requisite bits. > > We still pull the image we want to build just in case we can use the > cached data. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > tests/docker/docker.py | 16 +++++++++++----- > 1 file changed, 11 insertions(+), 5 deletions(-) > Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/docker/docker.py b/tests/docker/docker.py index 2d67bbd15a..c9f20d8d09 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -306,14 +306,18 @@ class Docker(object): checksum = _text_checksum(_dockerfile_preprocess(dockerfile)) if registry is not None: - # see if we can fetch a cache copy, may fail... - pull_args = ["pull", "%s/%s" % (registry, tag)] - if self._do(pull_args, quiet=quiet) == 0: + sources = re.findall("FROM qemu\/(.*)", dockerfile) + # Fetch any cache layers we can, may fail + for s in sources: + pull_args = ["pull", "%s/qemu/%s" % (registry, s)] + if self._do(pull_args, quiet=quiet) != 0: + registry = None + break + # Make substitutions + if registry is not None: dockerfile = dockerfile.replace("FROM qemu/", "FROM %s/qemu/" % (registry)) - else: - registry = None tmp_df = tempfile.NamedTemporaryFile(mode="w+t", encoding='utf-8', @@ -339,6 +343,8 @@ class Docker(object): build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"] if registry is not None: + pull_args = ["pull", "%s/%s" % (registry, tag)] + self._do(pull_args, quiet=quiet) cache = "%s/%s" % (registry, tag) build_args += ["--cache-from", cache] build_args += argv
This worked on a system that was already bootstrapped because the stage 2 images already existed even if they wouldn't be used. What we should have pulled down was the FROM line containers first because building on gitlab doesn't have the advantage of using our build system to build the pre-requisite bits. We still pull the image we want to build just in case we can use the cached data. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> --- tests/docker/docker.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)