| Message ID | 1457086720-30391-11-git-send-email-famz@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Fam Zheng <famz@redhat.com> writes: > The script is not named test-travis.sh so it won't run with "make > docker-run", because it can take too long. > > Run it with "make docker-run-travis.sh@ubuntu". 16:08 alex@zen/x86_64 [qemu.git/review/docker-v3] >make docker-run-travis.sh@ubuntu ARCHIVE qemu.tgz COPY RUNNER RUN travis.sh in ubuntu ./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or directory > > Signed-off-by: Fam Zheng <famz@redhat.com> > --- > tests/docker/travis | 21 +++++++++++++++++++++ > tests/docker/travis.py | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 69 insertions(+) > create mode 100755 tests/docker/travis > create mode 100755 tests/docker/travis.py > > diff --git a/tests/docker/travis b/tests/docker/travis > new file mode 100755 > index 0000000..d345393 > --- /dev/null > +++ b/tests/docker/travis > @@ -0,0 +1,21 @@ > +#!/bin/bash -e > +# > +# Mimic a travis testing matrix > +# > +# Copyright (c) 2016 Red Hat Inc. > +# > +# Authors: > +# Fam Zheng <famz@redhat.com> > +# > +# This work is licensed under the terms of the GNU GPL, version 2 > +# or (at your option) any later version. See the COPYING file in > +# the top-level directory. > + > +. common.rc > + > +requires pyyaml > +cmdfile=/tmp/travis_cmd_list.sh > +$QEMU_SRC/tests/docker/travis.py $QEMU_SRC/.travis.yml > $cmdfile > +chmod +x $cmdfile > +cd "$QEMU_SRC" > +$cmdfile > diff --git a/tests/docker/travis.py b/tests/docker/travis.py > new file mode 100755 > index 0000000..8dcc964 > --- /dev/null > +++ b/tests/docker/travis.py > @@ -0,0 +1,48 @@ > +#!/usr/bin/env python > +# > +# Travis YAML config parser > +# > +# Copyright (c) 2016 Red Hat Inc. > +# > +# Authors: > +# Fam Zheng <famz@redhat.com> > +# > +# This work is licensed under the terms of the GNU GPL, version 2 > +# or (at your option) any later version. See the COPYING file in > +# the top-level directory. > + > +import sys > +import yaml > +import itertools > + > +def load_yaml(fname): > + return yaml.load(open(fname, "r").read()) > + > +def conf_iter(conf): > + def env_to_list(env): > + return env if isinstance(env, list) else [env] > + global_env = conf["env"]["global"] > + for entry in conf["matrix"]["include"]: > + yield {"env": global_env + env_to_list(entry["env"]), > + "compiler": entry["compiler"]} > + for entry in itertools.product(conf["compiler"], > + conf["env"]["matrix"]): > + yield {"env": global_env + env_to_list(entry[1]), > + "compiler": entry[0]} > + > +def main(): > + if len(sys.argv) < 2: > + sys.stderr.write("Usage: %s <travis-file>\n" % sys.argv[0]) > + return 1 > + conf = load_yaml(sys.argv[1]) > + for config in conf_iter(conf): > + print "(" > + print "\n".join(config["env"]) > + print "alias cc=" + config["compiler"] > + print "\n".join(conf["before_script"]) > + print "\n".join(conf["script"]) > + print ")" > + return 0 > + > +if __name__ == "__main__": > + sys.exit(main()) -- Alex Bennée
On Fri, 03/11 16:14, Alex Bennée wrote: > > Fam Zheng <famz@redhat.com> writes: > > > The script is not named test-travis.sh so it won't run with "make > > docker-run", because it can take too long. > > > > Run it with "make docker-run-travis.sh@ubuntu". > > 16:08 alex@zen/x86_64 [qemu.git/review/docker-v3] >make docker-run-travis.sh@ubuntu > ARCHIVE qemu.tgz > COPY RUNNER > RUN travis.sh in ubuntu > ./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or directory Will update the commit message. Fam
Fam Zheng <famz@redhat.com> writes: > On Fri, 03/11 16:14, Alex Bennée wrote: >> >> Fam Zheng <famz@redhat.com> writes: >> >> > The script is not named test-travis.sh so it won't run with "make >> > docker-run", because it can take too long. >> > >> > Run it with "make docker-run-travis.sh@ubuntu". >> >> 16:08 alex@zen/x86_64 [qemu.git/review/docker-v3] >make docker-run-travis.sh@ubuntu >> ARCHIVE qemu.tgz >> COPY RUNNER >> RUN travis.sh in ubuntu >> ./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or directory > > Will update the commit message. I had a bit of a further play with this while trying to help with the recent Travis breakage. I realised we need to be clearer about what this does. It's not the same as running on travis, just a way of iterating through the travis build matrix on whatever image you happen to be on. It would be nice to have a travis image for local debug but that seems to be harder to do than I thought. I couldn't find any such images on the hub. > > Fam -- Alex Bennée
On Wed, 03/16 09:09, Alex Bennée wrote: > > Fam Zheng <famz@redhat.com> writes: > > > On Fri, 03/11 16:14, Alex Bennée wrote: > >> > >> Fam Zheng <famz@redhat.com> writes: > >> > >> > The script is not named test-travis.sh so it won't run with "make > >> > docker-run", because it can take too long. > >> > > >> > Run it with "make docker-run-travis.sh@ubuntu". > >> > >> 16:08 alex@zen/x86_64 [qemu.git/review/docker-v3] >make docker-run-travis.sh@ubuntu > >> ARCHIVE qemu.tgz > >> COPY RUNNER > >> RUN travis.sh in ubuntu > >> ./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or directory > > > > Will update the commit message. > > I had a bit of a further play with this while trying to help with the > recent Travis breakage. Did this help? > I realised we need to be clearer about what this > does. It's not the same as running on travis, just a way of iterating > through the travis build matrix on whatever image you happen to be on. Right. I have no idea how to precisely replicate travis environment, and I ignored python version, packages etc for simplicity, only the command matrix was simulated. But are there any major differences you are noticing? At least we are also on Ubuntu Trusty, the same as: https://docs.travis-ci.com/user/ci-environment/ Fam > > It would be nice to have a travis image for local debug but that seems > to be harder to do than I thought. I couldn't find any such images on > the hub.
Fam Zheng <famz@redhat.com> writes: > On Wed, 03/16 09:09, Alex Bennée wrote: >> >> Fam Zheng <famz@redhat.com> writes: >> >> > On Fri, 03/11 16:14, Alex Bennée wrote: >> >> >> >> Fam Zheng <famz@redhat.com> writes: >> >> >> >> > The script is not named test-travis.sh so it won't run with "make >> >> > docker-run", because it can take too long. >> >> > >> >> > Run it with "make docker-run-travis.sh@ubuntu". >> >> >> >> 16:08 alex@zen/x86_64 [qemu.git/review/docker-v3] >make docker-run-travis.sh@ubuntu >> >> ARCHIVE qemu.tgz >> >> COPY RUNNER >> >> RUN travis.sh in ubuntu >> >> ./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or directory >> > >> > Will update the commit message. >> >> I had a bit of a further play with this while trying to help with the >> recent Travis breakage. > > Did this help? Daniel fixed it before I could get something up and running. > >> I realised we need to be clearer about what this >> does. It's not the same as running on travis, just a way of iterating >> through the travis build matrix on whatever image you happen to be on. > > Right. I have no idea how to precisely replicate travis environment, and I > ignored python version, packages etc for simplicity, only the command matrix > was simulated. But are there any major differences you are noticing? At least > we are also on Ubuntu Trusty, the same as: > > https://docs.travis-ci.com/user/ci-environment/ No the current container based CI environment is Precise (Trusty is in beta). I tried creating a precise image from scratch but apt got confused about having both i386 and amd64 packages in package lists and I didn't get the bottom of it before upstream was fixed. > > Fam > >> >> It would be nice to have a travis image for local debug but that seems >> to be harder to do than I thought. I couldn't find any such images on >> the hub. -- Alex Bennée
diff --git a/tests/docker/travis b/tests/docker/travis new file mode 100755 index 0000000..d345393 --- /dev/null +++ b/tests/docker/travis @@ -0,0 +1,21 @@ +#!/bin/bash -e +# +# Mimic a travis testing matrix +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng <famz@redhat.com> +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +. common.rc + +requires pyyaml +cmdfile=/tmp/travis_cmd_list.sh +$QEMU_SRC/tests/docker/travis.py $QEMU_SRC/.travis.yml > $cmdfile +chmod +x $cmdfile +cd "$QEMU_SRC" +$cmdfile diff --git a/tests/docker/travis.py b/tests/docker/travis.py new file mode 100755 index 0000000..8dcc964 --- /dev/null +++ b/tests/docker/travis.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python +# +# Travis YAML config parser +# +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Fam Zheng <famz@redhat.com> +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +import sys +import yaml +import itertools + +def load_yaml(fname): + return yaml.load(open(fname, "r").read()) + +def conf_iter(conf): + def env_to_list(env): + return env if isinstance(env, list) else [env] + global_env = conf["env"]["global"] + for entry in conf["matrix"]["include"]: + yield {"env": global_env + env_to_list(entry["env"]), + "compiler": entry["compiler"]} + for entry in itertools.product(conf["compiler"], + conf["env"]["matrix"]): + yield {"env": global_env + env_to_list(entry[1]), + "compiler": entry[0]} + +def main(): + if len(sys.argv) < 2: + sys.stderr.write("Usage: %s <travis-file>\n" % sys.argv[0]) + return 1 + conf = load_yaml(sys.argv[1]) + for config in conf_iter(conf): + print "(" + print "\n".join(config["env"]) + print "alias cc=" + config["compiler"] + print "\n".join(conf["before_script"]) + print "\n".join(conf["script"]) + print ")" + return 0 + +if __name__ == "__main__": + sys.exit(main())
The script is not named test-travis.sh so it won't run with "make docker-run", because it can take too long. Run it with "make docker-run-travis.sh@ubuntu". Signed-off-by: Fam Zheng <famz@redhat.com> --- tests/docker/travis | 21 +++++++++++++++++++++ tests/docker/travis.py | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100755 tests/docker/travis create mode 100755 tests/docker/travis.py