| Message ID | 20190321113408.19929-1-lersek@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | bundle edk2 platform firmware with QEMU | expand |
On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek <lersek@redhat.com> wrote: > > Repo: https://github.com/lersek/qemu.git > Branch: edk2_build_v3 > > Version 2, that is: > > [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU > > was posted at: > > https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html > http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com > > Updates in v3 are noted on each patch individually, in the Notes > section. In summary, > > - I've picked up feedback tags from the v2 thread (see above), > > - I've replaced the xz compression with bz2 compression according to the > subthread > > Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms Thanks. Could you check that this works in the OpenBSD VM test? It should just be a matter of make -C build vm-build-openbsd (with 'build' replaced with whatever your build directory's name is): it will download the VM image and run it automatically. You'll want to be doing this as a user that can use KVM so that we do the build in a KVM VM rather than a slow emulated one :-)) thanks -- PMM
On 03/21/19 12:52, Peter Maydell wrote: > On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek <lersek@redhat.com> wrote: >> >> Repo: https://github.com/lersek/qemu.git >> Branch: edk2_build_v3 >> >> Version 2, that is: >> >> [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU >> >> was posted at: >> >> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html >> http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com >> >> Updates in v3 are noted on each patch individually, in the Notes >> section. In summary, >> >> - I've picked up feedback tags from the v2 thread (see above), >> >> - I've replaced the xz compression with bz2 compression according to the >> subthread >> >> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms > > Thanks. Could you check that this works in the OpenBSD VM test? > It should just be a matter of > make -C build vm-build-openbsd Actually I tried to do that before posting v3, following the instructions in "docs/devel/testing.rst": cd $QEMU_SRC/tests/vm ./openbsd --build-image --image /mnt/data/tmp/openbsd.img ./openbsd --debug --image /mnt/data/tmp/openbsd.img \ --build-qemu $QEMU_SRC The first "openbsd" invocation succeeded, but the second failed, with a complaint about some (cross?)compiler missing. Anyway I'll now retry with the make command you specify above. > (with 'build' replaced with whatever your build directory's name is): Aha! So first I guess I should run at least configure, for setting up an out-of-tree build, and then try the "make" command. > it will download the VM image and run it automatically. You'll > want to be doing this as a user that can use KVM so that we > do the build in a KVM VM rather than a slow emulated one :-)) Thanks, I'll report back. Laszlo
On Thu, 21 Mar 2019 at 16:55, Laszlo Ersek <lersek@redhat.com> wrote: > On 03/21/19 12:52, Peter Maydell wrote: > > Thanks. Could you check that this works in the OpenBSD VM test? > > It should just be a matter of > > make -C build vm-build-openbsd > > Actually I tried to do that before posting v3, following the > instructions in "docs/devel/testing.rst": > > cd $QEMU_SRC/tests/vm > ./openbsd --build-image --image /mnt/data/tmp/openbsd.img > ./openbsd --debug --image /mnt/data/tmp/openbsd.img \ > --build-qemu $QEMU_SRC This is in the "manual invocation" subsection of the docs. You want the "quickstart" subsection which has the same 'make vm-build-whatever' command I suggest. > Aha! So first I guess I should run at least configure, for setting up an > out-of-tree build, and then try the "make" command. You need to configure, yes. I think it should work on an in-tree build, but I've never tried it as I never use in-tree builds. (We're probably going to get rid of in-tree build support in 4.1.) thanks -- PMM
On 03/21/19 17:55, Laszlo Ersek wrote: > On 03/21/19 12:52, Peter Maydell wrote: >> On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek <lersek@redhat.com> wrote: >>> >>> Repo: https://github.com/lersek/qemu.git >>> Branch: edk2_build_v3 >>> >>> Version 2, that is: >>> >>> [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU >>> >>> was posted at: >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html >>> http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com >>> >>> Updates in v3 are noted on each patch individually, in the Notes >>> section. In summary, >>> >>> - I've picked up feedback tags from the v2 thread (see above), >>> >>> - I've replaced the xz compression with bz2 compression according to the >>> subthread >>> >>> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms >> >> Thanks. Could you check that this works in the OpenBSD VM test? >> It should just be a matter of >> make -C build vm-build-openbsd > > Actually I tried to do that before posting v3, following the > instructions in "docs/devel/testing.rst": > > cd $QEMU_SRC/tests/vm > ./openbsd --build-image --image /mnt/data/tmp/openbsd.img > ./openbsd --debug --image /mnt/data/tmp/openbsd.img \ > --build-qemu $QEMU_SRC > > The first "openbsd" invocation succeeded, but the second failed, with a > complaint about some (cross?)compiler missing. > > Anyway I'll now retry with the make command you specify above. Unfortunately, I got the exact same setup error: > make: Entering directory `.../tmp/qemu-build-opt.d' > VM-IMAGE openbsd > --2019-03-21 18:14:46-- http://download.patchew.org/openbsd-6.1-amd64.img.xz > Resolving download.patchew.org (download.patchew.org)... 140.211.15.4 > Connecting to download.patchew.org (download.patchew.org)|140.211.15.4|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: 675094624 (644M) [application/octet-stream] > Saving to: '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download' > > 100%[=====================================================================================================================================================================================================>] 675,094,624 12.0MB/s in 53s > > 2019-03-21 18:15:40 (12.1 MB/s) - '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download' saved [675094624/675094624] > > Extracting the image... > .../.cache/qemu-vm/images/openbsd.img.tmp.xz (1/1) > 100 % 643.8 MiB / 32.0 GiB = 0.020 253 MiB/s 2:09 > VM-BUILD openbsd > Cloning into '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot'... > done. > Checking out files: 100% (6748/6748), done. > Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc' > Cloning into '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/dtc'... > Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) registered for path 'ui/keycodemapdb' > Cloning into '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/ui/keycodemapdb'... > Submodule 'tests/fp/berkeley-softfloat-3' (https://github.com/cota/berkeley-softfloat-3) registered for path 'tests/fp/berkeley-softfloat-3' > Cloning into '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-softfloat-3'... > Submodule 'tests/fp/berkeley-testfloat-3' (https://github.com/cota/berkeley-testfloat-3) registered for path 'tests/fp/berkeley-testfloat-3' > Cloning into '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-testfloat-3'... > > ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist or does not work > > make: *** [vm-build-openbsd] Error 3 > make: Leaving directory `.../tmp/qemu-build-opt.d' Thanks, Laszlo
Le jeu. 21 mars 2019 18:22, Laszlo Ersek <lersek@redhat.com> a écrit : > On 03/21/19 17:55, Laszlo Ersek wrote: > > On 03/21/19 12:52, Peter Maydell wrote: > >> On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek <lersek@redhat.com> wrote: > >>> > >>> Repo: https://github.com/lersek/qemu.git > >>> Branch: edk2_build_v3 > >>> > >>> Version 2, that is: > >>> > >>> [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU > >>> > >>> was posted at: > >>> > >>> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html > >>> http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com > >>> > >>> Updates in v3 are noted on each patch individually, in the Notes > >>> section. In summary, > >>> > >>> - I've picked up feedback tags from the v2 thread (see above), > >>> > >>> - I've replaced the xz compression with bz2 compression according to > the > >>> subthread > >>> > >>> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms > >> > >> Thanks. Could you check that this works in the OpenBSD VM test? > >> It should just be a matter of > >> make -C build vm-build-openbsd > > > > Actually I tried to do that before posting v3, following the > > instructions in "docs/devel/testing.rst": > > > > cd $QEMU_SRC/tests/vm > > ./openbsd --build-image --image /mnt/data/tmp/openbsd.img > > ./openbsd --debug --image /mnt/data/tmp/openbsd.img \ > > --build-qemu $QEMU_SRC > These isn't the line I remember using with vmtest openbsd. I remember specifying host-cc and python. I guess you can find it in openbsd.py under tests/vm/. > > > The first "openbsd" invocation succeeded, but the second failed, with a > > complaint about some (cross?)compiler missing. > > > > Anyway I'll now retry with the make command you specify above. > > Unfortunately, I got the exact same setup error: > > > make: Entering directory `.../tmp/qemu-build-opt.d' > > VM-IMAGE openbsd > > --2019-03-21 18:14:46-- > http://download.patchew.org/openbsd-6.1-amd64.img.xz > > Resolving download.patchew.org (download.patchew.org)... 140.211.15.4 > > Connecting to download.patchew.org (download.patchew.org)|140.211.15.4|:80... > connected. > > HTTP request sent, awaiting response... 200 OK > > Length: 675094624 (644M) [application/octet-stream] > > Saving to: > '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download' > > > > > 100%[=====================================================================================================================================================================================================>] > 675,094,624 12.0MB/s in 53s > > > > 2019-03-21 18:15:40 (12.1 MB/s) - > '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download' > saved [675094624/675094624] > > > > Extracting the image... > > .../.cache/qemu-vm/images/openbsd.img.tmp.xz (1/1) > > 100 % 643.8 MiB / 32.0 GiB = 0.020 253 MiB/s 2:09 > > VM-BUILD openbsd > > Cloning into > '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot'... > > done. > > Checking out files: 100% (6748/6748), done. > > Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path > 'dtc' > > Cloning into > '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/dtc'... > > Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) > registered for path 'ui/keycodemapdb' > > Cloning into > '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/ui/keycodemapdb'... > > Submodule 'tests/fp/berkeley-softfloat-3' ( > https://github.com/cota/berkeley-softfloat-3) registered for path > 'tests/fp/berkeley-softfloat-3' > > Cloning into > '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-softfloat-3'... > > Submodule 'tests/fp/berkeley-testfloat-3' ( > https://github.com/cota/berkeley-testfloat-3) registered for path > 'tests/fp/berkeley-testfloat-3' > > Cloning into > '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-testfloat-3'... > > > > ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist or > does not work > > > > make: *** [vm-build-openbsd] Error 3 > > make: Leaving directory `.../tmp/qemu-build-opt.d' > To reach shell even on failure you have to use smth like: $ VERBOSE=1 make Or DEBUG=1. If default ./configure && make fails you get shell and can run again with nodefault options. You can check basevm.py in tests/. > Thanks, > Laszlo >
On 03/21/19 21:58, Philippe Mathieu-Daudé wrote: > Le jeu. 21 mars 2019 18:22, Laszlo Ersek <lersek@redhat.com> a écrit : >> Unfortunately, I got the exact same setup error: >>> ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist >>> or does not work > $ VERBOSE=1 make > Or DEBUG=1. If default ./configure && make fails you get shell and can > run again with nodefault options. You can check basevm.py in tests/. I used the "interactive shell" commands from "docs/devel/testing.rst", to look around inside the image. The preinstalled gcc is this one: > qemu-openbsd# gcc --version > gcc (GCC) 4.2.1 20070719 not what the "openbsd" test case refers to. Let's see what I could install myself: > qemu-openbsd# cat /etc/installurl > https://fastly.cdn.openbsd.org/pub/OpenBSD > qemu-openbsd# pkg_info -Q gcc > [...] > gcc-4.9.4p12 > [...] > qemu-openbsd# pkg_add gcc-4.9.4p12 And then this gives me "x86_64-unknown-openbsd6.4-gcc-4.9.4". Again, *not* what the "openbsd" script refers to. Then I look at the "openbsd" script itself, and realize it downloads: > cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz", > sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf') Hmmm. Can I perhaps *browse* the download directory on patchew, in Firefox? I sure can, directory listing is enabled: > README 25-Aug-2017 12:14 0 > freebsd-11.1-amd64.img.xz 03-Aug-2017 09:21 545422116 > netbsd-7.1-amd64.img.xz 03-Aug-2017 09:32 224195948 > openbsd-6.1-amd64.img.xz 27-Feb-2019 11:48 675094624 > openbsd-6.1-amd64.img.xz.old 16-Aug-2017 03:23 622579972 > patchew-db-small.tar.xz 15-Mar-2018 10:17 10451488 There it is. "openbsd-6.1-amd64.img.xz.old" has an mtime of "16-Aug-2017 03:23", which slightly precedes commit fdfaa33291eb ("tests: Add OpenBSD image", 2017-09-22). But someone must have added that ".old" suffix to the filename *recently*, and uploaded "openbsd-6.1-amd64.img.xz" in its place -- see the mtime on "openbsd-6.1-amd64.img.xz": "27-Feb-2019 11:48". I guess the "openbsd" script should have been updated too, in sync. :/ Bonus question: why doesn't the download abort with an sha256sum mismatch? Here's what I get on my end, after manually fetching both images: > 0a836e5013fd502c2c1e14f3f29b25bb1f1ee946abb27d3ca3596d678a728476 openbsd-6.1-amd64.img.xz > 8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf openbsd-6.1-amd64.img.xz.old The "old" image matches the checksum (added in original commit fdfaa33291eb, see above), but the new one doesn't. ... Aaargh, "basevm.py" only uses the checksum for local caching (i.e. for identifying stale vs. up-to-date local copies), not for verification! Cool, so let me try this. I'm going to download the xz.old file manually. Rename it to just xz. It will then match the built-in checksum, and will be used as a cached copy. Then I will try building my series in *that* ("old") VM. Thanks Laszlo
On 03/21/19 23:32, Laszlo Ersek wrote: > Cool, so let me try this. I'm going to download the xz.old file > manually. Rename it to just xz. It will then match the built-in > checksum, and will be used as a cached copy. Then I will try building my > series in *that* ("old") VM. Summary: (1) The image file at <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been recently uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone unknown to me, and its sha256sum doesn't match the sha256sum in the "tests/vm/openbsd" test script. This is why my earlier attempts at the OpenBSD build test have failed. And in fact I don't understand how it could work for anyone else -- the compiler that the "tests/vm/openbsd" script specifies is neither installed, nor available with "pkg_add", in this image. (2) Against the "old" image <http://download.patchew.org/openbsd-6.1-amd64.img.xz.old>, which indeed has the expected sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf, the build test *does* succeed. ( In order to make use of the old image, it has to be downloaded manually, then moved/renamed to: $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd because the last filename component must be the sha1sum of the URL itself, for the caching mechanism to recognize the compressed image: > $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \ > | sha1sum > bc4733f6c6e76931702528a515a1bf70eb8baecd - ) I'm attaching the log of the successful OpenBSD build test, which I captured with "screen" (see the "BUNZIP2" lines in it, in particular). Thanks, Laszlo
Le ven. 22 mars 2019 00:33, Laszlo Ersek <lersek@redhat.com> a écrit : > On 03/21/19 23:32, Laszlo Ersek wrote: > > > Cool, so let me try this. I'm going to download the xz.old file > > manually. Rename it to just xz. It will then match the built-in > > checksum, and will be used as a cached copy. Then I will try building my > > series in *that* ("old") VM. > > Summary: > > (1) The image file at > <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been recently > uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone > unknown to me, and its sha256sum doesn't match the sha256sum in the > "tests/vm/openbsd" test script. > > This is why my earlier attempts at the OpenBSD build test have failed. > Can someone include Fam/Paolo/Brad in this thread please? (I don't have their emails in my cellphone). Thanks. And in fact I don't understand how it could work for anyone else -- the > compiler that the "tests/vm/openbsd" script specifies is neither > installed, nor available with "pkg_add", in this image. > > > (2) Against the "old" image > <http://download.patchew.org/openbsd-6.1-amd64.img.xz.old>, which indeed > has the expected > sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf, > the build test *does* succeed. > > ( > > In order to make use of the old image, it has to be downloaded manually, > then moved/renamed to: > > $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd > > because the last filename component must be the sha1sum of the URL > itself, for the caching mechanism to recognize the compressed image: > > > $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \ > > | sha1sum > > bc4733f6c6e76931702528a515a1bf70eb8baecd - > > ) > > I'm attaching the log of the successful OpenBSD build test, which I > captured with "screen" (see the "BUNZIP2" lines in it, in particular). > > Thanks, > Laszlo >
On 03/22/19 08:02, Philippe Mathieu-Daudé wrote: > Le ven. 22 mars 2019 00:33, Laszlo Ersek <lersek@redhat.com> a écrit : > >> On 03/21/19 23:32, Laszlo Ersek wrote: >> >>> Cool, so let me try this. I'm going to download the xz.old file >>> manually. Rename it to just xz. It will then match the built-in >>> checksum, and will be used as a cached copy. Then I will try building my >>> series in *that* ("old") VM. >> >> Summary: >> >> (1) The image file at >> <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been recently >> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone >> unknown to me, and its sha256sum doesn't match the sha256sum in the >> "tests/vm/openbsd" test script. >> >> This is why my earlier attempts at the OpenBSD build test have failed. >> > > Can someone include Fam/Paolo/Brad in this thread please? (I don't have > their emails in my cellphone). Thanks. Done. Thanks Laszlo >> And in fact I don't understand how it could work for anyone else -- the >> compiler that the "tests/vm/openbsd" script specifies is neither >> installed, nor available with "pkg_add", in this image. >> >> >> (2) Against the "old" image >> <http://download.patchew.org/openbsd-6.1-amd64.img.xz.old>, which indeed >> has the expected >> sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf, >> the build test *does* succeed. >> >> ( >> >> In order to make use of the old image, it has to be downloaded manually, >> then moved/renamed to: >> >> $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd >> >> because the last filename component must be the sha1sum of the URL >> itself, for the caching mechanism to recognize the compressed image: >> >>> $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \ >>> | sha1sum >>> bc4733f6c6e76931702528a515a1bf70eb8baecd - >> >> ) >> >> I'm attaching the log of the successful OpenBSD build test, which I >> captured with "screen" (see the "BUNZIP2" lines in it, in particular). >> >> Thanks, >> Laszlo >> >
On 3/21/19 6:03 PM, Peter Maydell wrote:
> (We're probably going to get rid of in-tree build support in 4.1.)
Sorry for hijicking the thread, but can you point in me the discussion
on the list where was this discussed please? I'm unable to find it. As a
person who does in tree builds only I'm interested to learn more.
Thanks,
Michal
On Mon, 25 Mar 2019 at 08:29, Michal Privoznik <mprivozn@redhat.com> wrote: > > On 3/21/19 6:03 PM, Peter Maydell wrote: > > (We're probably going to get rid of in-tree build support in 4.1.) > > Sorry for hijicking the thread, but can you point in me the discussion > on the list where was this discussed please? I'm unable to find it. As a > person who does in tree builds only I'm interested to learn more. https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg01949.html and subthread. The short reasoning is that there's nothing you can do with an in-tree build that you can't do with an out-of-tree build (whereas the reverse is not true), and maintaining both mechanisms leads to lots of bugs where we accidentally break one of them because the developers are only using the other kind. So in-tree builds are more pain to support than they're worth. thanks -- PMM
Hi Peter, can you please comment: On 03/22/19 10:17, Laszlo Ersek wrote: > On 03/22/19 08:02, Philippe Mathieu-Daudé wrote: >> Le ven. 22 mars 2019 00:33, Laszlo Ersek <lersek@redhat.com> a écrit : >> >>> On 03/21/19 23:32, Laszlo Ersek wrote: >>> >>>> Cool, so let me try this. I'm going to download the xz.old file >>>> manually. Rename it to just xz. It will then match the built-in >>>> checksum, and will be used as a cached copy. Then I will try building my >>>> series in *that* ("old") VM. >>> >>> Summary: >>> >>> (1) The image file at >>> <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been recently >>> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone >>> unknown to me, and its sha256sum doesn't match the sha256sum in the >>> "tests/vm/openbsd" test script. >>> >>> This is why my earlier attempts at the OpenBSD build test have failed. >>> >> >> Can someone include Fam/Paolo/Brad in this thread please? (I don't have >> their emails in my cellphone). Thanks. > > Done. - do we have any idea what happened on download.patchew.org (i.e. why the image matching the script was replaced with an image not matching the script)? >>> And in fact I don't understand how it could work for anyone else -- the >>> compiler that the "tests/vm/openbsd" script specifies is neither >>> installed, nor available with "pkg_add", in this image. >>> >>> >>> (2) Against the "old" image >>> <http://download.patchew.org/openbsd-6.1-amd64.img.xz.old>, which indeed >>> has the expected >>> sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf, >>> the build test *does* succeed. >>> >>> ( >>> >>> In order to make use of the old image, it has to be downloaded manually, >>> then moved/renamed to: >>> >>> $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd >>> >>> because the last filename component must be the sha1sum of the URL >>> itself, for the caching mechanism to recognize the compressed image: >>> >>>> $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \ >>>> | sha1sum >>>> bc4733f6c6e76931702528a515a1bf70eb8baecd - >>> >>> ) >>> >>> I'm attaching the log of the successful OpenBSD build test, which I >>> captured with "screen" (see the "BUNZIP2" lines in it, in particular). - are you satisfied with the OpenBSD build testing I did (using the "old" image, matching the script)? Thanks Laszlo
On 25/03/19 11:40, Laszlo Ersek wrote: >>>> >>>> (1) The image file at >>>> <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been recently >>>> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone >>>> unknown to me, and its sha256sum doesn't match the sha256sum in the >>>> "tests/vm/openbsd" test script. >>>> >>>> This is why my earlier attempts at the OpenBSD build test have failed. >>>> >>> Can someone include Fam/Paolo/Brad in this thread please? (I don't have >>> their emails in my cellphone). Thanks. >> Done. > - do we have any idea what happened on download.patchew.org (i.e. why > the image matching the script was replaced with an image not matching > the script)? The update was requested by Daniel in this thread: https://patchew.org/QEMU/20181031025712.18602-1-famz@redhat.com/ Fam did the update, but forgot to update the sha256sum. Paolo
(Keeping Fam on the address list, adding Dan & Markus) On 03/27/19 14:18, Paolo Bonzini wrote: > On 25/03/19 11:40, Laszlo Ersek wrote: >>>>> >>>>> (1) The image file at >>>>> <http://download.patchew.org/openbsd-6.1-amd64.img.xz> has been >>>>> recently uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") >>>>> by someone unknown to me, and its sha256sum doesn't match the >>>>> sha256sum in the "tests/vm/openbsd" test script. >>>>> >>>>> This is why my earlier attempts at the OpenBSD build test have >>>>> failed. >>>>> >>>> Can someone include Fam/Paolo/Brad in this thread please? (I don't >>>> have their emails in my cellphone). Thanks. >>> Done. >> - do we have any idea what happened on download.patchew.org (i.e. why >> the image matching the script was replaced with an image not matching >> the script)? > > The update was requested by Daniel in this thread: > https://patchew.org/QEMU/20181031025712.18602-1-famz@redhat.com/ > > Fam did the update, but forgot to update the sha256sum. Thank you for tracking this down! But, it's not just the sha256sum that needs an update in "tests/vm/openbsd". In addition to that, the script refers to the compiler under the name x86_64-unknown-openbsd6.1-gcc-4.9.4 This compiler is not available in the updated OpenBSD disk image, either pre-installed, or from the network. Like I described here: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06277.html the preinstalled compiler ("gcc") is version 4.2.1, and the one that's available from the network, with "pkg_add", is called differently: x86_64-unknown-openbsd6.4-gcc-4.9.4 So, my take is that *both* the script is out-of-sync with the new OpenBSD image, *and* the OpenBSD image is unusable (for this particular test) and has never been verified. ... Which in turn raises the question: *before* Peter reported the "xz" failure first, against my series, at https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html what image had Peter been using? Because, as far as I see it, at that point there was *no* way for the OpenBSD build test to succeed, *regardless* of my series. The compiler must not have been available. In fact, what OpenBSD image is Peter using *right now*, for the build-QEMU-on-OpenBSD test? Again, I see no way how that could succeed right now. ... Honestly, given that the OpenBSD image was broken & out-of-sync to begin with, I feel it was questionable to block my series (= Phil's PULL) from 4.0, just because the series placed an *additional* requirement (namely "xz") on the OpenBSD image, which had *already* been in sore need of a functional C compiler. :/ ---*--- OK, let me be constructive here. Right now, the *current* "tests/vm/openbsd" script is unusable for *any* testing. (My testing did succeed agains the previous image, which is still available under a different name.) Who's going to fix the current image, and who's going to fix the script? And when? Regarding my own series, after all this churn, I'll stick with using bz2 in it; the xz->bz2 storage "loss" is minimal, about 1%. Thanks, Laszlo
On 27/03/19 15:31, Laszlo Ersek wrote: > OK, let me be constructive here. Right now, the *current* > "tests/vm/openbsd" script is unusable for *any* testing. (My testing did > succeed agains the previous image, which is still available under a > different name.) Who's going to fix the current image, and who's going > to fix the script? And when? For now I'll just revert the image to the old one. Paolo
On 03/27/19 16:06, Paolo Bonzini wrote: > On 27/03/19 15:31, Laszlo Ersek wrote: >> OK, let me be constructive here. Right now, the *current* >> "tests/vm/openbsd" script is unusable for *any* testing. (My testing did >> succeed agains the previous image, which is still available under a >> different name.) Who's going to fix the current image, and who's going >> to fix the script? And when? > > For now I'll just revert the image to the old one. Thank you! Should I propagate Daniel's original request to a Launchpad ticket? For example: Please install SDL 2 development packages in the OpenBSD image, because SDL 1.2 support has been removed in qemu-4.0 -- see <http://mid.mail-archive.com/20190122161236.GZ13143@redhat.com> and <https://wiki.qemu.org/ChangeLog/4.0#GUI>. Thank you again! Laszlo
On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek <lersek@redhat.com> wrote: > ... Which in turn raises the question: *before* Peter reported the "xz" > failure first, against my series, at > > https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html > > what image had Peter been using? Because, as far as I see it, at > that point there was *no* way for the OpenBSD build test to succeed, > *regardless* of my series. The compiler must not have been available. The tests/vm scripts cache the downloaded image. So my setup presumably has the old image cached and is continuing to use it. This is good for speed and avoiding lots of downloads, but it looks like it's had the unfortunate side effect that nobody noticed the breakage because everybody doing tests already had the old image in their local cache and wasn't using the broken new image :-( thanks -- PMM
On Wed, Mar 27, 2019 at 03:25:19PM +0000, Peter Maydell wrote: > On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek <lersek@redhat.com> wrote: > > ... Which in turn raises the question: *before* Peter reported the "xz" > > failure first, against my series, at > > > > https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html > > > > what image had Peter been using? Because, as far as I see it, at > > that point there was *no* way for the OpenBSD build test to succeed, > > *regardless* of my series. The compiler must not have been available. > > The tests/vm scripts cache the downloaded image. So my setup > presumably has the old image cached and is continuing to use it. > This is good for speed and avoiding lots of downloads, but it > looks like it's had the unfortunate side effect that nobody > noticed the breakage because everybody doing tests already > had the old image in their local cache and wasn't using the > broken new image :-( Perhaps the VM test scripts should do a "HEAD" request for the image every time to discover if it has been changed on the server, before honouring the local cache. Regards, Daniel
On 03/27/19 16:25, Peter Maydell wrote: > On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek <lersek@redhat.com> wrote: >> ... Which in turn raises the question: *before* Peter reported the "xz" >> failure first, against my series, at >> >> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html >> >> what image had Peter been using? Because, as far as I see it, at >> that point there was *no* way for the OpenBSD build test to succeed, >> *regardless* of my series. The compiler must not have been available. > > The tests/vm scripts cache the downloaded image. So my setup > presumably has the old image cached and is continuing to use it. > This is good for speed and avoiding lots of downloads, but it > looks like it's had the unfortunate side effect that nobody > noticed the breakage because everybody doing tests already > had the old image in their local cache and wasn't using the > broken new image :-( Mistery solved! Thank you very much for responding! Laszlo
On 03/27/19 16:30, Daniel P. Berrangé wrote: > On Wed, Mar 27, 2019 at 03:25:19PM +0000, Peter Maydell wrote: >> On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek <lersek@redhat.com> wrote: >>> ... Which in turn raises the question: *before* Peter reported the "xz" >>> failure first, against my series, at >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html >>> >>> what image had Peter been using? Because, as far as I see it, at >>> that point there was *no* way for the OpenBSD build test to succeed, >>> *regardless* of my series. The compiler must not have been available. >> >> The tests/vm scripts cache the downloaded image. So my setup >> presumably has the old image cached and is continuing to use it. >> This is good for speed and avoiding lots of downloads, but it >> looks like it's had the unfortunate side effect that nobody >> noticed the breakage because everybody doing tests already >> had the old image in their local cache and wasn't using the >> broken new image :-( > > Perhaps the VM test scripts should do a "HEAD" request for the image > every time to discover if it has been changed on the server, before > honouring the local cache. "curl -I" can do that. (Sorry if that's obvious -- I'm mentioning it because I used "curl -I" myself manually, as part of the earlier "investigation".) Thanks Laszlo
On 27/03/19 16:30, Daniel P. Berrangé wrote: > Perhaps the VM test scripts should do a "HEAD" request for the image > every time to discover if it has been changed on the server, before > honouring the local cache. Another possibility is to first download the shasum from download.patchew.org, and compare _that_ against the one that is stored locally, instead of hardcoding it in QEMU's repository. Paolo
On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: > On 27/03/19 16:30, Daniel P. Berrangé wrote: > > Perhaps the VM test scripts should do a "HEAD" request for the image > > every time to discover if it has been changed on the server, before > > honouring the local cache. > > Another possibility is to first download the shasum from > download.patchew.org, and compare _that_ against the one that is stored > locally, instead of hardcoding it in QEMU's repository. Personally I prefer the idea of having the shasum stored in the repo. That means that if we update git master to point to a newer image, previous stable branches will stick with their original image, rather than using a new image that may be incompatible with the stable branch Storing hash in git also means that if someone compromised the patchew server, they can't cause developer to run compromised images, without first also compromising git to change the hash. Regards, Daniel
On 27/03/19 17:05, Daniel P. Berrangé wrote: > On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: >> On 27/03/19 16:30, Daniel P. Berrangé wrote: >>> Perhaps the VM test scripts should do a "HEAD" request for the image >>> every time to discover if it has been changed on the server, before >>> honouring the local cache. >> >> Another possibility is to first download the shasum from >> download.patchew.org, and compare _that_ against the one that is stored >> locally, instead of hardcoding it in QEMU's repository. > > Personally I prefer the idea of having the shasum stored in the repo. > > That means that if we update git master to point to a newer image, > previous stable branches will stick with their original image, rather > than using a new image that may be incompatible with the stable branch > > Storing hash in git also means that if someone compromised the patchew > server, they can't cause developer to run compromised images, without > first also compromising git to change the hash. The two are not mutually exclusive. We can warn if the hash doesn't match against the one in QEMU, add a --force option, or whatever. Also, I have now created symlinks by hash at http://download.patchew.org/by-sha256sum in case someone finds them useful. Paolo
On 03/27/19 17:15, Paolo Bonzini wrote: > On 27/03/19 17:05, Daniel P. Berrangé wrote: >> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: >>> On 27/03/19 16:30, Daniel P. Berrangé wrote: >>>> Perhaps the VM test scripts should do a "HEAD" request for the image >>>> every time to discover if it has been changed on the server, before >>>> honouring the local cache. >>> >>> Another possibility is to first download the shasum from >>> download.patchew.org, and compare _that_ against the one that is stored >>> locally, instead of hardcoding it in QEMU's repository. >> >> Personally I prefer the idea of having the shasum stored in the repo. >> >> That means that if we update git master to point to a newer image, >> previous stable branches will stick with their original image, rather >> than using a new image that may be incompatible with the stable branch >> >> Storing hash in git also means that if someone compromised the patchew >> server, they can't cause developer to run compromised images, without >> first also compromising git to change the hash. > > The two are not mutually exclusive. We can warn if the hash doesn't > match against the one in QEMU, add a --force option, or whatever. > > Also, I have now created symlinks by hash at > http://download.patchew.org/by-sha256sum in case someone finds them useful. Isn't this risky? If someone replaces an image file (keeping its name), the old symlink will continue "working", but the hash stated by the symlink's name will not match the pointed-to image file. It would be nice to enforce a hash update on upload, or addressing files by content. ... Are we inventing a git submodule? :) Laszlo
On 27/03/19 17:24, Laszlo Ersek wrote: > On 03/27/19 17:15, Paolo Bonzini wrote: >> On 27/03/19 17:05, Daniel P. Berrangé wrote: >>> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: >>>> On 27/03/19 16:30, Daniel P. Berrangé wrote: >>>>> Perhaps the VM test scripts should do a "HEAD" request for the image >>>>> every time to discover if it has been changed on the server, before >>>>> honouring the local cache. >>>> >>>> Another possibility is to first download the shasum from >>>> download.patchew.org, and compare _that_ against the one that is stored >>>> locally, instead of hardcoding it in QEMU's repository. >>> >>> Personally I prefer the idea of having the shasum stored in the repo. >>> >>> That means that if we update git master to point to a newer image, >>> previous stable branches will stick with their original image, rather >>> than using a new image that may be incompatible with the stable branch >>> >>> Storing hash in git also means that if someone compromised the patchew >>> server, they can't cause developer to run compromised images, without >>> first also compromising git to change the hash. >> >> The two are not mutually exclusive. We can warn if the hash doesn't >> match against the one in QEMU, add a --force option, or whatever. >> >> Also, I have now created symlinks by hash at >> http://download.patchew.org/by-sha256sum in case someone finds them useful. > > Isn't this risky? If someone replaces an image file (keeping its name), > the old symlink will continue "working", but the hash stated by the > symlink's name will not match the pointed-to image file. Well, the idea is that if you use them you also double-check what you downloaded. :) Paolo
On 03/27/2019 01:15 PM, Paolo Bonzini wrote: > On 27/03/19 17:05, Daniel P. Berrangé wrote: >> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: >>> On 27/03/19 16:30, Daniel P. Berrangé wrote: >>>> Perhaps the VM test scripts should do a "HEAD" request for the image >>>> every time to discover if it has been changed on the server, before >>>> honouring the local cache. >>> Another possibility is to first download the shasum from >>> download.patchew.org, and compare _that_ against the one that is stored >>> locally, instead of hardcoding it in QEMU's repository. >> Personally I prefer the idea of having the shasum stored in the repo. >> >> That means that if we update git master to point to a newer image, >> previous stable branches will stick with their original image, rather >> than using a new image that may be incompatible with the stable branch >> >> Storing hash in git also means that if someone compromised the patchew >> server, they can't cause developer to run compromised images, without >> first also compromising git to change the hash. > The two are not mutually exclusive. We can warn if the hash doesn't > match against the one in QEMU, add a --force option, or whatever. I'm about to send a patch to make vm-test work with Python3. I can work on that image checking mechanism you folks have discussed, unless someone is already working on it. - Wainer > > Also, I have now created symlinks by hash at > http://download.patchew.org/by-sha256sum in case someone finds them useful. > > Paolo > >
On 27/03/19 19:08, Wainer dos Santos Moschetta wrote: > > > On 03/27/2019 01:15 PM, Paolo Bonzini wrote: >> On 27/03/19 17:05, Daniel P. Berrangé wrote: >>> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: >>>> On 27/03/19 16:30, Daniel P. Berrangé wrote: >>>>> Perhaps the VM test scripts should do a "HEAD" request for the image >>>>> every time to discover if it has been changed on the server, before >>>>> honouring the local cache. >>>> Another possibility is to first download the shasum from >>>> download.patchew.org, and compare _that_ against the one that is stored >>>> locally, instead of hardcoding it in QEMU's repository. >>> Personally I prefer the idea of having the shasum stored in the repo. >>> >>> That means that if we update git master to point to a newer image, >>> previous stable branches will stick with their original image, rather >>> than using a new image that may be incompatible with the stable branch >>> >>> Storing hash in git also means that if someone compromised the patchew >>> server, they can't cause developer to run compromised images, without >>> first also compromising git to change the hash. >> The two are not mutually exclusive. We can warn if the hash doesn't >> match against the one in QEMU, add a --force option, or whatever. > > I'm about to send a patch to make vm-test work with Python3. I can work > on that image checking mechanism you folks have discussed, unless > someone is already working on it. Absolutely not, go for it! Paolo
Le mer. 27 mars 2019 17:06, Daniel P. Berrangé <berrange@redhat.com> a écrit : > On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: > > On 27/03/19 16:30, Daniel P. Berrangé wrote: > > > Perhaps the VM test scripts should do a "HEAD" request for the image > > > every time to discover if it has been changed on the server, before > > > honouring the local cache. > > > > Another possibility is to first download the shasum from > > download.patchew.org, and compare _that_ against the one that is stored > > locally, instead of hardcoding it in QEMU's repository. > > Personally I prefer the idea of having the shasum stored in the repo. > > That means that if we update git master to point to a newer image, > previous stable branches will stick with their original image, rather > than using a new image that may be incompatible with the stable branch > > Storing hash in git also means that if someone compromised the patchew > server, they can't cause developer to run compromised images, without > first also compromising git to change the hash. > And we can git bisect too, right? > Regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| >
On 27/03/19 21:27, Philippe Mathieu-Daudé wrote: > > > Le mer. 27 mars 2019 17:06, Daniel P. Berrangé <berrange@redhat.com > <mailto:berrange@redhat.com>> a écrit : > > On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote: > > On 27/03/19 16:30, Daniel P. Berrangé wrote: > > > Perhaps the VM test scripts should do a "HEAD" request for the image > > > every time to discover if it has been changed on the server, before > > > honouring the local cache. > > > > Another possibility is to first download the shasum from > > download.patchew.org <http://download.patchew.org>, and compare > _that_ against the one that is stored > > locally, instead of hardcoding it in QEMU's repository. > > Personally I prefer the idea of having the shasum stored in the repo. > > That means that if we update git master to point to a newer image, > previous stable branches will stick with their original image, rather > than using a new image that may be incompatible with the stable branch > > Storing hash in git also means that if someone compromised the patchew > server, they can't cause developer to run compromised images, without > first also compromising git to change the hash. > > > And we can git bisect too, right? Only if the image is never overwritten with another one with the same name (which was also the idea behind storing the files by hash on the server). Paolo
Repo: https://github.com/lersek/qemu.git Branch: edk2_build_v3 Version 2, that is: [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU was posted at: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com Updates in v3 are noted on each patch individually, in the Notes section. In summary, - I've picked up feedback tags from the v2 thread (see above), - I've replaced the xz compression with bz2 compression according to the subthread Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms at: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06016.html http://mid.mail-archive.com/CAFEAcA-UcX6Brv3O28vcCw31bt=QpvbUwnPBBjhETm=Vfciuvw@mail.gmail.com I'm pasting a cumulative diffstat, and diff, between v2 and v3 below. Note that a normal diffstat for this v3 series is at the bottom of the cover letter, as usual. > Makefile | 6 +++--- > configure | 2 +- > pc-bios/README | 2 +- > pc-bios/edk2-aarch64-code.fd.bz2 | Bin 0 -> 1177603 bytes > pc-bios/edk2-aarch64-code.fd.xz | Bin 1146804 -> 0 bytes > pc-bios/edk2-arm-code.fd.bz2 | Bin 0 -> 1173662 bytes > pc-bios/edk2-arm-code.fd.xz | Bin 1147852 -> 0 bytes > pc-bios/edk2-arm-vars.fd.bz2 | Bin 0 -> 263 bytes > pc-bios/edk2-arm-vars.fd.xz | Bin 10008 -> 0 bytes > pc-bios/edk2-i386-code.fd.bz2 | Bin 0 -> 1688659 bytes > pc-bios/edk2-i386-code.fd.xz | Bin 1674764 -> 0 bytes > pc-bios/edk2-i386-secure-code.fd.bz2 | Bin 0 -> 1881979 bytes > pc-bios/edk2-i386-secure-code.fd.xz | Bin 1870024 -> 0 bytes > pc-bios/edk2-i386-vars.fd.bz2 | Bin 0 -> 190 bytes > pc-bios/edk2-i386-vars.fd.xz | Bin 320 -> 0 bytes > pc-bios/edk2-x86_64-code.fd.bz2 | Bin 0 -> 1669280 bytes > pc-bios/edk2-x86_64-code.fd.xz | Bin 1655276 -> 0 bytes > pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes > pc-bios/edk2-x86_64-secure-code.fd.xz | Bin 1889024 -> 0 bytes > roms/Makefile.edk2 | 6 +++--- > 20 files changed, 8 insertions(+), 8 deletions(-) > diff --git a/configure b/configure > index 5061f68653a3..df6e01ada833 100755 > --- a/configure > +++ b/configure > @@ -7865,7 +7865,7 @@ for bios_file in \ > $source_path/pc-bios/*.img \ > $source_path/pc-bios/openbios-* \ > $source_path/pc-bios/u-boot.* \ > - $source_path/pc-bios/edk2-*.fd.xz \ > + $source_path/pc-bios/edk2-*.fd.bz2 \ > $source_path/pc-bios/palcode-* > do > LINKS="$LINKS pc-bios/$(basename $bios_file)" > diff --git a/Makefile b/Makefile > index 6c5c1a1cad6e..f4562092e8dc 100644 > --- a/Makefile > +++ b/Makefile > @@ -296,9 +296,9 @@ ui/input-keymap-%.c: $(KEYCODEMAP_GEN) $(KEYCODEMAP_CSV) $(SRC_PATH)/ui/Makefile > $(KEYCODEMAP_GEN): .git-submodule-status > $(KEYCODEMAP_CSV): .git-submodule-status > > -edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.xz)) > -pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.xz > - $(call quiet-command,xz -d -c $< > $@,"UNXZ",$<) > +edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.bz2)) > +pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.bz2 > + $(call quiet-command,bzip2 -d -c $< > $@,"BUNZIP2",$<) > > # Don't try to regenerate Makefile or configure > # We don't generate any of them > diff --git a/roms/Makefile.edk2 b/roms/Makefile.edk2 > index e5c3036949c8..822c547fec64 100644 > --- a/roms/Makefile.edk2 > +++ b/roms/Makefile.edk2 > @@ -34,11 +34,11 @@ flashdevs := \ > arm-vars \ > i386-vars > > -all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.xz) \ > +all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \ > ../pc-bios/edk2-licenses.txt > > -../pc-bios/edk2-%.fd.xz: ../pc-bios/edk2-%.fd > - xz -9 -e -c $< > $@ > +../pc-bios/edk2-%.fd.bz2: ../pc-bios/edk2-%.fd > + bzip2 -9 -c $< > $@ > > # When the build completes, we need not keep the uncompressed flash device > # files. > diff --git a/pc-bios/README b/pc-bios/README > index f6acd3209610..807d95dd5418 100644 > --- a/pc-bios/README > +++ b/pc-bios/README > @@ -51,7 +51,7 @@ > - QemuMacDrivers (https://github.com/ozbenh/QemuMacDrivers) is a project to > provide virtualised drivers for PPC MacOS guests. > > -- The "edk2-*.fd.xz" images are platform firmware binaries and matching UEFI > +- The "edk2-*.fd.bz2" images are platform firmware binaries and matching UEFI > variable store templates built from the TianoCore community's EFI Development > Kit II project > <https://github.com/tianocore/tianocore.github.io/wiki/EDK-II>. The images > diff --git a/pc-bios/edk2-aarch64-code.fd.bz2 b/pc-bios/edk2-aarch64-code.fd.bz2 > new file mode 100644 > index 000000000000..b213334d9553 > Binary files /dev/null and b/pc-bios/edk2-aarch64-code.fd.bz2 differ > diff --git a/pc-bios/edk2-aarch64-code.fd.xz b/pc-bios/edk2-aarch64-code.fd.xz > deleted file mode 100644 > index 8ca34c56b51b..000000000000 > Binary files a/pc-bios/edk2-aarch64-code.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-arm-code.fd.bz2 b/pc-bios/edk2-arm-code.fd.bz2 > new file mode 100644 > index 000000000000..a494a8391eed > Binary files /dev/null and b/pc-bios/edk2-arm-code.fd.bz2 differ > diff --git a/pc-bios/edk2-arm-code.fd.xz b/pc-bios/edk2-arm-code.fd.xz > deleted file mode 100644 > index db2c20b71e29..000000000000 > Binary files a/pc-bios/edk2-arm-code.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-arm-vars.fd.bz2 b/pc-bios/edk2-arm-vars.fd.bz2 > new file mode 100644 > index 000000000000..2a5a9af31a6c > Binary files /dev/null and b/pc-bios/edk2-arm-vars.fd.bz2 differ > diff --git a/pc-bios/edk2-arm-vars.fd.xz b/pc-bios/edk2-arm-vars.fd.xz > deleted file mode 100644 > index 8c3a4b5c8bdc..000000000000 > Binary files a/pc-bios/edk2-arm-vars.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-i386-code.fd.bz2 b/pc-bios/edk2-i386-code.fd.bz2 > new file mode 100644 > index 000000000000..5440ca856db6 > Binary files /dev/null and b/pc-bios/edk2-i386-code.fd.bz2 differ > diff --git a/pc-bios/edk2-i386-code.fd.xz b/pc-bios/edk2-i386-code.fd.xz > deleted file mode 100644 > index f25dbec37e72..000000000000 > Binary files a/pc-bios/edk2-i386-code.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-i386-secure-code.fd.bz2 b/pc-bios/edk2-i386-secure-code.fd.bz2 > new file mode 100644 > index 000000000000..24ae88ac3e40 > Binary files /dev/null and b/pc-bios/edk2-i386-secure-code.fd.bz2 differ > diff --git a/pc-bios/edk2-i386-secure-code.fd.xz b/pc-bios/edk2-i386-secure-code.fd.xz > deleted file mode 100644 > index 11d499b90d85..000000000000 > Binary files a/pc-bios/edk2-i386-secure-code.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-i386-vars.fd.bz2 b/pc-bios/edk2-i386-vars.fd.bz2 > new file mode 100644 > index 000000000000..4a1bc4523880 > Binary files /dev/null and b/pc-bios/edk2-i386-vars.fd.bz2 differ > diff --git a/pc-bios/edk2-i386-vars.fd.xz b/pc-bios/edk2-i386-vars.fd.xz > deleted file mode 100644 > index 1656532daa6e..000000000000 > Binary files a/pc-bios/edk2-i386-vars.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-x86_64-code.fd.bz2 b/pc-bios/edk2-x86_64-code.fd.bz2 > new file mode 100644 > index 000000000000..6a0b1f983199 > Binary files /dev/null and b/pc-bios/edk2-x86_64-code.fd.bz2 differ > diff --git a/pc-bios/edk2-x86_64-code.fd.xz b/pc-bios/edk2-x86_64-code.fd.xz > deleted file mode 100644 > index 92398935a482..000000000000 > Binary files a/pc-bios/edk2-x86_64-code.fd.xz and /dev/null differ > diff --git a/pc-bios/edk2-x86_64-secure-code.fd.bz2 b/pc-bios/edk2-x86_64-secure-code.fd.bz2 > new file mode 100644 > index 000000000000..3ef6452c0375 > Binary files /dev/null and b/pc-bios/edk2-x86_64-secure-code.fd.bz2 differ > diff --git a/pc-bios/edk2-x86_64-secure-code.fd.xz b/pc-bios/edk2-x86_64-secure-code.fd.xz > deleted file mode 100644 > index a95891dcf134..000000000000 > Binary files a/pc-bios/edk2-x86_64-secure-code.fd.xz and /dev/null differ Thanks, Laszlo Laszlo Ersek (12): roms: lift "edk2-funcs.sh" from "tests/uefi-test-tools/build.sh" roms/edk2-funcs.sh: require gcc-4.8+ for building i386 and x86_64 tests/uefi-test-tools/build.sh: work around TianoCore#1607 roms/edk2: advance to tag edk2-stable201903 roms/edk2-funcs.sh: add the qemu_edk2_get_thread_count() function roms/Makefile: replace the $(EFIROM) target with "edk2-basetools" roms: build edk2 firmware binaries and variable store templates pc-bios: add edk2 firmware binaries and variable store templates pc-bios: document the edk2 firmware images; add firmware descriptors tests: add missing dependency to build QTEST_QEMU_BINARY, round 2 Makefile: install the edk2 firmware images and their descriptors MAINTAINERS: add the "EDK2 Firmware" subsystem .gitignore | 1 + MAINTAINERS | 12 + Makefile | 29 ++- configure | 1 + pc-bios/README | 11 + pc-bios/descriptors/50-edk2-i386-secure.json | 34 +++ pc-bios/descriptors/50-edk2-x86_64-secure.json | 35 +++ pc-bios/descriptors/60-edk2-aarch64.json | 31 +++ pc-bios/descriptors/60-edk2-arm.json | 31 +++ pc-bios/descriptors/60-edk2-i386.json | 33 +++ pc-bios/descriptors/60-edk2-x86_64.json | 34 +++ pc-bios/edk2-aarch64-code.fd.bz2 | Bin 0 -> 1177603 bytes pc-bios/edk2-arm-code.fd.bz2 | Bin 0 -> 1173662 bytes pc-bios/edk2-arm-vars.fd.bz2 | Bin 0 -> 263 bytes pc-bios/edk2-i386-code.fd.bz2 | Bin 0 -> 1688659 bytes pc-bios/edk2-i386-secure-code.fd.bz2 | Bin 0 -> 1881979 bytes pc-bios/edk2-i386-vars.fd.bz2 | Bin 0 -> 190 bytes pc-bios/edk2-licenses.txt | 209 ++++++++++++++++ pc-bios/edk2-x86_64-code.fd.bz2 | Bin 0 -> 1669280 bytes pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes roms/Makefile | 9 +- roms/Makefile.edk2 | 148 ++++++++++++ roms/edk2 | 2 +- roms/edk2-build.sh | 55 +++++ roms/edk2-funcs.sh | 253 ++++++++++++++++++++ tests/Makefile.include | 2 +- tests/uefi-test-tools/build.sh | 100 +------- 27 files changed, 934 insertions(+), 96 deletions(-) create mode 100644 pc-bios/descriptors/50-edk2-i386-secure.json create mode 100644 pc-bios/descriptors/50-edk2-x86_64-secure.json create mode 100644 pc-bios/descriptors/60-edk2-aarch64.json create mode 100644 pc-bios/descriptors/60-edk2-arm.json create mode 100644 pc-bios/descriptors/60-edk2-i386.json create mode 100644 pc-bios/descriptors/60-edk2-x86_64.json create mode 100644 pc-bios/edk2-aarch64-code.fd.bz2 create mode 100644 pc-bios/edk2-arm-code.fd.bz2 create mode 100644 pc-bios/edk2-arm-vars.fd.bz2 create mode 100644 pc-bios/edk2-i386-code.fd.bz2 create mode 100644 pc-bios/edk2-i386-secure-code.fd.bz2 create mode 100644 pc-bios/edk2-i386-vars.fd.bz2 create mode 100644 pc-bios/edk2-licenses.txt create mode 100644 pc-bios/edk2-x86_64-code.fd.bz2 create mode 100644 pc-bios/edk2-x86_64-secure-code.fd.bz2 create mode 100644 roms/Makefile.edk2 create mode 100755 roms/edk2-build.sh create mode 100644 roms/edk2-funcs.sh