| Message ID | 20211119171241.102173-3-bage@debian.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | debian: Fix xfsprogs FT(C)BFS | expand |
On Fri, Nov 19, 2021 at 06:12:41PM +0100, Bastian Germann wrote: > xfsprogs fails to cross build from source, because it attempts to build > its crc32 test with the build architecture compiler and thus fails > finding the liburcu, which is only requested for the host architecture. > While this test is useful for native builds, it is not that useful for > cross builds. Skip it by pre-creating the output file. I don't get it. The crcselftest does not use liburcu in any way, nor does it try to link against liburcu, so it should not fail because other parts of xfsprogs use liburcu. What's the build error that occurs? > Reported-by: Helmut Grohne <helmut@subdivi.de> > Signed-off-by: Bastian Germann <bage@debian.org> > --- > debian/rules | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/debian/rules b/debian/rules > index 6d5b82a8..28cc414d 100755 > --- a/debian/rules > +++ b/debian/rules > @@ -40,6 +40,9 @@ build-arch: built > build-indep: built > built: dibuild config > @echo "== dpkg-buildpackage: build" 1>&2 > +ifneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH)) > + touch --date=+3day libfrog/crc32selftest > +endif > $(MAKE) $(PMAKEFLAGS) default > touch built We need to fix the generic cross-build problem in the xfsprogs code, not slap a distro-specific build band-aid over it. Cheers, Dave.
Hi Dave, On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > I don't get it. The crcselftest does not use liburcu in > any way, nor does it try to link against liburcu, so it should not > fail because other parts of xfsprogs use liburcu. > > What's the build error that occurs? As the build log shows, that's not technically accurate. You can find logs of test builds for various architecture combinations at http://crossqa.debian.net/src/xfsprogs. This is also available as a link called "cross" in https://tracker.debian.org/xfsprogs. The relevant part is: | [TEST] CRC32 | In file included from crc32.c:35: | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory | 27 | #include <urcu.h> | | ^~~~~~~~ | compilation terminated. I failed to figure a good way of dropping either include directive. > We need to fix the generic cross-build problem in the xfsprogs code, > not slap a distro-specific build band-aid over it. I fully agree with this in principle. However, when I fail to find that upstreamable solution, I try to at least provide a Debian-specific solution to iterate from. Can you propose a way to drop either #include? Helmut
On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > Hi Dave, > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > I don't get it. The crcselftest does not use liburcu in > > any way, nor does it try to link against liburcu, so it should not > > fail because other parts of xfsprogs use liburcu. > > > > What's the build error that occurs? > > As the build log shows, that's not technically accurate. You can find > logs of test builds for various architecture combinations at > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > called "cross" in https://tracker.debian.org/xfsprogs. > > The relevant part is: > | [TEST] CRC32 > | In file included from crc32.c:35: > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > | 27 | #include <urcu.h> > | | ^~~~~~~~ > | compilation terminated. > > I failed to figure a good way of dropping either include directive. > > > We need to fix the generic cross-build problem in the xfsprogs code, > > not slap a distro-specific build band-aid over it. > > I fully agree with this in principle. However, when I fail to find that > upstreamable solution, I try to at least provide a Debian-specific > solution to iterate from. > > Can you propose a way to drop either #include? Frankly I don't really see the value in crc32selftest when cross compiling -- sure, the crc32c code works correctly on the build host, but that proves nothing about the (cross-)built binaries that end up in the package. The selftest code is modular enough that it's included in xfs_io and it seems to run in under 500us even on my ancient raspberry pi 3b+ downclocked to 600mhz. Why don't we just add it to mkfs and repair? Those tools shouldn't be run all that frequently, and now we'll know immediately if crc32c is broken on a user's system. Then we don't need the build-time selftest at all. --D > Helmut >
On Sat, Nov 20, 2021 at 09:15:48AM -0800, Darrick J. Wong wrote: > On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > > Hi Dave, > > > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > > I don't get it. The crcselftest does not use liburcu in > > > any way, nor does it try to link against liburcu, so it should not > > > fail because other parts of xfsprogs use liburcu. > > > > > > What's the build error that occurs? > > > > As the build log shows, that's not technically accurate. You can find > > logs of test builds for various architecture combinations at > > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > > called "cross" in https://tracker.debian.org/xfsprogs. > > > > The relevant part is: > > | [TEST] CRC32 > > | In file included from crc32.c:35: > > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > > | 27 | #include <urcu.h> > > | | ^~~~~~~~ > > | compilation terminated. > > > > I failed to figure a good way of dropping either include directive. > > > > > We need to fix the generic cross-build problem in the xfsprogs code, > > > not slap a distro-specific build band-aid over it. > > > > I fully agree with this in principle. However, when I fail to find that > > upstreamable solution, I try to at least provide a Debian-specific > > solution to iterate from. > > > > Can you propose a way to drop either #include? > > Frankly I don't really see the value in crc32selftest when cross > compiling -- sure, the crc32c code works correctly on the build host, > but that proves nothing about the (cross-)built binaries that end up in > the package. Yup, that's pretty much what I was getting at. > The selftest code is modular enough that it's included in xfs_io and it > seems to run in under 500us even on my ancient raspberry pi 3b+ > downclocked to 600mhz. Why don't we just add it to mkfs and repair? > Those tools shouldn't be run all that frequently, and now we'll know > immediately if crc32c is broken on a user's system. > > Then we don't need the build-time selftest at all. Sounds reasonable, my only concern is what do users do when xfs-repair fails the self test and they need to fix their filesystem? Cheers, Dave.
On Sun, Nov 21, 2021 at 09:20:09AM +1100, Dave Chinner wrote: > On Sat, Nov 20, 2021 at 09:15:48AM -0800, Darrick J. Wong wrote: > > On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > > > Hi Dave, > > > > > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > > > I don't get it. The crcselftest does not use liburcu in > > > > any way, nor does it try to link against liburcu, so it should not > > > > fail because other parts of xfsprogs use liburcu. > > > > > > > > What's the build error that occurs? > > > > > > As the build log shows, that's not technically accurate. You can find > > > logs of test builds for various architecture combinations at > > > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > > > called "cross" in https://tracker.debian.org/xfsprogs. > > > > > > The relevant part is: > > > | [TEST] CRC32 > > > | In file included from crc32.c:35: > > > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > > > | 27 | #include <urcu.h> > > > | | ^~~~~~~~ > > > | compilation terminated. > > > > > > I failed to figure a good way of dropping either include directive. > > > > > > > We need to fix the generic cross-build problem in the xfsprogs code, > > > > not slap a distro-specific build band-aid over it. > > > > > > I fully agree with this in principle. However, when I fail to find that > > > upstreamable solution, I try to at least provide a Debian-specific > > > solution to iterate from. > > > > > > Can you propose a way to drop either #include? > > > > Frankly I don't really see the value in crc32selftest when cross > > compiling -- sure, the crc32c code works correctly on the build host, > > but that proves nothing about the (cross-)built binaries that end up in > > the package. > > Yup, that's pretty much what I was getting at. Though this is pretty simple and should just work, too: --- a/libfrog/crc32.c +++ b/libfrog/crc32.c @@ -29,10 +29,11 @@ * match the hardware acceleration available on Intel CPUs. */ +#include <stdio.h> +#include <sys/types.h> #include <inttypes.h> #include <asm/types.h> #include <sys/time.h> -#include "platform_defs.h" /* For endian conversion routines */ #include "xfs_arch.h" #include "crc32defs.h" Cheers, Dave.
On Sun, Nov 21, 2021 at 09:20:09AM +1100, Dave Chinner wrote: > On Sat, Nov 20, 2021 at 09:15:48AM -0800, Darrick J. Wong wrote: > > On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > > > Hi Dave, > > > > > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > > > I don't get it. The crcselftest does not use liburcu in > > > > any way, nor does it try to link against liburcu, so it should not > > > > fail because other parts of xfsprogs use liburcu. > > > > > > > > What's the build error that occurs? > > > > > > As the build log shows, that's not technically accurate. You can find > > > logs of test builds for various architecture combinations at > > > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > > > called "cross" in https://tracker.debian.org/xfsprogs. > > > > > > The relevant part is: > > > | [TEST] CRC32 > > > | In file included from crc32.c:35: > > > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > > > | 27 | #include <urcu.h> > > > | | ^~~~~~~~ > > > | compilation terminated. > > > > > > I failed to figure a good way of dropping either include directive. > > > > > > > We need to fix the generic cross-build problem in the xfsprogs code, > > > > not slap a distro-specific build band-aid over it. > > > > > > I fully agree with this in principle. However, when I fail to find that > > > upstreamable solution, I try to at least provide a Debian-specific > > > solution to iterate from. > > > > > > Can you propose a way to drop either #include? > > > > Frankly I don't really see the value in crc32selftest when cross > > compiling -- sure, the crc32c code works correctly on the build host, > > but that proves nothing about the (cross-)built binaries that end up in > > the package. > > Yup, that's pretty much what I was getting at. > > > The selftest code is modular enough that it's included in xfs_io and it > > seems to run in under 500us even on my ancient raspberry pi 3b+ > > downclocked to 600mhz. Why don't we just add it to mkfs and repair? > > Those tools shouldn't be run all that frequently, and now we'll know > > immediately if crc32c is broken on a user's system. > > > > Then we don't need the build-time selftest at all. > > Sounds reasonable, my only concern is what do users do when > xfs-repair fails the self test and they need to fix their > filesystem? Find a system with a non-broken CPU, or complain to their support rep or the upstream mailing list. I'd rather they do that than let repair erroneously obliterate all the metadata with incorrect crc validation, or write garbage out that fails to mount immediately after. --D > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com
On Sun, Nov 21, 2021 at 09:27:19AM +1100, Dave Chinner wrote: > On Sun, Nov 21, 2021 at 09:20:09AM +1100, Dave Chinner wrote: > > On Sat, Nov 20, 2021 at 09:15:48AM -0800, Darrick J. Wong wrote: > > > On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > > > > Hi Dave, > > > > > > > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > > > > I don't get it. The crcselftest does not use liburcu in > > > > > any way, nor does it try to link against liburcu, so it should not > > > > > fail because other parts of xfsprogs use liburcu. > > > > > > > > > > What's the build error that occurs? > > > > > > > > As the build log shows, that's not technically accurate. You can find > > > > logs of test builds for various architecture combinations at > > > > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > > > > called "cross" in https://tracker.debian.org/xfsprogs. > > > > > > > > The relevant part is: > > > > | [TEST] CRC32 > > > > | In file included from crc32.c:35: > > > > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > > > > | 27 | #include <urcu.h> > > > > | | ^~~~~~~~ > > > > | compilation terminated. > > > > > > > > I failed to figure a good way of dropping either include directive. > > > > > > > > > We need to fix the generic cross-build problem in the xfsprogs code, > > > > > not slap a distro-specific build band-aid over it. > > > > > > > > I fully agree with this in principle. However, when I fail to find that > > > > upstreamable solution, I try to at least provide a Debian-specific > > > > solution to iterate from. > > > > > > > > Can you propose a way to drop either #include? > > > > > > Frankly I don't really see the value in crc32selftest when cross > > > compiling -- sure, the crc32c code works correctly on the build host, > > > but that proves nothing about the (cross-)built binaries that end up in > > > the package. > > > > Yup, that's pretty much what I was getting at. > > Though this is pretty simple and should just work, too: > > --- a/libfrog/crc32.c > +++ b/libfrog/crc32.c > @@ -29,10 +29,11 @@ > * match the hardware acceleration available on Intel CPUs. > */ > > +#include <stdio.h> > +#include <sys/types.h> > #include <inttypes.h> > #include <asm/types.h> > #include <sys/time.h> > -#include "platform_defs.h" > /* For endian conversion routines */ > #include "xfs_arch.h" > #include "crc32defs.h" That works for my system, though I don't do cross builds frequently so I don't really know if it fixes those outside of my toy environment. I'll send a bigger patch to add selftest to mkfs/repair tomorrow. --D > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com
On Mon, Nov 22, 2021 at 03:31:13PM -0800, Darrick J. Wong wrote: > On Sun, Nov 21, 2021 at 09:20:09AM +1100, Dave Chinner wrote: > > On Sat, Nov 20, 2021 at 09:15:48AM -0800, Darrick J. Wong wrote: > > > On Sat, Nov 20, 2021 at 08:32:40AM +0100, Helmut Grohne wrote: > > > > Hi Dave, > > > > > > > > On Sat, Nov 20, 2021 at 10:11:05AM +1100, Dave Chinner wrote: > > > > > I don't get it. The crcselftest does not use liburcu in > > > > > any way, nor does it try to link against liburcu, so it should not > > > > > fail because other parts of xfsprogs use liburcu. > > > > > > > > > > What's the build error that occurs? > > > > > > > > As the build log shows, that's not technically accurate. You can find > > > > logs of test builds for various architecture combinations at > > > > http://crossqa.debian.net/src/xfsprogs. This is also available as a link > > > > called "cross" in https://tracker.debian.org/xfsprogs. > > > > > > > > The relevant part is: > > > > | [TEST] CRC32 > > > > | In file included from crc32.c:35: > > > > | ../include/platform_defs.h:27:10: fatal error: urcu.h: No such file or directory > > > > | 27 | #include <urcu.h> > > > > | | ^~~~~~~~ > > > > | compilation terminated. > > > > > > > > I failed to figure a good way of dropping either include directive. > > > > > > > > > We need to fix the generic cross-build problem in the xfsprogs code, > > > > > not slap a distro-specific build band-aid over it. > > > > > > > > I fully agree with this in principle. However, when I fail to find that > > > > upstreamable solution, I try to at least provide a Debian-specific > > > > solution to iterate from. > > > > > > > > Can you propose a way to drop either #include? > > > > > > Frankly I don't really see the value in crc32selftest when cross > > > compiling -- sure, the crc32c code works correctly on the build host, > > > but that proves nothing about the (cross-)built binaries that end up in > > > the package. > > > > Yup, that's pretty much what I was getting at. > > > > > The selftest code is modular enough that it's included in xfs_io and it > > > seems to run in under 500us even on my ancient raspberry pi 3b+ > > > downclocked to 600mhz. Why don't we just add it to mkfs and repair? > > > Those tools shouldn't be run all that frequently, and now we'll know > > > immediately if crc32c is broken on a user's system. > > > > > > Then we don't need the build-time selftest at all. > > > > Sounds reasonable, my only concern is what do users do when > > xfs-repair fails the self test and they need to fix their > > filesystem? > > Find a system with a non-broken CPU, or complain to their support rep or > the upstream mailing list. I'd rather they do that than let repair > erroneously obliterate all the metadata with incorrect crc validation, > or write garbage out that fails to mount immediately after. Ok, sounds like a man page update is in order, then :) Cheers, Dave.
diff --git a/debian/rules b/debian/rules index 6d5b82a8..28cc414d 100755 --- a/debian/rules +++ b/debian/rules @@ -40,6 +40,9 @@ build-arch: built build-indep: built built: dibuild config @echo "== dpkg-buildpackage: build" 1>&2 +ifneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH)) + touch --date=+3day libfrog/crc32selftest +endif $(MAKE) $(PMAKEFLAGS) default touch built
xfsprogs fails to cross build from source, because it attempts to build its crc32 test with the build architecture compiler and thus fails finding the liburcu, which is only requested for the host architecture. While this test is useful for native builds, it is not that useful for cross builds. Skip it by pre-creating the output file. Reported-by: Helmut Grohne <helmut@subdivi.de> Signed-off-by: Bastian Germann <bage@debian.org> --- debian/rules | 3 +++ 1 file changed, 3 insertions(+)