| Message ID | 20180905190700.66123-1-ebiggers@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [resend] android-xfstests: fix finding FS_TYPE when userdata is on dm device | expand |
On Wed, Sep 05, 2018 at 12:07:00PM -0700, Eric Biggers wrote: > From: Eric Biggers <ebiggers@google.com> > > If the userdata filesystem is on a device-mapper device, then shrinking > the partition underneath it breaks 'blkid' probing, which > android-setup-partitions now uses find the filesystem type. Use blkid's > '-p' and '-S' options to do a low-level probe with explicitly specified > size, which still works in this case. > > Signed-off-by: Eric Biggers <ebiggers@google.com> Thanks, applied. - Ted
On Sat, Sep 15, 2018 at 03:36:57PM -0400, Theodore Y. Ts'o wrote: > On Wed, Sep 05, 2018 at 12:07:00PM -0700, Eric Biggers wrote: > > From: Eric Biggers <ebiggers@google.com> > > > > If the userdata filesystem is on a device-mapper device, then shrinking > > the partition underneath it breaks 'blkid' probing, which > > android-setup-partitions now uses find the filesystem type. Use blkid's > > '-p' and '-S' options to do a low-level probe with explicitly specified > > size, which still works in this case. > > > > Signed-off-by: Eric Biggers <ebiggers@google.com> > > Thanks, applied. > > - Ted Hi Ted, are you planning to push this out? - Eric
On Wed, Oct 03, 2018 at 04:37:18PM -0700, Eric Biggers wrote: > > Hi Ted, are you planning to push this out? > Oops, sorry. Pushed out now. - Ted
On Thu, Oct 4, 2018 at 5:35 AM Theodore Y. Ts'o <tytso@mit.edu> wrote: > > On Wed, Oct 03, 2018 at 04:37:18PM -0700, Eric Biggers wrote: > > > > Hi Ted, are you planning to push this out? > > > > Oops, sorry. Pushed out now. > Ted, FYI, this is pushed to github, but not kernel.org tree. Intentional? FYI2, I started trying to build a kvm-xfstests image with Debian buster. First thing I ran into with setup-buildchroot is that golang-1.8-go package is not available. I suppose golang is for syzbot? Their wiki recommends to use golang-go for debian based releases, which does work for buster. Now I hit an error in ./gen-image: W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb I see that this pkg doesn't exist in the mirror, but this pkg does exists (deb9u2): http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.deb Do you know what that is about? Thanks, Amir.
On Thu, Oct 11, 2018 at 03:01:36PM +0300, Amir Goldstein wrote: > > FYI, this is pushed to github, but not kernel.org tree. > Intentional? Nope, just another oversight; fixed now. > FYI2, I started trying to build a kvm-xfstests image with Debian buster. > First thing I ran into with setup-buildchroot is that golang-1.8-go > package is not available. > I suppose golang is for syzbot? Their wiki recommends to use > golang-go for debian based releases, which does work for buster. At the time syzbot required golang 1.8. I just checked, and according to their docs[1] syzbot now requires golang 1.9. I haven't been bothering quite as much with syzkaller lately because there are now C reproducers, and the syzkaller reproducers are not stable --- that is, if you don't use *precisely* the same version of syzkaller, down to the commit id, the syzkaller repro might not work at all. I've tried talking to Dmitry about providing at least *backwards* compatibility (e.g., that a newer syzkaller can use a repro produced by an older syzbot), but he's not willing to provide even that level of compatibility. [1] https://github.com/google/syzkaller/blob/master/docs/linux/setup.md So building syzkaller is currently optional, and the only time I would recommend it is that there apparently some cases where there is a syzkaller reproducer, but no C reproducer, which is very said given the complete lack of backwards compatibility. I suppose we could include git, the syzkaller git tree and the go compiler into the test VM, and build the appropriate version syzkaller for each repro --- but it would really bloat the kvm-xfstests, and that information isn't even encoded into the syzkaller repro file. :-( So it's all rather a mess at this point. Maybe this is something we could do just for the gce-xfstests test appliance. Patches appreciated. :-) On your other note, I haven't bothered using setup-buildchroot since my laptop environment is buster. Making setup-buildchroot install the newest possible version of go available for buster or stretch sounds like the best approach for now. > Now I hit an error in ./gen-image: > W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at > http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb > I see that this pkg doesn't exist in the mirror, but this pkg does > exists (deb9u2): > http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.deb > > Do you know what that is about? That's generally due to the mirror not being fully synced up so the Debian distro metadata files weren't synced up with the debian package files. If you try again in 30 minutes or so, it should resolve itself. - Ted
On Thu, Oct 11, 2018 at 5:56 PM Theodore Y. Ts'o <tytso@mit.edu> wrote: > > On Thu, Oct 11, 2018 at 03:01:36PM +0300, Amir Goldstein wrote: > > > > FYI, this is pushed to github, but not kernel.org tree. > > Intentional? > > Nope, just another oversight; fixed now. > > > FYI2, I started trying to build a kvm-xfstests image with Debian buster. > > First thing I ran into with setup-buildchroot is that golang-1.8-go > > package is not available. > > I suppose golang is for syzbot? Their wiki recommends to use > > golang-go for debian based releases, which does work for buster. > > At the time syzbot required golang 1.8. I just checked, and according > to their docs[1] syzbot now requires golang 1.9. I haven't been > bothering quite as much with syzkaller lately because there are now C > reproducers, and the syzkaller reproducers are not stable --- that is, > if you don't use *precisely* the same version of syzkaller, down to > the commit id, the syzkaller repro might not work at all. I've tried > talking to Dmitry about providing at least *backwards* compatibility > (e.g., that a newer syzkaller can use a repro produced by an older > syzbot), but he's not willing to provide even that level of > compatibility. > > [1] https://github.com/google/syzkaller/blob/master/docs/linux/setup.md > > So building syzkaller is currently optional, and the only time I would > recommend it is that there apparently some cases where there is a > syzkaller reproducer, but no C reproducer, which is very said given > the complete lack of backwards compatibility. > > I suppose we could include git, the syzkaller git tree and the go > compiler into the test VM, and build the appropriate version syzkaller > for each repro --- but it would really bloat the kvm-xfstests, and > that information isn't even encoded into the syzkaller repro file. :-( > That's a shame. > So it's all rather a mess at this point. Maybe this is something we > could do just for the gce-xfstests test appliance. Patches > appreciated. :-) > > On your other note, I haven't bothered using setup-buildchroot since > my laptop environment is buster. Making setup-buildchroot install the > newest possible version of go available for buster or stretch sounds > like the best approach for now. > That's what I did. will post one liner once everything is working. > > > Now I hit an error in ./gen-image: > > W: Couldn't download package openssl (ver 1.1.0f-3+deb9u1 arch amd64) at > > http://mirrors.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u1_amd64.deb > > I see that this pkg doesn't exist in the mirror, but this pkg does > > exists (deb9u2): > > http://mirrors.edge.kernel.org/debian/pool/main/o/openssl/openssl_1.1.0f-3+deb9u2_amd64.deb > > > > Do you know what that is about? > > That's generally due to the mirror not being fully synced up so the > Debian distro metadata files weren't synced up with the debian package > files. If you try again in 30 minutes or so, it should resolve > itself. > Note that the 2 packages I referred to above are different. One is +deb9u1 (missing) the other is +deb9u2 (available) I see that some packages downloaded have the +deb9u1 suffix some have the +deb9u2 suffix and some no suffix at all. I know absolutely zero about that distro black magic, so if you have a lead for me, please let me know. Thanks, Amir.
diff --git a/kvm-xfstests/test-appliance/android-setup-partitions b/kvm-xfstests/test-appliance/android-setup-partitions index 8d38303..ddaa252 100755 --- a/kvm-xfstests/test-appliance/android-setup-partitions +++ b/kvm-xfstests/test-appliance/android-setup-partitions @@ -259,7 +259,8 @@ get_fs_size() # userdata partition. That isn't necessary, since resizing the underlying # partition to a size smaller than the dm device just causes I/O requests to the # truncated region to fail, and normally there should be no I/O occurring beyond -# the end of the filesystem. +# the end of the filesystem. Exception: this makes 'blkid' stop reporting +# information about the device, unless blkid's -p and -S options are used. shrink_userdata_partition() { local fs_size part_size @@ -380,7 +381,9 @@ else fi # Type of the userdata filesystem, e.g. ext4 or f2fs -USERDATA_FS_TYPE=$(blkid -s TYPE -o value "$USERDATA_FS_DEV") +USERDATA_FS_TYPE=$(blkid -s TYPE -o value \ + -p -S $(get_partition_size "$USERDATA_RAW_DEV") \ + "$USERDATA_FS_DEV") if ! all_partitions_present ; then # Free up as much space as we can, then create the partitions.