From patchwork Thu Dec 19 14:55:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Harmstone X-Patchwork-Id: 13915134 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C335378F54 for ; Thu, 19 Dec 2024 14:56:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734620216; cv=none; b=rtvXptfePMKlux8THVWQM8rhjlgQOzq9+L46SNCnneTNQ9Vxvv2pEi8IY70Xmj3EjuxHbIMhqdvYDGVidmQ60nwgxWi/J93p2OxM57mqJSd6zEpAKc2/pGKNBaW8UP70/fxZTfOLhrxfNV45f0wlQP9oTm8m5AFCmKsgfkvQC0Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734620216; c=relaxed/simple; bh=nyPLepdPiFn6D+OGD+MjSqXBJcNJ9+UCod+pDt9/UQ0=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=JH3ovurZOWmg4GqO9mz06CJmBuwsUt+1lJp3rJjkI00UQhqetcpXkdDp/TO60fMeElrmYC4EUKa2Le9MO7+9XB+jsi8+wsWn5aZmTPpeycoRt/0O/7pjuhw1p76hVpx09NeNTG6d824/W4Cv6TN0XiOgGS2J0f/iNvZ/nUTvHk8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b=QAQW9e6k; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="QAQW9e6k" Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BJ7PCQF025747 for ; Thu, 19 Dec 2024 06:56:54 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to; s=facebook; bh=bKv5BuYbPe0RVZ5KawS7S2S fYc8+BUTb6VmUSA7oGGU=; b=QAQW9e6kauHtnyq0JbjyUzWcB1vmSQbEBekXG5D bli2BDbGfplXRxJ15/qiUXmoDJVsE2zCARSs+bIH/2WRjM7Jw8Debw+0g5ets/Mj jJKfGZuBibq3rflxfJQ9iY2LUYx5REX+5eGmnY/f3QvWKe+2393gAS+0fNMzbL2S 5buc= Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43md1uawgj-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 19 Dec 2024 06:56:53 -0800 (PST) Received: from twshared60378.16.frc2.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.14; Thu, 19 Dec 2024 14:56:51 +0000 Received: by devbig276.nha1.facebook.com (Postfix, from userid 660015) id DDBFE9B34819; Thu, 19 Dec 2024 14:56:48 +0000 (GMT) From: Mark Harmstone To: , CC: , , , Mark Harmstone Subject: [PATCH v3 1/2] configure: use pkg-config to find liburing Date: Thu, 19 Dec 2024 14:55:55 +0000 Message-ID: <20241219145608.3925261-1-maharmstone@fb.com> X-Mailer: git-send-email 2.43.5 Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: eCxzxXuu09ZGFxC8CqmNjKLyOPFsrAkP X-Proofpoint-GUID: eCxzxXuu09ZGFxC8CqmNjKLyOPFsrAkP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_02,2024-10-04_01,2024-09-30_01 Change our autoconf macros so that instead of checking for the presence of liburing.h, we use pkg-config. The benefit of this is that we can then check the version of liburing, and do conditional compilation based on this. There's a macro IO_URING_CHECK_VERSION already, but it's only in relatively recent versions of liburing.h. This replaces HAVE_URING_H, defined by AC_CHECK_HEADERS, with HAVE_URING. I also had to rename PKG_{MAJOR,MINOR,REVISION,BUILD} to start with PACKAGE_, to avoid "possibly undefined macro" errors; it looks like pkg-config assumes that anything called PKG_* is for its own use. Signed-off-by: Mark Harmstone --- VERSION | 8 ++++---- m4/package_globals.m4 | 4 ++-- m4/package_liburing.m4 | 6 +++++- release.sh | 2 +- src/feature.c | 4 ++-- src/vfs/idmapped-mounts.c | 6 +++--- src/vfs/idmapped-mounts.h | 2 +- src/vfs/tmpfs-idmapped-mounts.c | 6 +++--- src/vfs/utils.c | 4 ++-- src/vfs/utils.h | 6 +++--- src/vfs/vfstest.c | 6 +++--- 11 files changed, 29 insertions(+), 25 deletions(-) diff --git a/VERSION b/VERSION index 7294a002..afcab53e 100644 --- a/VERSION +++ b/VERSION @@ -1,7 +1,7 @@ # # This file is used by configure to get version information # -PKG_MAJOR=1 -PKG_MINOR=1 -PKG_REVISION=1 -PKG_BUILD=1 +PACKAGE_MAJOR=1 +PACKAGE_MINOR=1 +PACKAGE_REVISION=1 +PACKAGE_BUILD=1 diff --git a/m4/package_globals.m4 b/m4/package_globals.m4 index ce7a8c51..c8d5d124 100644 --- a/m4/package_globals.m4 +++ b/m4/package_globals.m4 @@ -9,9 +9,9 @@ AC_DEFUN([AC_PACKAGE_GLOBALS], AC_SUBST(pkg_name) . ./VERSION - pkg_version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} + pkg_version=${PACKAGE_MAJOR}.${PACKAGE_MINOR}.${PACKAGE_REVISION} AC_SUBST(pkg_version) - pkg_release=$PKG_BUILD + pkg_release=$PACKAGE_BUILD test -z "$BUILD_VERSION" || pkg_release="$BUILD_VERSION" AC_SUBST(pkg_release) diff --git a/m4/package_liburing.m4 b/m4/package_liburing.m4 index c92cc02a..0553966d 100644 --- a/m4/package_liburing.m4 +++ b/m4/package_liburing.m4 @@ -1,4 +1,8 @@ AC_DEFUN([AC_PACKAGE_WANT_URING], - [ AC_CHECK_HEADERS(liburing.h, [ have_uring=true ], [ have_uring=false ]) + [ PKG_CHECK_MODULES([LIBURING], [liburing], + [ AC_DEFINE([HAVE_LIBURING], [1], [Use liburing]) + have_uring=true + ], + [ have_uring=false ]) AC_SUBST(have_uring) ]) diff --git a/release.sh b/release.sh index 5b78ec79..70fbf47e 100644 --- a/release.sh +++ b/release.sh @@ -5,7 +5,7 @@ . ./VERSION -version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} +version=${PACKAGE_MAJOR}.${PACKAGE_MINOR}.${PACKAGE_REVISION} date=`date +"%-d %B %Y"` echo "Cleaning up" diff --git a/src/feature.c b/src/feature.c index 7e474ce5..7df36acf 100644 --- a/src/feature.c +++ b/src/feature.c @@ -42,7 +42,7 @@ #include #endif -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING #include #endif @@ -227,7 +227,7 @@ check_aio_support(void) static int check_uring_support(void) { -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING struct io_uring ring; int err; diff --git a/src/vfs/idmapped-mounts.c b/src/vfs/idmapped-mounts.c index f4dfc3f3..ed9992f9 100644 --- a/src/vfs/idmapped-mounts.c +++ b/src/vfs/idmapped-mounts.c @@ -2206,7 +2206,7 @@ out: } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int tcore_io_uring_idmapped(const struct vfstest_info *info) { int fret = -1; @@ -2743,7 +2743,7 @@ out_unmap: return fret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* Validate that protected symlinks work correctly on idmapped mounts. */ int tcore_protected_symlinks_idmapped_mounts(const struct vfstest_info *info) @@ -8859,7 +8859,7 @@ static const struct test_struct t_idmapped_mounts[] = { { tcore_hardlink_crossing_idmapped_mounts, true, "cross idmapped mount hardlink", }, { tcore_hardlink_from_idmapped_mount, true, "hardlinks from idmapped mounts", }, { tcore_hardlink_from_idmapped_mount_in_userns, true, "hardlinks from idmapped mounts in user namespace", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { tcore_io_uring_idmapped, true, "io_uring from idmapped mounts", }, { tcore_io_uring_idmapped_userns, true, "io_uring from idmapped mounts in user namespace", }, { tcore_io_uring_idmapped_unmapped, true, "io_uring from idmapped mounts with unmapped ids", }, diff --git a/src/vfs/idmapped-mounts.h b/src/vfs/idmapped-mounts.h index 4a2c7b39..688394c8 100644 --- a/src/vfs/idmapped-mounts.h +++ b/src/vfs/idmapped-mounts.h @@ -30,7 +30,7 @@ int tcore_fscaps_idmapped_mounts_in_userns_separate_userns(const struct vfstest_ int tcore_hardlink_crossing_idmapped_mounts(const struct vfstest_info *info); int tcore_hardlink_from_idmapped_mount(const struct vfstest_info *info); int tcore_hardlink_from_idmapped_mount_in_userns(const struct vfstest_info *info); -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int tcore_io_uring_idmapped(const struct vfstest_info *info); int tcore_io_uring_idmapped_userns(const struct vfstest_info *info); int tcore_io_uring_idmapped_unmapped(const struct vfstest_info *info); diff --git a/src/vfs/tmpfs-idmapped-mounts.c b/src/vfs/tmpfs-idmapped-mounts.c index 0899aed9..d8212bce 100644 --- a/src/vfs/tmpfs-idmapped-mounts.c +++ b/src/vfs/tmpfs-idmapped-mounts.c @@ -167,7 +167,7 @@ static int tmpfs_hardlink_from_idmapped_mount_in_userns(const struct vfstest_inf return tmpfs_nested_mount_setup(info, tcore_hardlink_from_idmapped_mount_in_userns); } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING static int tmpfs_io_uring_idmapped(const struct vfstest_info *info) { return tmpfs_nested_mount_setup(info, tcore_io_uring_idmapped); @@ -184,7 +184,7 @@ static int tmpfs_io_uring_idmapped_unmapped_userns(const struct vfstest_info *in { return tmpfs_nested_mount_setup(info, tcore_io_uring_idmapped_unmapped_userns); } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ static int tmpfs_protected_symlinks_idmapped_mounts(const struct vfstest_info *info) { @@ -272,7 +272,7 @@ static const struct test_struct t_tmpfs[] = { { tmpfs_hardlink_crossing_idmapped_mounts, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs cross idmapped mount hardlink", }, { tmpfs_hardlink_from_idmapped_mount, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs hardlinks from idmapped mounts", }, { tmpfs_hardlink_from_idmapped_mount_in_userns, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs hardlinks from idmapped mounts in user namespace", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { tmpfs_io_uring_idmapped, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts", }, { tmpfs_io_uring_idmapped_userns, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts in user namespace", }, { tmpfs_io_uring_idmapped_unmapped, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts with unmapped ids", }, diff --git a/src/vfs/utils.c b/src/vfs/utils.c index 0ab5de15..c1c7951c 100644 --- a/src/vfs/utils.c +++ b/src/vfs/utils.c @@ -502,7 +502,7 @@ out: return fret; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, int cred_id, bool with_link, int *ret_cqe) { @@ -555,7 +555,7 @@ int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, out: return ret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* caps_up - raise all permitted caps */ int caps_up(void) diff --git a/src/vfs/utils.h b/src/vfs/utils.h index 872fd96f..c086885a 100644 --- a/src/vfs/utils.h +++ b/src/vfs/utils.h @@ -25,7 +25,7 @@ #include #endif -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING #include #endif @@ -349,11 +349,11 @@ static inline bool switch_fsids(uid_t fsuid, gid_t fsgid) return true; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING extern int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, int cred_id, bool with_link, int *ret_cqe); -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ extern int chown_r(int fd, const char *path, uid_t uid, gid_t gid); extern int rm_r(int fd, const char *path); diff --git a/src/vfs/vfstest.c b/src/vfs/vfstest.c index f842117d..e0c897bb 100644 --- a/src/vfs/vfstest.c +++ b/src/vfs/vfstest.c @@ -1222,7 +1222,7 @@ out: return fret; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING static int io_uring(const struct vfstest_info *info) { int fret = -1; @@ -1495,7 +1495,7 @@ out_unmap: return fret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* The following tests are concerned with setgid inheritance. These can be * filesystem type specific. For xfs, if a new file or directory or node is @@ -2349,7 +2349,7 @@ static const struct option longopts[] = { static const struct test_struct t_basic[] = { { fscaps, T_REQUIRE_USERNS, "fscaps on regular mounts", }, { hardlink_crossing_mounts, 0, "cross mount hardlink", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { io_uring, 0, "io_uring", }, { io_uring_userns, T_REQUIRE_USERNS, "io_uring in user namespace", }, #endif From patchwork Thu Dec 19 14:55:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Harmstone X-Patchwork-Id: 13915135 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B6DC78F23 for ; Thu, 19 Dec 2024 14:57:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734620227; cv=none; b=c4NYMNPW3qwuOjh4JoyFpA47tFlrtXwmPTEc3ZvAm1J4/R1N7JIvHP2FaI60pLC6EKXmKD2zccaeC//u7WwKie2FRFYsA/9+VI7SA/Eyg+9I3aozzUxSh43pcQiaGVKD/2LtVuKINuE8GcGrSnm82RVjcqvkGDH4ckA5926f3mk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734620227; c=relaxed/simple; bh=ae74+hUzafgbuvxa3Fdr7Tpwli25tDVkgXR4GpEERcE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mcp4WdbYZaUoYd7J4HiUMba+xw3arHl+MW7qE+4UUBHiNVhuwrQFRzLVSBbMPVMM6eBTZSrNZ6HSMfjvLb9o0ZRR9d/3BK0PfJfRsxMr81lrpOdOinroPI/DAk0Od8mvxcZD4P/lRAPlw63yqgzKLhVm3DnEzXeubPuemJOkli8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b=dKTcYPur; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="dKTcYPur" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BJAtjki004268 for ; Thu, 19 Dec 2024 06:57:04 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=facebook; bh=p ek+n0EYIWeqGfv1pa9bherhVafK0EzVY7ErJil47Rs=; b=dKTcYPurzvkTq0ScX dmMttgoFddQ3feUmYdtoTa1SsKsOAfLZmgk/4PNPrFlEMb4XizRB19yBV4/DweAL k4tzz/YSv2QaclPXvcSAtSe2c8c+Hle7H8bljhPepuW7jtpVaGw1ng8HgoEwTenH ncHAwgC0uBiMTwG/u/CbpiEDwA= Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43mj5ks82p-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 19 Dec 2024 06:57:03 -0800 (PST) Received: from twshared40462.17.frc2.facebook.com (2620:10d:c0a8:fe::f072) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.14; Thu, 19 Dec 2024 14:56:51 +0000 Received: by devbig276.nha1.facebook.com (Postfix, from userid 660015) id E440B9B3481A; Thu, 19 Dec 2024 14:56:48 +0000 (GMT) From: Mark Harmstone To: , CC: , , , Mark Harmstone Subject: [PATCH v3 2/2] btrfs: add test for encoded reads Date: Thu, 19 Dec 2024 14:55:56 +0000 Message-ID: <20241219145608.3925261-2-maharmstone@fb.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241219145608.3925261-1-maharmstone@fb.com> References: <20241219145608.3925261-1-maharmstone@fb.com> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: brZKrfF5AyDN2pZSlxPj7pqweeaZfVci X-Proofpoint-GUID: brZKrfF5AyDN2pZSlxPj7pqweeaZfVci X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 Add btrfs/333 and its helper programs btrfs_encoded_read and btrfs_encoded_write, in order to test encoded reads. We use the BTRFS_IOC_ENCODED_WRITE ioctl to write random data into a compressed extent, then use the BTRFS_IOC_ENCODED_READ ioctl to check that it matches what we've written. If the new io_uring interface for encoded reads is supported, we also check that that matches the ioctl. Note that what we write isn't valid compressed data, so any non-encoded reads on these files will fail. Signed-off-by: Mark Harmstone --- This should now work on systems with old versions of liburing, and systems with old versions of the btrfs.h header. I've also included Daniel Vacek's suggestions for reducing the amount of time spent doing dd. .gitignore | 2 + m4/package_liburing.m4 | 2 + src/Makefile | 3 +- src/btrfs_encoded_read.c | 203 +++++++++++++++++++++++++++++++++ src/btrfs_encoded_write.c | 234 ++++++++++++++++++++++++++++++++++++++ tests/btrfs/333 | 220 +++++++++++++++++++++++++++++++++++ tests/btrfs/333.out | 2 + 7 files changed, 665 insertions(+), 1 deletion(-) create mode 100644 src/btrfs_encoded_read.c create mode 100644 src/btrfs_encoded_write.c create mode 100755 tests/btrfs/333 create mode 100644 tests/btrfs/333.out diff --git a/.gitignore b/.gitignore index f16173d9..efd47773 100644 --- a/.gitignore +++ b/.gitignore @@ -62,6 +62,8 @@ tags /src/attr_replace_test /src/attr-list-by-handle-cursor-test /src/bstat +/src/btrfs_encoded_read +/src/btrfs_encoded_write /src/bulkstat_null_ocount /src/bulkstat_unlink_test /src/bulkstat_unlink_test_modified diff --git a/m4/package_liburing.m4 b/m4/package_liburing.m4 index 0553966d..7fbf4a5f 100644 --- a/m4/package_liburing.m4 +++ b/m4/package_liburing.m4 @@ -1,6 +1,8 @@ AC_DEFUN([AC_PACKAGE_WANT_URING], [ PKG_CHECK_MODULES([LIBURING], [liburing], [ AC_DEFINE([HAVE_LIBURING], [1], [Use liburing]) + AC_DEFINE_UNQUOTED([LIBURING_MAJOR_VERSION], [`$PKG_CONFIG --modversion liburing | cut -d. -f1`], [liburing major version]) + AC_DEFINE_UNQUOTED([LIBURING_MINOR_VERSION], [`$PKG_CONFIG --modversion liburing | cut -d. -f2`], [liburing minor version]) have_uring=true ], [ have_uring=false ]) diff --git a/src/Makefile b/src/Makefile index a0396332..b42b8147 100644 --- a/src/Makefile +++ b/src/Makefile @@ -34,7 +34,8 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ - uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment + uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ + btrfs_encoded_read btrfs_encoded_write EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \ diff --git a/src/btrfs_encoded_read.c b/src/btrfs_encoded_read.c new file mode 100644 index 00000000..2e4079b0 --- /dev/null +++ b/src/btrfs_encoded_read.c @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) Meta Platforms, Inc. and affiliates. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "config.h" + +#ifdef HAVE_LIBURING +#include +#endif + +/* IORING_OP_URING_CMD defined from liburing 2.2 onwards */ +#if defined(HAVE_LIBURING) && (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 2)) +#define IORING_OP_URING_CMD 46 +#endif + +#ifndef BTRFS_IOC_ENCODED_READ +struct btrfs_ioctl_encoded_io_args { + const struct iovec *iov; + unsigned long iovcnt; + __s64 offset; + __u64 flags; + __u64 len; + __u64 unencoded_len; + __u64 unencoded_offset; + __u32 compression; + __u32 encryption; + __u8 reserved[64]; +}; + +#define BTRFS_IOC_ENCODED_READ _IOR(BTRFS_IOCTL_MAGIC, 64, struct btrfs_ioctl_encoded_io_args) +#endif + +#define BTRFS_MAX_COMPRESSED 131072 +#define QUEUE_DEPTH 1 + +static int encoded_read_ioctl(const char *filename, long long offset) +{ + int ret, fd; + char buf[BTRFS_MAX_COMPRESSED]; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + + fd = open(filename, O_RDONLY); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + return 1; + } + + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.flags = 0; + + ret = ioctl(fd, BTRFS_IOC_ENCODED_READ, &enc); + + if (ret < 0) { + printf("%i\n", -errno); + close(fd); + return 0; + } + + close(fd); + + printf("%i\n", ret); + printf("%llu\n", enc.len); + printf("%llu\n", enc.unencoded_len); + printf("%llu\n", enc.unencoded_offset); + printf("%u\n", enc.compression); + printf("%u\n", enc.encryption); + + fwrite(buf, ret, 1, stdout); + + return 0; +} + +static int encoded_read_io_uring(const char *filename, long long offset) +{ +#ifdef HAVE_LIBURING + int ret, fd; + char buf[BTRFS_MAX_COMPRESSED]; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + struct io_uring ring; + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + + io_uring_queue_init(QUEUE_DEPTH, &ring, 0); + + fd = open(filename, O_RDONLY); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + ret = 1; + goto out_uring; + } + + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.flags = 0; + + sqe = io_uring_get_sqe(&ring); + if (!sqe) { + fprintf(stderr, "io_uring_get_sqe failed\n"); + ret = 1; + goto out_close; + } + + io_uring_prep_rw(IORING_OP_URING_CMD, sqe, fd, &enc, sizeof(enc), 0); + + /* sqe->cmd_op union'd to sqe->off from liburing 2.3 onwards */ +#if (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 3)) + sqe->off = BTRFS_IOC_ENCODED_READ; +#else + sqe->cmd_op = BTRFS_IOC_ENCODED_READ; +#endif + + io_uring_submit(&ring); + + ret = io_uring_wait_cqe(&ring, &cqe); + if (ret < 0) { + fprintf(stderr, "io_uring_wait_cqe returned %i\n", ret); + ret = 1; + goto out_close; + } + + io_uring_cqe_seen(&ring, cqe); + + if (cqe->res < 0) { + printf("%i\n", cqe->res); + ret = 0; + goto out_close; + } + + printf("%i\n", cqe->res); + printf("%llu\n", enc.len); + printf("%llu\n", enc.unencoded_len); + printf("%llu\n", enc.unencoded_offset); + printf("%u\n", enc.compression); + printf("%u\n", enc.encryption); + + fwrite(buf, cqe->res, 1, stdout); + + ret = 0; + +out_close: + close(fd); + +out_uring: + io_uring_queue_exit(&ring); + + return ret; +#else + fprintf(stderr, "liburing not linked in\n"); + return 1; +#endif +} + +static void usage() +{ + fprintf(stderr, "Usage: btrfs_encoded_read ioctl|io_uring filename offset\n"); +} + +int main(int argc, char *argv[]) +{ + const char *filename; + long long offset; + + if (argc != 4) { + usage(); + return 1; + } + + filename = argv[2]; + + offset = atoll(argv[3]); + if (offset == 0 && errno != 0) { + usage(); + return 1; + } + + if (!strcmp(argv[1], "ioctl")) { + return encoded_read_ioctl(filename, offset); + } else if (!strcmp(argv[1], "io_uring")) { + return encoded_read_io_uring(filename, offset); + } else { + usage(); + return 1; + } +} diff --git a/src/btrfs_encoded_write.c b/src/btrfs_encoded_write.c new file mode 100644 index 00000000..1b063fa1 --- /dev/null +++ b/src/btrfs_encoded_write.c @@ -0,0 +1,234 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) Meta Platforms, Inc. and affiliates. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "config.h" + +#ifdef HAVE_LIBURING +#include +#endif + +/* IORING_OP_URING_CMD defined from liburing 2.2 onwards */ +#if defined(HAVE_LIBURING) && (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 2)) +#define IORING_OP_URING_CMD 46 +#endif + +#ifndef BTRFS_IOC_ENCODED_WRITE +struct btrfs_ioctl_encoded_io_args { + const struct iovec *iov; + unsigned long iovcnt; + __s64 offset; + __u64 flags; + __u64 len; + __u64 unencoded_len; + __u64 unencoded_offset; + __u32 compression; + __u32 encryption; + __u8 reserved[64]; +}; + +#define BTRFS_IOC_ENCODED_WRITE _IOW(BTRFS_IOCTL_MAGIC, 64, struct btrfs_ioctl_encoded_io_args) +#endif + +#define BTRFS_MAX_COMPRESSED 131072 +#define QUEUE_DEPTH 1 + +static int encoded_write_ioctl(const char *filename, long long offset, + long long len, long long unencoded_len, + long long unencoded_offset, int compression, + char *buf, size_t size) +{ + int ret, fd; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + + fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0644); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + return 1; + } + + iov.iov_base = buf; + iov.iov_len = size; + + memset(&enc, 0, sizeof(enc)); + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.len = len; + enc.unencoded_len = unencoded_len; + enc.unencoded_offset = unencoded_offset; + enc.compression = compression; + + ret = ioctl(fd, BTRFS_IOC_ENCODED_WRITE, &enc); + + if (ret < 0) { + printf("%i\n", -errno); + close(fd); + return 0; + } + + printf("%i\n", ret); + + close(fd); + + return 0; +} + +static int encoded_write_io_uring(const char *filename, long long offset, + long long len, long long unencoded_len, + long long unencoded_offset, int compression, + char *buf, size_t size) +{ +#ifdef HAVE_LIBURING + int ret, fd; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + struct io_uring ring; + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + + io_uring_queue_init(QUEUE_DEPTH, &ring, 0); + + fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0644); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + ret = 1; + goto out_uring; + } + + iov.iov_base = buf; + iov.iov_len = size; + + memset(&enc, 0, sizeof(enc)); + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.len = len; + enc.unencoded_len = unencoded_len; + enc.unencoded_offset = unencoded_offset; + enc.compression = compression; + + sqe = io_uring_get_sqe(&ring); + if (!sqe) { + fprintf(stderr, "io_uring_get_sqe failed\n"); + ret = 1; + goto out_close; + } + + io_uring_prep_rw(IORING_OP_URING_CMD, sqe, fd, &enc, sizeof(enc), 0); + + /* sqe->cmd_op union'd to sqe->off from liburing 2.3 onwards */ +#if (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 3)) + sqe->off = BTRFS_IOC_ENCODED_WRITE; +#else + sqe->cmd_op = BTRFS_IOC_ENCODED_WRITE; +#endif + + io_uring_submit(&ring); + + ret = io_uring_wait_cqe(&ring, &cqe); + if (ret < 0) { + fprintf(stderr, "io_uring_wait_cqe returned %i\n", ret); + ret = 1; + goto out_close; + } + + io_uring_cqe_seen(&ring, cqe); + + if (cqe->res < 0) { + printf("%i\n", cqe->res); + ret = 0; + goto out_close; + } + + printf("%i\n", cqe->res); + + ret = 0; + +out_close: + close(fd); + +out_uring: + io_uring_queue_exit(&ring); + + return ret; +#else + fprintf(stderr, "liburing not linked in\n"); + return 1; +#endif +} + +static void usage() +{ + fprintf(stderr, "Usage: btrfs_encoded_write ioctl|io_uring filename offset len unencoded_len unencoded_offset compression\n"); +} + +int main(int argc, char *argv[]) +{ + const char *filename; + long long offset, len, unencoded_len, unencoded_offset; + int compression; + char buf[BTRFS_MAX_COMPRESSED]; + size_t size; + + if (argc != 8) { + usage(); + return 1; + } + + filename = argv[2]; + + offset = atoll(argv[3]); + if (offset == 0 && errno != 0) { + usage(); + return 1; + } + + len = atoll(argv[4]); + if (len == 0 && errno != 0) { + usage(); + return 1; + } + + unencoded_len = atoll(argv[5]); + if (unencoded_len == 0 && errno != 0) { + usage(); + return 1; + } + + unencoded_offset = atoll(argv[6]); + if (unencoded_offset == 0 && errno != 0) { + usage(); + return 1; + } + + compression = atoi(argv[7]); + if (compression == 0 && errno != 0) { + usage(); + return 1; + } + + size = fread(buf, 1, BTRFS_MAX_COMPRESSED, stdin); + + if (!strcmp(argv[1], "ioctl")) { + return encoded_write_ioctl(filename, offset, len, unencoded_len, + unencoded_offset, compression, buf, + size); + } else if (!strcmp(argv[1], "io_uring")) { + return encoded_write_io_uring(filename, offset, len, + unencoded_len, unencoded_offset, + compression, buf, size); + } else { + usage(); + return 1; + } +} diff --git a/tests/btrfs/333 b/tests/btrfs/333 new file mode 100755 index 00000000..d7fbb7c7 --- /dev/null +++ b/tests/btrfs/333 @@ -0,0 +1,220 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test No. btrfs/333 +# +# Test btrfs encoded reads + +. ./common/preamble +_begin_fstest auto quick compress rw + +. ./common/filter +. ./common/btrfs + +_supported_fs btrfs + +do_encoded_read() { + local fn=$1 + local type=$2 + local exp_ret=$3 + local exp_len=$4 + local exp_unencoded_len=$5 + local exp_unencoded_offset=$6 + local exp_compression=$7 + local exp_md5=$8 + + local tmpfile=`mktemp` + + echo "running btrfs_encoded_read $type $fn 0 > $tmpfile" >>$seqres.full + src/btrfs_encoded_read $type $fn 0 > $tmpfile + + if [[ $? -ne 0 ]]; then + echo "btrfs_encoded_read failed" >>$seqres.full + rm $tmpfile + return 1 + fi + + exec {FD}< $tmpfile + + read -u ${FD} ret + + if [[ $ret == -95 && $type -eq "io_uring" ]]; then + echo "btrfs io_uring encoded read failed with -EOPNOTSUPP, skipping" >>$seqres.full + exec {FD}<&- + return 1 + fi + + if [[ $ret -lt 0 ]]; then + if [[ $ret == -1 ]]; then + echo "btrfs encoded read failed with -EPERM; are you running as root?" >>$seqres.full + else + echo "btrfs encoded read failed (errno $ret)" >>$seqres.full + fi + exec {FD}<&- + return 1 + fi + + local status=0 + + if [[ $ret -ne $exp_ret ]]; then + echo "$fn: btrfs encoded read returned $ret, expected $exp_ret" >>$seqres.full + status=1 + fi + + read -u ${FD} len + read -u ${FD} unencoded_len + read -u ${FD} unencoded_offset + read -u ${FD} compression + read -u ${FD} encryption + + local filesize=`stat -c%s $tmpfile` + local datafile=`mktemp` + + tail -c +$((1+$filesize-$ret)) $tmpfile > $datafile + + exec {FD}<&- + rm $tmpfile + + local md5=`md5sum $datafile | cut -d ' ' -f 1` + rm $datafile + + if [[ $len -ne $exp_len ]]; then + echo "$fn: btrfs encoded read had len of $len, expected $exp_len" >>$seqres.full + status=1 + fi + + if [[ $unencoded_len -ne $exp_unencoded_len ]]; then + echo "$fn: btrfs encoded read had unencoded_len of $unencoded_len, expected $exp_unencoded_len" >>$seqres.full + status=1 + fi + + if [[ $unencoded_offset -ne $exp_unencoded_offset ]]; then + echo "$fn: btrfs encoded read had unencoded_offset of $unencoded_offset, expected $exp_unencoded_offset" >>$seqres.full + status=1 + fi + + if [[ $compression -ne $exp_compression ]]; then + echo "$fn: btrfs encoded read had compression of $compression, expected $exp_compression" >>$seqres.full + status=1 + fi + + if [[ $encryption -ne 0 ]]; then + echo "$fn: btrfs encoded read had encryption of $encryption, expected 0" >>$seqres.full + status=1 + fi + + if [[ $md5 != $exp_md5 ]]; then + echo "$fn: data returned had hash of $md5, expected $exp_md5" >>$seqres.full + status=1 + fi + + return $status +} + +do_encoded_write() { + local fn=$1 + local exp_ret=$2 + local len=$3 + local unencoded_len=$4 + local unencoded_offset=$5 + local compression=$6 + local data_file=$7 + + local tmpfile=`mktemp` + + echo "running btrfs_encoded_write ioctl $fn 0 $len $unencoded_len $unencoded_offset $compression < $data_file > $tmpfile" >>$seqres.full + src/btrfs_encoded_write ioctl $fn 0 $len $unencoded_len $unencoded_offset $compression < $data_file > $tmpfile + + if [[ $? -ne 0 ]]; then + echo "btrfs_encoded_write failed" >>$seqres.full + rm $tmpfile + return 1 + fi + + exec {FD}< $tmpfile + + read -u ${FD} ret + + if [[ $ret -lt 0 ]]; then + if [[ $ret == -1 ]]; then + echo "btrfs encoded write failed with -EPERM; are you running as root?" >>$seqres.full + else + echo "btrfs encoded write failed (errno $ret)" >>$seqres.full + fi + exec {FD}<&- + return 1 + fi + + exec {FD}<&- + rm $tmpfile + + return 0 +} + +test_file() { + local size=$1 + local len=$2 + local unencoded_len=$3 + local unencoded_offset=$4 + local compression=$5 + + local tmpfile=`mktemp -p $SCRATCH_MNT` + local randfile=`mktemp` + + dd if=/dev/urandom of=$randfile bs=$size count=1 status=none + local md5=`md5sum $randfile | cut -d ' ' -f 1` + + do_encoded_write $tmpfile $size $len $unencoded_len $unencoded_offset \ + $compression $randfile || _fail "encoded write ioctl failed" + + rm $randfile + + do_encoded_read $tmpfile ioctl $size $len $unencoded_len \ + $unencoded_offset $compression $md5 || _fail "encoded read ioctl failed" + do_encoded_read $tmpfile io_uring $size $len $unencoded_len \ + $unencoded_offset $compression $md5 || _fail "encoded read io_uring failed" + + rm $tmpfile +} + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +sector_size=$(_scratch_btrfs_sectorsize) + +if [[ $sector_size -ne 4096 && $sector_size -ne 65536 ]]; then + _notrun "sector size $sector_size not supported by this test" +fi + +_scratch_mount "-o max_inline=2048" + +if [[ $sector_size -eq 4096 ]]; then + test_file 40960 97966 98304 0 1 # zlib + test_file 40960 97966 98304 0 2 # zstd + test_file 40960 97966 98304 0 3 # lzo 4k + test_file 40960 97966 110592 4096 1 # bookended zlib + test_file 40960 97966 110592 4096 2 # bookended zstd + test_file 40960 97966 110592 4096 3 # bookended lzo 4k +elif [[ $sector_size -eq 65536 ]]; then + test_file 65536 97966 131072 0 1 # zlib + test_file 65536 97966 131072 0 2 # zstd + test_file 65536 97966 131072 0 7 # lzo 64k + # can't test bookended extents on 64k, as max is only 2 sectors long +fi + +# btrfs won't create inline files unless PAGE_SIZE == sector size +if [[ "$(_get_page_size)" -eq $sector_size ]]; then + test_file 892 1931 1931 0 1 # inline zlib + test_file 892 1931 1931 0 2 # inline zstd + + if [[ $sector_size -eq 4096 ]]; then + test_file 892 1931 1931 0 3 # inline lzo 4k + elif [[ $sector_size -eq 65536 ]]; then + test_file 892 1931 1931 0 7 # inline lzo 64k + fi +fi + +_scratch_unmount + +echo Silence is golden +status=0 +exit diff --git a/tests/btrfs/333.out b/tests/btrfs/333.out new file mode 100644 index 00000000..60a15898 --- /dev/null +++ b/tests/btrfs/333.out @@ -0,0 +1,2 @@ +QA output created by 333 +Silence is golden