From patchwork Thu Nov 17 00:29:34 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 9433249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 14DC26021C for ; Thu, 17 Nov 2016 00:29:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0511B29198 for ; Thu, 17 Nov 2016 00:29:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE084291AF; Thu, 17 Nov 2016 00:29:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78DDE29198 for ; Thu, 17 Nov 2016 00:29:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936405AbcKQA3o (ORCPT ); Wed, 16 Nov 2016 19:29:44 -0500 Received: from mail-pg0-f52.google.com ([74.125.83.52]:36535 "EHLO mail-pg0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754296AbcKQA3n (ORCPT ); Wed, 16 Nov 2016 19:29:43 -0500 X-Greylist: delayed 86132 seconds by postgrey-1.27 at vger.kernel.org; Wed, 16 Nov 2016 19:29:43 EST Received: by mail-pg0-f52.google.com with SMTP id f188so84458663pgc.3 for ; Wed, 16 Nov 2016 16:29:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=TCA7M3IGSsxSwJA7bsxHiP6H9dA19W4DiDKsBm40VB8=; b=nIWAes1tBJb8LeBQUjTsYan7tBdhxTFice2lNZE1Qjq5LUJ3Ahb8fvrWbUNYkT1V5P habrneYwYbMYy6jDHsdRiNQRMoK/ireAwe0dck+H688G3qQGwpDuDAXIdoWe1S5l0fCF cSY1St6PAN7sk3qN832yNmf3RNwpjo/T+a52/q0cAzy+SvgZ68yhpAGl5vOVRzccQsKh hsFDwuLlO11blHRiUxPXa3iejH92GrVuk2C/AENVWnhIRSUll3x/FrT8sMIzMsy65Vr9 R5vW1hXvdPjz0smvrKeD5henUWb93MwDYed0EpRlxKkgSHJHyZuh16mo9bwLGRoTw1n3 pV9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=TCA7M3IGSsxSwJA7bsxHiP6H9dA19W4DiDKsBm40VB8=; b=I2M3WOoGZugfmArCirNv8g0xzk1rKs7H/aK8xlQDy4s73nNrzeshrkk1DPPDjdIOrg BWiN+z8JV+1QFL/mA6rXCUYzm9NnWrpe4hPf6A9Chs3sLiFH30aDrRnZv5HStDgf609D oualJpM6nHGfxDYyIB8MJVbMgeV0GSUTyWVI0+cUCbkf8DPObOeoU5fzKMNsp5z6jN4E QkyZgiqtfxe3qJAX1V4QgwsdjAajcVwsmLDiiN28hQkY0lcgivknl3S2WrBxCYj4zNVA YoPwcNelp3pIuzXF2fzUmFiDRSPeCycFX0tYRXRmNZ1gprQk9i/UOuQD5jWT6sFOG13X 4EQA== X-Gm-Message-State: ABUngvfZzZuc7ZSuE12mBkr8XnAN9uhiu97OrOjdRSUWMgwya4t5bVbkgFjcInAQOVFpCCbx X-Received: by 10.99.62.13 with SMTP id l13mr866710pga.112.1479342582283; Wed, 16 Nov 2016 16:29:42 -0800 (PST) Received: from vader.thefacebook.com ([2620:10d:c090:180::7e92]) by smtp.gmail.com with ESMTPSA id r74sm368224pfl.79.2016.11.16.16.29.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Nov 2016 16:29:41 -0800 (PST) From: Omar Sandoval To: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH] generic: test concurrent non-overlapping direct I/O on the same extents Date: Wed, 16 Nov 2016 16:29:34 -0800 Message-Id: <1edb45d8895de8fe347e800a5fd22b0f6eafb181.1479342541.git.osandov@fb.com> X-Mailer: git-send-email 2.10.2 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval There have been a couple of logic bugs in `btrfs_get_extent()` which could lead to spurious -EEXIST errors from read or write. This test exercises those conditions by having two threads race to add an extent to the extent map. This is fixed by Linux commit 8dff9c853410 ("Btrfs: deal with duplciates during extent_map insertion in btrfs_get_extent") and the patch "Btrfs: deal with existing encompassing extent map in btrfs_get_extent()" (http://marc.info/?l=linux-btrfs&m=147873402311143&w=2). Although the bug is Btrfs-specific, nothing about the test is. Signed-off-by: Omar Sandoval --- .gitignore | 1 + src/Makefile | 2 +- src/dio-interleaved.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/390 | 76 +++++++++++++++++++++++++++++++++++++++ tests/generic/390.out | 2 ++ tests/generic/group | 1 + 6 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 src/dio-interleaved.c create mode 100755 tests/generic/390 create mode 100644 tests/generic/390.out diff --git a/.gitignore b/.gitignore index 915d2d8..b8d13a0 100644 --- a/.gitignore +++ b/.gitignore @@ -44,6 +44,7 @@ /src/bulkstat_unlink_test_modified /src/dbtest /src/devzero +/src/dio-interleaved /src/dirperf /src/dirstress /src/dmiperf diff --git a/src/Makefile b/src/Makefile index dd51216..4056496 100644 --- a/src/Makefile +++ b/src/Makefile @@ -21,7 +21,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \ seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \ renameat2 t_getcwd e4compact test-nextquota punch-alternating \ - attr-list-by-handle-cursor-test listxattr + attr-list-by-handle-cursor-test listxattr dio-interleaved SUBDIRS = diff --git a/src/dio-interleaved.c b/src/dio-interleaved.c new file mode 100644 index 0000000..831a191 --- /dev/null +++ b/src/dio-interleaved.c @@ -0,0 +1,98 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static pthread_barrier_t barrier; + +static unsigned long extent_size; +static unsigned long num_extents; + +struct dio_thread_data { + int fd; + int thread_id; +}; + +static void *dio_thread(void *arg) +{ + struct dio_thread_data *data = arg; + off_t off; + ssize_t ret; + void *buf; + + if ((errno = posix_memalign(&buf, extent_size / 2, extent_size / 2))) { + perror("malloc"); + return NULL; + } + memset(buf, 0, extent_size / 2); + + off = (num_extents - 1) * extent_size; + if (data->thread_id) + off += extent_size / 2; + while (off >= 0) { + pthread_barrier_wait(&barrier); + + ret = pread(data->fd, buf, extent_size / 2, off); + if (ret == -1) + perror("pread"); + + off -= extent_size; + } + + free(buf); + return NULL; +} + +int main(int argc, char **argv) +{ + struct dio_thread_data data[2]; + pthread_t thread; + int fd; + + if (argc != 4) { + fprintf(stderr, "usage: %s SECTORSIZE NUM_EXTENTS PATH\n", + argv[0]); + return EXIT_FAILURE; + } + + extent_size = strtoul(argv[1], NULL, 0); + num_extents = strtoul(argv[2], NULL, 0); + + errno = pthread_barrier_init(&barrier, NULL, 2); + if (errno) { + perror("pthread_barrier_init"); + return EXIT_FAILURE; + } + + fd = open(argv[3], O_RDONLY | O_DIRECT); + if (fd == -1) { + perror("open"); + return EXIT_FAILURE; + } + + data[0].fd = fd; + data[0].thread_id = 0; + errno = pthread_create(&thread, NULL, dio_thread, &data[0]); + if (errno) { + perror("pthread_create"); + close(fd); + return EXIT_FAILURE; + } + + data[1].fd = fd; + data[1].thread_id = 1; + dio_thread(&data[1]); + + pthread_join(thread, NULL); + + close(fd); + return EXIT_SUCCESS; +} diff --git a/tests/generic/390 b/tests/generic/390 new file mode 100755 index 0000000..0ef6537 --- /dev/null +++ b/tests/generic/390 @@ -0,0 +1,76 @@ +#! /bin/bash +# FS QA Test 390 +# +# Test two threads doing non-overlapping direct I/O in the same extents. +# Motivated by a bug in Btrfs' direct I/O get_block function which would lead +# to spurious -EEXIST failures from direct I/O reads. +# +#----------------------------------------------------------------------- +# Copyright (c) 2016 Facebook. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + rm -f "$testfile" +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +_supported_fs generic +_supported_os Linux +_require_test +_require_xfs_io_command "falloc" +_require_test_program "dio-interleaved" + +extent_size="$(($(stat -f -c '%S' "$TEST_DIR") * 2))" +num_extents=1024 +testfile="$TEST_DIR/$$-testfile" + +truncate -s 0 "$testfile" +for ((off = 0; off < num_extents * extent_size; off += extent_size)); do + xfs_io -c "falloc $off $extent_size" "$testfile" +done + +# To reproduce the Btrfs bug, the extent map must not be cached in memory. +sync +echo 3 > /proc/sys/vm/drop_caches + +"$here/src/dio-interleaved" "$extent_size" "$num_extents" "$testfile" + +echo "Silence is golden" + +# success, all done +status=0 +exit diff --git a/tests/generic/390.out b/tests/generic/390.out new file mode 100644 index 0000000..3c7b405 --- /dev/null +++ b/tests/generic/390.out @@ -0,0 +1,2 @@ +QA output created by 390 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index 08007d7..d137d01 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -392,3 +392,4 @@ 387 auto clone 388 auto log metadata 389 auto quick acl +390 auto quick rw