From patchwork Thu Feb 22 18:34:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10236129 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A55A260209 for ; Thu, 22 Feb 2018 18:35:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99ECA2866D for ; Thu, 22 Feb 2018 18:35:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C92228ADD; Thu, 22 Feb 2018 18:35:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB1032866D for ; Thu, 22 Feb 2018 18:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750967AbeBVSex (ORCPT ); Thu, 22 Feb 2018 13:34:53 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:48492 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750821AbeBVSev (ORCPT ); Thu, 22 Feb 2018 13:34:51 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w1MIWFD1059778; Thu, 22 Feb 2018 18:34:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2017-10-26; bh=7Rj1p4zliCGYpVcotwF3VYok8V5lUxKQnHg0be7tJew=; b=A5z/s+sphWMm+4eZGZTUg/nAip/prZTRlzUKID0gGi4GXc23pW2dsZPnMKbV8rJ7xLfr B8lzuvWA6hMC87RJfQ088PcLOY1i0wtmBk7I19+hhFwqxhXTaKQ/to2BnYfZcsjP3IsN m9CNg7uISrZE2czUXf8BxuBZVpJM+UWfONiaQeXdccyneTOSjdVQlv9+gc1b3m5lSJVh 3F+iNmatZH1wc3+Ag9l7j9Mb62WAHB9tVjnruNry0nJ+oyf7IpED09FKktXq+B8Miqzi Hqrtq77OKkYFVZGZHFPsFqCrzCHh1gVgMfS254ulT4pGmkg4DDeYcEqw5I5Z3JVzhmxL sw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2ga322g59m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Feb 2018 18:34:47 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w1MIYkY5018019 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 22 Feb 2018 18:34:46 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w1MIYkYh004717; Thu, 22 Feb 2018 18:34:46 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 22 Feb 2018 10:34:46 -0800 Date: Thu, 22 Feb 2018 10:34:45 -0800 From: "Darrick J. Wong" To: Luis Henriques Cc: eguan@redhat.com, linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: Re: [PATCH v2 6/8] fsstress: implement the clonerange/deduperange ioctls Message-ID: <20180222183445.GG9827@magnolia> References: <151314499003.18893.8687182548758898133.stgit@magnolia> <151314503583.18893.15475795025536691678.stgit@magnolia> <20171215020731.GM6896@magnolia> <20180222160614.t2afwtdsouxliqie@hermes.olymp> <20180222172741.GD9827@magnolia> <20180222181731.jlddocrjlaaauaf3@hermes.olymp> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180222181731.jlddocrjlaaauaf3@hermes.olymp> User-Agent: Mutt/1.5.24 (2015-08-30) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8812 signatures=668677 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1802220233 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Feb 22, 2018 at 06:17:31PM +0000, Luis Henriques wrote: > On Thu, Feb 22, 2018 at 09:27:41AM -0800, Darrick J. Wong wrote: > > On Thu, Feb 22, 2018 at 04:06:14PM +0000, Luis Henriques wrote: > > > On Thu, Dec 14, 2017 at 06:07:31PM -0800, Darrick J. Wong wrote: > > > > > > > > > > > > > +void > > > > +clonerange_f( > > > > + int opno, > > > > + long r) > > > > +{ > > > > > > > > > > > > > + /* Calculate offsets */ > > > > + len = (random() % FILELEN_MAX) + 1; > > > > + len &= ~(stat1.st_blksize - 1); > > > > + if (len == 0) > > > > + len = stat1.st_blksize; > > > > + if (len > stat1.st_size) > > > > + len = stat1.st_size; > > > > + > > > > + lr = ((__int64_t)random() << 32) + random(); > > > > + if (stat1.st_size == len) > > > > + off1 = 0; > > > > + else > > > > + off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE)); > > > > + off1 %= maxfsize; > > > > + off1 &= ~(stat1.st_blksize - 1); > > > > + > > > > + /* > > > > + * If srcfile == destfile, randomly generate destination ranges > > > > + * until we find one that doesn't overlap the source range. > > > > + */ > > > > + do { > > > > + lr = ((__int64_t)random() << 32) + random(); > > > > + off2 = (off64_t)(lr % MIN(stat2.st_size + (1024 * 1024), MAXFSIZE)); > > > > + off2 %= maxfsize; > > > > + off2 &= ~(stat2.st_blksize - 1); > > > > + } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len); > > > > > > I started seeing hangs in generic/013 on cephfs. After spending some > > > time looking, I found that this loops forever. And the reason seems to > > > be that stat1.st_blksize is too big for this filesystem (4M) -- when > > > doing: > > > > "Too big for this filesystem"? > > > > Uh... maybe you'd better start by giving me more stat buffer info -- > > what's st_size? > > > > > off1 &= ~(stat1.st_blksize - 1); > > > > These bits round the start offset down to block granularity, since clone > > range implementations generally require that the ranges align to block > > boundaries. > > > > (Though AFAICT ceph doesn't support clone range anyway...) > > > > So reading between the lines, is the problem here that ceph advertises a > > blocksize of 4M and fsstress calls clonerange_f with files that are > > smaller than 4M in size, so the only possible offsets with a 4M > > blocksize are zero and that's why we end up looping forever? > > Brilliantly described! That is *exactly* what I'm seeing and failed to > describe. I guess I could use FSSTRESS_AVOID to work around this issue, > but there are probably better options. Better to patch fsstress.c against this bug. :) Does the following patch help? --D Tested-by: Luis Henriques --- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 935f5de..e107099 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -2222,6 +2222,7 @@ clonerange_f( off64_t lr; off64_t off1; off64_t off2; + off64_t max_off2; size_t len; int v1; int v2; @@ -2305,9 +2306,10 @@ clonerange_f( * If srcfile == destfile, randomly generate destination ranges * until we find one that doesn't overlap the source range. */ + max_off2 = MIN(stat2.st_size + (1024ULL * stat2.st_blksize), MAXFSIZE); do { lr = ((int64_t)random() << 32) + random(); - off2 = (off64_t)(lr % MIN(stat2.st_size + (1024 * 1024), MAXFSIZE)); + off2 = (off64_t)(lr % max_off2); off2 %= maxfsize; off2 &= ~(stat2.st_blksize - 1); } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len);