From patchwork Fri Mar 1 23:27:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10836225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1DF21390 for ; Fri, 1 Mar 2019 23:27:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAA492F050 for ; Fri, 1 Mar 2019 23:27:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE6EE2F0AF; Fri, 1 Mar 2019 23:27:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3082E2F050 for ; Fri, 1 Mar 2019 23:27:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726002AbfCAX1x (ORCPT ); Fri, 1 Mar 2019 18:27:53 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:42480 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725982AbfCAX1x (ORCPT ); Fri, 1 Mar 2019 18:27:53 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x21NOD7C122734 for ; Fri, 1 Mar 2019 23:27:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Zth1NGGSsCVwHjnGePldTcS74IF04vfaJ48uMpMQ23E=; b=h1N6omkszZyK85LFyIO0zLMA9004y+ft5oZ8ysB/Pf7JYdN9GzZjz3ezX/TE/GM3u0rY csNAHHfCcyKe3cztV4kZZP32YAXPjnG2mdXHxlVhnlcq/hEuY5MvbceMi5tg2JPd0MwN T3rG8rpL63fkslb9dTjx08wojPCQ1YAQkcU5zKKySf4n6fMO/CmVOzCr0cFS+Uxt9lPN B3yhhHdGU0v2eMWBX2NLx/SWDJeG4HsI2F6659lpDeKtcLG1YT8K90RXurLOrmfGNasQ IT8Oya2fIiE7/UWyWgtqr5JDgls8iE+Y5k725Lb+0L+qI5pCLdxkkQFXCCsL2PQ3SAqv dA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qtupeteks-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 01 Mar 2019 23:27:52 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x21NRkri009532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 1 Mar 2019 23:27:46 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x21NRj7M007753 for ; Fri, 1 Mar 2019 23:27:46 GMT Received: from localhost (/10.159.229.175) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 01 Mar 2019 15:27:45 -0800 Subject: [PATCH 08/23] xfs_scrub: don't expose internal pool state From: "Darrick J. Wong" To: darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Fri, 01 Mar 2019 15:27:37 -0800 Message-ID: <155148285771.16677.2023619487236682038.stgit@magnolia> In-Reply-To: <155148280859.16677.6057998944865066232.stgit@magnolia> References: <155148280859.16677.6057998944865066232.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9182 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903010161 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In xfs_scrub, the read/verify pool tries to coalesce the media verification requests into a smaller number of large IOs. There's no need to force callers to keep track of this internal state, so just move all that into read_verify.c. Signed-off-by: Darrick J. Wong --- scrub/phase6.c | 23 +++++++---------------- scrub/read_verify.c | 40 ++++++++++++++++++++++++++++++++++------ scrub/read_verify.h | 16 ++++------------ 3 files changed, 45 insertions(+), 34 deletions(-) diff --git a/scrub/phase6.c b/scrub/phase6.c index ead48d77..fe121769 100644 --- a/scrub/phase6.c +++ b/scrub/phase6.c @@ -9,7 +9,6 @@ #include #include "handle.h" #include "path.h" -#include "ptvar.h" #include "workqueue.h" #include "xfs_scrub.h" #include "common.h" @@ -290,7 +289,6 @@ xfs_report_verify_errors( struct xfs_verify_extent { struct read_verify_pool *readverify; - struct ptvar *rvstate; struct bitmap *d_bad; /* bytes */ struct bitmap *r_bad; /* bytes */ }; @@ -424,13 +422,13 @@ xfs_check_rmap( /* Schedule the read verify command for (eventual) running. */ disk = xfs_dev_to_disk(ctx, map->fmr_device); - read_verify_schedule_io(ve->readverify, ptvar_get(ve->rvstate), disk, - map->fmr_physical, map->fmr_length, ve); + read_verify_schedule_io(ve->readverify, disk, map->fmr_physical, + map->fmr_length, ve); out: /* Is this the last extent? Fire off the read. */ if (map->fmr_flags & FMR_OF_LAST) - read_verify_force_io(ve->readverify, ptvar_get(ve->rvstate)); + read_verify_force_io(ve->readverify); return true; } @@ -450,16 +448,10 @@ xfs_scan_blocks( struct xfs_verify_extent ve; bool moveon; - ve.rvstate = ptvar_init(scrub_nproc(ctx), sizeof(struct read_verify)); - if (!ve.rvstate) { - str_errno(ctx, ctx->mntpoint); - return false; - } - moveon = bitmap_init(&ve.d_bad); if (!moveon) { str_errno(ctx, ctx->mntpoint); - goto out_ve; + goto out; } moveon = bitmap_init(&ve.r_bad); @@ -469,7 +461,8 @@ xfs_scan_blocks( } ve.readverify = read_verify_pool_init(ctx, ctx->geo.blocksize, - xfs_check_rmap_ioerr, disk_heads(ctx->datadev)); + xfs_check_rmap_ioerr, disk_heads(ctx->datadev), + scrub_nproc(ctx)); if (!ve.readverify) { moveon = false; str_info(ctx, ctx->mntpoint, @@ -489,7 +482,6 @@ _("Could not create media verifier.")); bitmap_free(&ve.r_bad); bitmap_free(&ve.d_bad); - ptvar_free(ve.rvstate); return moveon; out_pool: @@ -498,8 +490,7 @@ _("Could not create media verifier.")); bitmap_free(&ve.r_bad); out_dbad: bitmap_free(&ve.d_bad); -out_ve: - ptvar_free(ve.rvstate); +out: return moveon; } diff --git a/scrub/read_verify.c b/scrub/read_verify.c index 75cb53ca..b5774736 100644 --- a/scrub/read_verify.c +++ b/scrub/read_verify.c @@ -7,6 +7,7 @@ #include #include #include +#include "ptvar.h" #include "workqueue.h" #include "path.h" #include "xfs_scrub.h" @@ -36,22 +37,40 @@ /* Tolerate 64k holes in adjacent read verify requests. */ #define RVP_IO_BATCH_LOCALITY (65536) +struct read_verify { + void *io_end_arg; + struct disk *io_disk; + uint64_t io_start; /* bytes */ + uint64_t io_length; /* bytes */ +}; + struct read_verify_pool { struct workqueue wq; /* thread pool */ struct scrub_ctx *ctx; /* scrub context */ void *readbuf; /* read buffer */ struct ptcounter *verified_bytes; + struct ptvar *rvstate; /* combines read requests */ read_verify_ioerr_fn_t ioerr_fn; /* io error callback */ size_t miniosz; /* minimum io size, bytes */ }; -/* Create a thread pool to run read verifiers. */ +/* + * Create a thread pool to run read verifiers. + * + * @miniosz is the minimum size of an IO to expect (in bytes). + * @ioerr_fn will be called when IO errors occur. + * @nproc is the maximum number of verify requests that may be sent to a disk + * at any given time. + * @submitter_threads is the number of threads that may be sending verify + * requests at any given time. + */ struct read_verify_pool * read_verify_pool_init( struct scrub_ctx *ctx, size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc) + unsigned int nproc, + unsigned int submitter_threads) { struct read_verify_pool *rvp; bool ret; @@ -71,14 +90,20 @@ read_verify_pool_init( rvp->miniosz = miniosz; rvp->ctx = ctx; rvp->ioerr_fn = ioerr_fn; + rvp->rvstate = ptvar_init(submitter_threads, + sizeof(struct read_verify)); + if (rvp->rvstate == NULL) + goto out_counter; /* Run in the main thread if we only want one thread. */ if (nproc == 1) nproc = 0; ret = workqueue_create(&rvp->wq, (struct xfs_mount *)rvp, nproc); if (ret) - goto out_counter; + goto out_rvstate; return rvp; +out_rvstate: + ptvar_free(rvp->rvstate); out_counter: ptcounter_free(rvp->verified_bytes); out_buf: @@ -101,6 +126,7 @@ void read_verify_pool_destroy( struct read_verify_pool *rvp) { + ptvar_free(rvp->rvstate); ptcounter_free(rvp->verified_bytes); free(rvp->readbuf); free(rvp); @@ -186,16 +212,17 @@ _("Could not queue read-verify work.")); bool read_verify_schedule_io( struct read_verify_pool *rvp, - struct read_verify *rv, struct disk *disk, uint64_t start, uint64_t length, void *end_arg) { + struct read_verify *rv; uint64_t req_end; uint64_t rv_end; assert(rvp->readbuf); + rv = ptvar_get(rvp->rvstate); req_end = start + length; rv_end = rv->io_start + rv->io_length; @@ -229,12 +256,13 @@ read_verify_schedule_io( /* Force any stashed IOs into the verifier. */ bool read_verify_force_io( - struct read_verify_pool *rvp, - struct read_verify *rv) + struct read_verify_pool *rvp) { + struct read_verify *rv; bool moveon; assert(rvp->readbuf); + rv = ptvar_get(rvp->rvstate); if (rv->io_length == 0) return true; diff --git a/scrub/read_verify.h b/scrub/read_verify.h index 38f1cd1a..1e7fd83f 100644 --- a/scrub/read_verify.h +++ b/scrub/read_verify.h @@ -16,21 +16,13 @@ typedef void (*read_verify_ioerr_fn_t)(struct scrub_ctx *ctx, struct read_verify_pool *read_verify_pool_init(struct scrub_ctx *ctx, size_t miniosz, read_verify_ioerr_fn_t ioerr_fn, - unsigned int nproc); + unsigned int nproc, unsigned int submitter_threads); void read_verify_pool_flush(struct read_verify_pool *rvp); void read_verify_pool_destroy(struct read_verify_pool *rvp); -struct read_verify { - void *io_end_arg; - struct disk *io_disk; - uint64_t io_start; /* bytes */ - uint64_t io_length; /* bytes */ -}; - -bool read_verify_schedule_io(struct read_verify_pool *rvp, - struct read_verify *rv, struct disk *disk, uint64_t start, - uint64_t length, void *end_arg); -bool read_verify_force_io(struct read_verify_pool *rvp, struct read_verify *rv); +bool read_verify_schedule_io(struct read_verify_pool *rvp, struct disk *disk, + uint64_t start, uint64_t length, void *end_arg); +bool read_verify_force_io(struct read_verify_pool *rvp); uint64_t read_verify_bytes(struct read_verify_pool *rvp); #endif /* XFS_SCRUB_READ_VERIFY_H_ */