From patchwork Tue May 22 19:09:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10419425 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F26A46053B for ; Tue, 22 May 2018 19:09:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFBA42907E for ; Tue, 22 May 2018 19:09:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D1F9729081; Tue, 22 May 2018 19:09:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F3C92907E for ; Tue, 22 May 2018 19:09:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751740AbeEVTJi (ORCPT ); Tue, 22 May 2018 15:09:38 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:54140 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751411AbeEVTJg (ORCPT ); Tue, 22 May 2018 15:09:36 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0F346401EF08; Tue, 22 May 2018 19:09:36 +0000 (UTC) Received: from localhost (unknown [10.18.25.149]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B8DFF2166BD6; Tue, 22 May 2018 19:09:34 +0000 (UTC) Date: Tue, 22 May 2018 15:09:34 -0400 From: Mike Snitzer To: Christoph Hellwig Cc: Kent Overstreet , Jens Axboe , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, colyli@suse.de, darrick.wong@oracle.com, clm@fb.com, bacik@fb.com, linux-xfs@vger.kernel.org, drbd-dev@lists.linbit.com, linux-btrfs@vger.kernel.org, linux-raid@vger.kernel.org, neilb@suse.com Subject: Re: [PATCH 00/13] convert block layer to bioset_init()/mempool_init() Message-ID: <20180522190933.GA25904@redhat.com> References: <4b343aef-e11c-73ba-1d88-7e73ca838cad@kernel.dk> <20180521150439.GA19379@redhat.com> <61e30dcf-a01c-f47d-087a-12930caf9aef@kernel.dk> <20180521151817.GA19454@redhat.com> <20180521160907.GA19553@redhat.com> <20180521173724.GG11495@kmo-pixel> <20180521182427.GA20013@redhat.com> <20180521233855.GI11495@kmo-pixel> <20180522064118.GA18704@infradead.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180522064118.GA18704@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 22 May 2018 19:09:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Tue, 22 May 2018 19:09:36 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'msnitzer@redhat.com' RCPT:'' Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, May 22 2018 at 2:41am -0400, Christoph Hellwig wrote: > On Mon, May 21, 2018 at 07:38:55PM -0400, Kent Overstreet wrote: > > On Mon, May 21, 2018 at 02:24:32PM -0400, Mike Snitzer wrote: > > > Every single data structure change in this series should be reviewed for > > > unforeseen alignment consequences. Jens seemed to say that is > > > worthwhile. Not sure if he'll do it or we divide it up. If we divide > > > it up a temp topic branch should be published for others to inspect. > > > > > > Could be alignment hasn't been a historic concern for a bunch of the > > > data structures changed in this series.. if so then all we can do is fix > > > up any obvious potential for false sharing. > > > > Honestly, I almost never worry about alignment... the very few times I do care, > > I use __cacheline_aligned_in_smp. > > And that generally is the right stratgey. If Mike really doesn't want > a change we can just open code the kmalloc for the bio set there, the > important point is that we should not keep the old API around for no > good reason. Again, I never said I didn't want these changes. I just thought it worthwhile to mention the potential for alignment quirks arising. Reality is false sharing is quite uncommon. So my concern was/is pretty niche and unlikely to be applicable. That said, I did take the opportunity to look at all the DM structures that were modified. The bio_set and mempool_t structs are so large that they span multiple cachelines as is. So I focused on eliminating unnecessary spanning of cachelines (by non-bio_set and non-mempool_t members) and eliminated most holes in DM structs. This is the result: http://people.redhat.com/msnitzer/dm-4.18-struct-reorg/ Compare before.txt vs after_kent.txt vs after_mike.txt Nothing staggering or special. Just something I'll add once Kent's latest changes land. But, I also eliminated 2 4-byte holes in struct bio_set, Jens please feel free to pick this up (if you think it useful): From: Mike Snitzer Subject: [PATCH] block: eliminate 2 4-byte holes in bio_set structure Signed-off-by: Mike Snitzer --- include/linux/bio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 5e472fc..e6fc692 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -735,7 +735,6 @@ static inline void bio_inc_remaining(struct bio *bio) struct bio_set { struct kmem_cache *bio_slab; - unsigned int front_pad; mempool_t bio_pool; mempool_t bvec_pool; @@ -743,6 +742,7 @@ struct bio_set { mempool_t bio_integrity_pool; mempool_t bvec_integrity_pool; #endif + unsigned int front_pad; /* * Deadlock avoidance for stacking block drivers: see comments in