From patchwork Sat Jun 27 07:31:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B167618 for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 38AD12088E for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="L6XusXQS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38AD12088E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9566D6B0071; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 90CDD6B0073; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F3AC6B0074; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 646766B0071 for ; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1FAE12C81 for ; Sat, 27 Jun 2020 07:32:32 +0000 (UTC) X-FDA: 76974174144.16.able96_1317e1126e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id DA071100E690B for ; Sat, 27 Jun 2020 07:32:31 +0000 (UTC) X-Spam-Summary: 30,2,0,2c0cbff4a6c65a1a,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3871:3874:4250:4321:5007:6261:6653:7903:9592:10010:11026:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:12986:13069:13161:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21451:21627:21990:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfocx3zrncy5zcrjon585zgjs15ocwjm8ondu1pnfcpftpur6rxtbb5seb88t.tcp1x73scahyhc3uyhkfhwdo66se54fj38ub6o95oaes5tphqspehd44mixgf9q.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: able96_1317e1126e5c X-Filterd-Recvd-Size: 3538 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=NlgrihBG7jLl3wBy3//te4S7VLbqg8KMSQ1SCyyAmko=; b=L6XusXQSYNrkbPY1ZdnER6a0Nu Py0nKSkGK+tqq//lZU3f6lPIlLqG7AwZDlD80X6uiQQANhC9Ki8YdsEqf9AWVeG36X7qV8wvG7yQN l7JSHH5pmxXnSXhQtKZ5GfBXasr2Ed+pL5T8qBFltJr8Q3fMZih0pDDvewnEKHD28qc0qPykDOhkh RVbyPFhCwYMvDErZaaz3FWTQvFdWCrRfJCEysY6/qtq9tPk+ZEZvyZNQrZszBQBru7LQTLFoLMubm FzUcKEhDSMKiq+Zp2Io/VybxiZLnRBMn+cv+M6efXuH/gGtFhunXjbuF7DZ8bshY/XNpJ5WpkEbdO U9MgZkRQ==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jm-0006la-8A; Sat, 27 Jun 2020 07:32:07 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 01/14] dm: use bio_uninit instead of bio_disassociate_blkg Date: Sat, 27 Jun 2020 09:31:46 +0200 Message-Id: <20200627073159.2447325-2-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: DA071100E690B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bio_uninit is the proper API to clean up a BIO that has been allocated on stack or inside a structure that doesn't come from the BIO allocator. Switch dm to use that instead of bio_disassociate_blkg, which really is an implementation detail. Note that the bio_uninit calls are also moved to the two callers of __send_empty_flush, so that they better pair with the bio_init calls used to initialize them. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- drivers/md/dm.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 109e81f33edb2d..e44473fe0f4873 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1452,9 +1452,6 @@ static int __send_empty_flush(struct clone_info *ci) BUG_ON(bio_has_data(ci->bio)); while ((ti = dm_table_get_target(ci->map, target_nr++))) __send_duplicate_bios(ci, ti, ti->num_flush_bios, NULL); - - bio_disassociate_blkg(ci->bio); - return 0; } @@ -1642,6 +1639,7 @@ static blk_qc_t __split_and_process_bio(struct mapped_device *md, ci.bio = &flush_bio; ci.sector_count = 0; error = __send_empty_flush(&ci); + bio_uninit(ci.bio); /* dec_pending submits any data associated with flush */ } else if (op_is_zone_mgmt(bio_op(bio))) { ci.bio = bio; @@ -1716,6 +1714,7 @@ static blk_qc_t __process_bio(struct mapped_device *md, struct dm_table *map, ci.bio = &flush_bio; ci.sector_count = 0; error = __send_empty_flush(&ci); + bio_uninit(ci.bio); /* dec_pending submits any data associated with flush */ } else { struct dm_target_io *tio; From patchwork Sat Jun 27 07:31:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A34A7138C for ; Sat, 27 Jun 2020 07:32:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 706682081A for ; Sat, 27 Jun 2020 07:32:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="C+ZyH4XC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 706682081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3AFCA6B0062; Sat, 27 Jun 2020 03:32:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 360326B0068; Sat, 27 Jun 2020 03:32:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24C206B006C; Sat, 27 Jun 2020 03:32:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 0B8A66B0062 for ; Sat, 27 Jun 2020 03:32:28 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id CEF911EF1 for ; Sat, 27 Jun 2020 07:32:26 +0000 (UTC) X-FDA: 76974173892.04.work36_5f0146226e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id AE60D8008400 for ; Sat, 27 Jun 2020 07:32:26 +0000 (UTC) X-Spam-Summary: 1,0,0,64f10a2613f3b360,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:966:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:4321:4385:4605:5007:6117:6261:6653:7875:7903:9592:10004:11026:11473:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:12986:13894:14096:14181:14394:14721:21080:21451:21627:21990:30012:30054:30091,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yggoprscoudzcxf7r4x74c9pwfooch6ay33fgu3omeiosztkdwqp3mxgn4ta9.5fq11efrdagbmagzewuxx3xci33def13skpyo9g7a9oqbujpsafky5ecxx4knd5.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: work36_5f0146226e5c X-Filterd-Recvd-Size: 4352 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TqBMh8wKMnVF/qZj/bM6cnnHs8Yim2J3gFC2ELItu+s=; b=C+ZyH4XC/PkkkGo8OrL4qvcn92 TnizwBndOG4NHHJpYFLRpu8LYRZqZAN2x6fPq3HSY0c0038FlIUnkfpTHi3Qt2VhrxPFglng/X4Cc aZdSwTUR1u/eINbQC2X9UvbpnjV1m7wgCSeWJCDjROxe87fuIzT0EQqaXmxcSmrm4yq1F+bzu8VUN 1HvaMG5Z4RE93Payt3MIpxhHccumVWuOutqXqy60zCifApzwCB2YgfFIETkyIrT0uDpmMHAqwsDmS jmmV+CbyN7d3VFXs3yi5MBQmvswTJ9K17V+dHSM5wowN9H+wv4o0R17ejZHH5EoXtFjTRlYlOK53+ nUeLzM7Q==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jo-0006m0-Ro; Sat, 27 Jun 2020 07:32:09 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 02/14] block: remove bio_disassociate_blkg Date: Sat, 27 Jun 2020 09:31:47 +0200 Message-Id: <20200627073159.2447325-3-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: AE60D8008400 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bio_disassociate_blkg has two callers, of which one immediately assigns a new value to >bi_blkg. Just open code the function in the two callers. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/bio.c | 27 ++++++++------------------- include/linux/bio.h | 2 -- 2 files changed, 8 insertions(+), 21 deletions(-) diff --git a/block/bio.c b/block/bio.c index fb5533416fa670..8aef4460b32e0e 100644 --- a/block/bio.c +++ b/block/bio.c @@ -234,8 +234,12 @@ struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, void bio_uninit(struct bio *bio) { - bio_disassociate_blkg(bio); - +#ifdef CONFIG_BLK_CGROUP + if (bio->bi_blkg) { + blkg_put(bio->bi_blkg); + bio->bi_blkg = NULL; + } +#endif if (bio_integrity(bio)) bio_integrity_free(bio); @@ -1625,21 +1629,6 @@ EXPORT_SYMBOL(bioset_init_from_src); #ifdef CONFIG_BLK_CGROUP -/** - * bio_disassociate_blkg - puts back the blkg reference if associated - * @bio: target bio - * - * Helper to disassociate the blkg from @bio if a blkg is associated. - */ -void bio_disassociate_blkg(struct bio *bio) -{ - if (bio->bi_blkg) { - blkg_put(bio->bi_blkg); - bio->bi_blkg = NULL; - } -} -EXPORT_SYMBOL_GPL(bio_disassociate_blkg); - /** * __bio_associate_blkg - associate a bio with the a blkg * @bio: target bio @@ -1656,8 +1645,8 @@ EXPORT_SYMBOL_GPL(bio_disassociate_blkg); */ static void __bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg) { - bio_disassociate_blkg(bio); - + if (bio->bi_blkg) + blkg_put(bio->bi_blkg); bio->bi_blkg = blkg_tryget_closest(blkg); } diff --git a/include/linux/bio.h b/include/linux/bio.h index 0282f8aa85935c..4cd229e175c058 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -497,13 +497,11 @@ static inline void bio_associate_blkg_from_page(struct bio *bio, #endif #ifdef CONFIG_BLK_CGROUP -void bio_disassociate_blkg(struct bio *bio); void bio_associate_blkg(struct bio *bio); void bio_associate_blkg_from_css(struct bio *bio, struct cgroup_subsys_state *css); void bio_clone_blkg_association(struct bio *dst, struct bio *src); #else /* CONFIG_BLK_CGROUP */ -static inline void bio_disassociate_blkg(struct bio *bio) { } static inline void bio_associate_blkg(struct bio *bio) { } static inline void bio_associate_blkg_from_css(struct bio *bio, struct cgroup_subsys_state *css) From patchwork Sat Jun 27 07:31:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 905B6618 for ; Sat, 27 Jun 2020 07:32:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 53A93208C7 for ; Sat, 27 Jun 2020 07:32:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="NllTbXB2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 53A93208C7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D2736B006E; Sat, 27 Jun 2020 03:32:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 55B606B0071; Sat, 27 Jun 2020 03:32:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 449D26B0072; Sat, 27 Jun 2020 03:32:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id 313ED6B0071 for ; Sat, 27 Jun 2020 03:32:31 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E208F2C81 for ; Sat, 27 Jun 2020 07:32:30 +0000 (UTC) X-FDA: 76974174060.16.rain95_531620b26e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id B4C54100E690B for ; Sat, 27 Jun 2020 07:32:30 +0000 (UTC) X-Spam-Summary: 1,0,0,85f92d79541d41b8,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:968:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:3874:4250:5007:6261:6653:8660:9592:10004:11026:11473:11658:11914:12043:12050:12114:12160:12297:12438:12517:12519:12555:12895:13069:13148:13230:13255:13311:13357:13894:14096:14181:14384:14394:14721:21080:21451:21627:21939:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yreb1ti56xb9ohfjjhrgcj9wk4fyce8k8k7ebx4te3z97a895gn98idqfapaj.1igcrwjudp6pt41xqp7j4zb8d58gizuj78qfdhj3sx1x74373cpnfemp8srtjok.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: rain95_531620b26e5c X-Filterd-Recvd-Size: 2984 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=te67Kz6q7FtaNJUJ111t8gsYne197tgP/jW4K8xX9vA=; b=NllTbXB2YDFcVqcI043iyFsHwr tt77jKmSatOzYm5rqro2913myOlHulKnX4dt+8MOfTpepgR6TO2fYBzD4y3xZLLq57sFsuQKwjUn/ lLsBuzEeKDXVHq14z0Y+4mUXx0ae20B6us+olRYV4IeSBgdmadhqE/mKZGOP0xLqG7DAOykeARTW8 Vjwxz8hyIJzVL46ScOCngCuMa5pAPZ9e4ExDnntp/DtIrhC5V8t0SDq3ssA0P/EEZesARi7JZ4vl/ YcKpLXYo8gmWNoJberB6ct7R0JEf97ic7E/wfDM/BF56/51DV1p8WH5aKgRth6mdXE0o4D8tMK60/ N0hDQYQg==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jr-0006mC-8O; Sat, 27 Jun 2020 07:32:12 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 03/14] block: really clone the block cgroup in bio_clone_blkg_association Date: Sat, 27 Jun 2020 09:31:48 +0200 Message-Id: <20200627073159.2447325-4-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: B4C54100E690B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bio_clone_blkg_association is supposed to clone the associatation, but actually ends up doing a search with a tryget. As we know we have a reference on the source cgroup just get an unconditional additional reference to it and call it a day. That also removes the need for a RCU critical section. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/bio.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/bio.c b/block/bio.c index 8aef4460b32e0e..e1d01acce8070c 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1737,12 +1737,12 @@ EXPORT_SYMBOL_GPL(bio_associate_blkg); */ void bio_clone_blkg_association(struct bio *dst, struct bio *src) { - rcu_read_lock(); - - if (src->bi_blkg) - __bio_associate_blkg(dst, src->bi_blkg); - - rcu_read_unlock(); + if (src->bi_blkg) { + if (dst->bi_blkg) + blkg_put(dst->bi_blkg); + blkg_get(src->bi_blkg); + dst->bi_blkg = src->bi_blkg; + } } EXPORT_SYMBOL_GPL(bio_clone_blkg_association); #endif /* CONFIG_BLK_CGROUP */ From patchwork Sat Jun 27 07:31:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0825D618 for ; Sat, 27 Jun 2020 07:32:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C90FD2081A for ; Sat, 27 Jun 2020 07:32:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Y+/+SN3N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C90FD2081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D4D96B0073; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0532B6B0074; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EACE56B0078; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id B61766B0074 for ; Sat, 27 Jun 2020 03:32:32 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 634AA181AC9C6 for ; Sat, 27 Jun 2020 07:32:32 +0000 (UTC) X-FDA: 76974174144.22.war99_001135b26e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 344D318038E67 for ; Sat, 27 Jun 2020 07:32:32 +0000 (UTC) X-Spam-Summary: 1,0,0,4d1bf768df4c4403,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2005:2195:2196:2199:2200:2393:2559:2562:2693:2899:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3871:4250:4321:4385:5007:6117:6119:6261:6653:7875:7903:9592:10004:11026:11473:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12683:12895:12986:13255:13894:14096:14110:14181:14394:14721:21080:21451:21627:21987:30012:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yr483mn7phqtrhs9b576cppi9propfy8yfpnamjciwe7jcjwgattsrdacbn6d.ztaq1pmebsjtsd48ka33ij19pq4fh4acmq5px5pukgzd4tyczcp18tham8xd5kf.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMM ARY:none X-HE-Tag: war99_001135b26e5c X-Filterd-Recvd-Size: 4629 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=LREORKkL9EawusRPfY+RfGgkga9SJryaB1y7sBbv+3c=; b=Y+/+SN3Nj38jfxO2i3lNtkEEEh p5PU9kxKUEStA039sBaU80FUIIvTpcHFt36RI1FknpmJhMWGKFgiwg7s+Ms4DQI1MtXDPeUM0wIlV 38tOKbh9wlcT5Kj6Ny5CNIX5um8hs4EbmbRw5gwgMH3tffQVM7ubNGIG8NC+S/yEQMYZDHuLWxqmi 9IPc4iePF4IzCvsADXfxFap/nSeBhF4g2Upoi5yYBu1877xEsCRJU4tl1kUnbUmTivqgDceaxmw7y X4I/EqORE71u36SwJMR6c7nexLlV+8Lg6IfBUPe9FwHt6SC1jFfbKqSNrFRrIuHZR74kmPbEayDff UkYW2/YQ==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jt-0006mN-VO; Sat, 27 Jun 2020 07:32:14 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 04/14] block: merge __bio_associate_blkg into bio_associate_blkg_from_css Date: Sat, 27 Jun 2020 09:31:49 +0200 Message-Id: <20200627073159.2447325-5-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 344D318038E67 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Merge __bio_associate_blkg into the only caller, which allows to slightly reduce the RCU crticial section and better explain the code flow. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/bio.c | 45 +++++++++++++-------------------------------- 1 file changed, 13 insertions(+), 32 deletions(-) diff --git a/block/bio.c b/block/bio.c index e1d01acce8070c..bc8de2432e3645 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1628,52 +1628,33 @@ int bioset_init_from_src(struct bio_set *bs, struct bio_set *src) EXPORT_SYMBOL(bioset_init_from_src); #ifdef CONFIG_BLK_CGROUP - -/** - * __bio_associate_blkg - associate a bio with the a blkg - * @bio: target bio - * @blkg: the blkg to associate - * - * This tries to associate @bio with the specified @blkg. Association failure - * is handled by walking up the blkg tree. Therefore, the blkg associated can - * be anything between @blkg and the root_blkg. This situation only happens - * when a cgroup is dying and then the remaining bios will spill to the closest - * alive blkg. - * - * A reference will be taken on the @blkg and will be released when @bio is - * freed. - */ -static void __bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg) -{ - if (bio->bi_blkg) - blkg_put(bio->bi_blkg); - bio->bi_blkg = blkg_tryget_closest(blkg); -} - /** * bio_associate_blkg_from_css - associate a bio with a specified css * @bio: target bio * @css: target css * * Associate @bio with the blkg found by combining the css's blkg and the - * request_queue of the @bio. This falls back to the queue's root_blkg if - * the association fails with the css. + * request_queue of the @bio. An association failure is handled by walking up + * the blkg tree. Therefore, the blkg associated can be anything between @blkg + * and q->root_blkg. This situation only happens when a cgroup is dying and + * then the remaining bios will spill to the closest alive blkg. + * + * A reference will be taken on the blkg and will be released when @bio is + * freed. */ void bio_associate_blkg_from_css(struct bio *bio, struct cgroup_subsys_state *css) { struct request_queue *q = bio->bi_disk->queue; - struct blkcg_gq *blkg; + struct blkcg_gq *blkg = q->root_blkg; - rcu_read_lock(); + if (bio->bi_blkg) + blkg_put(bio->bi_blkg); - if (!css || !css->parent) - blkg = q->root_blkg; - else + rcu_read_lock(); + if (css && css->parent) blkg = blkg_lookup_create(css_to_blkcg(css), q); - - __bio_associate_blkg(bio, blkg); - + bio->bi_blkg = blkg_tryget_closest(blkg); rcu_read_unlock(); } EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); From patchwork Sat Jun 27 07:31:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5064A138C for ; Sat, 27 Jun 2020 07:32:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1446A2081A for ; Sat, 27 Jun 2020 07:32:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lXKR3KOv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1446A2081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A7426B0074; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 953C46B007B; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CE236B007D; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id 44C326B007B for ; Sat, 27 Jun 2020 03:32:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0203D181AC9C6 for ; Sat, 27 Jun 2020 07:32:33 +0000 (UTC) X-FDA: 76974174186.05.field00_620ccdf26e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id CE9031826B6A6 for ; Sat, 27 Jun 2020 07:32:32 +0000 (UTC) X-Spam-Summary: 1,0,0,b44d767e72ac1556,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:2901:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:4321:4605:5007:6117:6119:6261:6653:7875:7903:9592:10004:11026:11473:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:12986:13894:14096:14181:14394:14721:21080:21324:21451:21627:21990:30012:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04ygi8xwa8dnozohq7fxmdj6dxq1rope1yjbwpzn18t4uf989f516gz6ct7ktd8.6zuztdx817xnhhr1ibti4huke7do7hdyymy7c65ja9z7r5pngb78ujtk1kkun1n.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: field00_620ccdf26e5c X-Filterd-Recvd-Size: 4885 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Xpio8MsBnjn0ZZX6kn1sCcutxgSaexamcwAr94sVSxo=; b=lXKR3KOvyolkJvQ5vU5UcEIzmG NYPk7kN3ZRmanvN7d8Wa79vMwEVXOCKwhA5cRXqxRnIzRaQKFfeyijCzahSTeBwTtWxQdOXhVpyO/ YuHacuqeUBjX3qf3iAdjhLyYUt1l/IQAcaFJsrw2gxdW0HYASxszrpqF1yn41OUOea6/fVJZr/T6K 60U+ok5Yg/ke/5eSyH/QCSPyx1bDiWE8jYVYGCoahVg4SZqCwkcX6+WR85l+0dHx4s82sh1IT7KeT P3vou1PSNgXzRlYwf4Fvs1JsOZLLoK4ZGYC11tzwf5gfVLbj7i+vGui4+co9JaY7GjpfoIjre4B5O CO34+0jQ==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jw-0006me-CB; Sat, 27 Jun 2020 07:32:18 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 05/14] block: move bio_associate_blkg_from_page to mm/page_io.c Date: Sat, 27 Jun 2020 09:31:50 +0200 Message-Id: <20200627073159.2447325-6-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: CE9031826B6A6 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bio_associate_blkg_from_page is a special purpose helper for swap bios that doesn't need access to bio internals. Move it to the swap code instead of having it in bio.c. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Johannes Thumshirn --- block/bio.c | 26 -------------------------- include/linux/bio.h | 7 ------- mm/page_io.c | 17 +++++++++++++++++ 3 files changed, 17 insertions(+), 33 deletions(-) diff --git a/block/bio.c b/block/bio.c index bc8de2432e3645..901d22715dd4f3 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1659,32 +1659,6 @@ void bio_associate_blkg_from_css(struct bio *bio, } EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); -#ifdef CONFIG_MEMCG -/** - * bio_associate_blkg_from_page - associate a bio with the page's blkg - * @bio: target bio - * @page: the page to lookup the blkcg from - * - * Associate @bio with the blkg from @page's owning memcg and the respective - * request_queue. If cgroup_e_css returns %NULL, fall back to the queue's - * root_blkg. - */ -void bio_associate_blkg_from_page(struct bio *bio, struct page *page) -{ - struct cgroup_subsys_state *css; - - if (!page->mem_cgroup) - return; - - rcu_read_lock(); - - css = cgroup_e_css(page->mem_cgroup->css.cgroup, &io_cgrp_subsys); - bio_associate_blkg_from_css(bio, css); - - rcu_read_unlock(); -} -#endif /* CONFIG_MEMCG */ - /** * bio_associate_blkg - associate a bio with a blkg * @bio: target bio diff --git a/include/linux/bio.h b/include/linux/bio.h index 4cd229e175c058..c6d76538292644 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -489,13 +489,6 @@ do { \ #define bio_dev(bio) \ disk_devt((bio)->bi_disk) -#if defined(CONFIG_MEMCG) && defined(CONFIG_BLK_CGROUP) -void bio_associate_blkg_from_page(struct bio *bio, struct page *page); -#else -static inline void bio_associate_blkg_from_page(struct bio *bio, - struct page *page) { } -#endif - #ifdef CONFIG_BLK_CGROUP void bio_associate_blkg(struct bio *bio); void bio_associate_blkg_from_css(struct bio *bio, diff --git a/mm/page_io.c b/mm/page_io.c index e8726f3e3820bf..ccda7679008851 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -277,6 +277,23 @@ static inline void count_swpout_vm_event(struct page *page) count_vm_events(PSWPOUT, hpage_nr_pages(page)); } +#if defined(CONFIG_MEMCG) && defined(CONFIG_BLK_CGROUP) +static void bio_associate_blkg_from_page(struct bio *bio, struct page *page) +{ + struct cgroup_subsys_state *css; + + if (!page->mem_cgroup) + return; + + rcu_read_lock(); + css = cgroup_e_css(page->mem_cgroup->css.cgroup, &io_cgrp_subsys); + bio_associate_blkg_from_css(bio, css); + rcu_read_unlock(); +} +#else +#define bio_associate_blkg_from_page(bio, page) do { } while (0) +#endif /* CONFIG_MEMCG && CONFIG_BLK_CGROUP */ + int __swap_writepage(struct page *page, struct writeback_control *wbc, bio_end_io_t end_write_func) { From patchwork Sat Jun 27 07:31:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9D97138C for ; Sat, 27 Jun 2020 07:32:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6ADF32081A for ; Sat, 27 Jun 2020 07:32:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="t6EZGTFG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6ADF32081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BA1426B007B; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B4E446B007E; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A163B6B0080; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 834046B007B for ; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 755B7824556B for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-FDA: 76974174312.24.limit64_040e3c326e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 454821A4A0 for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-Spam-Summary: 1,0,0,d36221b70b3df011,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:1:2:41:69:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2005:2195:2196:2199:2200:2393:2553:2559:2562:2693:2899:2903:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3872:3874:4051:4321:4385:4605:5007:6117:6119:6261:6653:7875:7903:9592:10004:11026:11232:11473:11658:11914:12043:12160:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13894:14096:14110:14394:21080:21451:21627:21740:21987:21990:30012:30054:30090,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y867obmsco3p3eob58s47yrgkirypd7mknokjiw6qhm6yyi7jdbrys4q3n1e4.ud6x8auiyoa1sh1cxc8wiykwton7bwh5w4zkxpap9gzaxjnjsetokg8jaq7powf.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0: 0,LFtime X-HE-Tag: limit64_040e3c326e5c X-Filterd-Recvd-Size: 10358 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=nuU16mRXYOwpUOjAQFerIPbRDEcrM6bzFgPQvCmulVk=; b=t6EZGTFGBUvjVe9FE06mcfQP9/ n767z99i4jIPRAXgH9/mbSMB7z2/YITi3XlIWlAfZ5cEVNhbPatenibpnHRDe/sl6/rEwIBXSRZBh B5R6PgNKGt+bqJ46jUTvqi5oHz7KhhZcabwle7BRHv23uOHRjQ2KQu6uSFklyLr/I4Mao10HglTSW 7bvpNcxEXH1jO1UB9hjBc3/AnBPBaQ2+hLaVIO6krs64UiNsJp6N3UnZA7+qwoRU69G/KRrUCowuq embVsxiF/hDPQndhKoOtkLYTDv8CKCU0vL1mV9afQzj5X0EUgRI16ySiZNq9IhV1d2iyKgpiJqeMJ U18LHFig==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jz-0006n0-L2; Sat, 27 Jun 2020 07:32:21 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 06/14] block: move the bio cgroup associatation helpers to blk-cgroup.c Date: Sat, 27 Jun 2020 09:31:51 +0200 Message-Id: <20200627073159.2447325-7-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 454821A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Keep the cgroup code together. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/bio.c | 75 --------------------------- block/blk-cgroup.c | 103 ++++++++++++++++++++++++++++++++++++- include/linux/blk-cgroup.h | 30 ----------- 3 files changed, 101 insertions(+), 107 deletions(-) diff --git a/block/bio.c b/block/bio.c index 901d22715dd4f3..fc1299f9d86a24 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1627,81 +1627,6 @@ int bioset_init_from_src(struct bio_set *bs, struct bio_set *src) } EXPORT_SYMBOL(bioset_init_from_src); -#ifdef CONFIG_BLK_CGROUP -/** - * bio_associate_blkg_from_css - associate a bio with a specified css - * @bio: target bio - * @css: target css - * - * Associate @bio with the blkg found by combining the css's blkg and the - * request_queue of the @bio. An association failure is handled by walking up - * the blkg tree. Therefore, the blkg associated can be anything between @blkg - * and q->root_blkg. This situation only happens when a cgroup is dying and - * then the remaining bios will spill to the closest alive blkg. - * - * A reference will be taken on the blkg and will be released when @bio is - * freed. - */ -void bio_associate_blkg_from_css(struct bio *bio, - struct cgroup_subsys_state *css) -{ - struct request_queue *q = bio->bi_disk->queue; - struct blkcg_gq *blkg = q->root_blkg; - - if (bio->bi_blkg) - blkg_put(bio->bi_blkg); - - rcu_read_lock(); - if (css && css->parent) - blkg = blkg_lookup_create(css_to_blkcg(css), q); - bio->bi_blkg = blkg_tryget_closest(blkg); - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); - -/** - * bio_associate_blkg - associate a bio with a blkg - * @bio: target bio - * - * Associate @bio with the blkg found from the bio's css and request_queue. - * If one is not found, bio_lookup_blkg() creates the blkg. If a blkg is - * already associated, the css is reused and association redone as the - * request_queue may have changed. - */ -void bio_associate_blkg(struct bio *bio) -{ - struct cgroup_subsys_state *css; - - rcu_read_lock(); - - if (bio->bi_blkg) - css = &bio_blkcg(bio)->css; - else - css = blkcg_css(); - - bio_associate_blkg_from_css(bio, css); - - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(bio_associate_blkg); - -/** - * bio_clone_blkg_association - clone blkg association from src to dst bio - * @dst: destination bio - * @src: source bio - */ -void bio_clone_blkg_association(struct bio *dst, struct bio *src) -{ - if (src->bi_blkg) { - if (dst->bi_blkg) - blkg_put(dst->bi_blkg); - blkg_get(src->bi_blkg); - dst->bi_blkg = src->bi_blkg; - } -} -EXPORT_SYMBOL_GPL(bio_clone_blkg_association); -#endif /* CONFIG_BLK_CGROUP */ - static void __init biovec_init_slabs(void) { int i; diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 0ecc897b225c96..bb0607bfd771cd 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -328,7 +328,7 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, * Returns the blkg or the closest blkg if blkg_create() fails as it walks * down from root. */ -struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, +static struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, struct request_queue *q) { struct blkcg_gq *blkg; @@ -377,7 +377,7 @@ struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, * This looks up or creates the blkg representing the unique pair * of the blkcg and the request_queue. */ -struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, +static struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, struct request_queue *q) { struct blkcg_gq *blkg = blkg_lookup(blkcg, q); @@ -1727,6 +1727,105 @@ void blkcg_add_delay(struct blkcg_gq *blkg, u64 now, u64 delta) atomic64_add(delta, &blkg->delay_nsec); } +/** + * blkg_tryget_closest - try and get a blkg ref on the closet blkg + * @blkg: blkg to get + * + * This needs to be called rcu protected. As the failure mode here is to walk + * up the blkg tree, this ensure that the blkg->parent pointers are always + * valid. This returns the blkg that it ended up taking a reference on or %NULL + * if no reference was taken. + */ +static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg) +{ + struct blkcg_gq *ret_blkg = NULL; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + while (blkg) { + if (blkg_tryget(blkg)) { + ret_blkg = blkg; + break; + } + blkg = blkg->parent; + } + + return ret_blkg; +} + +/** + * bio_associate_blkg_from_css - associate a bio with a specified css + * @bio: target bio + * @css: target css + * + * Associate @bio with the blkg found by combining the css's blkg and the + * request_queue of the @bio. An association failure is handled by walking up + * the blkg tree. Therefore, the blkg associated can be anything between @blkg + * and q->root_blkg. This situation only happens when a cgroup is dying and + * then the remaining bios will spill to the closest alive blkg. + * + * A reference will be taken on the blkg and will be released when @bio is + * freed. + */ +void bio_associate_blkg_from_css(struct bio *bio, + struct cgroup_subsys_state *css) +{ + struct request_queue *q = bio->bi_disk->queue; + struct blkcg_gq *blkg = q->root_blkg; + + if (bio->bi_blkg) + blkg_put(bio->bi_blkg); + + rcu_read_lock(); + if (css && css->parent) + blkg = blkg_lookup_create(css_to_blkcg(css), q); + bio->bi_blkg = blkg_tryget_closest(blkg); + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); + +/** + * bio_associate_blkg - associate a bio with a blkg + * @bio: target bio + * + * Associate @bio with the blkg found from the bio's css and request_queue. + * If one is not found, bio_lookup_blkg() creates the blkg. If a blkg is + * already associated, the css is reused and association redone as the + * request_queue may have changed. + */ +void bio_associate_blkg(struct bio *bio) +{ + struct cgroup_subsys_state *css; + + rcu_read_lock(); + + if (bio->bi_blkg) + css = &bio_blkcg(bio)->css; + else + css = blkcg_css(); + + bio_associate_blkg_from_css(bio, css); + + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(bio_associate_blkg); + +/** + * bio_clone_blkg_association - clone blkg association from src to dst bio + * @dst: destination bio + * @src: source bio + */ +void bio_clone_blkg_association(struct bio *dst, struct bio *src) +{ + if (src->bi_blkg) { + if (dst->bi_blkg) + blkg_put(dst->bi_blkg); + blkg_get(src->bi_blkg); + dst->bi_blkg = src->bi_blkg; + } +} +EXPORT_SYMBOL_GPL(bio_clone_blkg_association); + static int __init blkcg_init(void) { blkcg_punt_bio_wq = alloc_workqueue("blkcg_punt_bio", diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index a57ebe2f00abd8..60df97202314c7 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -183,10 +183,6 @@ extern bool blkcg_debug_stats; struct blkcg_gq *blkg_lookup_slowpath(struct blkcg *blkcg, struct request_queue *q, bool update_hint); -struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q); -struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q); int blkcg_init_queue(struct request_queue *q); void blkcg_exit_queue(struct request_queue *q); @@ -480,32 +476,6 @@ static inline bool blkg_tryget(struct blkcg_gq *blkg) return blkg && percpu_ref_tryget(&blkg->refcnt); } -/** - * blkg_tryget_closest - try and get a blkg ref on the closet blkg - * @blkg: blkg to get - * - * This needs to be called rcu protected. As the failure mode here is to walk - * up the blkg tree, this ensure that the blkg->parent pointers are always - * valid. This returns the blkg that it ended up taking a reference on or %NULL - * if no reference was taken. - */ -static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg) -{ - struct blkcg_gq *ret_blkg = NULL; - - WARN_ON_ONCE(!rcu_read_lock_held()); - - while (blkg) { - if (blkg_tryget(blkg)) { - ret_blkg = blkg; - break; - } - blkg = blkg->parent; - } - - return ret_blkg; -} - /** * blkg_put - put a blkg reference * @blkg: blkg to put From patchwork Sat Jun 27 07:31:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D68B618 for ; Sat, 27 Jun 2020 07:32:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 116D62081A for ; Sat, 27 Jun 2020 07:32:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YdxLWQTA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 116D62081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 44C7C6B007E; Sat, 27 Jun 2020 03:32:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3FC8D6B0081; Sat, 27 Jun 2020 03:32:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C6026B0082; Sat, 27 Jun 2020 03:32:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 057D26B0081 for ; Sat, 27 Jun 2020 03:32:38 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EEC43181AC9C6 for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-FDA: 76974174312.06.rod30_520ee1826e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id C787710040CBF for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-Spam-Summary: 1,0,0,e30465b54ceb1fc6,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:2901:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3871:3872:3874:4250:4321:4605:5007:6117:6119:6261:6653:7875:7903:9592:10004:11026:11473:11658:11914:12043:12160:12296:12297:12438:12517:12519:12555:12683:12895:12986:13894:14096:14110:14181:14394:14721:21080:21324:21433:21451:21627:21990:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfwhanzumcjp54r7anafczjnhntopzu7qhscu1xjukm7d6aywujezn13gtoec.8w8shuiubd1fquqm84nm4q3zaikgatzyuefc8dpj49t3c48xc1jnn5qt656unbp.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: rod30_520ee1826e5c X-Filterd-Recvd-Size: 4783 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=S7ew7tFSTCy/AgXfhknQUe2b8dthYpBxIvPKvi74jmI=; b=YdxLWQTAi5Ieyyatgq0h9uJfn1 BGXkahQmOZuu9szIz6bz4ij6CxtHBH4Ab+5mc0WAE/Lz8VHb2l3vz9Nv11Uau3jfnvvMhlNtPph2W 476eG96my0sNe0X6xcuGOWkoLNTLAI+BVMhRNE0mZwr/vfKAhSeeNtn3WYUlfeHKbQ9zEBNfM1+jG v8F/RPXkYoIfuas7wxO0yuB2C3eigotYi1AfVik7M7Aq4YZM4wdxg5AJyNa4tU1cYZ+Q1Y+i+4Y16 g6mVz15f41EyQ+5CSe/9nlppdQ+Cj70QyR+cStzXHF091tiiVKIUcfiuT+A4vIT+UoaeT0mQtmyXX 0VA1lnlw==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5K3-0006nC-48; Sat, 27 Jun 2020 07:32:24 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 07/14] block: merge blkg_lookup_create and __blkg_lookup_create Date: Sat, 27 Jun 2020 09:31:52 +0200 Message-Id: <20200627073159.2447325-8-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: C787710040CBF X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: No good reason to keep these two functions split. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/blk-cgroup.c | 49 +++++++++++++++++----------------------------- 1 file changed, 18 insertions(+), 31 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index bb0607bfd771cd..6aedb73ffbf473 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -316,30 +316,35 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, } /** - * __blkg_lookup_create - lookup blkg, try to create one if not there + * blkg_lookup_create - lookup blkg, try to create one if not there * @blkcg: blkcg of interest * @q: request_queue of interest * * Lookup blkg for the @blkcg - @q pair. If it doesn't exist, try to * create one. blkg creation is performed recursively from blkcg_root such * that all non-root blkg's have access to the parent blkg. This function - * should be called under RCU read lock and @q->queue_lock. + * should be called under RCU read lock and takes @q->queue_lock. * * Returns the blkg or the closest blkg if blkg_create() fails as it walks * down from root. */ -static struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q) +static struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, + struct request_queue *q) { struct blkcg_gq *blkg; + unsigned long flags; WARN_ON_ONCE(!rcu_read_lock_held()); - lockdep_assert_held(&q->queue_lock); - blkg = __blkg_lookup(blkcg, q, true); + blkg = blkg_lookup(blkcg, q); if (blkg) return blkg; + spin_lock_irqsave(&q->queue_lock, flags); + blkg = __blkg_lookup(blkcg, q, true); + if (blkg) + goto found; + /* * Create blkgs walking down from blkcg_root to @blkcg, so that all * non-root blkgs have access to their parents. Returns the closest @@ -362,34 +367,16 @@ static struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, } blkg = blkg_create(pos, q, NULL); - if (IS_ERR(blkg)) - return ret_blkg; + if (IS_ERR(blkg)) { + blkg = ret_blkg; + break; + } if (pos == blkcg) - return blkg; - } -} - -/** - * blkg_lookup_create - find or create a blkg - * @blkcg: target block cgroup - * @q: target request_queue - * - * This looks up or creates the blkg representing the unique pair - * of the blkcg and the request_queue. - */ -static struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q) -{ - struct blkcg_gq *blkg = blkg_lookup(blkcg, q); - - if (unlikely(!blkg)) { - unsigned long flags; - - spin_lock_irqsave(&q->queue_lock, flags); - blkg = __blkg_lookup_create(blkcg, q); - spin_unlock_irqrestore(&q->queue_lock, flags); + break; } +found: + spin_unlock_irqrestore(&q->queue_lock, flags); return blkg; } From patchwork Sat Jun 27 07:31:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E13F8618 for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A51BD2088E for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DUd0a08w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A51BD2088E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 44C716B0081; Sat, 27 Jun 2020 03:32:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 387D56B0083; Sat, 27 Jun 2020 03:32:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24EC56B0085; Sat, 27 Jun 2020 03:32:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id F019E6B0081 for ; Sat, 27 Jun 2020 03:32:47 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A24AE824556B for ; Sat, 27 Jun 2020 07:32:47 +0000 (UTC) X-FDA: 76974174774.30.crate01_0008fbf26e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 7EDB7180B3C85 for ; Sat, 27 Jun 2020 07:32:47 +0000 (UTC) X-Spam-Summary: 1,0,0,d7fc892e96045d28,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3868:3870:3871:3874:4321:5007:6117:6119:6261:6653:10004:11026:11473:11658:11914:12043:12160:12296:12297:12438:12517:12519:12555:12895:12986:13069:13161:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21451:21627:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yg7wgxxg73w7ozr94cahonpmu1sop4i7y6hn5ajxtgdqr9posxgo4noqqo78h.j986fnn5fmrt63kgji5zqgcjen5iqd69zkmwcyqryd77sbswwypgw7bgjboww9n.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: crate01_0008fbf26e5c X-Filterd-Recvd-Size: 3188 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=GGiSOAs9cZu8YPYPdf6bT1AGgpeRSLCWtTfRUcMw8po=; b=DUd0a08wTpSbY0adkAupOokAWp +Ij/hgTuW5RW75ccKxqhkrzy9fhx8QsEhlJKu3R416Iffa4LObjU5Dt9pIPT/J6Dt9Re9JjgdyQ5c LPBjuoM0NSylWVyDxwhs9szHi+BMAKgqpVlGWUY3IES6andSNvUGwycll6w2IEIuBKXqlCiFjjicR YpbE2VIphHNWEyGkkQnFmwGIheMgfuzckSiee4Nb5Y5RgYozISDJWrYobSvebkTt7YNEmmqTA2lqy QH9+idB2SLdYLMAEi4Rfo7HZjjf4bHV5w8hynB0ab7WZ7U/rr5ZkDf1cWePpT/dHugveyfSyOJXqA 1YK96cEw==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5K8-0006nQ-7F; Sat, 27 Jun 2020 07:32:29 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 08/14] block: bypass blkg_tryget_closest for the root_blkg Date: Sat, 27 Jun 2020 09:31:53 +0200 Message-Id: <20200627073159.2447325-9-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 7EDB7180B3C85 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The root_blkg is only torn down at the very end of removing a queue. So in the I/O submission path is always has a life reference and we can just grab another one using blkg_get instead of doing a tryget and parent walk that won't lead anywhere. Signed-off-by: Christoph Hellwig --- block/blk-cgroup.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 6aedb73ffbf473..0912820d4f8194 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1758,16 +1758,21 @@ void bio_associate_blkg_from_css(struct bio *bio, struct cgroup_subsys_state *css) { struct request_queue *q = bio->bi_disk->queue; - struct blkcg_gq *blkg = q->root_blkg; if (bio->bi_blkg) blkg_put(bio->bi_blkg); - rcu_read_lock(); - if (css && css->parent) + if (css && css->parent) { + struct blkcg_gq *blkg; + + rcu_read_lock(); blkg = blkg_lookup_create(css_to_blkcg(css), q); - bio->bi_blkg = blkg_tryget_closest(blkg); - rcu_read_unlock(); + bio->bi_blkg = blkg_tryget_closest(blkg); + rcu_read_unlock(); + } else { + blkg_get(q->root_blkg); + bio->bi_blkg = q->root_blkg; + } } EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); From patchwork Sat Jun 27 07:31:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDCD9138C for ; Sat, 27 Jun 2020 07:32:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB2292081A for ; Sat, 27 Jun 2020 07:32:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="T1TDJ0/J" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB2292081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A19ED6B0083; Sat, 27 Jun 2020 03:32:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 97A046B0087; Sat, 27 Jun 2020 03:32:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 784336B0088; Sat, 27 Jun 2020 03:32:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id 549986B0083 for ; Sat, 27 Jun 2020 03:32:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 48AE8180AD820 for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) X-FDA: 76974174858.27.angle58_391668526e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 259633D663 for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) X-Spam-Summary: 1,0,0,5863de48baa44879,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:2899:2903:3138:3139:3140:3141:3142:3354:3865:3867:3868:3871:3872:3874:4250:4321:5007:6117:6119:6261:6653:7901:7903:9592:10004:11026:11232:11473:11658:11914:12043:12160:12296:12297:12438:12517:12519:12555:12683:12895:13894:14096:14110:14181:14394:14721:21080:21324:21451:21627:21990:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yrqkc7dcsy83i33p83sunurd4bjycq9ntpfjyd3jxpznedmz6u1o8xriewneq.u5ceoy9tzsdz1xbf9pohy9k5n9ecdfcxicop8bnkzcujgiu49eziyjbe63ogy64.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: angle58_391668526e5c X-Filterd-Recvd-Size: 4566 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=EZ7JERHE6/tvgP05nSclplXZEEiO4tv/M9Y5/s349gU=; b=T1TDJ0/J5jy3E+tsjXEkkkQ57r DhTeVGJD6SyfqR7bG4EpVAZ4on6AuyexkciKBJq5jqo6yUrmcUIWGJFK18KEzuQjvYqHVKxsJAv7O CrWwF/zVPEGAGC11hBr6joU3qnoPHNwqJfip9BYtxtGUfFQ9nWnOQKO8uoCzQ0ERlalMvcsD9VPTd TZ02agzs3BQbsAFYWFHNFA9IB+Zx12YJRY0Kh5i2WO13j3+uy0OuRvvC8IEj3b+v5ZoXTdGzyQt7l sBnCrwjHbU9OdYu405TDLDw6RTpQiloFPnA1ntOXm91l3yJZI0shWxBSBteMEqGJwOMjzSq27xYrR R+6srrvQ==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KB-0006na-7G; Sat, 27 Jun 2020 07:32:32 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 09/14] block: move the initial blkg lookup into blkg_tryget_closest Date: Sat, 27 Jun 2020 09:31:54 +0200 Message-Id: <20200627073159.2447325-10-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 259633D663 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: By moving the initial blkg lookup into blkg_tryget_closest we get a nicely self contained routines that does all the RCU locking. Signed-off-by: Christoph Hellwig --- block/blk-cgroup.c | 33 ++++++++++++++------------------- 1 file changed, 14 insertions(+), 19 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 0912820d4f8194..d21ec2acd716e7 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1716,19 +1716,20 @@ void blkcg_add_delay(struct blkcg_gq *blkg, u64 now, u64 delta) /** * blkg_tryget_closest - try and get a blkg ref on the closet blkg - * @blkg: blkg to get + * @bio: target bio + * @css: target css * - * This needs to be called rcu protected. As the failure mode here is to walk - * up the blkg tree, this ensure that the blkg->parent pointers are always - * valid. This returns the blkg that it ended up taking a reference on or %NULL - * if no reference was taken. + * As the failure mode here is to walk up the blkg tree, this ensure that the + * blkg->parent pointers are always valid. This returns the blkg that it ended + * up taking a reference on or %NULL if no reference was taken. */ -static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg) +static inline struct blkcg_gq *blkg_tryget_closest(struct bio *bio, + struct cgroup_subsys_state *css) { - struct blkcg_gq *ret_blkg = NULL; - - WARN_ON_ONCE(!rcu_read_lock_held()); + struct blkcg_gq *blkg, *ret_blkg = NULL; + rcu_read_lock(); + blkg = blkg_lookup_create(css_to_blkcg(css), bio->bi_disk->queue); while (blkg) { if (blkg_tryget(blkg)) { ret_blkg = blkg; @@ -1736,6 +1737,7 @@ static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg) } blkg = blkg->parent; } + rcu_read_unlock(); return ret_blkg; } @@ -1757,21 +1759,14 @@ static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg) void bio_associate_blkg_from_css(struct bio *bio, struct cgroup_subsys_state *css) { - struct request_queue *q = bio->bi_disk->queue; - if (bio->bi_blkg) blkg_put(bio->bi_blkg); if (css && css->parent) { - struct blkcg_gq *blkg; - - rcu_read_lock(); - blkg = blkg_lookup_create(css_to_blkcg(css), q); - bio->bi_blkg = blkg_tryget_closest(blkg); - rcu_read_unlock(); + bio->bi_blkg = blkg_tryget_closest(bio, css); } else { - blkg_get(q->root_blkg); - bio->bi_blkg = q->root_blkg; + blkg_get(bio->bi_disk->queue->root_blkg); + bio->bi_blkg = bio->bi_disk->queue->root_blkg; } } EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); From patchwork Sat Jun 27 07:31:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B86A3618 for ; Sat, 27 Jun 2020 07:32:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7C4C22088E for ; Sat, 27 Jun 2020 07:32:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Pg+uD5yS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C4C22088E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7ACD76B0087; Sat, 27 Jun 2020 03:32:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6E4D56B0089; Sat, 27 Jun 2020 03:32:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 588A56B008A; Sat, 27 Jun 2020 03:32:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id DDAD46B0089 for ; Sat, 27 Jun 2020 03:32:50 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D60581EF1 for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) X-FDA: 76974174858.06.frame99_021292826e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id AA468100404E9 for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) X-Spam-Summary: 1,0,0,7e0809b31b942c48,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:69:355:379:541:800:960:968:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3867:3870:3871:3872:4321:5007:6117:6261:6653:7903:9592:10004:11026:11232:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:12986:13069:13255:13311:13357:13894:14181:14384:14394:14721:21080:21451:21627:21740:30054:30070,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04ygp7yka4zk3sfpcin7mbrbazxkdyppatgrop4ntqe98jq5gx4nw8k9ejnh5i4.wcghq3tcpnffcys9qfe1gk938zp8j9h5bn4nww9dc81mq4uizt5mdsujrw74tnz.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: frame99_021292826e5c X-Filterd-Recvd-Size: 3021 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=nSd3C/DSNsC2O/2jsqExpsnu1gd7FF5TZdmvZuWRNPs=; b=Pg+uD5ySHobKze+G3z72vgMqje I3LI6dZfO3nOCLTW9PT7d6ugzTiNDCEtNY0mB8yeFck0uVAD2NK5kK59O4flLxaLCnECUhOYCji/N 23QiHSqYjFn0jgYdIdjxCpjH5iZhuxzwRMC4wcK+4Twctub06mZx70N39Cbqk/hfMBAkUlPxfTQ2g 31pSU1TgdBefiv2HiZm9UGCDAiZyPFWA+BRsBAHsiC++udMAeoOwBhV/B6DEPUDWSu+2pKW9N+h9T 9oxhhPsi7vxIWB8qxOh9/U4FNAjdd0QkyaliIKRQ+QJF/o7aLfD7PoAqxpk6jCQ1OXTSVMnAoGcrS q7oYemlA==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KD-0006nj-M4; Sat, 27 Jun 2020 07:32:35 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 10/14] blk-cgroup: remove the !bio->bi_blkg check in blkcg_bio_issue_check Date: Sat, 27 Jun 2020 09:31:55 +0200 Message-Id: <20200627073159.2447325-11-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: AA468100404E9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is purely a sanity check for grave programming errors. Remove it to simplify further work in this area. Signed-off-by: Christoph Hellwig --- include/linux/blk-cgroup.h | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index 60df97202314c7..8e86b598316c10 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -543,24 +543,11 @@ static inline void blkcg_bio_issue_init(struct bio *bio) static inline bool blkcg_bio_issue_check(struct request_queue *q, struct bio *bio) { - struct blkcg_gq *blkg; + struct blkcg_gq *blkg = bio->bi_blkg; bool throtl = false; rcu_read_lock(); - - if (!bio->bi_blkg) { - char b[BDEVNAME_SIZE]; - - WARN_ONCE(1, - "no blkg associated for bio on block-device: %s\n", - bio_devname(bio, b)); - bio_associate_blkg(bio); - } - - blkg = bio->bi_blkg; - throtl = blk_throtl_bio(q, blkg, bio); - if (!throtl) { struct blkg_iostat_set *bis; int rwd, cpu; From patchwork Sat Jun 27 07:31:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3CBC138C for ; Sat, 27 Jun 2020 07:32:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 818BA2081A for ; Sat, 27 Jun 2020 07:32:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ct6ICbrd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 818BA2081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A8F406B0089; Sat, 27 Jun 2020 03:32:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F29F6B008C; Sat, 27 Jun 2020 03:32:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F5DE6B0092; Sat, 27 Jun 2020 03:32:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 631CE6B0089 for ; Sat, 27 Jun 2020 03:32:55 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5C787824556B for ; Sat, 27 Jun 2020 07:32:54 +0000 (UTC) X-FDA: 76974175068.17.spade76_620e60926e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 3711B180D0180 for ; Sat, 27 Jun 2020 07:32:54 +0000 (UTC) X-Spam-Summary: 1,0,0,0b04234e86fdfe4e,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1539:1567:1711:1714:1730:1747:1777:1792:2393:2559:2562:2895:3138:3139:3140:3141:3142:3867:3868:3871:3872:5007:6261:6653:7875:7903:10004:11026:11473:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:13069:13311:13357:13894:14181:14384:14394:14721:21080:21451:21627:30012:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yf6hx8eftrnm6igqy4y8dsseozmyp1ib46z63q4u7ikewr1t3sy953i3s7urn.e9h3bp7z7jhr7d1xs51iw5keodexgibgujhqi9cw83bcf8iomcnxaaui1kyhtst.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: spade76_620e60926e5c X-Filterd-Recvd-Size: 2531 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=/aQSpo4n3xdtcdftvKvpaQvh2VA9RA7aMy8HIr4zPtc=; b=Ct6ICbrdi+RdZnvBXmsBtJLXgd JyygBSI7zac+fAOFdtzFfO5pB3w5jxxzoRyF3zvrexxkDBNYOfEBbAHTAaZaOcFQ4YsnFdnEW9lCA redVfqYvEHSd2b+evoixC9bsJJ3lipbPl9SX/+Usc95saeL0OIIc2NZGNhV52k3SVtDbtXSnWFZIE ZyiuwFNKcsJgY/seQavqFWtu9yydvuhdZcQRPYZtdGVCV5h/VUHHyenRHE2wo1+bS9V4pSQmmOUag dJx3sW2qxWhk1UdRNYfGgF8MHs+8WRmq9wnFoCbgKD9djg5otbgJvNW7VYmRzFgoPqFXSDe/ZNnnt v6cdokcA==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KH-0006o1-7C; Sat, 27 Jun 2020 07:32:38 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 11/14] cgroup: unexport cgroup_rstat_updated Date: Sat, 27 Jun 2020 09:31:56 +0200 Message-Id: <20200627073159.2447325-12-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 3711B180D0180 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: cgroup_rstat_updated is only used by core block code, no need to export it. Signed-off-by: Christoph Hellwig --- kernel/cgroup/rstat.c | 1 - 1 file changed, 1 deletion(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index b6397a186ce9c8..d51175cedfca4f 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -64,7 +64,6 @@ void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) raw_spin_unlock_irqrestore(cpu_lock, flags); } -EXPORT_SYMBOL_GPL(cgroup_rstat_updated); /** * cgroup_rstat_cpu_pop_updated - iterate and dismantle rstat_cpu updated tree From patchwork Sat Jun 27 07:31:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 429F3618 for ; Sat, 27 Jun 2020 07:33:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 05D322081A for ; Sat, 27 Jun 2020 07:33:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lBBFTQqt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05D322081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CDF866B008C; Sat, 27 Jun 2020 03:32:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C63BA6B0093; Sat, 27 Jun 2020 03:32:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2E3B6B0095; Sat, 27 Jun 2020 03:32:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 942AB6B008C for ; Sat, 27 Jun 2020 03:32:59 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5A269180AD801 for ; Sat, 27 Jun 2020 07:32:59 +0000 (UTC) X-FDA: 76974175278.04.straw06_61016e526e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 38BB38009669 for ; Sat, 27 Jun 2020 07:32:59 +0000 (UTC) X-Spam-Summary: 1,0,0,6fbeec2ffb8789d4,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3874:4321:4605:5007:6117:6261:6653:10004:11026:11232:11473:11658:11914:12043:12114:12160:12296:12297:12438:12517:12519:12555:12895:12986:13069:13161:13229:13255:13311:13357:13894:14096:14181:14384:14394:14721:14877:21080:21451:21627:21990:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y8nbfon687ff1jrcf6sp3pn6joroppitrwaogiw3qijbym8gq4swprbwt68qa.robygu8sntpq46qpkja1qyi7796a9g4uxq4zhf4t3ok7dpsrf8fo5mzzxp3jpua.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: straw06_61016e526e5c X-Filterd-Recvd-Size: 3576 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TotXMoDr4SMB2qAUQWeT1iUDwtwRZjQn0sDaTbNw3KE=; b=lBBFTQqtqPDmKcz4C2xD/zM/hu Lrzy51/shbHq+GN4MpENNU+6yV67bBHqSSioPVy0RchQm5VxOyJgLQP6umNIPnNZ2sR945Vfxjpd/ /hue1Aei3ALd+WQ2tq3K3FcM4HyMRuyCPeIuUQ7cwJigY8EEgTF8r0aD2/qwQt91Vywh+ZIu0toG+ +PkK+1vEPRG9l1s0mKPgGsfxyDttV9haep1R+Uwaj/JQWy5QRXi8xPwioRwTEP/ScYyGwmFxbkJ/c hXdX1GFMy79Vf4ofCFXTs3Pd+ijSmZ0PHxDPNtultydJoEPMMuE9W7sagWDR6VRHNNYHh5ENAddnC YrPOgNRg==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KO-0006oz-Bj; Sat, 27 Jun 2020 07:32:46 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 12/14] blk-cgroup: move rcu locking from blkcg_bio_issue_check to blk_throtl_bio Date: Sat, 27 Jun 2020 09:31:57 +0200 Message-Id: <20200627073159.2447325-13-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 38BB38009669 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The only thing in blkcg_bio_issue_check that needs to be under rcu_read_lock is blk_throtl_bio, so move the locking there. Signed-off-by: Christoph Hellwig --- block/blk-throttle.c | 3 ++- include/linux/blk-cgroup.h | 2 -- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 209fdd8939fba6..ac008345050010 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2168,7 +2168,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, bool throttled = false; struct throtl_data *td = tg->td; - WARN_ON_ONCE(!rcu_read_lock_held()); + rcu_read_lock(); /* see throtl_charge_bio() */ if (bio_flagged(bio, BIO_THROTTLED)) @@ -2273,6 +2273,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, if (throttled || !td->track_bio_latency) bio->bi_issue.value |= BIO_ISSUE_THROTL_SKIP_LATENCY; #endif + rcu_read_unlock(); return throttled; } diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index 8e86b598316c10..8ab043c911f233 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -546,7 +546,6 @@ static inline bool blkcg_bio_issue_check(struct request_queue *q, struct blkcg_gq *blkg = bio->bi_blkg; bool throtl = false; - rcu_read_lock(); throtl = blk_throtl_bio(q, blkg, bio); if (!throtl) { struct blkg_iostat_set *bis; @@ -582,7 +581,6 @@ static inline bool blkcg_bio_issue_check(struct request_queue *q, blkcg_bio_issue_init(bio); - rcu_read_unlock(); return !throtl; } From patchwork Sat Jun 27 07:31:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98A7D618 for ; Sat, 27 Jun 2020 07:33:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 595942081A for ; Sat, 27 Jun 2020 07:33:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="u0A0PX8v" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 595942081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3133F6B0093; Sat, 27 Jun 2020 03:33:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 29DA26B0096; Sat, 27 Jun 2020 03:33:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1419B6B0098; Sat, 27 Jun 2020 03:33:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id EA9C66B0093 for ; Sat, 27 Jun 2020 03:33:08 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E14B81EE6 for ; Sat, 27 Jun 2020 07:33:07 +0000 (UTC) X-FDA: 76974175614.09.tail83_5d0e16a26e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id C42BD180AD801 for ; Sat, 27 Jun 2020 07:33:07 +0000 (UTC) X-Spam-Summary: 1,0,0,df0ed970c9fbdb8d,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3865:3866:3868:3870:3871:3872:3874:4050:4120:4321:4384:4605:5007:6117:6119:6261:6653:7903:8603:9592:10004:11026:11473:11658:11914:12043:12114:12160:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13894:13972:14394:14877:21080:21433:21450:21451:21627:21740:21990:30012:30054,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y8uzummjufeco9fe1c9neoehpjkoc5m75m8bx4brrhsagmk1ueuf58jfaep57.dmopxubs1614f9sowxtuxcrqjbp7nob5a3zjdubnwtqo4g4ugfwttrxicfabuey.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: tail83_5d0e16a26e5c X-Filterd-Recvd-Size: 9085 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:33:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=neOgU92RsrWDuksyQ2Z1wFdOnnGrSBcBhrDlZznOqq0=; b=u0A0PX8v6CDjtRXkekDJ9GPZcA FENXXFhI2izz8lcBcKb5aWOhXCPP2z4UVHieA7JWGSgOpclTA8JLr2Pp1pX2NQ83uXTaPwMmz5vAU U2X35b2VMZAlui75iR9AdUpKd9NGKPiN9uhpWEHxitnCXeatyhWsBdhNbQfPMM+0yjf4gSx3Z3HaH GWNHb/bJK5RuN1uXcAatmVe3NAk4BLtr/tn3Z2XPqLxGW1D9WJjqmZ2uo9zwBhUeXVOc4oJhuCyfR pxw/1+5SLMZzgs5r+Y0Gbv0CQqwFjeqFeXcCMxPp7rSw3Wq8fhJlCzQsJb5+fN9oNwPgV6zq4T/cc YpgPDRLQ==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KS-0006q3-6c; Sat, 27 Jun 2020 07:32:51 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 13/14] blk-cgroup: remove blkcg_bio_issue_check Date: Sat, 27 Jun 2020 09:31:58 +0200 Message-Id: <20200627073159.2447325-14-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: C42BD180AD801 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: blkcg_bio_issue_check is a giant inline function that does three entirely different things. Factor out the blk-cgroup related bio initalization into a new helper, and the open code the sequence in the only caller, relying on the fact that all the actual functionality is stubbed out for non-cgroup builds. Signed-off-by: Christoph Hellwig --- block/blk-cgroup.c | 34 +++++++++++++++++++++++ block/blk-core.c | 7 ++++- block/blk-throttle.c | 5 ++-- block/blk.h | 2 ++ include/linux/blk-cgroup.h | 56 ++------------------------------------ 5 files changed, 47 insertions(+), 57 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index d21ec2acd716e7..1ce94afc03bcfd 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1813,6 +1813,40 @@ void bio_clone_blkg_association(struct bio *dst, struct bio *src) } EXPORT_SYMBOL_GPL(bio_clone_blkg_association); +static int blk_cgroup_io_type(struct bio *bio) +{ + if (op_is_discard(bio->bi_opf)) + return BLKG_IOSTAT_DISCARD; + if (op_is_write(bio->bi_opf)) + return BLKG_IOSTAT_WRITE; + return BLKG_IOSTAT_READ; +} + +void blk_cgroup_bio_start(struct bio *bio) +{ + int rwd = blk_cgroup_io_type(bio), cpu; + struct blkg_iostat_set *bis; + + cpu = get_cpu(); + bis = per_cpu_ptr(bio->bi_blkg->iostat_cpu, cpu); + u64_stats_update_begin(&bis->sync); + + /* + * If the bio is flagged with BIO_CGROUP_ACCT it means this is a split + * bio and we would have already accounted for the size of the bio. + */ + if (!bio_flagged(bio, BIO_CGROUP_ACCT)) { + bio_set_flag(bio, BIO_CGROUP_ACCT); + bis->cur.bytes[rwd] += bio->bi_iter.bi_size; + } + bis->cur.ios[rwd]++; + + u64_stats_update_end(&bis->sync); + if (cgroup_subsys_on_dfl(io_cgrp_subsys)) + cgroup_rstat_updated(bio->bi_blkg->blkcg->css.cgroup, cpu); + put_cpu(); +} + static int __init blkcg_init(void) { blkcg_punt_bio_wq = alloc_workqueue("blkcg_punt_bio", diff --git a/block/blk-core.c b/block/blk-core.c index a9769c1a287546..76cfd5709f66cd 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1073,8 +1073,13 @@ generic_make_request_checks(struct bio *bio) if (unlikely(!current->io_context)) create_task_io_context(current, GFP_ATOMIC, q->node); - if (!blkcg_bio_issue_check(q, bio)) + if (blk_throtl_bio(bio)) { + blkcg_bio_issue_init(bio); return false; + } + + blk_cgroup_bio_start(bio); + blkcg_bio_issue_init(bio); if (!bio_flagged(bio, BIO_TRACE_COMPLETION)) { trace_block_bio_queue(q, bio); diff --git a/block/blk-throttle.c b/block/blk-throttle.c index ac008345050010..9d00f62c05ecdf 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2158,9 +2158,10 @@ static inline void throtl_update_latency_buckets(struct throtl_data *td) } #endif -bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, - struct bio *bio) +bool blk_throtl_bio(struct bio *bio) { + struct request_queue *q = bio->bi_disk->queue; + struct blkcg_gq *blkg = bio->bi_blkg; struct throtl_qnode *qn = NULL; struct throtl_grp *tg = blkg_to_tg(blkg ?: q->root_blkg); struct throtl_service_queue *sq; diff --git a/block/blk.h b/block/blk.h index 3a120a070dac82..41a50880c94e98 100644 --- a/block/blk.h +++ b/block/blk.h @@ -288,10 +288,12 @@ int create_task_io_context(struct task_struct *task, gfp_t gfp_mask, int node); extern int blk_throtl_init(struct request_queue *q); extern void blk_throtl_exit(struct request_queue *q); extern void blk_throtl_register_queue(struct request_queue *q); +bool blk_throtl_bio(struct bio *bio); #else /* CONFIG_BLK_DEV_THROTTLING */ static inline int blk_throtl_init(struct request_queue *q) { return 0; } static inline void blk_throtl_exit(struct request_queue *q) { } static inline void blk_throtl_register_queue(struct request_queue *q) { } +static inline bool blk_throtl_bio(struct bio *bio) { return false; } #endif /* CONFIG_BLK_DEV_THROTTLING */ #ifdef CONFIG_BLK_DEV_THROTTLING_LOW extern ssize_t blk_throtl_sample_time_show(struct request_queue *q, char *page); diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index 8ab043c911f233..431b2d18bf4074 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -517,14 +517,6 @@ static inline void blkg_put(struct blkcg_gq *blkg) if (((d_blkg) = __blkg_lookup(css_to_blkcg(pos_css), \ (p_blkg)->q, false))) -#ifdef CONFIG_BLK_DEV_THROTTLING -extern bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, - struct bio *bio); -#else -static inline bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, - struct bio *bio) { return false; } -#endif - bool __blkcg_punt_bio_submit(struct bio *bio); static inline bool blkcg_punt_bio_submit(struct bio *bio) @@ -540,50 +532,6 @@ static inline void blkcg_bio_issue_init(struct bio *bio) bio_issue_init(&bio->bi_issue, bio_sectors(bio)); } -static inline bool blkcg_bio_issue_check(struct request_queue *q, - struct bio *bio) -{ - struct blkcg_gq *blkg = bio->bi_blkg; - bool throtl = false; - - throtl = blk_throtl_bio(q, blkg, bio); - if (!throtl) { - struct blkg_iostat_set *bis; - int rwd, cpu; - - if (op_is_discard(bio->bi_opf)) - rwd = BLKG_IOSTAT_DISCARD; - else if (op_is_write(bio->bi_opf)) - rwd = BLKG_IOSTAT_WRITE; - else - rwd = BLKG_IOSTAT_READ; - - cpu = get_cpu(); - bis = per_cpu_ptr(blkg->iostat_cpu, cpu); - u64_stats_update_begin(&bis->sync); - - /* - * If the bio is flagged with BIO_CGROUP_ACCT it means this is a - * split bio and we would have already accounted for the size of - * the bio. - */ - if (!bio_flagged(bio, BIO_CGROUP_ACCT)) { - bio_set_flag(bio, BIO_CGROUP_ACCT); - bis->cur.bytes[rwd] += bio->bi_iter.bi_size; - } - bis->cur.ios[rwd]++; - - u64_stats_update_end(&bis->sync); - if (cgroup_subsys_on_dfl(io_cgrp_subsys)) - cgroup_rstat_updated(blkg->blkcg->css.cgroup, cpu); - put_cpu(); - } - - blkcg_bio_issue_init(bio); - - return !throtl; -} - static inline void blkcg_use_delay(struct blkcg_gq *blkg) { if (WARN_ON_ONCE(atomic_read(&blkg->use_delay) < 0)) @@ -657,6 +605,7 @@ static inline void blkcg_clear_delay(struct blkcg_gq *blkg) atomic_dec(&blkg->blkcg->css.cgroup->congestion_count); } +void blk_cgroup_bio_start(struct bio *bio); void blkcg_add_delay(struct blkcg_gq *blkg, u64 now, u64 delta); void blkcg_schedule_throttle(struct request_queue *q, bool use_memdelay); void blkcg_maybe_throttle_current(void); @@ -710,8 +659,7 @@ static inline void blkg_put(struct blkcg_gq *blkg) { } static inline bool blkcg_punt_bio_submit(struct bio *bio) { return false; } static inline void blkcg_bio_issue_init(struct bio *bio) { } -static inline bool blkcg_bio_issue_check(struct request_queue *q, - struct bio *bio) { return true; } +static inline void blk_cgroup_bio_start(struct bio *bio) { } #define blk_queue_for_each_rl(rl, q) \ for ((rl) = &(q)->root_rl; (rl); (rl) = NULL) From patchwork Sat Jun 27 07:31:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11629333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0DFB6161F for ; Sat, 27 Jun 2020 07:33:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF7F1214D8 for ; Sat, 27 Jun 2020 07:33:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="cbYqXmVe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF7F1214D8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 44D366B0096; Sat, 27 Jun 2020 03:33:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3D9116B0099; Sat, 27 Jun 2020 03:33:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 202746B009A; Sat, 27 Jun 2020 03:33:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id ED5B46B0096 for ; Sat, 27 Jun 2020 03:33:09 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BB4FA2C81 for ; Sat, 27 Jun 2020 07:33:09 +0000 (UTC) X-FDA: 76974175698.05.rod51_241396326e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 96601180295AD for ; Sat, 27 Jun 2020 07:33:09 +0000 (UTC) X-Spam-Summary: 1,0,0,866f8a68bfc10a3a,d41d8cd98f00b204,batv+3c141e6ecba9ad0677a1+6152+infradead.org+hch@casper.srs.infradead.org,,RULES_HIT:41:355:379:541:800:960:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1539:1711:1714:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3350:3865:4321:5007:6117:6261:6653:7903:10004:11026:11658:11914:12043:12160:12296:12297:12438:12517:12519:12555:12895:12986:13069:13311:13357:13894:14096:14181:14384:14394:14721:21080:21451:21627:30054:30070,0,RBL:90.155.50.34:@casper.srs.infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04ygysmzd3phgzyjcjknz8x7epjhdocjkceczp1huawcyzbtiduz9rt9nowmo8s.stxkeabozak8g4cz8755fa9cuk3kzkkhi7gmduq47qiqsr6qwax65iqk9t1f9i1.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: rod51_241396326e5c X-Filterd-Recvd-Size: 2669 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:33:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vmpoA1D7dyjV06bzwDyN/+k+GU++bNP7VnywwlpIVGc=; b=cbYqXmVeSoBvWX/9hZmJT+Lpgl l5Y6Vcm2ctIJzcPBTMX3VolpG677Sf4JJWJZfg14gMcYc1m3NFSPX26I/Vdufl8Wj4OJD/jl2Umzw 27pEigOY5nvjxvYMvM49rAPEbxY55yRTlPi3Lbfj1+TngLfjOcWXmlbk3hcRzSGoz6hhvGr0okR7W GCJr60AtGTdEOj8PSSltng1UW/O17eCtnu6LH/wkHVpdPf76uXKO6uYRjV3pBB02z8yKJ9PAQx81q mu+xZlNzRT5mjjmmiZE8tNwZpxaMdqqkKx8I//a0cFfvkQn/kdUEIwH5bGq7AXsO0yq6LLvd29UPA 2/yaXMOg==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5KY-0006qv-AN; Sat, 27 Jun 2020 07:32:56 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 14/14] blk-cgroup: remove a dead check in blk_throtl_bio Date: Sat, 27 Jun 2020 09:31:59 +0200 Message-Id: <20200627073159.2447325-15-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 96601180295AD X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bios must have a valid block group by the time they are submitted. Signed-off-by: Christoph Hellwig --- block/blk-throttle.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 9d00f62c05ecdf..ad37043297ed58 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2163,7 +2163,7 @@ bool blk_throtl_bio(struct bio *bio) struct request_queue *q = bio->bi_disk->queue; struct blkcg_gq *blkg = bio->bi_blkg; struct throtl_qnode *qn = NULL; - struct throtl_grp *tg = blkg_to_tg(blkg ?: q->root_blkg); + struct throtl_grp *tg = blkg_to_tg(blkg); struct throtl_service_queue *sq; bool rw = bio_data_dir(bio); bool throttled = false;