From patchwork Fri May 8 04:44:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11535509 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E925714B4 for ; Fri, 8 May 2020 04:44:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CD92820720 for ; Fri, 8 May 2020 04:44:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hOK6OqHR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725681AbgEHEod (ORCPT ); Fri, 8 May 2020 00:44:33 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:37687 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725825AbgEHEoc (ORCPT ); Fri, 8 May 2020 00:44:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588913070; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e8gNEy7m2O7/4KGfkJAlFm0quuKItdj5MuWr8hU03s8=; b=hOK6OqHRJwhmdDsg7wXuwUodGNxVyyaoapm1qskEbFyzMx+fHLUcAoNZSfRatlS/NbJ0QU dd3LbOl1E+kE0bQZ2lkVu52fS02y7EhrtSJI5+nlupCawppvaOU7F8SCkWOuAutIla8bM1 YL5WU55JO8emOWbhUFcmIBlKLcfTVnA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-303-rKDaZwK1Of2fGdge4Ti-0A-1; Fri, 08 May 2020 00:44:28 -0400 X-MC-Unique: rKDaZwK1Of2fGdge4Ti-0A-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4B86F473; Fri, 8 May 2020 04:44:27 +0000 (UTC) Received: from localhost (ovpn-8-27.pek2.redhat.com [10.72.8.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id A663C5D9CA; Fri, 8 May 2020 04:44:23 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Ming Lei , Yufen Yu , Christoph Hellwig , Hou Tao Subject: [PATCH V2 1/4] block: fix use-after-free on cached last_lookup partition Date: Fri, 8 May 2020 12:44:04 +0800 Message-Id: <20200508044407.1371907-2-ming.lei@redhat.com> In-Reply-To: <20200508044407.1371907-1-ming.lei@redhat.com> References: <20200508044407.1371907-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org delete_partition() clears the cached last_lookup partition. However the .last_lookup cache may be overwritten by one IO path after it is cleared from delete_partition(). Then another IO path may use the cached deleting partition after hd_struct_free() is called, then use-after-free is triggered on the cached partition. Fixes the issue by the following approach: 1) always get the partition's refcount via hd_struct_try_get() before setting .last_lookup 2) move clearing .last_lookup from delete_partition() to hd_struct_free() which is the release handle of the partition's percpu-refcount, so that no IO path can cache deleteing partition via .last_lookup. It is one candidate approach of Yufen's patch[1] which adds overhead in fast path by indirect lookup which may introduce one extra cacheline in IO path. Also this patch relies on percpu-refcount's protection, and it is easier to understand and verify. [1] https://lore.kernel.org/linux-block/20200109013551.GB9655@ming.t460p/T/#t Reported-by: Yufen Yu Cc: Christoph Hellwig Cc: Hou Tao Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/blk-core.c | 12 ------------ block/genhd.c | 15 ++++++++++++--- block/partitions/core.c | 12 +++++++++++- 3 files changed, 23 insertions(+), 16 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ec50d7e6be21..826a8980997d 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1363,18 +1363,6 @@ void blk_account_io_start(struct request *rq, bool new_io) part_stat_inc(part, merges[rw]); } else { part = disk_map_sector_rcu(rq->rq_disk, blk_rq_pos(rq)); - if (!hd_struct_try_get(part)) { - /* - * The partition is already being removed, - * the request will be accounted on the disk only - * - * We take a reference on disk->part0 although that - * partition will never be deleted, so we can treat - * it as any other partition. - */ - part = &rq->rq_disk->part0; - hd_struct_get(part); - } part_inc_in_flight(rq->q, part, rw); rq->part = part; } diff --git a/block/genhd.c b/block/genhd.c index c05d509877fa..ec57d5d7a64d 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -344,11 +344,12 @@ static inline int sector_in_part(struct hd_struct *part, sector_t sector) * primarily used for stats accounting. * * CONTEXT: - * RCU read locked. The returned partition pointer is valid only - * while preemption is disabled. + * RCU read locked. The returned partition pointer is always valid + * because its refcount is grabbed. * * RETURNS: * Found partition on success, part0 is returned if no partition matches + * or the matched partition is being deleted. */ struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector) { @@ -359,17 +360,25 @@ struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector) ptbl = rcu_dereference(disk->part_tbl); part = rcu_dereference(ptbl->last_lookup); - if (part && sector_in_part(part, sector)) + if (part && sector_in_part(part, sector) && hd_struct_try_get(part)) return part; for (i = 1; i < ptbl->len; i++) { part = rcu_dereference(ptbl->part[i]); if (part && sector_in_part(part, sector)) { + /* + * only live partition can be cached for lookup, + * so use-after-free on cached & deleting partition + * can be avoided + */ + if (!hd_struct_try_get(part)) + break; rcu_assign_pointer(ptbl->last_lookup, part); return part; } } + hd_struct_get(&disk->part0); return &disk->part0; } diff --git a/block/partitions/core.c b/block/partitions/core.c index c085bf85509b..f4000dac23ef 100644 --- a/block/partitions/core.c +++ b/block/partitions/core.c @@ -288,6 +288,12 @@ static void hd_struct_free_work(struct work_struct *work) static void hd_struct_free(struct percpu_ref *ref) { struct hd_struct *part = container_of(ref, struct hd_struct, ref); + struct gendisk *disk = part_to_disk(part); + struct disk_part_tbl *ptbl = + rcu_dereference_protected(disk->part_tbl, 1); + + rcu_assign_pointer(ptbl->last_lookup, NULL); + put_device(disk_to_dev(disk)); INIT_RCU_WORK(&part->rcu_work, hd_struct_free_work); queue_rcu_work(system_wq, &part->rcu_work); @@ -309,8 +315,12 @@ void delete_partition(struct gendisk *disk, struct hd_struct *part) struct disk_part_tbl *ptbl = rcu_dereference_protected(disk->part_tbl, 1); + /* + * ->part_tbl is referenced in this part's release handler, so + * we have to hold the disk device + */ + get_device(disk_to_dev(part_to_disk(part))); rcu_assign_pointer(ptbl->part[part->partno], NULL); - rcu_assign_pointer(ptbl->last_lookup, NULL); kobject_put(part->holder_dir); device_del(part_to_dev(part)); From patchwork Fri May 8 04:44:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11535511 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0202D912 for ; Fri, 8 May 2020 04:44:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF094208D6 for ; Fri, 8 May 2020 04:44:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KRkLaGjS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725829AbgEHEoh (ORCPT ); Fri, 8 May 2020 00:44:37 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:52628 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725825AbgEHEoh (ORCPT ); Fri, 8 May 2020 00:44:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588913076; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N/oaaQ4C9wcrt0h4NLM933HxrSlP7xIww0UEPzR3D7U=; b=KRkLaGjSjbf/Fcg+pqGJzX2ZqNmTuzFiAH8kQ+n1pzmR7nERkY8kbcf7av+23QTe8zUUnI hJqqVo0aDEy9o8I21gx2Ig+Nq2yUgLcHniM/LcqdaWEJAizO06optUMn3lUkqZI/6iJP5i tWc4ffHbWJd4+mxeHhM8uoNdu9ciEW8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-458-ZPgXhurjO6GDkCIPn6i-zg-1; Fri, 08 May 2020 00:44:34 -0400 X-MC-Unique: ZPgXhurjO6GDkCIPn6i-zg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8344E1005510; Fri, 8 May 2020 04:44:33 +0000 (UTC) Received: from localhost (ovpn-8-27.pek2.redhat.com [10.72.8.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 389F710013BD; Fri, 8 May 2020 04:44:29 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Ming Lei , Yufen Yu , Christoph Hellwig , Hou Tao Subject: [PATCH V2 2/4] block: only define 'nr_sects_seq' in hd_part for 32bit SMP Date: Fri, 8 May 2020 12:44:05 +0800 Message-Id: <20200508044407.1371907-3-ming.lei@redhat.com> In-Reply-To: <20200508044407.1371907-1-ming.lei@redhat.com> References: <20200508044407.1371907-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The seqcount of 'nr_sects_seq' is only needed in case of 32bit SMP, so define it just for 32bit SMP. Cc: Yufen Yu Cc: Christoph Hellwig Cc: Hou Tao Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- block/genhd.c | 2 +- block/partitions/core.c | 2 +- include/linux/genhd.h | 9 +++++++++ 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index ec57d5d7a64d..bf8cbb033d64 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1729,7 +1729,7 @@ struct gendisk *__alloc_disk_node(int minors, int node_id) * TODO: Ideally set_capacity() and get_capacity() should be * converted to make use of bd_mutex and sequence counters. */ - seqcount_init(&disk->part0.nr_sects_seq); + hd_sects_seq_init(&disk->part0); if (hd_ref_init(&disk->part0)) { hd_free_part(&disk->part0); kfree(disk); diff --git a/block/partitions/core.c b/block/partitions/core.c index f4000dac23ef..ec81986b358e 100644 --- a/block/partitions/core.c +++ b/block/partitions/core.c @@ -392,7 +392,7 @@ static struct hd_struct *add_partition(struct gendisk *disk, int partno, goto out_free; } - seqcount_init(&p->nr_sects_seq); + hd_sects_seq_init(p); pdev = part_to_dev(p); p->start_sect = start; diff --git a/include/linux/genhd.h b/include/linux/genhd.h index f9c226f9546a..b4744035ae58 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -68,7 +68,9 @@ struct hd_struct { * can be non-atomic on 32bit machines with 64bit sector_t. */ sector_t nr_sects; +#if BITS_PER_LONG==32 && defined(CONFIG_SMP) seqcount_t nr_sects_seq; +#endif sector_t alignment_offset; unsigned int discard_alignment; struct device __dev; @@ -274,6 +276,13 @@ static inline void disk_put_part(struct hd_struct *part) put_device(part_to_dev(part)); } +static inline void hd_sects_seq_init(struct hd_struct *p) +{ +#if BITS_PER_LONG==32 && defined(CONFIG_SMP) + seqcount_init(&p->nr_sects_seq); +#endif +} + /* * Smarter partition iterator without context limits. */ From patchwork Fri May 8 04:44:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11535513 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B242314B4 for ; Fri, 8 May 2020 04:44:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9128F208D6 for ; Fri, 8 May 2020 04:44:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e8choH6L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725865AbgEHEor (ORCPT ); Fri, 8 May 2020 00:44:47 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:38308 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725825AbgEHEoq (ORCPT ); Fri, 8 May 2020 00:44:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588913085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eqE+VZT6iP5BBQICf6hdfbtPcveHLASDX8yMLp9dHYE=; b=e8choH6Ll+vBelim8b3f+M8tdczTup0TQFuVoStXSH0B9yfAk7hZxwlTb1aDAtnWpY4ATZ NfZD2q4WV63+bbTUZ20e457p4OOdMpzCGC+RGncfFk5d1UWmq1TG+34TaDtcNQJhIOVWig 9psMM6PUIPMCWPRrCqXueAdgGU2GcH4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-134-meqgznt8OkeUCDmfNPERMA-1; Fri, 08 May 2020 00:44:42 -0400 X-MC-Unique: meqgznt8OkeUCDmfNPERMA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2EA380058A; Fri, 8 May 2020 04:44:40 +0000 (UTC) Received: from localhost (ovpn-8-27.pek2.redhat.com [10.72.8.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 69BD064456; Fri, 8 May 2020 04:44:36 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Ming Lei , Yufen Yu , Christoph Hellwig , Hou Tao Subject: [PATCH V2 3/4] block: re-organize fields of 'struct hd_part' Date: Fri, 8 May 2020 12:44:06 +0800 Message-Id: <20200508044407.1371907-4-ming.lei@redhat.com> In-Reply-To: <20200508044407.1371907-1-ming.lei@redhat.com> References: <20200508044407.1371907-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Put all fields accessed in IO path together at the beginning of the struct, so that all can be fetched in single cacheline. Cc: Yufen Yu Cc: Christoph Hellwig Cc: Hou Tao Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei --- include/linux/genhd.h | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/include/linux/genhd.h b/include/linux/genhd.h index b4744035ae58..a9384449465a 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -71,6 +71,14 @@ struct hd_struct { #if BITS_PER_LONG==32 && defined(CONFIG_SMP) seqcount_t nr_sects_seq; #endif + unsigned long stamp; +#ifdef CONFIG_SMP + struct disk_stats __percpu *dkstats; +#else + struct disk_stats dkstats; +#endif + struct percpu_ref ref; + sector_t alignment_offset; unsigned int discard_alignment; struct device __dev; @@ -80,13 +88,6 @@ struct hd_struct { #ifdef CONFIG_FAIL_MAKE_REQUEST int make_it_fail; #endif - unsigned long stamp; -#ifdef CONFIG_SMP - struct disk_stats __percpu *dkstats; -#else - struct disk_stats dkstats; -#endif - struct percpu_ref ref; struct rcu_work rcu_work; }; From patchwork Fri May 8 04:44:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11535515 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA98B14B4 for ; Fri, 8 May 2020 04:44:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B172920720 for ; Fri, 8 May 2020 04:44:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Rm/tZaGq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725891AbgEHEow (ORCPT ); Fri, 8 May 2020 00:44:52 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:51843 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725825AbgEHEov (ORCPT ); Fri, 8 May 2020 00:44:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588913090; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Yg3mtnBZlEMX2Q3h3gaf+uOD+kLSGxth7jwMUbwN5A=; b=Rm/tZaGqPP1XxAsA9qz/E++UmX38G6dtXfUeqGGcY/9cisBf1Zdr5UkNcYw5y1cX3JzCsc ohkkk3GfY3P0cEt6tW2CpsKdsugGh29IiCjrlt8j7xtZhhuhDmYilmg2o2Q2Myq6mAX2zY gCgPN1K9Fs8DOv0er793WrAtGtbj74k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-87-btPvrZWOPySOjTaMSMf8zg-1; Fri, 08 May 2020 00:44:49 -0400 X-MC-Unique: btPvrZWOPySOjTaMSMf8zg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B6EE3107ACF4; Fri, 8 May 2020 04:44:47 +0000 (UTC) Received: from localhost (ovpn-8-27.pek2.redhat.com [10.72.8.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FE3F5C1B0; Fri, 8 May 2020 04:44:44 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Ming Lei , Yufen Yu , Christoph Hellwig , Hou Tao Subject: [PATCH V2 4/4] block: don't hold part0's refcount in IO path Date: Fri, 8 May 2020 12:44:07 +0800 Message-Id: <20200508044407.1371907-5-ming.lei@redhat.com> In-Reply-To: <20200508044407.1371907-1-ming.lei@redhat.com> References: <20200508044407.1371907-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org gendisk can't be gone when there is IO activity, so not hold part0's refcount in IO path. Cc: Yufen Yu Cc: Christoph Hellwig Cc: Hou Tao Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig --- block/blk-core.c | 3 ++- block/blk-merge.c | 3 ++- block/genhd.c | 4 ++-- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 826a8980997d..56cc61354671 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1343,7 +1343,8 @@ void blk_account_io_done(struct request *req, u64 now) part_stat_add(part, nsecs[sgrp], now - req->start_time_ns); part_dec_in_flight(req->q, part, rq_data_dir(req)); - hd_struct_put(part); + if (part->partno) + hd_struct_put(part); part_stat_unlock(); } } diff --git a/block/blk-merge.c b/block/blk-merge.c index a04e991b5ded..b44caa28ec04 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -672,7 +672,8 @@ static void blk_account_io_merge(struct request *req) part_dec_in_flight(req->q, part, rq_data_dir(req)); - hd_struct_put(part); + if (part->partno) + hd_struct_put(part); part_stat_unlock(); } } diff --git a/block/genhd.c b/block/genhd.c index bf8cbb033d64..d97b95d1a2fd 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -345,7 +345,8 @@ static inline int sector_in_part(struct hd_struct *part, sector_t sector) * * CONTEXT: * RCU read locked. The returned partition pointer is always valid - * because its refcount is grabbed. + * because its refcount is grabbed except for part0, which lifetime + * is same with the disk. * * RETURNS: * Found partition on success, part0 is returned if no partition matches @@ -378,7 +379,6 @@ struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector) return part; } } - hd_struct_get(&disk->part0); return &disk->part0; }