From patchwork Fri Oct 16 00:42:19 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 7410321 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: X-Original-To: patchwork-dm-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 580BE9F302 for ; Fri, 16 Oct 2015 00:45:54 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2EC2B2069A for ; Fri, 16 Oct 2015 00:45:53 +0000 (UTC) Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB53920699 for ; Fri, 16 Oct 2015 00:45:51 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id t9G0gUxh016770; Thu, 15 Oct 2015 20:42:30 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id t9G0gTI4005313 for ; Thu, 15 Oct 2015 20:42:29 -0400 Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t9G0gSYn030079 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 15 Oct 2015 20:42:28 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx1.redhat.com (Postfix) with ESMTP id 06A54C0BFD2D for ; Fri, 16 Oct 2015 00:42:21 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 15 Oct 2015 17:42:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,687,1437462000"; d="scan'208";a="812185986" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by fmsmga001.fm.intel.com with ESMTP; 15 Oct 2015 17:42:21 -0700 Received: from orsmsx114.amr.corp.intel.com (10.22.240.10) by ORSMSX101.amr.corp.intel.com (10.22.225.128) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 15 Oct 2015 17:42:20 -0700 Received: from orsmsx107.amr.corp.intel.com ([169.254.1.53]) by ORSMSX114.amr.corp.intel.com ([169.254.8.204]) with mapi id 14.03.0248.002; Thu, 15 Oct 2015 17:42:20 -0700 From: "Williams, Dan J" To: "martin.petersen@oracle.com" Thread-Topic: [PATCH v2 10/12] block: move blk_integrity to request_queue Thread-Index: AQHRB4T3vUAo14JLH0Kz+OotymDVpZ5tvVSA Date: Fri, 16 Oct 2015 00:42:19 +0000 Message-ID: <1444956139.2737.7.camel@intel.com> References: <20151015195939.20721.23101.stgit@dwillia2-desk3.jf.intel.com> <20151015200034.20721.23425.stgit@dwillia2-desk3.jf.intel.com> In-Reply-To: <20151015200034.20721.23425.stgit@dwillia2-desk3.jf.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.139] Content-ID: <24074458BB173143B7F486C77D86846F@intel.com> MIME-Version: 1.0 X-RedHat-Spam-Score: -4.33 (BAYES_50, DCC_REPUT_13_19, RCVD_IN_DNSWL_HI, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS, T_RP_MATCHES_RCVD, URIBL_BLOCKED) 192.55.52.88 mga01.intel.com 192.55.52.88 mga01.intel.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-MIME-Autoconverted: from base64 to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id t9G0gTI4005313 X-loop: dm-devel@redhat.com Cc: "axboe@fb.com" , "linux-nvdimm@lists.01.org" , "linux-nvme@lists.infradead.org" , "linux-raid@vger.kernel.org" , "dm-devel@redhat.com" , "hch@lst.de" Subject: Re: [dm-devel] [PATCH v2 10/12] block: move blk_integrity to request_queue X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: device-mapper development List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, 2015-10-15 at 16:00 -0400, Dan Williams wrote: > A trace like the following proceeds a crash in bio_integrity_process() > when it goes to use an already freed blk_integrity profile. > > BUG: unable to handle kernel paging request at ffff8800d31b10d8 > IP: [] 0xffff8800d31b10d8 > PGD 2f65067 PUD 21fffd067 PMD 80000000d30001e3 > Oops: 0011 [#1] SMP > Dumping ftrace buffer: > --------------------------------- > ndctl-2222 2.... 44526245us : disk_release: pmem1s > systemd--2223 4.... 44573945us : bio_integrity_endio: pmem1s > <...>-409 4.... 44574005us : bio_integrity_process: pmem1s > --------------------------------- > [..] > Call Trace: > [] ? bio_integrity_process+0x159/0x2d0 > [] bio_integrity_verify_fn+0x36/0x60 > [] process_one_work+0x1cc/0x4e0 > > Given that a request_queue is pinned while i/o is in flight and that a > gendisk is allowed to have a shorter lifetime, move blk_integrity to > request_queue to satisfy requests arriving after the gendisk has been > torn down. > > Cc: Christoph Hellwig > Cc: Martin K. Petersen > [martin: fix the CONFIG_BLK_DEV_INTEGRITY=n case] > Tested-by: Ross Zwisler > Signed-off-by: Dan Williams > --- > Documentation/ABI/testing/sysfs-block | 12 +++--- > block/blk-integrity.c | 63 +++++++++++++++++---------------- > block/blk-sysfs.c | 4 ++ > block/genhd.c | 2 - > block/partition-generic.c | 2 + > drivers/md/dm-table.c | 4 +- > drivers/md/md.c | 4 +- > drivers/nvdimm/core.c | 2 + > drivers/nvme/host/pci.c | 8 +++- > drivers/scsi/sd_dif.c | 2 + > fs/block_dev.c | 2 + > include/linux/blkdev.h | 16 ++++++-- > include/linux/genhd.h | 16 +++----- > 13 files changed, 72 insertions(+), 65 deletions(-) > Martin pointed out that I broke compatibility by changing the sysfs layout of the integrity attributes. Rather than add a sysfs-link we can make this patch simpler by only moving struct blk_integrity to request_queue and leave integrity_kobj where it is in gendisk. Also undo the conversion of blk_integrity apis take a request_queue. Through this simplification I also noticed that I had broken blk_integrity_compare() Here's the much smaller v3, and I also refreshed for-4.4/blk-integrity 8<----- Subject: block: move blk_integrity to request_queue From: Dan Williams A trace like the following proceeds a crash in bio_integrity_process() when it goes to use an already freed blk_integrity profile. BUG: unable to handle kernel paging request at ffff8800d31b10d8 IP: [] 0xffff8800d31b10d8 PGD 2f65067 PUD 21fffd067 PMD 80000000d30001e3 Oops: 0011 [#1] SMP Dumping ftrace buffer: --------------------------------- ndctl-2222 2.... 44526245us : disk_release: pmem1s systemd--2223 4.... 44573945us : bio_integrity_endio: pmem1s <...>-409 4.... 44574005us : bio_integrity_process: pmem1s --------------------------------- [..] Call Trace: [] ? bio_integrity_process+0x159/0x2d0 [] bio_integrity_verify_fn+0x36/0x60 [] process_one_work+0x1cc/0x4e0 Given that a request_queue is pinned while i/o is in flight and that a gendisk is allowed to have a shorter lifetime, move blk_integrity to request_queue to satisfy requests arriving after the gendisk has been torn down. Cc: Christoph Hellwig Cc: Martin K. Petersen [martin: fix the CONFIG_BLK_DEV_INTEGRITY=n case] Tested-by: Ross Zwisler Signed-off-by: Dan Williams --- block/blk-integrity.c | 14 +++++++------- drivers/nvme/host/pci.c | 2 +- include/linux/blkdev.h | 6 +++++- include/linux/genhd.h | 1 - 4 files changed, 13 insertions(+), 10 deletions(-) -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel diff --git a/block/blk-integrity.c b/block/blk-integrity.c index 4615a3386798..5d339ae64d56 100644 --- a/block/blk-integrity.c +++ b/block/blk-integrity.c @@ -142,8 +142,8 @@ EXPORT_SYMBOL(blk_rq_map_integrity_sg); */ int blk_integrity_compare(struct gendisk *gd1, struct gendisk *gd2) { - struct blk_integrity *b1 = &gd1->integrity; - struct blk_integrity *b2 = &gd2->integrity; + struct blk_integrity *b1 = &gd1->queue->integrity; + struct blk_integrity *b2 = &gd2->queue->integrity; if (!b1->profile && !b2->profile) return 0; @@ -246,7 +246,7 @@ static ssize_t integrity_attr_show(struct kobject *kobj, struct attribute *attr, char *page) { struct gendisk *disk = container_of(kobj, struct gendisk, integrity_kobj); - struct blk_integrity *bi = &disk->integrity; + struct blk_integrity *bi = &disk->queue->integrity; struct integrity_sysfs_entry *entry = container_of(attr, struct integrity_sysfs_entry, attr); @@ -258,7 +258,7 @@ static ssize_t integrity_attr_store(struct kobject *kobj, size_t count) { struct gendisk *disk = container_of(kobj, struct gendisk, integrity_kobj); - struct blk_integrity *bi = &disk->integrity; + struct blk_integrity *bi = &disk->queue->integrity; struct integrity_sysfs_entry *entry = container_of(attr, struct integrity_sysfs_entry, attr); ssize_t ret = 0; @@ -397,7 +397,7 @@ static struct kobj_type integrity_ktype = { */ void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template) { - struct blk_integrity *bi = &disk->integrity; + struct blk_integrity *bi = &disk->queue->integrity; bi->flags = BLK_INTEGRITY_VERIFY | BLK_INTEGRITY_GENERATE | template->flags; @@ -420,13 +420,13 @@ EXPORT_SYMBOL(blk_integrity_register); void blk_integrity_unregister(struct gendisk *disk) { blk_integrity_revalidate(disk); - memset(&disk->integrity, 0, sizeof(struct blk_integrity)); + memset(&disk->queue->integrity, 0, sizeof(struct blk_integrity)); } EXPORT_SYMBOL(blk_integrity_unregister); void blk_integrity_revalidate(struct gendisk *disk) { - struct blk_integrity *bi = &disk->integrity; + struct blk_integrity *bi = &disk->queue->integrity; if (!(disk->flags & GENHD_FL_UP)) return; diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 5578de67f406..2a39314fc61a 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -538,7 +538,7 @@ static void nvme_dif_remap(struct request *req, virt = bip_get_seed(bip); phys = nvme_block_nr(ns, blk_rq_pos(req)); nlb = (blk_rq_bytes(req) >> ns->lba_shift); - ts = ns->disk->integrity.tuple_size; + ts = ns->disk->queue->integrity.tuple_size; for (i = 0; i < nlb; i++, virt++, phys++) { pi = (struct t10_pi_tuple *)p; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 3e0465257d68..cf57884db4b7 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -369,6 +369,10 @@ struct request_queue { */ struct kobject mq_kobj; +#ifdef CONFIG_BLK_DEV_INTEGRITY + struct blk_integrity integrity; +#endif /* CONFIG_BLK_DEV_INTEGRITY */ + #ifdef CONFIG_PM struct device *dev; int rpm_status; @@ -1481,7 +1485,7 @@ extern bool blk_integrity_merge_bio(struct request_queue *, struct request *, static inline struct blk_integrity *blk_get_integrity(struct gendisk *disk) { - struct blk_integrity *bi = &disk->integrity; + struct blk_integrity *bi = &disk->queue->integrity; if (!bi->profile) return NULL; diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 82f4911e0ad8..847cc1d91634 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -210,7 +210,6 @@ struct gendisk { atomic_t sync_io; /* RAID */ struct disk_events *ev; #ifdef CONFIG_BLK_DEV_INTEGRITY - struct blk_integrity integrity; struct kobject integrity_kobj; #endif /* CONFIG_BLK_DEV_INTEGRITY */ int node_id;