From patchwork Wed Jun 29 14:51:39 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Matias_Bj=C3=B8rling?= X-Patchwork-Id: 9205559 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BE3A26075F for ; Wed, 29 Jun 2016 14:56:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE3D228669 for ; Wed, 29 Jun 2016 14:56:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1D2A2866B; Wed, 29 Jun 2016 14:56:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CBBD728669 for ; Wed, 29 Jun 2016 14:56:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5TEqPDk003208; Wed, 29 Jun 2016 10:52:25 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5TEq4Ds009801 for ; Wed, 29 Jun 2016 10:52:04 -0400 Received: from mx1.redhat.com (ext-mx01.extmail.prod.ext.phx2.redhat.com [10.5.110.25]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u5TEq4qr031565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 29 Jun 2016 10:52:04 -0400 Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 015CF7F6A2 for ; Wed, 29 Jun 2016 14:52:03 +0000 (UTC) Received: by mail-wm0-f68.google.com with SMTP id 187so14956415wmz.1 for ; Wed, 29 Jun 2016 07:52:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bjorling.me; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R2c5icW2SQjcjtAg9wyQYe0gV+MjObcoA0SIkumH4ao=; b=yFvxXvAmleM9XzXe0Wds7W2z055TKDAZdnKrVGowKl62pqlE609cCWt3w/JUUMmBGu ZpGvchx+gptlI7QuLWfud5AlXe8or+Bxpnu83duju74s1HjSalX/xEXoWQuY406xHeFW OBQZHgIYeo7aVlDqLWZ7iGxKp80iCzcWUh/yg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R2c5icW2SQjcjtAg9wyQYe0gV+MjObcoA0SIkumH4ao=; b=PuZZycRPQ3R4r5LWlhj877mNkU33tICIwhkeS5KVGFcOVLglO5CAC8y0Wp6VwTSwA3 HHyHQyvCKnzmdDYFUoYT1TWgKVo68ry4wL8a/i9MD3b6+eFryE5Th0yggCXSLiQmn/aH HS4OCyAwSF4sMvQUNEnJ6gH9Pnb5ACiBEBGo/U/8O8beVkHrQH2MvrVRgCiyM6B4Rm/4 /qJHBDA8bl1WRg3SGG07rNdjOcaiRbillbPyULU+FhPjFrWlIvFqa5GbrKoiA9o8CTlb nnEPMJ90+44FxfkP5AH6WO24HkYdRAZTIH5k8YtrtltUQq1lAeUKRrsfBtjPD16Q3Q4R 9bug== X-Gm-Message-State: ALyK8tJR+DHAodt9XE35UMpS2c6bfk/GLUmel/+QUTv3YCFKlbKModKtxxuo2NVa8nKqbA== X-Received: by 10.28.52.142 with SMTP id b136mr9237095wma.35.1467211921602; Wed, 29 Jun 2016 07:52:01 -0700 (PDT) Received: from Macroninja.cnexlabs.com (6164211-cl69.boa.fiberby.dk. [193.106.164.211]) by smtp.gmail.com with ESMTPSA id t1sm4112703wjy.3.2016.06.29.07.52.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 29 Jun 2016 07:52:01 -0700 (PDT) From: =?UTF-8?q?Matias=20Bj=C3=B8rling?= To: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, axboe@fb.com, keith.busch@intel.com, linux-nvme@lists.infradead.org, dm-devel@redhat.com Date: Wed, 29 Jun 2016 16:51:39 +0200 Message-Id: <1467211901-26707-5-git-send-email-m@bjorling.me> In-Reply-To: <1467211901-26707-1-git-send-email-m@bjorling.me> References: <1467211901-26707-1-git-send-email-m@bjorling.me> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 29 Jun 2016 14:52:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 29 Jun 2016 14:52:03 +0000 (UTC) for IP:'74.125.82.68' DOMAIN:'mail-wm0-f68.google.com' HELO:'mail-wm0-f68.google.com' FROM:'m@bjorling.me' RCPT:'' X-RedHat-Spam-Score: -0.121 (BAYES_50, DCC_REPUT_13_19, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 74.125.82.68 mail-wm0-f68.google.com 74.125.82.68 mail-wm0-f68.google.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.75 on 10.5.110.25 X-loop: dm-devel@redhat.com Cc: =?UTF-8?q?Matias=20Bj=C3=B8rling?= Subject: [dm-devel] [PATCH 4/6] lightnvm: let drivers control the lifetime of nvm_dev X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Virus-Scanned: ClamAV using ClamSMTP LightNVM does not currently expose the device driver sysfs specific entries to user-space, as the device driver skips the initialization of gendisk. To enable a device driver sysfs entries to be exposed, we need a struct device to attach it to. To allow both the device driver and LightNVM to access the same struct device, we need the device driver to track the lifetime of the nvm_dev structure. This patch refactors the two users of LightNVM (NVMe and null_blk), enables them to allocate and free nvm_dev, and at last removes gendisk usage when a LightNVM device is used. Signed-off-by: Matias Bjørling --- drivers/block/null_blk.c | 22 ++++++++++++++++++++-- drivers/lightnvm/core.c | 34 +++++++--------------------------- drivers/nvme/host/core.c | 34 ++++++++++++++-------------------- drivers/nvme/host/lightnvm.c | 33 ++++++++++++++++++++++++++------- drivers/nvme/host/nvme.h | 8 +++++--- include/linux/lightnvm.h | 15 +++++++++------ 6 files changed, 81 insertions(+), 65 deletions(-) diff --git a/drivers/block/null_blk.c b/drivers/block/null_blk.c index 3117df1..5a0da8b 100644 --- a/drivers/block/null_blk.c +++ b/drivers/block/null_blk.c @@ -34,6 +34,7 @@ struct nullb { unsigned int index; struct request_queue *q; struct gendisk *disk; + struct nvm_dev *ndev; struct blk_mq_tag_set tag_set; struct hrtimer timer; unsigned int queue_depth; @@ -550,12 +551,29 @@ static struct nvm_dev_ops null_lnvm_dev_ops = { static int null_nvm_register(struct nullb *nullb) { - return nvm_register(nullb->q, nullb->disk_name, &null_lnvm_dev_ops); + struct nvm_dev *dev; + int rv; + + dev = nvm_alloc_dev(0); + if (!dev) + return -ENOMEM; + + dev->q = nullb->q; + memcpy(dev->name, nullb->disk_name, DISK_NAME_LEN); + dev->ops = &null_lnvm_dev_ops; + + rv = nvm_register(dev); + if (rv) { + kfree(dev); + return rv; + } + nullb->ndev = dev; + return 0; } static void null_nvm_unregister(struct nullb *nullb) { - nvm_unregister(nullb->disk_name); + nvm_unregister(nullb->ndev); } #else static int null_nvm_register(struct nullb *nullb) diff --git a/drivers/lightnvm/core.c b/drivers/lightnvm/core.c index 25c5df9..0654c06 100644 --- a/drivers/lightnvm/core.c +++ b/drivers/lightnvm/core.c @@ -660,23 +660,15 @@ static void nvm_exit(struct nvm_dev *dev) pr_info("nvm: successfully unloaded\n"); } -int nvm_register(struct request_queue *q, char *disk_name, - struct nvm_dev_ops *ops) +struct nvm_dev *nvm_alloc_dev(int node) +{ + return kzalloc_node(sizeof(struct nvm_dev), GFP_KERNEL, node); +} + +int nvm_register(struct nvm_dev *dev) { - struct nvm_dev *dev; int ret; - if (!ops->identity) - return -EINVAL; - - dev = kzalloc(sizeof(struct nvm_dev), GFP_KERNEL); - if (!dev) - return -ENOMEM; - - dev->q = q; - dev->ops = ops; - strncpy(dev->name, disk_name, DISK_NAME_LEN); - ret = nvm_init(dev); if (ret) goto err_init; @@ -714,29 +706,17 @@ int nvm_register(struct request_queue *q, char *disk_name, return 0; err_init: kfree(dev->lun_map); - kfree(dev); return ret; } EXPORT_SYMBOL(nvm_register); -void nvm_unregister(char *disk_name) +void nvm_unregister(struct nvm_dev *dev) { - struct nvm_dev *dev; - down_write(&nvm_lock); - dev = nvm_find_nvm_dev(disk_name); - if (!dev) { - pr_err("nvm: could not find device %s to unregister\n", - disk_name); - up_write(&nvm_lock); - return; - } - list_del(&dev->devices); up_write(&nvm_lock); nvm_exit(dev); - kfree(dev); } EXPORT_SYMBOL(nvm_unregister); diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f615b6b..fe135d9 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -137,12 +137,14 @@ static void nvme_free_ns(struct kref *kref) { struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref); - if (ns->type == NVME_NS_LIGHTNVM) - nvme_nvm_unregister(ns->queue, ns->disk->disk_name); + if (ns->ndev) + nvme_nvm_unregister(ns); - spin_lock(&dev_list_lock); - ns->disk->private_data = NULL; - spin_unlock(&dev_list_lock); + if (ns->disk) { + spin_lock(&dev_list_lock); + ns->disk->private_data = NULL; + spin_unlock(&dev_list_lock); + } put_disk(ns->disk); ida_simple_remove(&ns->ctrl->ns_ida, ns->instance); @@ -788,8 +790,7 @@ static void nvme_config_discard(struct nvme_ns *ns) static int nvme_revalidate_ns(struct nvme_ns *ns, struct nvme_id_ns **id) { if (nvme_identify_ns(ns->ctrl, ns->ns_id, id)) { - dev_warn(disk_to_dev(ns->disk), "%s: Identify failure\n", - __func__); + dev_warn(ns->ctrl->dev, "%s: Identify failure\n", __func__); return -ENODEV; } @@ -1473,18 +1474,11 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) sprintf(disk_name, "nvme%dn%d", ctrl->instance, ns->instance); if (nvme_nvm_ns_supported(ns, id)) { - if (nvme_nvm_register(ns->queue, disk_name)) { - dev_warn(ctrl->dev, - "%s: LightNVM init failure\n", __func__); + if (nvme_nvm_register(ns, disk_name, node)) { + dev_warn(ctrl->dev, "%s: LightNVM init failure\n", + __func__); goto out_free_id; } - - disk = alloc_disk_node(0, node); - if (!disk) - goto out_free_id; - memcpy(disk->disk_name, disk_name, DISK_NAME_LEN); - ns->disk = disk; - ns->type = NVME_NS_LIGHTNVM; } else { disk = alloc_disk_node(0, node); if (!disk) @@ -1532,7 +1526,7 @@ static void nvme_ns_remove(struct nvme_ns *ns) if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags)) return; - if (ns->disk->flags & GENHD_FL_UP) { + if (ns->disk && ns->disk->flags & GENHD_FL_UP) { if (blk_get_integrity(ns->disk)) blk_integrity_unregister(ns->disk); sysfs_remove_group(&disk_to_dev(ns->disk)->kobj, @@ -1552,7 +1546,7 @@ static void nvme_validate_ns(struct nvme_ctrl *ctrl, unsigned nsid) ns = nvme_find_ns(ctrl, nsid); if (ns) { - if (revalidate_disk(ns->disk)) + if (ns->disk && revalidate_disk(ns->disk)) nvme_ns_remove(ns); } else nvme_alloc_ns(ctrl, nsid); @@ -1856,7 +1850,7 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl) * Revalidating a dead namespace sets capacity to 0. This will * end buffered writers dirtying pages that can't be synced. */ - if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags)) + if (ns->disk && !test_and_set_bit(NVME_NS_DEAD, &ns->flags)) revalidate_disk(ns->disk); blk_set_queue_dying(ns->queue); diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c index ba51602..eb780ef 100644 --- a/drivers/nvme/host/lightnvm.c +++ b/drivers/nvme/host/lightnvm.c @@ -474,9 +474,8 @@ static inline void nvme_nvm_rqtocmd(struct request *rq, struct nvm_rq *rqd, c->ph_rw.length = cpu_to_le16(rqd->nr_ppas - 1); if (rqd->opcode == NVM_OP_HBWRITE || rqd->opcode == NVM_OP_HBREAD) - /* momentarily hardcode the shift configuration. lba_shift from - * nvm_dev will be available in a follow-up patch */ - c->hb_rw.slba = cpu_to_le64(rqd->bio->bi_iter.bi_sector >> 3); + c->hb_rw.slba = cpu_to_le64(nvme_block_nr(ns, + rqd->bio->bi_iter.bi_sector)); } static void nvme_nvm_end_io(struct request *rq, int error) @@ -593,14 +592,34 @@ static struct nvm_dev_ops nvme_nvm_dev_ops = { .max_phys_sect = 64, }; -int nvme_nvm_register(struct request_queue *q, char *disk_name) +int nvme_nvm_register(struct nvme_ns *ns, char *disk_name, int node) { - return nvm_register(q, disk_name, &nvme_nvm_dev_ops); + struct request_queue *q = ns->queue; + struct nvm_dev *dev; + + dev = nvm_alloc_dev(node); + if (!dev) + return -ENOMEM; + + dev->q = q; + memcpy(dev->name, disk_name, DISK_NAME_LEN); + dev->ops = &nvme_nvm_dev_ops; + ns->ndev = dev; + + ret = nvm_register(dev); + + ns->lba_shift = ilog2(dev->sec_size) - 9; + + if (sysfs_create_group(&dev->dev.kobj, attrs)) + pr_warn("%s: failed to create sysfs group for identification\n", + ns->disk->disk_name); + return ret; } -void nvme_nvm_unregister(struct request_queue *q, char *disk_name) +void nvme_nvm_unregister(struct nvme_ns *ns) { - nvm_unregister(disk_name); + nvm_unregister(ns->ndev); + kfree(ns->ndev); } /* move to shared place when used in multiple places. */ diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 282421f..7b49b45 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -18,6 +18,7 @@ #include #include #include +#include enum { /* @@ -122,6 +123,7 @@ struct nvme_ns { struct nvme_ctrl *ctrl; struct request_queue *queue; struct gendisk *disk; + struct nvm_dev *ndev; struct kref kref; int instance; @@ -133,7 +135,6 @@ struct nvme_ns { u16 ms; bool ext; u8 pi_type; - int type; unsigned long flags; #define NVME_NS_REMOVING 0 @@ -269,12 +270,13 @@ int nvme_nvm_ns_supported(struct nvme_ns *ns, struct nvme_id_ns *id); int nvme_nvm_register(struct request_queue *q, char *disk_name); void nvme_nvm_unregister(struct request_queue *q, char *disk_name); #else -static inline int nvme_nvm_register(struct request_queue *q, char *disk_name) +static inline int nvme_nvm_register(struct nvme_ns *ns, char *disk_name, + int node) { return 0; } -static inline void nvme_nvm_unregister(struct request_queue *q, char *disk_name) {}; +static inline void nvme_nvm_unregister(struct nvme_ns *ns) {}; static inline int nvme_nvm_ns_supported(struct nvme_ns *ns, struct nvme_id_ns *id) { diff --git a/include/linux/lightnvm.h b/include/linux/lightnvm.h index ba78b83..5afc263 100644 --- a/include/linux/lightnvm.h +++ b/include/linux/lightnvm.h @@ -524,9 +524,9 @@ extern struct nvm_block *nvm_get_blk(struct nvm_dev *, struct nvm_lun *, unsigned long); extern void nvm_put_blk(struct nvm_dev *, struct nvm_block *); -extern int nvm_register(struct request_queue *, char *, - struct nvm_dev_ops *); -extern void nvm_unregister(char *); +extern struct nvm_dev *nvm_alloc_dev(int); +extern int nvm_register(struct nvm_dev *); +extern void nvm_unregister(struct nvm_dev *); void nvm_mark_blk(struct nvm_dev *dev, struct ppa_addr ppa, int type); @@ -575,11 +575,14 @@ extern int nvm_dev_factory(struct nvm_dev *, int flags); #else /* CONFIG_NVM */ struct nvm_dev_ops; -static inline int nvm_register(struct request_queue *q, char *disk_name, - struct nvm_dev_ops *ops) +static inline struct nvm_dev *nvm_alloc_dev(int node) +{ + return ERR_PTR(-EINVAL); +} +static inline int nvm_register(struct nvm_dev *dev) { return -EINVAL; } -static inline void nvm_unregister(char *disk_name) {} +static inline void nvm_unregister(struct nvm_dev *dev) {} #endif /* CONFIG_NVM */ #endif /* LIGHTNVM.H */