From patchwork Wed Jul 24 21:57:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5AE0A1399 for ; Wed, 24 Jul 2019 21:57:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A3D11FFB2 for ; Wed, 24 Jul 2019 21:57:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E969288D4; Wed, 24 Jul 2019 21:57:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E7E85288CA for ; Wed, 24 Jul 2019 21:57:49 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 27DF52194EB77; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B31CA212DA5FC for ; Wed, 24 Jul 2019 15:00:13 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602204" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:46 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 01/13] libdaxctl: add interfaces to get ctx and check device state Date: Wed, 24 Jul 2019 15:57:29 -0600 Message-Id: <20190724215741.18556-2-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs: daxctl_dev_get_ctx daxctl_dev_is_enabled Cc: Dan Williams Reviewed-by: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 30 ++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 38 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index c2e3a52..70f896b 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -559,6 +559,36 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled(const char *drvpath) +{ + struct stat st; + + if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode)) + return 0; + else + return 1; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char *path = dev->dev_buf; + int len = dev->buf_len; + + if (snprintf(path, len, "%s/driver", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + daxctl_dev_get_devname(dev)); + return 0; + } + + return is_enabled(path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ + return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) { dax_devices_init(region); diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 84d3a69..c4af9a7 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 1d13ea2..e20ccb4 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev); int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); +int daxctl_dev_is_enabled(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Wed Jul 24 21:57:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6ADBD912 for ; Wed, 24 Jul 2019 21:57:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A32A1FFB2 for ; Wed, 24 Jul 2019 21:57:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4EE04288DB; Wed, 24 Jul 2019 21:57:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 72A871FFB2 for ; Wed, 24 Jul 2019 21:57:51 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 68EB72194EB7D; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E2A02212DA5ED for ; Wed, 24 Jul 2019 15:00:13 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602207" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:46 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 02/13] libdaxctl: add interfaces to enable/disable devices Date: Wed, 24 Jul 2019 15:57:30 -0600 Message-Id: <20190724215741.18556-3-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add new libdaxctl interfaces to disable a device_dax based device, and to enable it into the given mode. The modes available are 'devdax', and 'system-ram', where devdax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device making it usable as normal memory. This adds the following new interfaces: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; Cc: Dave Hansen Cc: Dan Williams Reviewed-by: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 21 +++ daxctl/lib/libdaxctl.c | 234 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 3 + 5 files changed, 263 insertions(+), 1 deletion(-) diff --git a/daxctl/lib/Makefile.am b/daxctl/lib/Makefile.am index d3d4852..9f0e444 100644 --- a/daxctl/lib/Makefile.am +++ b/daxctl/lib/Makefile.am @@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index 4a462e7..120137f 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -13,6 +13,8 @@ #ifndef _LIBDAXCTL_PRIVATE_H_ #define _LIBDAXCTL_PRIVATE_H_ +#include + #define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem { @@ -26,6 +28,17 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +enum daxctl_dev_mode { + DAXCTL_DEV_MODE_DEVDAX = 0, + DAXCTL_DEV_MODE_RAM, + DAXCTL_DEV_MODE_END, +}; + +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX] = "device_dax", + [DAXCTL_DEV_MODE_RAM] = "kmem", +}; + /** * struct daxctl_region - container for dax_devices */ @@ -53,6 +66,14 @@ struct daxctl_dev { char *dev_path; struct list_node list; unsigned long long size; + struct kmod_module *module; + struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod(struct kmod_ctx *kmod_ctx) +{ + return kmod_ctx ? 0 : -ENXIO; +} + #endif /* _LIBDAXCTL_PRIVATE_H_ */ diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 70f896b..85c4fbe 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -46,6 +46,7 @@ struct daxctl_ctx { void *userdata; int regions_init; struct list_head regions; + struct kmod_ctx *kmod_ctx; }; /** @@ -84,20 +85,32 @@ DAXCTL_EXPORT void daxctl_set_userdata(struct daxctl_ctx *ctx, void *userdata) */ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx) { + struct kmod_ctx *kmod_ctx; struct daxctl_ctx *c; + int rc = 0; c = calloc(1, sizeof(struct daxctl_ctx)); if (!c) return -ENOMEM; + kmod_ctx = kmod_new(NULL, NULL); + if (check_kmod(kmod_ctx) != 0) { + rc = -ENXIO; + goto out; + } + c->refcount = 1; log_init(&c->ctx, "libdaxctl", "DAXCTL_LOG"); info(c, "ctx %p created\n", c); dbg(c, "log_priority=%d\n", c->ctx.log_priority); *ctx = c; list_head_init(&c->regions); + c->kmod_ctx = kmod_ctx; return 0; +out: + free(c); + return rc; } /** @@ -132,6 +145,7 @@ DAXCTL_EXPORT void daxctl_unref(struct daxctl_ctx *ctx) list_for_each_safe(&ctx->regions, region, _r, list) free_region(region, &ctx->regions); + kmod_unref(ctx->kmod_ctx); info(ctx, "context %p released\n", ctx); free(ctx); } @@ -189,6 +203,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head) { if (head) list_del_from(head, &dev->list); + kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf); free(dev->dev_path); free(dev); @@ -306,6 +321,27 @@ DAXCTL_EXPORT struct daxctl_region *daxctl_new_region(struct daxctl_ctx *ctx, return region; } +static struct kmod_list *to_module_list(struct daxctl_ctx *ctx, + const char *alias) +{ + struct kmod_list *list = NULL; + int rc; + + if (!ctx->kmod_ctx || !alias) + return NULL; + if (alias[0] == 0) + return NULL; + + rc = kmod_module_new_from_lookup(ctx->kmod_ctx, alias, &list); + if (rc < 0 || !list) { + dbg(ctx, "failed to find modules for alias: %s %d list: %s\n", + alias, rc, list ? "populated" : "empty"); + return NULL; + } + + return list; +} + static void *add_dax_dev(void *parent, int id, const char *daxdev_base) { const char *devname = devpath_to_devname(daxdev_base); @@ -315,6 +351,7 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) struct daxctl_dev *dev, *dev_dup; char buf[SYSFS_ATTR_SIZE]; struct stat st; + int rc; if (!path) return NULL; @@ -346,6 +383,14 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) goto err_read; dev->buf_len = strlen(daxdev_base) + 50; + sprintf(path, "%s/modalias", daxdev_base); + rc = sysfs_read_attr(ctx, path, buf); + /* older kernels may be lack the modalias attribute */ + if (rc < 0 && rc != -ENOENT) + goto err_read; + if (rc == 0) + dev->kmod_list = to_module_list(ctx, buf); + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -569,6 +614,92 @@ static int is_enabled(const char *drvpath) return 1; } +static int daxctl_bind(struct daxctl_ctx *ctx, const char *devname, + const char *mod_name) +{ + DIR *dir; + int rc = 0; + char path[200]; + struct dirent *de; + const int len = sizeof(path); + + if (!devname) { + err(ctx, "missing devname\n"); + return -EINVAL; + } + + if (snprintf(path, len, "/sys/bus/dax/drivers") >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + dir = opendir(path); + if (!dir) { + err(ctx, "%s: opendir(\"%s\") failed\n", devname, path); + return -ENXIO; + } + + while ((de = readdir(dir)) != NULL) { + char *drv_path; + + if (de->d_ino == 0) + continue; + if (de->d_name[0] == '.') + continue; + if (strcmp(de->d_name, mod_name) != 0) + continue; + + if (asprintf(&drv_path, "%s/%s/new_id", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + + if (asprintf(&drv_path, "%s/%s/bind", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + break; + } + closedir(dir); + + if (rc) { + dbg(ctx, "%s: bind failed\n", devname); + return rc; + } + return 0; +} + +static int daxctl_unbind(struct daxctl_ctx *ctx, const char *devpath) +{ + const char *devname = devpath_to_devname(devpath); + char path[200]; + const int len = sizeof(path); + int rc; + + if (snprintf(path, len, "%s/driver/remove_id", devpath) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + rc = sysfs_write_attr(ctx, path, devname); + if (rc) + return rc; + + if (snprintf(path, len, "%s/driver/unbind", devpath) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + return sysfs_write_attr(ctx, path, devname); + +} + DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) { struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); @@ -584,6 +715,109 @@ DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) return is_enabled(path); } +static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev, + const char *mod_name) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + struct kmod_list *iter; + int rc = -ENXIO; + + if (dev->kmod_list == NULL) { + err(ctx, "%s: a modalias lookup list was not created\n", + devname); + return rc; + } + + kmod_list_foreach(iter, dev->kmod_list) { + struct kmod_module *mod = kmod_module_get_module(iter); + const char *name = kmod_module_get_name(mod); + + if (strcmp(name, mod_name) != 0) { + kmod_module_unref(mod); + continue; + } + dbg(ctx, "%s inserting module: %s\n", devname, name); + rc = kmod_module_probe_insert_module(mod, + KMOD_PROBE_APPLY_BLACKLIST, NULL, NULL, NULL, + NULL); + if (rc < 0) { + err(ctx, "%s: insert failure: %d\n", devname, rc); + return rc; + } + dev->module = mod; + } + + if (rc == -ENXIO) + err(ctx, "%s: Unable to find module: %s in alias list\n", + devname, mod_name); + return rc; +} + +static int daxctl_dev_enable(struct daxctl_dev *dev, enum daxctl_dev_mode mode) +{ + struct daxctl_region *region = daxctl_dev_get_region(dev); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mod_name = dax_modules[mode]; + int rc; + + if (daxctl_dev_is_enabled(dev)) + return 0; + + if (mode >= DAXCTL_DEV_MODE_END || mod_name == NULL) { + err(ctx, "%s: Invalid mode: %d\n", devname, mode); + return -EINVAL; + } + + rc = daxctl_insert_kmod_for_mode(dev, mod_name); + if (rc) + return rc; + + rc = daxctl_bind(ctx, devname, mod_name); + if (!daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to enable\n", devname); + return rc ? rc : -ENXIO; + } + + region->devices_init = 0; + dax_devices_init(region); + rc = 0; + dbg(ctx, "%s: enabled\n", devname); + return rc; +} + +DAXCTL_EXPORT int daxctl_dev_enable_devdax(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_DEVDAX); +} + +DAXCTL_EXPORT int daxctl_dev_enable_ram(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_RAM); +} + +DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + + if (!daxctl_dev_is_enabled(dev)) + return 0; + + daxctl_unbind(ctx, dev->dev_path); + + if (daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to disable\n", devname); + return -EBUSY; + } + + kmod_module_unref(dev->module); + dbg(ctx, "%s: disabled\n", devname); + + return 0; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) { return dev->region->ctx; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index c4af9a7..19904a2 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index e20ccb4..407f459 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); +int daxctl_dev_disable(struct daxctl_dev *dev); +int daxctl_dev_enable_devdax(struct daxctl_dev *dev); +int daxctl_dev_enable_ram(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Wed Jul 24 21:57:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057657 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8CFBA1399 for ; Wed, 24 Jul 2019 21:57:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C0841FFB2 for ; Wed, 24 Jul 2019 21:57:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 705B6288D4; Wed, 24 Jul 2019 21:57:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 96A361FFB2 for ; Wed, 24 Jul 2019 21:57:52 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B2FE52194D3AE; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 36BE0212DA5FC for ; Wed, 24 Jul 2019 15:00:14 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602210" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:47 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 03/13] libdaxctl: add an interface to retrieve the device resource Date: Wed, 24 Jul 2019 15:57:31 -0600 Message-Id: <20190724215741.18556-4-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin , Tony Luck Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add an interface to retrieve the 'resource' attribute for a dax device. Attempt to retrieve it as usual via sysfs, but since older kernels may be missing this attribute, as a fallback, attempt to retrieve it from /proc/iomem Cc: Dan Williams [fscanf format string problem and diagnosis] Reported-by: Tony Luck Reviewed-by: Dan Williams Signed-off-by: Vishal Verma --- Makefile.am | 3 ++- daxctl/lib/Makefile.am | 2 ++ daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 12 +++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/iomem.c | 37 ++++++++++++++++++++++++++++++++++ util/iomem.h | 12 +++++++++++ 8 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 util/iomem.c create mode 100644 util/iomem.h diff --git a/Makefile.am b/Makefile.am index df8797e..8d10a10 100644 --- a/Makefile.am +++ b/Makefile.am @@ -74,6 +74,7 @@ libutil_a_SOURCES = \ util/wrapper.c \ util/filter.c \ util/bitmap.c \ - util/abspath.c + util/abspath.c \ + util/iomem.c nobase_include_HEADERS = daxctl/libdaxctl.h diff --git a/daxctl/lib/Makefile.am b/daxctl/lib/Makefile.am index 9f0e444..7704b1b 100644 --- a/daxctl/lib/Makefile.am +++ b/daxctl/lib/Makefile.am @@ -9,6 +9,8 @@ lib_LTLIBRARIES = libdaxctl.la libdaxctl_la_SOURCES =\ ../libdaxctl.h \ libdaxctl-private.h \ + ../../util/iomem.c \ + ../../util/iomem.h \ ../../util/sysfs.c \ ../../util/sysfs.h \ ../../util/log.c \ diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index 120137f..fee67d1 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -65,6 +65,7 @@ struct daxctl_dev { size_t buf_len; char *dev_path; struct list_node list; + unsigned long long resource; unsigned long long size; struct kmod_module *module; struct kmod_list *kmod_list; diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 85c4fbe..9bbf256 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -24,6 +24,7 @@ #include #include +#include #include #include "libdaxctl-private.h" @@ -369,6 +370,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) dev->major = major(st.st_rdev); dev->minor = minor(st.st_rdev); + sprintf(path, "%s/resource", daxdev_base); + if (sysfs_read_attr(ctx, path, buf) == 0) + dev->resource = strtoull(buf, NULL, 0); + else + dev->resource = iomem_get_dev_resource(ctx, daxdev_base); + sprintf(path, "%s/size", daxdev_base); if (sysfs_read_attr(ctx, path, buf) < 0) goto err_read; @@ -878,6 +885,11 @@ DAXCTL_EXPORT int daxctl_dev_get_minor(struct daxctl_dev *dev) return dev->minor; } +DAXCTL_EXPORT unsigned long long daxctl_dev_get_resource(struct daxctl_dev *dev) +{ + return dev->resource; +} + DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev) { return dev->size; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 19904a2..1692624 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -58,4 +58,5 @@ global: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_get_resource; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 407f459..adf55f3 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -66,6 +66,7 @@ int daxctl_dev_get_id(struct daxctl_dev *dev); const char *daxctl_dev_get_devname(struct daxctl_dev *dev); int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); +unsigned long long daxctl_dev_get_resource(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); diff --git a/util/iomem.c b/util/iomem.c new file mode 100644 index 0000000..a3c23f5 --- /dev/null +++ b/util/iomem.c @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ + +#include +#include +#include +#include +#include +#include + +unsigned long long __iomem_get_dev_resource(struct log_ctx *ctx, + const char *devpath) +{ + const char *devname = devpath_to_devname(devpath); + FILE *fp = fopen("/proc/iomem", "r"); + unsigned long long res; + char name[256]; + + if (fp == NULL) { + log_err(ctx, "%s: open /proc/iomem: %s\n", devname, + strerror(errno)); + return 0; + } + + while (fscanf(fp, "%llx-%*x : %254[^\n]\n", &res, name) == 2) { + if (strcmp(name, devname) == 0) { + log_dbg(ctx, "%s: got resource via iomem: %#llx\n", + devname, res); + fclose(fp); + return res; + } + } + + log_dbg(ctx, "%s: not found in iomem\n", devname); + fclose(fp); + return 0; +} diff --git a/util/iomem.h b/util/iomem.h new file mode 100644 index 0000000..aaaf6a7 --- /dev/null +++ b/util/iomem.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_IOMEM_H_ +#define _NDCTL_IOMEM_H_ + +struct log_ctx; +unsigned long long __iomem_get_dev_resource(struct log_ctx *ctx, + const char *path); + +#define iomem_get_dev_resource(c, p) __iomem_get_dev_resource(&(c)->ctx, (p)) + +#endif /* _NDCTL_IOMEM_H_ */ From patchwork Wed Jul 24 21:57:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3079E912 for ; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E05A288CA for ; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1269C288E1; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2E57C288CA for ; Wed, 24 Jul 2019 21:57:53 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 18C7C212DC5B4; Wed, 24 Jul 2019 15:00:17 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 90EF021959CB2 for ; Wed, 24 Jul 2019 15:00:14 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602214" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:47 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 04/13] libdaxctl: add a 'daxctl_memory' object for memory based operations Date: Wed, 24 Jul 2019 15:57:32 -0600 Message-Id: <20190724215741.18556-5-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Introduce a new 'daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes. Add libdaxctl APIs to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device. This adds the following new interfaces: daxctl_dev_get_target_node daxctl_dev_get_memory; daxctl_memory_get_dev; daxctl_memory_get_node_path; daxctl_memory_get_block_size; daxctl_memory_set_online daxctl_memory_set_offline daxctl_memory_is_online Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams [for the memblock-already-online TOCTOU hole] Reported-by: Fan Du Tested-by: Fan Du Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl-private.h | 16 ++ daxctl/lib/libdaxctl.c | 407 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 8 + daxctl/libdaxctl.h | 10 + 4 files changed, 441 insertions(+) diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index fee67d1..46a5782 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -39,6 +39,11 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum memory_state { + MEM_OFFLINE, + MEM_ONLINE, +}; + /** * struct daxctl_region - container for dax_devices */ @@ -70,8 +75,19 @@ struct daxctl_dev { struct kmod_module *module; struct kmod_list *kmod_list; struct daxctl_region *region; + struct daxctl_memory *mem; + int target_node; +}; + +struct daxctl_memory { + struct daxctl_dev *dev; + void *mem_buf; + size_t buf_len; + char *node_path; + unsigned long block_size; }; + static inline int check_kmod(struct kmod_ctx *kmod_ctx) { return kmod_ctx ? 0 : -ENXIO; diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 9bbf256..4684a07 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -200,6 +200,15 @@ DAXCTL_EXPORT void daxctl_region_get_uuid(struct daxctl_region *region, uuid_t u uuid_copy(uu, region->uuid); } +static void free_mem(struct daxctl_dev *dev) +{ + if (dev && dev->mem) { + free(dev->mem->node_path); + free(dev->mem); + dev->mem = NULL; + } +} + static void free_dev(struct daxctl_dev *dev, struct list_head *head) { if (head) @@ -207,6 +216,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head) kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf); free(dev->dev_path); + free_mem(dev); free(dev); } @@ -343,6 +353,96 @@ static struct kmod_list *to_module_list(struct daxctl_ctx *ctx, return list; } +static int dev_is_system_ram_capable(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char *mod_path, *mod_base; + char path[200]; + int rc = 0; + const int len = sizeof(path); + + if (!daxctl_dev_is_enabled(dev)) + return rc; + + if (snprintf(path, len, "%s/driver/module", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + mod_path = realpath(path, NULL); + if (!mod_path) { + rc = -errno; + err(ctx, "%s: unable to determine module: %s\n", devname, + strerror(errno)); + return rc; + } + + mod_base = basename(mod_path); + if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_RAM]) == 0) + rc = 1; + else if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_DEVDAX]) == 0) + rc = 0; + + free(mod_path); + return rc; +} + +/* + * This checks for the device to be in system-ram mode, so calling + * daxctl_dev_get_memory() on a devdax mode device will always return NULL. + */ +static struct daxctl_memory *daxctl_dev_alloc_mem(struct daxctl_dev *dev) +{ + const char *size_path = "/sys/devices/system/memory/block_size_bytes"; + const char *node_base = "/sys/devices/system/node/node"; + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + struct daxctl_memory *mem; + char buf[SYSFS_ATTR_SIZE]; + int node_num; + + if (!dev_is_system_ram_capable(dev)) + return NULL; + + mem = calloc(1, sizeof(*mem)); + if (!mem) + return NULL; + + mem->dev = dev; + + if (sysfs_read_attr(ctx, size_path, buf) == 0) { + mem->block_size = strtoul(buf, NULL, 16); + if (mem->block_size == 0 || mem->block_size == ULONG_MAX) { + err(ctx, "%s: Unable to determine memblock size: %s\n", + devname, strerror(errno)); + mem->block_size = 0; + } + } + + node_num = daxctl_dev_get_target_node(dev); + if (node_num >= 0) { + if (asprintf(&mem->node_path, "%s%d", node_base, + node_num) < 0) { + err(ctx, "%s: Unable to set node_path\n", devname); + goto err_mem; + } + } + + mem->mem_buf = calloc(1, strlen(node_base) + 256); + if (!mem->mem_buf) + goto err_node; + mem->buf_len = strlen(node_base) + 256; + + return mem; + +err_node: + free(mem->node_path); +err_mem: + free(mem); + return NULL; +} + static void *add_dax_dev(void *parent, int id, const char *daxdev_base) { const char *devname = devpath_to_devname(daxdev_base); @@ -398,6 +498,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) if (rc == 0) dev->kmod_list = to_module_list(ctx, buf); + sprintf(path, "%s/target_node", daxdev_base); + if (sysfs_read_attr(ctx, path, buf) == 0) + dev->target_node = strtol(buf, NULL, 0); + else + dev->target_node = -1; + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -812,6 +918,9 @@ DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) if (!daxctl_dev_is_enabled(dev)) return 0; + /* If there is a memory object, first free that */ + free_mem(dev); + daxctl_unbind(ctx, dev->dev_path); if (daxctl_dev_is_enabled(dev)) { @@ -894,3 +1003,301 @@ DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev) { return dev->size; } + +DAXCTL_EXPORT int daxctl_dev_get_target_node(struct daxctl_dev *dev) +{ + return dev->target_node; +} + +DAXCTL_EXPORT struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev) +{ + if (dev->mem) + return dev->mem; + else + return daxctl_dev_alloc_mem(dev); +} + +DAXCTL_EXPORT struct daxctl_dev *daxctl_memory_get_dev(struct daxctl_memory *mem) +{ + return mem->dev; +} + +DAXCTL_EXPORT const char *daxctl_memory_get_node_path(struct daxctl_memory *mem) +{ + return mem->node_path; +} + +DAXCTL_EXPORT unsigned long daxctl_memory_get_block_size(struct daxctl_memory *mem) +{ + return mem->block_size; +} + +static int online_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "online_movable"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* + * if already online, possibly due to kernel config or a udev rule, + * there is nothing to do and we can skip over the memblock + */ + if (strncmp(buf, "online", 6) == 0) + return 1; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) { + /* + * While we performed an already-online check above, there + * is still a TOCTOU hole where someone (such as a udev rule) + * may have raced to online the memory. In such a case, + * the sysfs store will fail, however we can check for this + * by simply reading the state again. If it changed to the + * desired state, then we don't have to error out. + */ + if (sysfs_read_attr(ctx, path, buf) == 0) { + if (strncmp(buf, "online", 6) == 0) + return 1; + } + err(ctx, "%s: Failed to online %s: %s\n", + devname, path, strerror(-rc)); + } + return rc; +} + +static int offline_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "offline"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* if already offline, there is nothing to do */ + if (strncmp(buf, "offline", 7) == 0) + return 1; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) { + /* Close the TOCTOU hole like in online_one_memblock() above */ + if (sysfs_read_attr(ctx, path, buf) == 0) { + if (strncmp(buf, "offline", 7) == 0) + return 1; + } + err(ctx, "%s: Failed to offline %s: %s\n", + devname, path, strerror(-rc)); + } + return rc; +} + +static bool memblock_in_dev(struct daxctl_dev *dev, const char *memblock) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *mem_base = "/sys/devices/system/memory/"; + unsigned long long memblock_res, dev_start, dev_end; + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + unsigned long memblock_size; + int path_len = mem->buf_len; + char buf[SYSFS_ATTR_SIZE]; + unsigned long phys_index; + char *path = mem->mem_buf; + + if (snprintf(path, path_len, "%s/%s/phys_index", + mem_base, memblock) < 0) + return false; + + if (sysfs_read_attr(ctx, path, buf) == 0) { + phys_index = strtoul(buf, NULL, 16); + if (phys_index == 0 || phys_index == ULONG_MAX) { + err(ctx, "%s: %s: Unable to determine phys_index: %s\n", + devname, memblock, strerror(errno)); + return false; + } + } else { + err(ctx, "%s: %s: Unable to determine phys_index: %s\n", + devname, memblock, strerror(errno)); + return false; + } + + dev_start = daxctl_dev_get_resource(dev); + if (!dev_start) { + err(ctx, "%s: Unable to determine resource\n", devname); + return false; + } + dev_end = dev_start + daxctl_dev_get_size(dev); + + memblock_size = daxctl_memory_get_block_size(mem); + if (!memblock_size) { + err(ctx, "%s: Unable to determine memory block size\n", + devname); + return false; + } + memblock_res = phys_index * memblock_size; + + if (memblock_res >= dev_start && memblock_res <= dev_end) + return true; + + return false; +} + +static int daxctl_memory_set_state(struct daxctl_memory *mem, + enum memory_state state) +{ + struct daxctl_dev *dev = daxctl_memory_get_dev(mem); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *node_path; + int rc, changed = 0; + struct dirent *de; + DIR *node_dir; + + node_path = daxctl_memory_get_node_path(mem); + if (!node_path) { + err(ctx, "%s: Failed to get node_path\n", devname); + return -ENXIO; + } + + node_dir = opendir(node_path); + if (!node_dir) + return -errno; + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *path = mem->mem_buf; + int len = mem->buf_len; + + if (strncmp(de->d_name, "memory", 6) == 0) { + if (!memblock_in_dev(dev, de->d_name)) + continue; + rc = snprintf(path, len, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + if (state == MEM_ONLINE) + rc = online_one_memblock(dev, path); + else if (state == MEM_OFFLINE) + rc = offline_one_memblock(dev, path); + else + rc = -EINVAL; + if (rc < 0) + goto out_dir; + if (rc == 0) + changed++; + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = changed; + +out_dir: + closedir(node_dir); + return rc; +} + +DAXCTL_EXPORT int daxctl_memory_set_online(struct daxctl_memory *mem) +{ + return daxctl_memory_set_state(mem, MEM_ONLINE); +} + +DAXCTL_EXPORT int daxctl_memory_set_offline(struct daxctl_memory *mem) +{ + return daxctl_memory_set_state(mem, MEM_OFFLINE); +} + +static int memblock_is_online(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + if (strncmp(buf, "online", 6) == 0) + return 1; + + return 0; +} + +DAXCTL_EXPORT int daxctl_memory_is_online(struct daxctl_memory *mem) +{ + struct daxctl_dev *dev = daxctl_memory_get_dev(mem); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int rc, num_online = 0; + const char *node_path; + struct dirent *de; + DIR *node_dir; + + node_path = daxctl_memory_get_node_path(mem); + if (!node_path) { + err(ctx, "%s: Failed to get node_path\n", devname); + return -ENXIO; + } + + node_dir = opendir(node_path); + if (!node_dir) + return -errno; + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *path = mem->mem_buf; + int len = mem->buf_len; + + if (strncmp(de->d_name, "memory", 6) == 0) { + if (!memblock_in_dev(dev, de->d_name)) + continue; + rc = snprintf(path, len, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + rc = memblock_is_online(dev, path); + if (rc < 0) { + err(ctx, "%s: Unable to determine state: %s\n", + devname, path); + goto out_dir; + } + if (rc > 0) + num_online++; + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = num_online; + +out_dir: + closedir(node_dir); + return rc; +} diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 1692624..53eb700 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -59,4 +59,12 @@ global: daxctl_dev_enable_devdax; daxctl_dev_enable_ram; daxctl_dev_get_resource; + daxctl_dev_get_target_node; + daxctl_dev_get_memory; + daxctl_memory_get_dev; + daxctl_memory_get_node_path; + daxctl_memory_get_block_size; + daxctl_memory_set_online; + daxctl_memory_set_offline; + daxctl_memory_is_online; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index adf55f3..d844359 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -73,6 +73,16 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev); int daxctl_dev_disable(struct daxctl_dev *dev); int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); +int daxctl_dev_get_target_node(struct daxctl_dev *dev); + +struct daxctl_memory; +struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev); +struct daxctl_dev *daxctl_memory_get_dev(struct daxctl_memory *mem); +const char *daxctl_memory_get_node_path(struct daxctl_memory *mem); +unsigned long daxctl_memory_get_block_size(struct daxctl_memory *mem); +int daxctl_memory_set_online(struct daxctl_memory *mem); +int daxctl_memory_set_offline(struct daxctl_memory *mem); +int daxctl_memory_is_online(struct daxctl_memory *mem); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Wed Jul 24 21:57:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D31FE1399 for ; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C163C288CA for ; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B5A84288E1; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7639F288CA for ; Wed, 24 Jul 2019 21:57:54 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 62472212DC5BA; Wed, 24 Jul 2019 15:00:17 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D97C1212DC5A4 for ; Wed, 24 Jul 2019 15:00:14 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602217" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:47 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 05/13] daxctl/list: add target_node for device listings Date: Wed, 24 Jul 2019 15:57:33 -0600 Message-Id: <20190724215741.18556-6-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the node to which the new memory belongs. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/util/json.c b/util/json.c index babdc8c..f521337 100644 --- a/util/json.c +++ b/util/json.c @@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, { const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; + int node; jdev = json_object_new_object(); if (!devname || !jdev) @@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, if (jobj) json_object_object_add(jdev, "size", jobj); + node = daxctl_dev_get_target_node(dev); + if (node >= 0) { + jobj = json_object_new_int(node); + if (jobj) + json_object_object_add(jdev, "target_node", jobj); + } + return jdev; } From patchwork Wed Jul 24 21:57:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DA6B912 for ; Wed, 24 Jul 2019 21:57:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B55C1FFB2 for ; Wed, 24 Jul 2019 21:57:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FE1F288D4; Wed, 24 Jul 2019 21:57:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F3A871FFB2 for ; Wed, 24 Jul 2019 21:57:55 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id BB6A8212DC5BF; Wed, 24 Jul 2019 15:00:17 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 35548212DC5A4 for ; Wed, 24 Jul 2019 15:00:15 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602223" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:48 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 06/13] daxctl/list: display the mode for a dax device Date: Wed, 24 Jul 2019 15:57:34 -0600 Message-Id: <20190724215741.18556-7-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a reconfigure-device command, allow JSON listings to display the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json(), also daxctl-list) to print the mode in device listings via a 'daxctl-list' command or immediately after a mode change. Add a 'state' attribute to the json listings for devices, since a device could end up in a state where it is not bound to any driver, and hence, 'disabled'. The state attribute is only displayed for disabled devices. Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 2 ++ util/json.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 4684a07..5f8d2d0 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -12,6 +12,8 @@ */ #include #include +#include +#include #include #include #include diff --git a/util/json.c b/util/json.c index f521337..9f80b5b 100644 --- a/util/json.c +++ b/util/json.c @@ -269,6 +269,7 @@ struct json_object *util_dimm_to_json(struct ndctl_dimm *dimm, struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, unsigned long flags) { + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; int node; @@ -292,6 +293,19 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, json_object_object_add(jdev, "target_node", jobj); } + if (mem) + jobj = json_object_new_string("system-ram"); + else + jobj = json_object_new_string("devdax"); + if (jobj) + json_object_object_add(jdev, "mode", jobj); + + if (!daxctl_dev_is_enabled(dev)) { + jobj = json_object_new_string("disabled"); + if (jobj) + json_object_object_add(jdev, "state", jobj); + } + return jdev; } From patchwork Wed Jul 24 21:57:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057663 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D66D914DB for ; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C48DE1FFB2 for ; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B8E88288D4; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 66C4D1FFB2 for ; Wed, 24 Jul 2019 21:57:58 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 0DA1F2194EB7A; Wed, 24 Jul 2019 15:00:18 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 87D66212DC5AD for ; Wed, 24 Jul 2019 15:00:15 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602229" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:48 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 07/13] daxctl: add a new reconfigure-device command Date: Wed, 24 Jul 2019 15:57:35 -0600 Message-Id: <20190724215741.18556-8-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 347 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 351 insertions(+) create mode 100644 daxctl/device.c diff --git a/daxctl/Makefile.am b/daxctl/Makefile.am index 94f73f9..66dcc7f 100644 --- a/daxctl/Makefile.am +++ b/daxctl/Makefile.am @@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 00ef5e9..756ba2a 100644 --- a/daxctl/builtin.h +++ b/daxctl/builtin.h @@ -6,4 +6,5 @@ struct daxctl_ctx; int cmd_list(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_migrate(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx); #endif /* _DAXCTL_BUILTIN_H_ */ diff --git a/daxctl/daxctl.c b/daxctl/daxctl.c index 2e41747..e1ba7b8 100644 --- a/daxctl/daxctl.c +++ b/daxctl/daxctl.c @@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, }; int main(int argc, const char **argv) diff --git a/daxctl/device.c b/daxctl/device.c new file mode 100644 index 0000000..a71ebbe --- /dev/null +++ b/daxctl/device.c @@ -0,0 +1,347 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static struct { + const char *dev; + const char *mode; + int region_id; + bool no_online; + bool do_offline; + bool human; + bool verbose; +} param = { + .region_id = -1, +}; + +enum dev_mode { + DAXCTL_DEV_MODE_UNKNOWN, + DAXCTL_DEV_MODE_DEVDAX, + DAXCTL_DEV_MODE_RAM, +}; + +static enum dev_mode reconfig_mode = DAXCTL_DEV_MODE_UNKNOWN; +static unsigned long flags; + +enum device_action { + ACTION_RECONFIG, +}; + +#define BASE_OPTIONS() \ +OPT_INTEGER('r', "region", ¶m.region_id, "restrict to the given region"), \ +OPT_BOOLEAN('u', "human", ¶m.human, "use human friendly number formats"), \ +OPT_BOOLEAN('v', "verbose", ¶m.verbose, "emit more debug messages") + +#define RECONFIG_OPTIONS() \ +OPT_STRING('m', "mode", ¶m.mode, "mode", "mode to switch the device to"), \ +OPT_BOOLEAN('N', "no-online", ¶m.no_online, \ + "don't auto-online memory sections"), \ +OPT_BOOLEAN('O', "attempt-offline", ¶m.do_offline, \ + "attempt to offline memory sections") + +static const struct option reconfig_options[] = { + BASE_OPTIONS(), + RECONFIG_OPTIONS(), + OPT_END(), +}; + +static const char *parse_device_options(int argc, const char **argv, + enum device_action action, const struct option *options, + const char *usage, struct daxctl_ctx *ctx) +{ + const char * const u[] = { + usage, + NULL + }; + int i, rc = 0; + + argc = parse_options(argc, argv, options, u, 0); + + /* Handle action-agnostic non-option arguments */ + if (argc == 0) { + char *action_string; + + switch (action) { + case ACTION_RECONFIG: + action_string = "reconfigure"; + break; + default: + action_string = "<>"; + break; + } + fprintf(stderr, "specify a device to %s, or \"all\"\n", + action_string); + rc = -EINVAL; + } + for (i = 1; i < argc; i++) { + fprintf(stderr, "unknown extra parameter \"%s\"\n", argv[i]); + rc = -EINVAL; + } + + if (rc) { + usage_with_options(u, options); + return NULL; + } + + /* Handle action-agnostic options */ + if (param.verbose) + daxctl_set_log_priority(ctx, LOG_DEBUG); + if (param.human) + flags |= UTIL_JSON_HUMAN; + + /* Handle action-specific options */ + switch (action) { + case ACTION_RECONFIG: + if (!param.mode) { + fprintf(stderr, "error: a 'mode' option is required\n"); + usage_with_options(u, reconfig_options); + rc = -EINVAL; + } + if (strcmp(param.mode, "system-ram") == 0) { + reconfig_mode = DAXCTL_DEV_MODE_RAM; + if (param.do_offline) { + fprintf(stderr, + "can't --attempt-offline for system-ram mode\n"); + rc = -EINVAL; + } + } else if (strcmp(param.mode, "devdax") == 0) { + reconfig_mode = DAXCTL_DEV_MODE_DEVDAX; + if (param.no_online) { + fprintf(stderr, + "can't --no-online for devdax mode\n"); + rc = -EINVAL; + } + } + break; + } + if (rc) { + usage_with_options(u, options); + return NULL; + } + + return argv[0]; +} + +static int disable_devdax_device(struct daxctl_dev *dev) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + int rc; + + if (mem) { + fprintf(stderr, "%s is in system-ram mode, nothing to do\n", + devname); + return 1; + } + rc = daxctl_dev_disable(dev); + if (rc) { + fprintf(stderr, "%s: disable failed: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + return rc; + } + return 0; +} + +static int reconfig_mode_system_ram(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_memory *mem; + int rc, skip_enable = 0; + + if (daxctl_dev_is_enabled(dev)) { + rc = disable_devdax_device(dev); + if (rc < 0) + return rc; + if (rc > 0) + skip_enable = 1; + } + + if (!skip_enable) { + rc = daxctl_dev_enable_ram(dev); + if (rc) + return rc; + } + + if (param.no_online) + return 0; + + mem = daxctl_dev_get_memory(dev); + if (!mem) { + fprintf(stderr, "%s: unable to get the memory object: %s\n", + devname, strerror(-rc)); + return -ENXIO; + } + rc = daxctl_memory_set_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to online memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: onlined %d memory sections\n", + devname, rc); + + return 0; +} + +static int disable_system_ram_device(struct daxctl_dev *dev) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + int rc; + + if (!mem) { + fprintf(stderr, "%s: already in devdax mode, nothing to do\n", + devname); + return 1; + } + + if (param.do_offline) { + rc = daxctl_memory_set_offline(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to offline memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: offlined %d memory sections\n", + devname, rc); + } + + rc = daxctl_memory_is_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to determine online state: %s\n", + devname, strerror(-rc)); + return rc; + } + if (rc > 0) { + if (param.verbose) { + fprintf(stderr, "%s: found %d memory sections online\n", + devname, rc); + fprintf(stderr, "%s: refusing to change modes\n", + devname); + } + return -EBUSY; + } + rc = daxctl_dev_disable(dev); + if (rc) { + fprintf(stderr, "%s: disable failed: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + return rc; + } + return 0; +} + +static int reconfig_mode_devdax(struct daxctl_dev *dev) +{ + int rc; + + if (daxctl_dev_is_enabled(dev)) { + rc = disable_system_ram_device(dev); + if (rc) + return rc; + } + + rc = daxctl_dev_enable_devdax(dev); + if (rc) + return rc; + + return 0; +} + +static int do_reconfig(struct daxctl_dev *dev, enum dev_mode mode, + struct json_object *jdevs) +{ + struct json_object *jdev; + int rc = 0; + + switch (mode) { + case DAXCTL_DEV_MODE_RAM: + rc = reconfig_mode_system_ram(dev); + break; + case DAXCTL_DEV_MODE_DEVDAX: + rc = reconfig_mode_devdax(dev); + break; + default: + fprintf(stderr, "%s: unknown mode requested: %d\n", + daxctl_dev_get_devname(dev), mode); + rc = -EINVAL; + } + + jdev = util_daxctl_dev_to_json(dev, flags); + if (jdevs && jdev) + json_object_array_add(jdevs, jdev); + + return rc; +} + +static int do_xaction_device(const char *device, enum device_action action, + struct daxctl_ctx *ctx, int *processed) +{ + struct json_object *jdevs = json_object_new_array(); + struct daxctl_region *region; + struct daxctl_dev *dev; + int rc = -ENXIO; + + *processed = 0; + + daxctl_region_foreach(ctx, region) { + if (param.region_id >= 0 && param.region_id + != daxctl_region_get_id(region)) + continue; + + daxctl_dev_foreach(region, dev) { + if (!util_daxctl_dev_filter(dev, device)) + continue; + + switch (action) { + case ACTION_RECONFIG: + rc = do_reconfig(dev, reconfig_mode, jdevs); + if (rc == 0) + (*processed)++; + break; + default: + rc = -EINVAL; + break; + } + } + } + + /* + * jdevs is the containing json array for all devices we are reporting + * on. It therefore needs to be outside the region/device iterators, + * and passed in to the do_ functions to add their objects to + */ + if (jdevs) + util_display_json_array(stdout, jdevs, flags); + + return rc; +} + +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl reconfigure-device []"; + const char *device = parse_device_options(argc, argv, ACTION_RECONFIG, + reconfig_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_RECONFIG, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error reconfiguring devices: %s\n", + strerror(-rc)); + + fprintf(stderr, "reconfigured %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} From patchwork Wed Jul 24 21:57:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057661 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37681912 for ; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 232E5288CA for ; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 17B66288E1; Wed, 24 Jul 2019 21:57:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9E2E5288CA for ; Wed, 24 Jul 2019 21:57:58 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 4FDCF2194D3B8; Wed, 24 Jul 2019 15:00:18 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D885B212DC5B3 for ; Wed, 24 Jul 2019 15:00:15 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602232" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:48 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 08/13] Documentation/daxctl: add a man page for daxctl-reconfigure-device Date: Wed, 24 Jul 2019 15:57:36 -0600 Message-Id: <20190724215741.18556-9-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a man page describing the new daxctl-reconfigure-device command. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 139 ++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt diff --git a/Documentation/daxctl/Makefile.am b/Documentation/daxctl/Makefile.am index 6aba035..715fbad 100644 --- a/Documentation/daxctl/Makefile.am +++ b/Documentation/daxctl/Makefile.am @@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS) diff --git a/Documentation/daxctl/daxctl-reconfigure-device.txt b/Documentation/daxctl/daxctl-reconfigure-device.txt new file mode 100644 index 0000000..fb2b36b --- /dev/null +++ b/Documentation/daxctl/daxctl-reconfigure-device.txt @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-reconfigure-device(1) +============================ + +NAME +---- +daxctl-reconfigure-device - Reconfigure a dax device into a different mode + +SYNOPSIS +-------- +[verse] +'daxctl reconfigure-device' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] +---- + +* Reconfigure dax0.0 to devdax mode, attempt to offline the memory +---- +# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0 +{ + "chardev":"dax0.0", + "size":"15.63 GiB (16.78 GB)", + "target_node":2, + "mode":"devdax" +} +---- + +* Reconfigure all dax devices on region0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --region=0 all +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + }, + { + "chardev":"dax0.1", + "size":16777216000, + "target_node":3, + "mode":"system-ram" + } +] +---- + +* Run a process called 'some-service' using numactl to restrict its cpu +nodes to '0' and '1', and memory allocations to node 2 (determined using +daxctl_dev_get_target_node() or 'daxctl list') +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] + +# numactl --cpunodebind=0-1 --membind=2 -- some-service --opt1 --opt2 +---- + +DESCRIPTION +----------- + +Reconfigure the operational mode of a dax device. This can be used to convert +a regular 'devdax' mode device to the 'system-ram' mode which allows for the dax +range to be hot-plugged into the system as regular memory. + +NOTE: This is a destructive operation. Any data on the dax device *will* be +lost. + +NOTE: Device reconfiguration depends on the dax-bus device model. If dax-class is +in use (via the dax_pmem_compat driver), the reconfiguration will fail. See +linkdaxctl:daxctl-migrate-device-model[1] for more information. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-m:: +--mode=:: + Specify the mode to which the dax device(s) should be reconfigured. + - "system-ram": hotplug the device into system memory. + + - "devdax": switch to the normal "device dax" mode. This requires the + kernel to support hot-unplugging 'kmem' based memory. If this is not + available, a reboot is the only way to switch back to 'devdax' mode. + +-N:: +--no-online:: + By default, memory sections provided by system-ram devices will be + brought online automatically and immediately with the 'online_movable' + policy. Use this option to disable the automatic onlining behavior. + +-O:: +--attempt-offline:: + When converting from "system-ram" mode to "devdax", it is expected + that all the memory sections are first made offline. By default, + daxctl won't touch online memory. However with this option, attempt + to offline the memory on the NUMA node associated with the dax device + before converting it back to "devdax" mode. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-list[1],daxctl-migrate-device-model[1] From patchwork Wed Jul 24 21:57:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEA39912 for ; Wed, 24 Jul 2019 21:58:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CC3D1FFB2 for ; Wed, 24 Jul 2019 21:58:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 914E0288D4; Wed, 24 Jul 2019 21:58:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 127291FFB2 for ; Wed, 24 Jul 2019 21:58:01 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 95B8B21945DDA; Wed, 24 Jul 2019 15:00:18 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 47CA32194EB7A for ; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602235" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:49 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 09/13] daxctl: add commands to online and offline memory Date: Wed, 24 Jul 2019 15:57:37 -0600 Message-Id: <20190724215741.18556-10-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add two new commands: daxctl-online-memory daxctl-offline-memory to manage the state of hot-plugged memory from the system-ram mode for dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user that reconfigures a device into the system-ram mode with the --no-online option, would have no way to later online the memory, and would have to resort to shell scripting to online them manually via sysfs. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- daxctl/builtin.h | 2 + daxctl/daxctl.c | 2 + daxctl/device.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 1 deletion(-) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 756ba2a..f5a0147 100644 --- a/daxctl/builtin.h +++ b/daxctl/builtin.h @@ -7,4 +7,6 @@ struct daxctl_ctx; int cmd_list(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_migrate(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_online_memory(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_offline_memory(int argc, const char **argv, struct daxctl_ctx *ctx); #endif /* _DAXCTL_BUILTIN_H_ */ diff --git a/daxctl/daxctl.c b/daxctl/daxctl.c index e1ba7b8..1ab0732 100644 --- a/daxctl/daxctl.c +++ b/daxctl/daxctl.c @@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, }; int main(int argc, const char **argv) diff --git a/daxctl/device.c b/daxctl/device.c index a71ebbe..64eff04 100644 --- a/daxctl/device.c +++ b/daxctl/device.c @@ -36,6 +36,8 @@ static unsigned long flags; enum device_action { ACTION_RECONFIG, + ACTION_ONLINE, + ACTION_OFFLINE, }; #define BASE_OPTIONS() \ @@ -56,6 +58,11 @@ static const struct option reconfig_options[] = { OPT_END(), }; +static const struct option memory_options[] = { + BASE_OPTIONS(), + OPT_END(), +}; + static const char *parse_device_options(int argc, const char **argv, enum device_action action, const struct option *options, const char *usage, struct daxctl_ctx *ctx) @@ -76,6 +83,12 @@ static const char *parse_device_options(int argc, const char **argv, case ACTION_RECONFIG: action_string = "reconfigure"; break; + case ACTION_ONLINE: + action_string = "online memory for"; + break; + case ACTION_OFFLINE: + action_string = "offline memory for"; + break; default: action_string = "<>"; break; @@ -124,6 +137,10 @@ static const char *parse_device_options(int argc, const char **argv, } } break; + case ACTION_ONLINE: + case ACTION_OFFLINE: + /* nothing special */ + break; } if (rc) { usage_with_options(u, options); @@ -286,10 +303,75 @@ static int do_reconfig(struct daxctl_dev *dev, enum dev_mode mode, return rc; } +static int do_xline(struct daxctl_dev *dev, enum device_action action) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + int rc, num_online; + + if (!daxctl_dev_is_enabled(dev)) { + fprintf(stderr, + "%s: memory operations not possible when disabled\n", + devname); + return -ENXIO; + } + + if (!mem) { + fprintf(stderr, + "%s: memory operations are not applicable in devdax mode\n", + devname); + return -ENXIO; + } + + /* We are enabled, and in the correct mode. Proceed. */ + num_online = daxctl_memory_is_online(mem); + if (num_online < 0) { + fprintf(stderr, "%s: unable to determine online state: %s\n", + devname, strerror(-num_online)); + return num_online; + } + + switch (action) { + case ACTION_ONLINE: + if (num_online > 0) + fprintf(stderr, "%s: %d section%s already online\n", + devname, num_online, + num_online == 1 ? "" : "s"); + rc = daxctl_memory_set_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to online memory: %s\n", + devname, strerror(-rc)); + return rc; + } + fprintf(stderr, "%s: %d new section%s onlined\n", devname, rc, + rc == 1 ? "" : "s"); + break; + case ACTION_OFFLINE: + if (num_online == 0) { + fprintf(stderr, "%s: all sections already offline\n", + devname); + return 0; + } + rc = daxctl_memory_set_offline(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to offline memory: %s\n", + devname, strerror(-rc)); + return rc; + } + fprintf(stderr, "%s: %d section%s offlined\n", devname, rc, + rc == 1 ? "" : "s"); + break; + default: + fprintf(stderr, "%s: invalid action: %d\n", devname, action); + return -EINVAL; + } + return 0; +} + static int do_xaction_device(const char *device, enum device_action action, struct daxctl_ctx *ctx, int *processed) { - struct json_object *jdevs = json_object_new_array(); + struct json_object *jdevs = NULL; struct daxctl_region *region; struct daxctl_dev *dev; int rc = -ENXIO; @@ -307,10 +389,23 @@ static int do_xaction_device(const char *device, enum device_action action, switch (action) { case ACTION_RECONFIG: + /* reconfig needs jdevs, initialize it once */ + if (!jdevs) + jdevs = json_object_new_array(); rc = do_reconfig(dev, reconfig_mode, jdevs); if (rc == 0) (*processed)++; break; + case ACTION_ONLINE: + rc = do_xline(dev, action); + if (rc == 0) + (*processed)++; + break; + case ACTION_OFFLINE: + rc = do_xline(dev, action); + if (rc == 0) + (*processed)++; + break; default: rc = -EINVAL; break; @@ -345,3 +440,37 @@ int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx) processed == 1 ? "" : "s"); return rc; } + +int cmd_online_memory(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl online-memory []"; + const char *device = parse_device_options(argc, argv, ACTION_ONLINE, + memory_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_ONLINE, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error onlining memory: %s\n", + strerror(-rc)); + + fprintf(stderr, "onlined memory for %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} + +int cmd_offline_memory(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl offline-memory []"; + const char *device = parse_device_options(argc, argv, ACTION_OFFLINE, + memory_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_OFFLINE, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error offlining memory: %s\n", + strerror(-rc)); + + fprintf(stderr, "offlined memory for %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} From patchwork Wed Jul 24 21:57:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9749912 for ; Wed, 24 Jul 2019 21:58:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7D641FFB2 for ; Wed, 24 Jul 2019 21:58:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CBEC5288D4; Wed, 24 Jul 2019 21:58:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 579C11FFB2 for ; Wed, 24 Jul 2019 21:58:01 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id D22AD212DC5D2; Wed, 24 Jul 2019 15:00:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B063C2194D387 for ; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602239" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:49 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 10/13] Documentation: Add man pages for daxctl-{on, off}line-memory Date: Wed, 24 Jul 2019 15:57:38 -0600 Message-Id: <20190724215741.18556-11-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add man pages for the two new commands: daxctl-online-memory, and daxctl-offline-memory. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma Reviewed-by: Dan Williams --- Documentation/daxctl/Makefile.am | 4 +- .../daxctl/daxctl-offline-memory.txt | 72 +++++++++++++++++ Documentation/daxctl/daxctl-online-memory.txt | 80 +++++++++++++++++++ 3 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt diff --git a/Documentation/daxctl/Makefile.am b/Documentation/daxctl/Makefile.am index 715fbad..37c3bde 100644 --- a/Documentation/daxctl/Makefile.am +++ b/Documentation/daxctl/Makefile.am @@ -29,7 +29,9 @@ man1_MANS = \ daxctl.1 \ daxctl-list.1 \ daxctl-migrate-device-model.1 \ - daxctl-reconfigure-device.1 + daxctl-reconfigure-device.1 \ + daxctl-online-memory.1 \ + daxctl-offline-memory.1 CLEANFILES = $(man1_MANS) diff --git a/Documentation/daxctl/daxctl-offline-memory.txt b/Documentation/daxctl/daxctl-offline-memory.txt new file mode 100644 index 0000000..ba06287 --- /dev/null +++ b/Documentation/daxctl/daxctl-offline-memory.txt @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-offline-memory(1) +======================== + +NAME +---- +daxctl-offline-memory - Offline the memory for a device that is in system-ram mode + +SYNOPSIS +-------- +[verse] +'daxctl offline-memory' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --human dax0.0 +{ + "chardev":"dax0.0", + "size":"7.87 GiB (8.45 GB)", + "target_node":2, + "mode":"system-ram" +} +---- + +* Offline the memory +---- +# daxctl offline-memory dax0.0 +dax0.0: 62 sections offlined +offlined memory for 1 device +---- + +DESCRIPTION +----------- + +Offline the memory sections associated with a device that has been converted +to the system-ram mode. If one or more blocks are already offline, attempt to +offline the remaining blocks. If all blocks were already offline, print a +message and return success without actually doing anything. + +This is complementary to the 'daxctl-online-memory' command, and may be used +when it is wished to offline the memory sections, but not convert the device +back to 'devdax' mode. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-reconfigure-device[1],daxctl-online-memory[1] diff --git a/Documentation/daxctl/daxctl-online-memory.txt b/Documentation/daxctl/daxctl-online-memory.txt new file mode 100644 index 0000000..5ac1cbf --- /dev/null +++ b/Documentation/daxctl/daxctl-online-memory.txt @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-online-memory(1) +======================= + +NAME +---- +daxctl-online-memory - Online the memory for a device that is in system-ram mode + +SYNOPSIS +-------- +[verse] +'daxctl online-memory' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online --human dax0.0 +{ + "chardev":"dax0.0", + "size":"7.87 GiB (8.45 GB)", + "target_node":2, + "mode":"system-ram" +} +---- + +* Online the memory separately +---- +# daxctl online-memory dax0.0 +dax0.0: 62 new sections onlined +onlined memory for 1 device +---- + +* Onlining memory when some sections were already online +---- +# daxctl online-memory dax0.0 +dax0.0: 1 section already online +dax0.0: 61 new sections onlined +onlined memory for 1 device +---- + +DESCRIPTION +----------- + +Online the memory sections associated with a device that has been converted +to the system-ram mode. If one or more blocks are already online, print a +message about them, and attempt to online the remaining blocks. + +This is complementary to the 'daxctl-reconfigure-device' command, when used with +the '--no-online' option to skip onlining memory sections immediately after the +reconfigure. In these scenarios, the memory can be onlined at a later time using +'daxctl-online-memory'. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-reconfigure-device[1],daxctl-offline-memory[1] From patchwork Wed Jul 24 21:57:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A7F6912 for ; Wed, 24 Jul 2019 21:58:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19ADC1FFB2 for ; Wed, 24 Jul 2019 21:58:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0DFD9288D4; Wed, 24 Jul 2019 21:58:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C63DD1FFB2 for ; Wed, 24 Jul 2019 21:58:02 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 1D0C0212DC5D6; Wed, 24 Jul 2019 15:00:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id DA6BC212DC5A7 for ; Wed, 24 Jul 2019 15:00:16 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602242" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:49 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 11/13] contrib/ndctl: fix region-id completions for daxctl Date: Wed, 24 Jul 2019 15:57:39 -0600 Message-Id: <20190724215741.18556-12-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'. Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly. While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options() Fixes: d6790a32f32c ("daxctl: Add bash-completion") Signed-off-by: Vishal Verma Reviewed-by: Dan Williams --- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index 396a344..cacee2d 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -531,8 +531,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp() @@ -561,10 +567,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return From patchwork Wed Jul 24 21:57:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057669 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27E031399 for ; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 169D81FFB2 for ; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0997F288D4; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EC0A3288CA for ; Wed, 24 Jul 2019 21:58:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 6AF08212DC5D4; Wed, 24 Jul 2019 15:00:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2D1F4212DC5B8 for ; Wed, 24 Jul 2019 15:00:17 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602245" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:50 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 12/13] contrib/ndctl: add bash-completion for the new daxctl commands Date: Wed, 24 Jul 2019 15:57:40 -0600 Message-Id: <20190724215741.18556-13-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add bash completion helpers for the new daxctl-reconfigure-device, daxctl-online-memory, and daxctl-offline-memory commands. Cc: Dan Williams Signed-off-by: Vishal Verma Reviewed-by: Dan Williams --- contrib/ndctl | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index cacee2d..680fe6a 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -547,7 +547,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} " @@ -572,6 +572,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;; @@ -582,8 +585,23 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + ;& + online-memory) + ;& + offline-memory) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main() From patchwork Wed Jul 24 21:57:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Verma X-Patchwork-Id: 11057671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBE1E1399 for ; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DABED288CA for ; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEC771FFB2; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 68BAB1FFB2 for ; Wed, 24 Jul 2019 21:58:05 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id BA089212DC5CC; Wed, 24 Jul 2019 15:00:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.115; helo=mga14.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 7CA28212DC5BD for ; Wed, 24 Jul 2019 15:00:17 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 14:57:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,304,1559545200"; d="scan'208";a="193602248" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga004.fm.intel.com with ESMTP; 24 Jul 2019 14:57:50 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v7 13/13] test: Add a unit test for daxctl-reconfigure-device and friends Date: Wed, 24 Jul 2019 15:57:41 -0600 Message-Id: <20190724215741.18556-14-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190724215741.18556-1-vishal.l.verma@intel.com> References: <20190724215741.18556-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a new unit test to test dax device reconfiguration and memory operations. This teaches test/common about daxctl, and adds an ACPI.NFIT bus variable. Since we have to operate on the ACPI.NFIT bus, the test is marked as destructive. Cc: Dan Williams Signed-off-by: Vishal Verma Reviewed-by: Dan Williams --- test/Makefile.am | 3 +- test/common | 19 ++++++++-- test/daxctl-devices.sh | 81 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+), 4 deletions(-) create mode 100755 test/daxctl-devices.sh diff --git a/test/Makefile.am b/test/Makefile.am index 874c4bb..84474d0 100644 --- a/test/Makefile.am +++ b/test/Makefile.am @@ -49,7 +49,8 @@ TESTS +=\ dax.sh \ device-dax \ device-dax-fio.sh \ - mmap.sh + mmap.sh \ + daxctl-devices.sh if ENABLE_KEYUTILS TESTS += security.sh diff --git a/test/common b/test/common index 1b9d3da..1814a0c 100644 --- a/test/common +++ b/test/common @@ -15,12 +15,25 @@ else exit 1 fi -# NFIT_TEST_BUS[01] +# DAXCTL # -NFIT_TEST_BUS0=nfit_test.0 -NFIT_TEST_BUS1=nfit_test.1 +if [ -f "../daxctl/daxctl" ] && [ -x "../daxctl/daxctl" ]; then + export DAXCTL=../daxctl/daxctl +elif [ -f "./daxctl/daxctl" ] && [ -x "./daxctl/daxctl" ]; then + export DAXCTL=./daxctl/daxctl +else + echo "Couldn't find an daxctl binary" + exit 1 +fi +# NFIT_TEST_BUS[01] +# +NFIT_TEST_BUS0="nfit_test.0" +NFIT_TEST_BUS1="nfit_test.1" +ACPI_BUS="ACPI.NFIT" +E820_BUS="e820" + # Functions # err diff --git a/test/daxctl-devices.sh b/test/daxctl-devices.sh new file mode 100755 index 0000000..cfd9362 --- /dev/null +++ b/test/daxctl-devices.sh @@ -0,0 +1,81 @@ +#!/bin/bash -Ex +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2019 Intel Corporation. All rights reserved. + +rc=77 +. ./common + +trap 'cleanup $LINENO' ERR + +cleanup() +{ + printf "Error at line %d\n" "$1" + [[ $testdev ]] && reset_dev + exit $rc +} + +find_testdev() +{ + local rc=77 + + # find a victim device + testbus="$ACPI_BUS" + testdev=$("$NDCTL" list -b "$testbus" -Ni | jq -er '.[0].dev | .//""') + if [[ ! $testdev ]]; then + printf "Unable to find a victim device\n" + exit "$rc" + fi + printf "Found victim dev: %s on bus: %s\n" "$testdev" "$testbus" +} + +setup_dev() +{ + test -n "$testbus" + test -n "$testdev" + + "$NDCTL" destroy-namespace -f -b "$testbus" "$testdev" + testdev=$("$NDCTL" create-namespace -b "$testbus" -m devdax -fe "$testdev" -s 256M | \ + jq -er '.dev') + test -n "$testdev" +} + +reset_dev() +{ + "$NDCTL" destroy-namespace -f -b "$testbus" "$testdev" +} + +daxctl_get_dev() +{ + "$NDCTL" list -n "$1" -X | jq -er '.[].daxregion.devices[0].chardev' +} + +daxctl_get_mode() +{ + "$DAXCTL" list -d "$1" | jq -er '.[].mode' +} + +daxctl_test() +{ + local daxdev + + daxdev=$(daxctl_get_dev "$testdev") + test -n "$daxdev" + + "$DAXCTL" reconfigure-device -N -m system-ram "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] + "$DAXCTL" online-memory "$daxdev" + "$DAXCTL" offline-memory "$daxdev" + "$DAXCTL" reconfigure-device -m devdax "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] + "$DAXCTL" reconfigure-device -m system-ram "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] + "$DAXCTL" reconfigure-device -O -m devdax "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] +} + +find_testdev +setup_dev +rc=1 +daxctl_test +reset_dev +exit 0