From patchwork Mon Sep 14 13:12:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773747 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEABE618 for ; Mon, 14 Sep 2020 13:13:10 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA1CC214F1 for ; Mon, 14 Sep 2020 13:13:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA1CC214F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 07E6E6E492; Mon, 14 Sep 2020 13:13:09 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1BA916E492 for ; Mon, 14 Sep 2020 13:13:07 +0000 (UTC) IronPort-SDR: jZYnx/5s43UBccQcdm8Iyeg1t2bfxJmd9ddiuDfbK09nIQIOtO+iuhyIhhWMCePM0GESteipDy Th77Xi+F14+w== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113413" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113413" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:06 -0700 IronPort-SDR: e/coP/EARJvmJMWy6O39RP5hrO3nb0qjJUAKGmMHXHdIVVI63UOtNcHFOp98clH+CYcBSGyeIY Pgw+9ewwclLw== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136299" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:05 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:48 +0100 Message-Id: <20200914131256.89465-2-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/9] drm/i915: Expose list of clients in sysfs X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Expose a list of clients with open file handles in sysfs. This will be a basis for a top-like utility showing per-client and per- engine GPU load. Currently we only expose each client's pid and name under opaque numbered directories in /sys/class/drm/card0/clients/. For instance: /sys/class/drm/card0/clients/3/name: Xorg /sys/class/drm/card0/clients/3/pid: 5664 v2: Chris Wilson: * Enclose new members into dedicated structs. * Protect against failed sysfs registration. v3: * sysfs_attr_init. v4: * Fix for internal clients. v5: * Use cyclic ida for client id. (Chris) * Do not leak pid reference. (Chris) * Tidy code with some locals. v6: * Use xa_alloc_cyclic to simplify locking. (Chris) * No need to unregister individial sysfs files. (Chris) * Rebase on top of fpriv kref. * Track client closed status and reflect in sysfs. v7: * Make drm_client more standalone concept. v8: * Simplify sysfs show. (Chris) * Always track name and pid. v9: * Fix cyclic id assignment. v10: * No need for a mutex around xa_alloc_cyclic. * Refactor sysfs into own function. * Unregister sysfs before freeing pid and name. * Move clients setup into own function. v11: * Call clients init directly from driver init. (Chris) v12: * Rebased after i915_gem_release removal. * Do not fail client add on id wrap. (Maciej) Signed-off-by: Tvrtko Ursulin Reviewed-by: Chris Wilson # v11 --- drivers/gpu/drm/i915/Makefile | 5 +- drivers/gpu/drm/i915/i915_drm_client.c | 179 +++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_drm_client.h | 64 +++++++++ drivers/gpu/drm/i915/i915_drv.c | 4 + drivers/gpu/drm/i915/i915_drv.h | 5 + drivers/gpu/drm/i915/i915_gem.c | 23 +++- drivers/gpu/drm/i915/i915_sysfs.c | 8 ++ 7 files changed, 283 insertions(+), 5 deletions(-) create mode 100644 drivers/gpu/drm/i915/i915_drm_client.c create mode 100644 drivers/gpu/drm/i915/i915_drm_client.h diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index e5574e506a5c..250f004d92bb 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -34,8 +34,9 @@ subdir-ccflags-y += -I$(srctree)/$(src) # Please keep these build lists sorted! # core driver code -i915-y += i915_drv.o \ - i915_config.o \ +i915-y += i915_config.o \ + i915_drm_client.o \ + i915_drv.o \ i915_irq.o \ i915_getparam.o \ i915_params.o \ diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c new file mode 100644 index 000000000000..137a5744f9c6 --- /dev/null +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2020 Intel Corporation + */ + +#include +#include +#include + +#include "i915_drm_client.h" +#include "i915_gem.h" +#include "i915_utils.h" + +void i915_drm_clients_init(struct i915_drm_clients *clients) +{ + clients->next_id = 0; + xa_init_flags(&clients->xarray, XA_FLAGS_ALLOC); +} + +static ssize_t +show_client_name(struct device *kdev, struct device_attribute *attr, char *buf) +{ + struct i915_drm_client *client = + container_of(attr, typeof(*client), attr.name); + + return snprintf(buf, PAGE_SIZE, + READ_ONCE(client->closed) ? "<%s>" : "%s", + client->name); +} + +static ssize_t +show_client_pid(struct device *kdev, struct device_attribute *attr, char *buf) +{ + struct i915_drm_client *client = + container_of(attr, typeof(*client), attr.pid); + + return snprintf(buf, PAGE_SIZE, + READ_ONCE(client->closed) ? "<%u>" : "%u", + pid_nr(client->pid)); +} + +static int +__client_register_sysfs(struct i915_drm_client *client) +{ + const struct { + const char *name; + struct device_attribute *attr; + ssize_t (*show)(struct device *dev, + struct device_attribute *attr, + char *buf); + } files[] = { + { "name", &client->attr.name, show_client_name }, + { "pid", &client->attr.pid, show_client_pid }, + }; + unsigned int i; + char buf[16]; + int ret; + + ret = scnprintf(buf, sizeof(buf), "%u", client->id); + if (ret == sizeof(buf)) + return -EINVAL; + + client->root = kobject_create_and_add(buf, client->clients->root); + if (!client->root) + return -ENOMEM; + + for (i = 0; i < ARRAY_SIZE(files); i++) { + struct device_attribute *attr = files[i].attr; + + sysfs_attr_init(&attr->attr); + + attr->attr.name = files[i].name; + attr->attr.mode = 0444; + attr->show = files[i].show; + + ret = sysfs_create_file(client->root, (struct attribute *)attr); + if (ret) + break; + } + + if (ret) + kobject_put(client->root); + + return ret; +} + +static void __client_unregister_sysfs(struct i915_drm_client *client) +{ + kobject_put(fetch_and_zero(&client->root)); +} + +static int +__i915_drm_client_register(struct i915_drm_client *client, + struct task_struct *task) +{ + struct i915_drm_clients *clients = client->clients; + char *name; + int ret; + + name = kstrdup(task->comm, GFP_KERNEL); + if (!name) + return -ENOMEM; + + client->pid = get_task_pid(task, PIDTYPE_PID); + client->name = name; + + if (!clients->root) + return 0; /* intel_fbdev_init registers a client before sysfs */ + + ret = __client_register_sysfs(client); + if (ret) + goto err_sysfs; + + return 0; + +err_sysfs: + put_pid(client->pid); + kfree(client->name); + + return ret; +} + +static void +__i915_drm_client_unregister(struct i915_drm_client *client) +{ + __client_unregister_sysfs(client); + + put_pid(fetch_and_zero(&client->pid)); + kfree(fetch_and_zero(&client->name)); +} + +struct i915_drm_client * +i915_drm_client_add(struct i915_drm_clients *clients, struct task_struct *task) +{ + struct i915_drm_client *client; + int ret; + + client = kzalloc(sizeof(*client), GFP_KERNEL); + if (!client) + return ERR_PTR(-ENOMEM); + + kref_init(&client->kref); + client->clients = clients; + + ret = xa_alloc_cyclic(&clients->xarray, &client->id, client, + xa_limit_32b, &clients->next_id, GFP_KERNEL); + if (ret < 0) + goto err_id; + + ret = __i915_drm_client_register(client, task); + if (ret) + goto err_register; + + return client; + +err_register: + xa_erase(&clients->xarray, client->id); +err_id: + kfree(client); + + return ERR_PTR(ret); +} + +void __i915_drm_client_free(struct kref *kref) +{ + struct i915_drm_client *client = + container_of(kref, typeof(*client), kref); + + __i915_drm_client_unregister(client); + xa_erase(&client->clients->xarray, client->id); + kfree_rcu(client, rcu); +} + +void i915_drm_client_close(struct i915_drm_client *client) +{ + GEM_BUG_ON(READ_ONCE(client->closed)); + WRITE_ONCE(client->closed, true); + i915_drm_client_put(client); +} diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h new file mode 100644 index 000000000000..af6998c74d4c --- /dev/null +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2020 Intel Corporation + */ + +#ifndef __I915_DRM_CLIENT_H__ +#define __I915_DRM_CLIENT_H__ + +#include +#include +#include +#include +#include +#include +#include + +struct i915_drm_clients { + struct xarray xarray; + u32 next_id; + + struct kobject *root; +}; + +struct i915_drm_client { + struct kref kref; + + struct rcu_head rcu; + + unsigned int id; + struct pid *pid; + char *name; + bool closed; + + struct i915_drm_clients *clients; + + struct kobject *root; + struct { + struct device_attribute pid; + struct device_attribute name; + } attr; +}; + +void i915_drm_clients_init(struct i915_drm_clients *clients); + +static inline struct i915_drm_client * +i915_drm_client_get(struct i915_drm_client *client) +{ + kref_get(&client->kref); + return client; +} + +void __i915_drm_client_free(struct kref *kref); + +static inline void i915_drm_client_put(struct i915_drm_client *client) +{ + kref_put(&client->kref, __i915_drm_client_free); +} + +void i915_drm_client_close(struct i915_drm_client *client); + +struct i915_drm_client *i915_drm_client_add(struct i915_drm_clients *clients, + struct task_struct *task); + +#endif /* !__I915_DRM_CLIENT_H__ */ diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 94e00e450683..4a16edfe5b6b 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -69,6 +69,7 @@ #include "gt/intel_rc6.h" #include "i915_debugfs.h" +#include "i915_drm_client.h" #include "i915_drv.h" #include "i915_ioc32.h" #include "i915_irq.h" @@ -339,6 +340,8 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv) i915_gem_init_early(dev_priv); + i915_drm_clients_init(&dev_priv->clients); + /* This must be called before any calls to HAS_PCH_* */ intel_detect_pch(dev_priv); @@ -1017,6 +1020,7 @@ static void i915_driver_postclose(struct drm_device *dev, struct drm_file *file) struct drm_i915_file_private *file_priv = file->driver_priv; i915_gem_context_close(file); + i915_drm_client_close(file_priv->client); kfree_rcu(file_priv, rcu); diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index ef75acda9bff..e548ff3b85f9 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -91,6 +91,7 @@ #include "intel_wakeref.h" #include "intel_wopcm.h" +#include "i915_drm_client.h" #include "i915_gem.h" #include "i915_gem_gtt.h" #include "i915_gpu_error.h" @@ -223,6 +224,8 @@ struct drm_i915_file_private { /** ban_score: Accumulated score of all ctx bans and fast hangs. */ atomic_t ban_score; unsigned long hang_timestamp; + + struct i915_drm_client *client; }; /* Interface history: @@ -1217,6 +1220,8 @@ struct drm_i915_private { struct i915_pmu pmu; + struct i915_drm_clients clients; + struct i915_hdcp_comp_master *hdcp_master; bool hdcp_comp_added; diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index bb0c12975f38..39182c85c5b7 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1267,6 +1267,8 @@ void i915_gem_cleanup_early(struct drm_i915_private *dev_priv) GEM_BUG_ON(!llist_empty(&dev_priv->mm.free_list)); GEM_BUG_ON(atomic_read(&dev_priv->mm.free_count)); drm_WARN_ON(&dev_priv->drm, dev_priv->mm.shrink_count); + drm_WARN_ON(&dev_priv->drm, !xa_empty(&dev_priv->clients.xarray)); + xa_destroy(&dev_priv->clients.xarray); } int i915_gem_freeze(struct drm_i915_private *dev_priv) @@ -1319,25 +1321,40 @@ int i915_gem_freeze_late(struct drm_i915_private *i915) int i915_gem_open(struct drm_i915_private *i915, struct drm_file *file) { struct drm_i915_file_private *file_priv; - int ret; + struct i915_drm_client *client; + int ret = -ENOMEM; DRM_DEBUG("\n"); file_priv = kzalloc(sizeof(*file_priv), GFP_KERNEL); if (!file_priv) - return -ENOMEM; + goto err_alloc; + + client = i915_drm_client_add(&i915->clients, current); + if (IS_ERR(client)) { + ret = PTR_ERR(client); + goto err_client; + } file->driver_priv = file_priv; file_priv->dev_priv = i915; file_priv->file = file; + file_priv->client = client; file_priv->bsd_engine = -1; file_priv->hang_timestamp = jiffies; ret = i915_gem_context_open(i915, file); if (ret) - kfree(file_priv); + goto err_context; + + return 0; +err_context: + i915_drm_client_close(client); +err_client: + kfree(file_priv); +err_alloc: return ret; } diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c index 45d32ef42787..b7d4a6d2dd5c 100644 --- a/drivers/gpu/drm/i915/i915_sysfs.c +++ b/drivers/gpu/drm/i915/i915_sysfs.c @@ -560,6 +560,11 @@ void i915_setup_sysfs(struct drm_i915_private *dev_priv) struct device *kdev = dev_priv->drm.primary->kdev; int ret; + dev_priv->clients.root = + kobject_create_and_add("clients", &kdev->kobj); + if (!dev_priv->clients.root) + DRM_ERROR("Per-client sysfs setup failed\n"); + #ifdef CONFIG_PM if (HAS_RC6(dev_priv)) { ret = sysfs_merge_group(&kdev->kobj, @@ -627,4 +632,7 @@ void i915_teardown_sysfs(struct drm_i915_private *dev_priv) sysfs_unmerge_group(&kdev->kobj, &rc6_attr_group); sysfs_unmerge_group(&kdev->kobj, &rc6p_attr_group); #endif + + if (dev_priv->clients.root) + kobject_put(dev_priv->clients.root); } From patchwork Mon Sep 14 13:12:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773753 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCC46618 for ; Mon, 14 Sep 2020 13:13:16 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 98199208E4 for ; Mon, 14 Sep 2020 13:13:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98199208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D783F6E499; Mon, 14 Sep 2020 13:13:14 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 75C946E21C for ; Mon, 14 Sep 2020 13:13:08 +0000 (UTC) IronPort-SDR: sHCatt/Fe8htoG1Q9QS1Lk+KQJl45IkN99HqKTj+pa75yU79mizSYY7LHTGosTzMgkGHsZtPk3 MFNEGbHTa13A== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113417" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113417" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:08 -0700 IronPort-SDR: zrIignqasbltN5Op3TmlJVxNQ8uhP0zO5Ru/UqIYgVDDWZAo9STxYnEPmCqpDLhCEjwXFLsALb 4vOJRHcIGI/g== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136322" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:06 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:49 +0100 Message-Id: <20200914131256.89465-3-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/9] drm/i915: Update client name on context create X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Some clients have the DRM fd passed to them over a socket by the X server. Grab the real client and pid when they create their first context and update the exposed data for more useful enumeration. To enable lockless access to client name and pid data from the following patches, we also make these fields rcu protected. In this way asynchronous code paths where both contexts which remain after the client exit, and access to client name and pid as they are getting updated due context creation running in parallel with name/pid queries. v2: * Do not leak the pid reference and borrow context idr_lock. (Chris) v3: * More avoiding leaks. (Chris) v4: * Move update completely to drm client. (Chris) * Do not lose previous client data on failure to re-register and simplify update to only touch what it needs. v5: * Reuse ext_data local. (Chris) Signed-off-by: Tvrtko Ursulin Reviewed-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 + drivers/gpu/drm/i915/i915_drm_client.c | 110 +++++++++++++++++--- drivers/gpu/drm/i915/i915_drm_client.h | 10 +- 3 files changed, 111 insertions(+), 14 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index cf5ecbde9e06..5919cc5f8348 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -74,6 +74,7 @@ #include "gt/intel_engine_user.h" #include "gt/intel_ring.h" +#include "i915_drm_client.h" #include "i915_gem_context.h" #include "i915_globals.h" #include "i915_trace.h" @@ -2388,6 +2389,10 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data, return -EIO; } + ret = i915_drm_client_update(ext_data.fpriv->client, current); + if (ret) + return ret; + ext_data.ctx = i915_gem_create_context(i915, args->flags); if (IS_ERR(ext_data.ctx)) return PTR_ERR(ext_data.ctx); diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index 137a5744f9c6..617509c2e887 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -7,6 +7,9 @@ #include #include +#include + +#include "i915_drv.h" #include "i915_drm_client.h" #include "i915_gem.h" #include "i915_utils.h" @@ -22,10 +25,15 @@ show_client_name(struct device *kdev, struct device_attribute *attr, char *buf) { struct i915_drm_client *client = container_of(attr, typeof(*client), attr.name); + int ret; + + rcu_read_lock(); + ret = snprintf(buf, PAGE_SIZE, + READ_ONCE(client->closed) ? "<%s>" : "%s", + rcu_dereference(client->name)); + rcu_read_unlock(); - return snprintf(buf, PAGE_SIZE, - READ_ONCE(client->closed) ? "<%s>" : "%s", - client->name); + return ret; } static ssize_t @@ -33,10 +41,15 @@ show_client_pid(struct device *kdev, struct device_attribute *attr, char *buf) { struct i915_drm_client *client = container_of(attr, typeof(*client), attr.pid); + int ret; - return snprintf(buf, PAGE_SIZE, - READ_ONCE(client->closed) ? "<%u>" : "%u", - pid_nr(client->pid)); + rcu_read_lock(); + ret = snprintf(buf, PAGE_SIZE, + READ_ONCE(client->closed) ? "<%u>" : "%u", + pid_nr(rcu_dereference(client->pid))); + rcu_read_unlock(); + + return ret; } static int @@ -101,8 +114,8 @@ __i915_drm_client_register(struct i915_drm_client *client, if (!name) return -ENOMEM; - client->pid = get_task_pid(task, PIDTYPE_PID); - client->name = name; + rcu_assign_pointer(client->pid, get_task_pid(task, PIDTYPE_PID)); + rcu_assign_pointer(client->name, name); if (!clients->root) return 0; /* intel_fbdev_init registers a client before sysfs */ @@ -114,8 +127,8 @@ __i915_drm_client_register(struct i915_drm_client *client, return 0; err_sysfs: - put_pid(client->pid); - kfree(client->name); + put_pid(rcu_replace_pointer(client->pid, NULL, true)); + kfree(rcu_replace_pointer(client->name, NULL, true)); return ret; } @@ -125,8 +138,8 @@ __i915_drm_client_unregister(struct i915_drm_client *client) { __client_unregister_sysfs(client); - put_pid(fetch_and_zero(&client->pid)); - kfree(fetch_and_zero(&client->name)); + put_pid(rcu_replace_pointer(client->pid, NULL, true)); + kfree(rcu_replace_pointer(client->name, NULL, true)); } struct i915_drm_client * @@ -140,6 +153,7 @@ i915_drm_client_add(struct i915_drm_clients *clients, struct task_struct *task) return ERR_PTR(-ENOMEM); kref_init(&client->kref); + mutex_init(&client->update_lock); client->clients = clients; ret = xa_alloc_cyclic(&clients->xarray, &client->id, client, @@ -177,3 +191,75 @@ void i915_drm_client_close(struct i915_drm_client *client) WRITE_ONCE(client->closed, true); i915_drm_client_put(client); } + +struct client_update_free { + struct rcu_head rcu; + struct pid *pid; + char *name; +}; + +static void __client_update_free(struct rcu_head *rcu) +{ + struct client_update_free *old = container_of(rcu, typeof(*old), rcu); + + put_pid(old->pid); + kfree(old->name); + kfree(old); +} + +int +i915_drm_client_update(struct i915_drm_client *client, + struct task_struct *task) +{ + struct drm_i915_private *i915 = + container_of(client->clients, typeof(*i915), clients); + struct client_update_free *old; + struct pid *pid; + char *name; + int ret; + + old = kmalloc(sizeof(*old), GFP_KERNEL); + if (!old) + return -ENOMEM; + + ret = mutex_lock_interruptible(&client->update_lock); + if (ret) + goto out_free; + + pid = get_task_pid(task, PIDTYPE_PID); + if (!pid) + goto out_pid; + if (pid == rcu_access_pointer(client->pid)) + goto out_name; + + name = kstrdup(task->comm, GFP_KERNEL); + if (!name) { + drm_notice(&i915->drm, + "Failed to update client id=%u,name=%s,pid=%u! (%d)\n", + client->id, + client->name, + pid_nr(rcu_access_pointer(client->pid)), + ret); + goto out_name; + } + + init_rcu_head(&old->rcu); + + old->pid = rcu_replace_pointer(client->pid, pid, true); + old->name = rcu_replace_pointer(client->name, name, true); + + mutex_unlock(&client->update_lock); + + call_rcu(&old->rcu, __client_update_free); + + return 0; + +out_name: + put_pid(pid); +out_pid: + mutex_unlock(&client->update_lock); +out_free: + kfree(old); + + return ret; +} diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index af6998c74d4c..11b48383881d 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -26,9 +27,11 @@ struct i915_drm_client { struct rcu_head rcu; + struct mutex update_lock; /* Serializes name and pid updates. */ + unsigned int id; - struct pid *pid; - char *name; + struct pid __rcu *pid; + char __rcu *name; bool closed; struct i915_drm_clients *clients; @@ -61,4 +64,7 @@ void i915_drm_client_close(struct i915_drm_client *client); struct i915_drm_client *i915_drm_client_add(struct i915_drm_clients *clients, struct task_struct *task); +int i915_drm_client_update(struct i915_drm_client *client, + struct task_struct *task); + #endif /* !__I915_DRM_CLIENT_H__ */ From patchwork Mon Sep 14 13:12:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0ACE16CA for ; Mon, 14 Sep 2020 13:13:13 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D90F321D1B for ; Mon, 14 Sep 2020 13:13:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D90F321D1B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5B1086E497; Mon, 14 Sep 2020 13:13:12 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id D791D6E497 for ; Mon, 14 Sep 2020 13:13:09 +0000 (UTC) IronPort-SDR: ZDBY6WQg3H632tCzNdGNRRTUzDrQPRq2RLoJlwOWXaytw5db0P2881SoPls9Gwfu6rbzy7CkMv I3PR9uqGcRPw== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113420" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113420" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:09 -0700 IronPort-SDR: d6NBe3hsnsbtV++CRQmE60EI3PhsPZBd51huj0pOdSbkoAwB2NWzqlLQX/pYuukkkUb10L2UU/ qM2hHOnx50Fw== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136340" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:08 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:50 +0100 Message-Id: <20200914131256.89465-4-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 3/9] drm/i915: Make GEM contexts track DRM clients X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin If we make GEM contexts keep a reference to i915_drm_client for the whole of their lifetime, we can consolidate the current task pid and name usage by getting it from the client. v2: * Don't bother supporting selftests contexts from debugfs. (Chris) v3: * Trivial rebase. v4: * Fully initialize context before adding to the list. (Lucas) Signed-off-by: Tvrtko Ursulin Reviewed-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 19 +++++++++--- .../gpu/drm/i915/gem/i915_gem_context_types.h | 13 ++------- drivers/gpu/drm/i915/i915_debugfs.c | 29 +++++++------------ drivers/gpu/drm/i915/i915_gpu_error.c | 22 ++++++++------ 4 files changed, 41 insertions(+), 42 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 5919cc5f8348..df5488998d53 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -336,8 +336,13 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx) static void i915_gem_context_free(struct i915_gem_context *ctx) { + struct i915_drm_client *client = ctx->client; + GEM_BUG_ON(!i915_gem_context_is_closed(ctx)); + if (client) + i915_drm_client_put(client); + spin_lock(&ctx->i915->gem.contexts.lock); list_del(&ctx->link); spin_unlock(&ctx->i915->gem.contexts.lock); @@ -348,7 +353,6 @@ static void i915_gem_context_free(struct i915_gem_context *ctx) if (ctx->timeline) intel_timeline_put(ctx->timeline); - put_pid(ctx->pid); mutex_destroy(&ctx->mutex); kfree_rcu(ctx, rcu); @@ -936,6 +940,7 @@ static int gem_context_register(struct i915_gem_context *ctx, u32 *id) { struct drm_i915_private *i915 = ctx->i915; + struct i915_drm_client *client; struct i915_address_space *vm; int ret; @@ -947,15 +952,21 @@ static int gem_context_register(struct i915_gem_context *ctx, WRITE_ONCE(vm->file, fpriv); /* XXX */ mutex_unlock(&ctx->mutex); - ctx->pid = get_task_pid(current, PIDTYPE_PID); + client = i915_drm_client_get(fpriv->client); + + rcu_read_lock(); snprintf(ctx->name, sizeof(ctx->name), "%s[%d]", - current->comm, pid_nr(ctx->pid)); + rcu_dereference(client->name), + pid_nr(rcu_dereference(client->pid))); + rcu_read_unlock(); /* And finally expose ourselves to userspace via the idr */ ret = xa_alloc(&fpriv->context_xa, id, ctx, xa_limit_32b, GFP_KERNEL); if (ret) goto err_pid; + ctx->client = client; + spin_lock(&i915->gem.contexts.lock); list_add_tail(&ctx->link, &i915->gem.contexts.list); spin_unlock(&i915->gem.contexts.lock); @@ -963,7 +974,7 @@ static int gem_context_register(struct i915_gem_context *ctx, return 0; err_pid: - put_pid(fetch_and_zero(&ctx->pid)); + i915_drm_client_put(client); return ret; } diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h index ae14ca24a11f..31a6a30f7ea8 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h @@ -96,20 +96,13 @@ struct i915_gem_context { */ struct i915_address_space __rcu *vm; - /** - * @pid: process id of creator - * - * Note that who created the context may not be the principle user, - * as the context may be shared across a local socket. However, - * that should only affect the default context, all contexts created - * explicitly by the client are expected to be isolated. - */ - struct pid *pid; - /** link: place with &drm_i915_private.context_list */ struct list_head link; struct llist_node free_link; + /** client: struct i915_drm_client */ + struct i915_drm_client *client; + /** * @ref: reference count * diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 784219962193..1df87d9735fc 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -331,17 +331,15 @@ static void print_context_stats(struct seq_file *m, .vm = rcu_access_pointer(ctx->vm), }; struct drm_file *file = ctx->file_priv->file; - struct task_struct *task; char name[80]; rcu_read_lock(); + idr_for_each(&file->object_idr, per_file_stats, &stats); - rcu_read_unlock(); - rcu_read_lock(); - task = pid_task(ctx->pid ?: file->pid, PIDTYPE_PID); snprintf(name, sizeof(name), "%s", - task ? task->comm : ""); + rcu_dereference(ctx->client->name)); + rcu_read_unlock(); print_file_stats(m, name, stats); @@ -1073,20 +1071,13 @@ static int i915_context_status(struct seq_file *m, void *unused) spin_unlock(&i915->gem.contexts.lock); seq_puts(m, "HW context "); - if (ctx->pid) { - struct task_struct *task; - - task = get_pid_task(ctx->pid, PIDTYPE_PID); - if (task) { - seq_printf(m, "(%s [%d]) ", - task->comm, task->pid); - put_task_struct(task); - } - } else if (IS_ERR(ctx->file_priv)) { - seq_puts(m, "(deleted) "); - } else { - seq_puts(m, "(kernel) "); - } + + rcu_read_lock(); + seq_printf(m, "(%s [%d]) %s", + rcu_dereference(ctx->client->name), + pid_nr(rcu_dereference(ctx->client->pid)), + ctx->client->closed ? "(closed) " : ""); + rcu_read_unlock(); seq_putc(m, ctx->remap_slice ? 'R' : 'r'); seq_putc(m, '\n'); diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c index 3e6cbb0d1150..8e468942f943 100644 --- a/drivers/gpu/drm/i915/i915_gpu_error.c +++ b/drivers/gpu/drm/i915/i915_gpu_error.c @@ -1229,7 +1229,9 @@ static void record_request(const struct i915_request *request, ctx = rcu_dereference(request->context->gem_context); if (ctx) - erq->pid = pid_nr(ctx->pid); + erq->pid = I915_SELFTEST_ONLY(!ctx->client) ? + 0 : + pid_nr(rcu_dereference(ctx->client->pid)); } rcu_read_unlock(); } @@ -1250,23 +1252,25 @@ static bool record_context(struct i915_gem_context_coredump *e, const struct i915_request *rq) { struct i915_gem_context *ctx; - struct task_struct *task; bool simulated; rcu_read_lock(); + ctx = rcu_dereference(rq->context->gem_context); if (ctx && !kref_get_unless_zero(&ctx->ref)) ctx = NULL; - rcu_read_unlock(); - if (!ctx) + if (!ctx) { + rcu_read_unlock(); return true; + } - rcu_read_lock(); - task = pid_task(ctx->pid, PIDTYPE_PID); - if (task) { - strcpy(e->comm, task->comm); - e->pid = task->pid; + if (I915_SELFTEST_ONLY(!ctx->client)) { + strcpy(e->comm, "[kernel]"); + } else { + strcpy(e->comm, rcu_dereference(ctx->client->name)); + e->pid = pid_nr(rcu_dereference(ctx->client->pid)); } + rcu_read_unlock(); e->sched_attr = ctx->sched; From patchwork Mon Sep 14 13:12:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773751 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C3726CA for ; Mon, 14 Sep 2020 13:13:15 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7CEE1214F1 for ; Mon, 14 Sep 2020 13:13:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CEE1214F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D7AD36E49A; Mon, 14 Sep 2020 13:13:14 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8CFB16E497 for ; Mon, 14 Sep 2020 13:13:11 +0000 (UTC) IronPort-SDR: ruE9gNrovtrOCpzl890UyE9M4ZIT9/vdWvHnOu1/goxRNe+WnArlHIqB/V4q6IN+FmMp9xwyb8 nW2ACFsrswBQ== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113424" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113424" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:11 -0700 IronPort-SDR: 5msE/fxyWGnUje2RjHjiA/GRSbt8EqFs5ejlsr9HKEE7Z+En+nD4NtyuGDKoMtMFwRHkSH7a6s DnZJeXiqu9YQ== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136361" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:09 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:51 +0100 Message-Id: <20200914131256.89465-5-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 4/9] drm/i915: Track runtime spent in unreachable intel_contexts X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin As contexts are abandoned we want to remember how much GPU time they used (per class) so later we can used it for smarter purposes. Signed-off-by: Tvrtko Ursulin Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 13 ++++++++++++- drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 5 +++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index df5488998d53..a12e926444e1 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -253,7 +253,19 @@ static void free_engines_rcu(struct rcu_head *rcu) { struct i915_gem_engines *engines = container_of(rcu, struct i915_gem_engines, rcu); + struct i915_gem_context *ctx = engines->ctx; + struct i915_gem_engines_iter it; + struct intel_context *ce; + + /* Transfer accumulated runtime to the parent GEM context. */ + for_each_gem_engine(ce, engines, it) { + unsigned int class = ce->engine->uabi_class; + GEM_BUG_ON(class >= ARRAY_SIZE(ctx->past_runtime)); + atomic64_add(ce->runtime.total, &ctx->past_runtime[class]); + } + + i915_gem_context_put(ctx); i915_sw_fence_fini(&engines->fence); free_engines(engines); } @@ -274,7 +286,6 @@ engines_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state) list_del(&engines->link); spin_unlock_irqrestore(&ctx->stale.lock, flags); } - i915_gem_context_put(engines->ctx); break; case FENCE_FREE: diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h index 31a6a30f7ea8..e473984b52c8 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h @@ -178,6 +178,11 @@ struct i915_gem_context { spinlock_t lock; struct list_head engines; } stale; + + /** + * @past_runtime: Accumulation of freed intel_context pphwsp runtimes. + */ + atomic64_t past_runtime[MAX_ENGINE_CLASS + 1]; }; #endif /* __I915_GEM_CONTEXT_TYPES_H__ */ From patchwork Mon Sep 14 13:12:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773759 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EC406CA for ; Mon, 14 Sep 2020 13:13:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2938A208E4 for ; Mon, 14 Sep 2020 13:13:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2938A208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DAE656E4A1; Mon, 14 Sep 2020 13:13:18 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3BA0F6E499 for ; Mon, 14 Sep 2020 13:13:14 +0000 (UTC) IronPort-SDR: M9ETJTFJSw8G11bYOZ5YKkZJ0YEmQOIYgcXPnBhApPDMAzVrH1MzZK+XkY3Ukm/sOVFtZ0PpZv rFT4+GhvAT6Q== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113434" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113434" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:13 -0700 IronPort-SDR: w38uEcCQPTrbDWsbdhtxT5ACW8JlvhPnwVQPS3VyZT4M99WGedkO7qiTVB904QByPSocfAaYj7 xQMJTPz2Z7rQ== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136374" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:11 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:52 +0100 Message-Id: <20200914131256.89465-6-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 5/9] drm/i915: Track runtime spent in closed GEM contexts X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin As GEM contexts are closed we want to have the DRM client remember how much GPU time they used (per class) so later we can used it for smarter purposes. Signed-off-by: Tvrtko Ursulin Reviewed-by: Mika Kuoppala --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 12 +++++++++++- drivers/gpu/drm/i915/i915_drm_client.h | 7 +++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index a12e926444e1..a50b83659399 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -351,8 +351,18 @@ static void i915_gem_context_free(struct i915_gem_context *ctx) GEM_BUG_ON(!i915_gem_context_is_closed(ctx)); - if (client) + if (client) { + unsigned int i; + + /* Transfer accumulated runtime to the parent drm client. */ + BUILD_BUG_ON(ARRAY_SIZE(client->past_runtime) != + ARRAY_SIZE(ctx->past_runtime)); + for (i = 0; i < ARRAY_SIZE(client->past_runtime); i++) + atomic64_add(atomic64_read(&ctx->past_runtime[i]), + &client->past_runtime[i]); + i915_drm_client_put(client); + } spin_lock(&ctx->i915->gem.contexts.lock); list_del(&ctx->link); diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 11b48383881d..29b116606596 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -15,6 +15,8 @@ #include #include +#include "gt/intel_engine_types.h" + struct i915_drm_clients { struct xarray xarray; u32 next_id; @@ -41,6 +43,11 @@ struct i915_drm_client { struct device_attribute pid; struct device_attribute name; } attr; + + /** + * @past_runtime: Accumulation of pphwsp runtimes from closed contexts. + */ + atomic64_t past_runtime[MAX_ENGINE_CLASS + 1]; }; void i915_drm_clients_init(struct i915_drm_clients *clients); From patchwork Mon Sep 14 13:12:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773755 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A4A5618 for ; Mon, 14 Sep 2020 13:13:19 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EB7BB208E4 for ; Mon, 14 Sep 2020 13:13:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EB7BB208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 706966E49C; Mon, 14 Sep 2020 13:13:18 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id E53A06E49B for ; Mon, 14 Sep 2020 13:13:15 +0000 (UTC) IronPort-SDR: 0wIHBMCG3E3S7JXH6YO1TRL4UC6jR2vBU+Q+3UK2F/otahQN7xQNhvFDJKYmU4BjTxvKoZWntE 9KNIYsi0/6UQ== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113437" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113437" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:15 -0700 IronPort-SDR: zRr2eeMpF6HM7ql8feNtMYDHV8FHHQM4NBRBc/xBH8C9W8a+z6kyCWmTUb6NkmHzseXpBL42qu p4+yxVL2OQuQ== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136390" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:14 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:53 +0100 Message-Id: <20200914131256.89465-7-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 6/9] drm/i915: Track all user contexts per client X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin We soon want to start answering questions like how much GPU time is the context belonging to a client which exited still using. To enable this we start tracking all context belonging to a client on a separate list. v2: * Keep adding to gem.contexts.list as last. (Lucas) Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 8 ++++++++ drivers/gpu/drm/i915/gem/i915_gem_context_types.h | 3 +++ drivers/gpu/drm/i915/i915_drm_client.c | 3 +++ drivers/gpu/drm/i915/i915_drm_client.h | 5 +++++ 4 files changed, 19 insertions(+) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index a50b83659399..8a5b36418860 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -354,6 +354,10 @@ static void i915_gem_context_free(struct i915_gem_context *ctx) if (client) { unsigned int i; + spin_lock(&client->ctx_lock); + list_del_rcu(&ctx->client_link); + spin_unlock(&client->ctx_lock); + /* Transfer accumulated runtime to the parent drm client. */ BUILD_BUG_ON(ARRAY_SIZE(client->past_runtime) != ARRAY_SIZE(ctx->past_runtime)); @@ -988,6 +992,10 @@ static int gem_context_register(struct i915_gem_context *ctx, ctx->client = client; + spin_lock(&client->ctx_lock); + list_add_tail_rcu(&ctx->client_link, &client->ctx_list); + spin_unlock(&client->ctx_lock); + spin_lock(&i915->gem.contexts.lock); list_add_tail(&ctx->link, &i915->gem.contexts.list); spin_unlock(&i915->gem.contexts.lock); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h index e473984b52c8..d65e3bd3c8bb 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h @@ -103,6 +103,9 @@ struct i915_gem_context { /** client: struct i915_drm_client */ struct i915_drm_client *client; + /** link: &drm_client.context_list */ + struct list_head client_link; + /** * @ref: reference count * diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index 617509c2e887..d99673508f8c 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -154,6 +154,9 @@ i915_drm_client_add(struct i915_drm_clients *clients, struct task_struct *task) kref_init(&client->kref); mutex_init(&client->update_lock); + spin_lock_init(&client->ctx_lock); + INIT_LIST_HEAD(&client->ctx_list); + client->clients = clients; ret = xa_alloc_cyclic(&clients->xarray, &client->id, client, diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 29b116606596..0be27aa9bbda 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -9,10 +9,12 @@ #include #include #include +#include #include #include #include #include +#include #include #include "gt/intel_engine_types.h" @@ -36,6 +38,9 @@ struct i915_drm_client { char __rcu *name; bool closed; + spinlock_t ctx_lock; /* For add/remove from ctx_list. */ + struct list_head ctx_list; /* List of contexts belonging to client. */ + struct i915_drm_clients *clients; struct kobject *root; From patchwork Mon Sep 14 13:12:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4985814B7 for ; Mon, 14 Sep 2020 13:13:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B17E208E4 for ; Mon, 14 Sep 2020 13:13:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B17E208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BAAF66E49B; Mon, 14 Sep 2020 13:13:18 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 60C2B6E491 for ; Mon, 14 Sep 2020 13:13:17 +0000 (UTC) IronPort-SDR: dOJUuGIxoF4cXKp9W4s5XQ78Pb+Q1oGzprFbG2ZOHGPVtstH2vhhIzgdzH1VE0O45rtteHnNJp xrEz0PI242GA== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113439" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113439" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:17 -0700 IronPort-SDR: zoDORZZM4O6uImfJyapvdfcCFmnOCqNfTlg5BIYaWpPz1hW3fss3MQUgKCsDpRhHrBvaiMhTsC C9oD8x4IdFpw== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136402" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:15 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:54 +0100 Message-Id: <20200914131256.89465-8-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 7/9] drm/i915: Expose per-engine client busyness X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Expose per-client and per-engine busyness under the previously added sysfs client root. The new files are one per-engine instance and located under the 'busy' directory. Each contains a monotonically increasing nano-second resolution times each client's jobs were executing on the GPU. This enables userspace to create a top-like tool for GPU utilization: ========================================================================== intel-gpu-top - 935/ 935 MHz; 0% RC6; 14.73 Watts; 1097 irqs/s IMC reads: 1401 MiB/s IMC writes: 4 MiB/s ENGINE BUSY MI_SEMA MI_WAIT Render/3D/0 63.73% |███████████████████ | 3% 0% Blitter/0 9.53% |██▊ | 6% 0% Video/0 39.32% |███████████▊ | 16% 0% Video/1 15.62% |████▋ | 0% 0% VideoEnhance/0 0.00% | | 0% 0% PID NAME RCS BCS VCS VECS 4084 gem_wsim |█████▌ ||█ || || | 4086 gem_wsim |█▌ || ||███ || | ========================================================================== v2: Use intel_context_engine_get_busy_time. v3: New directory structure. v4: Rebase. v5: sysfs_attr_init. v6: Small tidy in i915_gem_add_client. v7: Rebase to be engine class based. v8: * Always enable stats. * Walk all client contexts. v9: * Skip unsupported engine classes. (Chris) * Use scheduler caps. (Chris) v10: * Use pphwsp runtime only. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_drm_client.c | 110 ++++++++++++++++++++++++- drivers/gpu/drm/i915/i915_drm_client.h | 11 +++ 2 files changed, 120 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index d99673508f8c..e21d97d810a5 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -9,8 +9,13 @@ #include +#include + #include "i915_drv.h" #include "i915_drm_client.h" +#include "gem/i915_gem_context.h" +#include "gt/intel_engine_user.h" +#include "i915_drv.h" #include "i915_gem.h" #include "i915_utils.h" @@ -52,6 +57,104 @@ show_client_pid(struct device *kdev, struct device_attribute *attr, char *buf) return ret; } +static u64 +pphwsp_busy_add(struct i915_gem_context *ctx, unsigned int class) +{ + struct i915_gem_engines *engines = rcu_dereference(ctx->engines); + struct i915_gem_engines_iter it; + struct intel_context *ce; + u64 total = 0; + + for_each_gem_engine(ce, engines, it) { + if (ce->engine->uabi_class == class) + total += ce->runtime.total; + } + + return total; +} + +static ssize_t +show_client_busy(struct device *kdev, struct device_attribute *attr, char *buf) +{ + struct i915_engine_busy_attribute *i915_attr = + container_of(attr, typeof(*i915_attr), attr); + unsigned int class = i915_attr->engine_class; + struct i915_drm_client *client = i915_attr->client; + u64 total = atomic64_read(&client->past_runtime[class]); + struct list_head *list = &client->ctx_list; + struct i915_gem_context *ctx; + + rcu_read_lock(); + list_for_each_entry_rcu(ctx, list, client_link) { + total += atomic64_read(&ctx->past_runtime[class]); + total += pphwsp_busy_add(ctx, class); + } + rcu_read_unlock(); + + total *= RUNTIME_INFO(i915_attr->i915)->cs_timestamp_period_ns; + + return snprintf(buf, PAGE_SIZE, "%llu\n", total); +} + +static const char * const uabi_class_names[] = { + [I915_ENGINE_CLASS_RENDER] = "0", + [I915_ENGINE_CLASS_COPY] = "1", + [I915_ENGINE_CLASS_VIDEO] = "2", + [I915_ENGINE_CLASS_VIDEO_ENHANCE] = "3", +}; + +static int +__client_register_sysfs_busy(struct i915_drm_client *client) +{ + struct i915_drm_clients *clients = client->clients; + struct drm_i915_private *i915 = + container_of(clients, typeof(*i915), clients); + unsigned int i; + int ret = 0; + + if (!HAS_LOGICAL_RING_CONTEXTS(i915)) + return 0; + + client->busy_root = kobject_create_and_add("busy", client->root); + if (!client->busy_root) + return -ENOMEM; + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) { + struct i915_engine_busy_attribute *i915_attr = + &client->attr.busy[i]; + struct device_attribute *attr = &i915_attr->attr; + + if (!intel_engine_lookup_user(i915, i, 0)) + continue; + + i915_attr->client = client; + i915_attr->i915 = i915; + i915_attr->engine_class = i; + + sysfs_attr_init(&attr->attr); + + attr->attr.name = uabi_class_names[i]; + attr->attr.mode = 0444; + attr->show = show_client_busy; + + ret = sysfs_create_file(client->busy_root, + (struct attribute *)attr); + if (ret) + goto err; + } + + return 0; + +err: + kobject_put(client->busy_root); + return ret; +} + +static void __client_unregister_sysfs_busy(struct i915_drm_client *client) +{ + kobject_put(fetch_and_zero(&client->busy_root)); +} + static int __client_register_sysfs(struct i915_drm_client *client) { @@ -88,9 +191,12 @@ __client_register_sysfs(struct i915_drm_client *client) ret = sysfs_create_file(client->root, (struct attribute *)attr); if (ret) - break; + goto out; } + ret = __client_register_sysfs_busy(client); + +out: if (ret) kobject_put(client->root); @@ -99,6 +205,8 @@ __client_register_sysfs(struct i915_drm_client *client) static void __client_unregister_sysfs(struct i915_drm_client *client) { + __client_unregister_sysfs_busy(client); + kobject_put(fetch_and_zero(&client->root)); } diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 0be27aa9bbda..98c5fc24fcb4 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -26,6 +26,15 @@ struct i915_drm_clients { struct kobject *root; }; +struct i915_drm_client; + +struct i915_engine_busy_attribute { + struct device_attribute attr; + struct drm_i915_private *i915; + struct i915_drm_client *client; + unsigned int engine_class; +}; + struct i915_drm_client { struct kref kref; @@ -44,9 +53,11 @@ struct i915_drm_client { struct i915_drm_clients *clients; struct kobject *root; + struct kobject *busy_root; struct { struct device_attribute pid; struct device_attribute name; + struct i915_engine_busy_attribute busy[MAX_ENGINE_CLASS + 1]; } attr; /** From patchwork Mon Sep 14 13:12:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773763 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C4CB6CA for ; Mon, 14 Sep 2020 13:13:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF128214F1 for ; Mon, 14 Sep 2020 13:13:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF128214F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 493256E4A7; Mon, 14 Sep 2020 13:13:24 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id D32286E49F for ; Mon, 14 Sep 2020 13:13:18 +0000 (UTC) IronPort-SDR: cX/0Vq41JD6m9ysjKV02AotKk97l0uM+til0eMQ0YKIVVOTha7ZFAhIH1qdBWAVhKfa5rlC3BI vlY64ptcgd7Q== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113442" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113442" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:18 -0700 IronPort-SDR: UfbCTNqYjjPj7YSUBfUHkyMtiGh5EabLbOsjrl5gHcxTtR/nmktfuveQ//N1WW5/yj1lB7Me46 WvMNUgKAVUSg== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136410" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:17 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:55 +0100 Message-Id: <20200914131256.89465-9-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 8/9] drm/i915: Track context current active time X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin Track context active (on hardware) status together with the start timestamp. This will be used to provide better granularity of context runtime reporting in conjunction with already tracked pphwsp accumulated runtime. The latter is only updated on context save so does not give us visibility to any currently executing work. As part of the patch the existing runtime tracking data is moved under the new ce->stats member and updated under the seqlock. This provides the ability to atomically read out accumulated plus active runtime. v2: * Rename and make __intel_context_get_active_time unlocked. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 +- drivers/gpu/drm/i915/gt/intel_context.c | 11 +++- drivers/gpu/drm/i915/gt/intel_context.h | 6 +- drivers/gpu/drm/i915/gt/intel_context_types.h | 24 +++++--- drivers/gpu/drm/i915/gt/intel_lrc.c | 57 ++++++++++++++----- drivers/gpu/drm/i915/gt/selftest_lrc.c | 10 ++-- drivers/gpu/drm/i915/i915_drm_client.c | 2 +- drivers/gpu/drm/i915/i915_gpu_error.c | 4 +- 8 files changed, 84 insertions(+), 33 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 8a5b36418860..1f3c33f862fb 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -262,7 +262,8 @@ static void free_engines_rcu(struct rcu_head *rcu) unsigned int class = ce->engine->uabi_class; GEM_BUG_ON(class >= ARRAY_SIZE(ctx->past_runtime)); - atomic64_add(ce->runtime.total, &ctx->past_runtime[class]); + atomic64_add(ce->stats.runtime.total, + &ctx->past_runtime[class]); } i915_gem_context_put(ctx); diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c index d301dda1b261..471550e60a35 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.c +++ b/drivers/gpu/drm/i915/gt/intel_context.c @@ -369,7 +369,7 @@ intel_context_init(struct intel_context *ce, ce->sseu = engine->sseu; ce->ring = __intel_context_ring_size(SZ_4K); - ewma_runtime_init(&ce->runtime.avg); + ewma_runtime_init(&ce->stats.runtime.avg); ce->vm = i915_vm_get(engine->gt->vm); @@ -377,6 +377,7 @@ intel_context_init(struct intel_context *ce, INIT_LIST_HEAD(&ce->signals); mutex_init(&ce->pin_mutex); + seqlock_init(&ce->stats.lock); i915_active_init(&ce->active, __intel_context_active, __intel_context_retire); @@ -493,6 +494,14 @@ struct i915_request *intel_context_create_request(struct intel_context *ce) return rq; } +ktime_t __intel_context_get_active_time(struct intel_context *ce) +{ + struct intel_context_stats *stats = &ce->stats; + + return stats->active ? ktime_sub(ktime_get(), stats->start) : 0; +} + + #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) #include "selftest_context.c" #endif diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h index fda2eba81e22..1e76291485ac 100644 --- a/drivers/gpu/drm/i915/gt/intel_context.h +++ b/drivers/gpu/drm/i915/gt/intel_context.h @@ -251,7 +251,7 @@ static inline u64 intel_context_get_total_runtime_ns(struct intel_context *ce) const u32 period = RUNTIME_INFO(ce->engine->i915)->cs_timestamp_period_ns; - return READ_ONCE(ce->runtime.total) * period; + return READ_ONCE(ce->stats.runtime.total) * period; } static inline u64 intel_context_get_avg_runtime_ns(struct intel_context *ce) @@ -259,7 +259,9 @@ static inline u64 intel_context_get_avg_runtime_ns(struct intel_context *ce) const u32 period = RUNTIME_INFO(ce->engine->i915)->cs_timestamp_period_ns; - return mul_u32_u32(ewma_runtime_read(&ce->runtime.avg), period); + return mul_u32_u32(ewma_runtime_read(&ce->stats.runtime.avg), period); } +ktime_t __intel_context_get_active_time(struct intel_context *ce); + #endif /* __INTEL_CONTEXT_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h index 552cb57a2e8c..bdeb65c0f1a9 100644 --- a/drivers/gpu/drm/i915/gt/intel_context_types.h +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h @@ -12,6 +12,7 @@ #include #include #include +#include #include "i915_active_types.h" #include "i915_utils.h" @@ -81,14 +82,21 @@ struct intel_context { } lrc; u32 tag; /* cookie passed to HW to track this context on submission */ - /* Time on GPU as tracked by the hw. */ - struct { - struct ewma_runtime avg; - u64 total; - u32 last; - I915_SELFTEST_DECLARE(u32 num_underflow); - I915_SELFTEST_DECLARE(u32 max_underflow); - } runtime; + /** stats: Context GPU engine busyness tracking. */ + struct intel_context_stats { + seqlock_t lock; + bool active; + ktime_t start; + + /* Time on GPU as tracked by the hw. */ + struct { + struct ewma_runtime avg; + u64 total; + u32 last; + I915_SELFTEST_DECLARE(u32 num_underflow); + I915_SELFTEST_DECLARE(u32 max_underflow); + } runtime; + } stats; unsigned int active_count; /* protected by timeline->mutex */ diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 0412a44f25f2..9ce123450f1e 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1262,7 +1262,7 @@ static void restore_default_state(struct intel_context *ce, regs = memset(ce->lrc_reg_state, 0, engine->context_size - PAGE_SIZE); execlists_init_reg_state(regs, ce, engine, ce->ring, true); - ce->runtime.last = intel_context_get_runtime(ce); + ce->stats.runtime.last = intel_context_get_runtime(ce); } static void reset_active(struct i915_request *rq, @@ -1304,35 +1304,61 @@ static void reset_active(struct i915_request *rq, ce->lrc.desc |= CTX_DESC_FORCE_RESTORE; } -static void st_update_runtime_underflow(struct intel_context *ce, s32 dt) +static void +st_update_runtime_underflow(struct intel_context_stats *stats, s32 dt) { #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) - ce->runtime.num_underflow += dt < 0; - ce->runtime.max_underflow = max_t(u32, ce->runtime.max_underflow, -dt); + stats->runtime.num_underflow += dt < 0; + stats->runtime.max_underflow = + max_t(u32, stats->runtime.max_underflow, -dt); #endif } static void intel_context_update_runtime(struct intel_context *ce) { + struct intel_context_stats *stats = &ce->stats; u32 old; s32 dt; if (intel_context_is_barrier(ce)) return; - old = ce->runtime.last; - ce->runtime.last = intel_context_get_runtime(ce); - dt = ce->runtime.last - old; + old = stats->runtime.last; + stats->runtime.last = intel_context_get_runtime(ce); + dt = stats->runtime.last - old; if (unlikely(dt <= 0)) { CE_TRACE(ce, "runtime underflow: last=%u, new=%u, delta=%d\n", - old, ce->runtime.last, dt); - st_update_runtime_underflow(ce, dt); + old, stats->runtime.last, dt); + st_update_runtime_underflow(stats, dt); return; } - ewma_runtime_add(&ce->runtime.avg, dt); - ce->runtime.total += dt; + ewma_runtime_add(&stats->runtime.avg, dt); + stats->runtime.total += dt; +} + +static void intel_context_stats_start(struct intel_context *ce) +{ + struct intel_context_stats *stats = &ce->stats; + unsigned long flags; + + write_seqlock_irqsave(&stats->lock, flags); + stats->start = ktime_get(); + stats->active = true; + write_sequnlock_irqrestore(&stats->lock, flags); +} + +static void intel_context_stats_stop(struct intel_context *ce) +{ + struct intel_context_stats *stats = &ce->stats; + unsigned long flags; + + write_seqlock_irqsave(&stats->lock, flags); + stats->active = false; + stats->start = 0; + intel_context_update_runtime(ce); + write_sequnlock_irqrestore(&stats->lock, flags); } static inline struct intel_engine_cs * @@ -1410,7 +1436,7 @@ __execlists_schedule_out(struct i915_request *rq, struct intel_engine_cs * const engine, unsigned int ccid) { - struct intel_context * const ce = rq->context; + struct intel_context *ce = rq->context; /* * NB process_csb() is not under the engine->active.lock and hence @@ -1436,6 +1462,7 @@ __execlists_schedule_out(struct i915_request *rq, intel_context_update_runtime(ce); intel_engine_context_out(engine); + intel_context_stats_stop(ce); execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_OUT); if (engine->fw_domain && !atomic_dec_return(&engine->fw_active)) intel_uncore_forcewake_put(engine->uncore, engine->fw_domain); @@ -1995,15 +2022,19 @@ static unsigned long active_timeslice(const struct intel_engine_cs *engine) static void set_timeslice(struct intel_engine_cs *engine) { + struct intel_engine_execlists * const execlists = &engine->execlists; unsigned long duration; + if (*execlists->active) + intel_context_stats_start((*execlists->active)->context); + if (!intel_engine_has_timeslices(engine)) return; duration = active_timeslice(engine); ENGINE_TRACE(engine, "bump timeslicing, interval:%lu", duration); - set_timer_ms(&engine->execlists.timer, duration); + set_timer_ms(&execlists->timer, duration); } static void start_timeslice(struct intel_engine_cs *engine, int prio) diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c index 95d41c01d0e0..67f8083f40a3 100644 --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c @@ -6381,8 +6381,8 @@ static int __live_pphwsp_runtime(struct intel_engine_cs *engine) if (IS_ERR(ce)) return PTR_ERR(ce); - ce->runtime.num_underflow = 0; - ce->runtime.max_underflow = 0; + ce->stats.runtime.num_underflow = 0; + ce->stats.runtime.max_underflow = 0; do { unsigned int loop = 1024; @@ -6420,11 +6420,11 @@ static int __live_pphwsp_runtime(struct intel_engine_cs *engine) intel_context_get_avg_runtime_ns(ce)); err = 0; - if (ce->runtime.num_underflow) { + if (ce->stats.runtime.num_underflow) { pr_err("%s: pphwsp underflow %u time(s), max %u cycles!\n", engine->name, - ce->runtime.num_underflow, - ce->runtime.max_underflow); + ce->stats.runtime.num_underflow, + ce->stats.runtime.max_underflow); GEM_TRACE_DUMP(); err = -EOVERFLOW; } diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index e21d97d810a5..93265d1afaef 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -67,7 +67,7 @@ pphwsp_busy_add(struct i915_gem_context *ctx, unsigned int class) for_each_gem_engine(ce, engines, it) { if (ce->engine->uabi_class == class) - total += ce->runtime.total; + total += ce->stats.runtime.total; } return total; diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c index 8e468942f943..4262a2c72617 100644 --- a/drivers/gpu/drm/i915/i915_gpu_error.c +++ b/drivers/gpu/drm/i915/i915_gpu_error.c @@ -1277,8 +1277,8 @@ static bool record_context(struct i915_gem_context_coredump *e, e->guilty = atomic_read(&ctx->guilty_count); e->active = atomic_read(&ctx->active_count); - e->total_runtime = rq->context->runtime.total; - e->avg_runtime = ewma_runtime_read(&rq->context->runtime.avg); + e->total_runtime = rq->context->stats.runtime.total; + e->avg_runtime = ewma_runtime_read(&rq->context->stats.runtime.avg); simulated = i915_gem_context_no_error_capture(ctx); From patchwork Mon Sep 14 13:12:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 11773761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62FB4618 for ; Mon, 14 Sep 2020 13:13:24 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 445BF214F1 for ; Mon, 14 Sep 2020 13:13:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 445BF214F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B82116E4A6; Mon, 14 Sep 2020 13:13:23 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3F83F6E49D for ; Mon, 14 Sep 2020 13:13:20 +0000 (UTC) IronPort-SDR: pKxoWzXrud3/zM+J7DafMihYTy3Ic9EU0pjl53JTbTuC+FBkAP/1D1tVCckkRuD0z0nqA7KsBu z8m/aT2iBdYg== X-IronPort-AV: E=McAfee;i="6000,8403,9743"; a="159113446" X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="159113446" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:20 -0700 IronPort-SDR: y6KJVqtFjc8+Eo7ULVQHu1ZRYzeMFFTUD3IWFnmT2pJ+FnsmtyDm/oQ+wIyPJon+LUo6qGL3Gz 0kzO0efx4XVA== X-IronPort-AV: E=Sophos;i="5.76,426,1592895600"; d="scan'208";a="507136419" Received: from matancoh-mobl2.ger.corp.intel.com (HELO localhost.localdomain) ([10.255.198.45]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2020 06:13:18 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 14 Sep 2020 14:12:56 +0100 Message-Id: <20200914131256.89465-10-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> References: <20200914131256.89465-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 9/9] drm/i915: Prefer software tracked context busyness X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Tvrtko Ursulin When available prefer context tracked context busyness because it provides visibility into currently executing contexts as well. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_drm_client.c | 68 ++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index 93265d1afaef..0ba8e5974ed1 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -96,6 +96,61 @@ show_client_busy(struct device *kdev, struct device_attribute *attr, char *buf) return snprintf(buf, PAGE_SIZE, "%llu\n", total); } +static u64 +sw_busy_add(struct i915_gem_context *ctx, unsigned int class) +{ + struct i915_gem_engines *engines = rcu_dereference(ctx->engines); + u32 period_ns = RUNTIME_INFO(ctx->i915)->cs_timestamp_period_ns; + struct i915_gem_engines_iter it; + struct intel_context *ce; + u64 total = 0; + + for_each_gem_engine(ce, engines, it) { + struct intel_context_stats *stats; + unsigned int seq; + u64 t; + + if (ce->engine->uabi_class != class) + continue; + + stats = &ce->stats; + + do { + seq = read_seqbegin(&stats->lock); + t = ce->stats.runtime.total * period_ns; + t += __intel_context_get_active_time(ce); + } while (read_seqretry(&stats->lock, seq)); + + total += t; + } + + return total; +} + +static ssize_t +show_client_sw_busy(struct device *kdev, + struct device_attribute *attr, + char *buf) +{ + struct i915_engine_busy_attribute *i915_attr = + container_of(attr, typeof(*i915_attr), attr); + unsigned int class = i915_attr->engine_class; + struct i915_drm_client *client = i915_attr->client; + u32 period_ns = RUNTIME_INFO(i915_attr->i915)->cs_timestamp_period_ns; + u64 total = atomic64_read(&client->past_runtime[class]) * period_ns; + struct list_head *list = &client->ctx_list; + struct i915_gem_context *ctx; + + rcu_read_lock(); + list_for_each_entry_rcu(ctx, list, client_link) { + total += atomic64_read(&ctx->past_runtime[class]) * period_ns + + sw_busy_add(ctx, class); + } + rcu_read_unlock(); + + return snprintf(buf, PAGE_SIZE, "%llu\n", total); +} + static const char * const uabi_class_names[] = { [I915_ENGINE_CLASS_RENDER] = "0", [I915_ENGINE_CLASS_COPY] = "1", @@ -109,6 +164,8 @@ __client_register_sysfs_busy(struct i915_drm_client *client) struct i915_drm_clients *clients = client->clients; struct drm_i915_private *i915 = container_of(clients, typeof(*i915), clients); + bool sw_stats = i915->caps.scheduler & + I915_SCHEDULER_CAP_ENGINE_BUSY_STATS; unsigned int i; int ret = 0; @@ -135,18 +192,19 @@ __client_register_sysfs_busy(struct i915_drm_client *client) attr->attr.name = uabi_class_names[i]; attr->attr.mode = 0444; - attr->show = show_client_busy; + attr->show = sw_stats ? + show_client_sw_busy : show_client_busy; ret = sysfs_create_file(client->busy_root, (struct attribute *)attr); if (ret) - goto err; + goto out; } - return 0; +out: + if (ret) + kobject_put(client->busy_root); -err: - kobject_put(client->busy_root); return ret; }