From patchwork Tue Jul 19 17:18:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AA86CCA47F for ; Tue, 19 Jul 2022 17:18:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238597AbiGSRSn (ORCPT ); Tue, 19 Jul 2022 13:18:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238580AbiGSRSm (ORCPT ); Tue, 19 Jul 2022 13:18:42 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73CDE46DB3; Tue, 19 Jul 2022 10:18:41 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id b9so14163546pfp.10; Tue, 19 Jul 2022 10:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0sMFYlUXA/wvNrdz6yghIiBT/tIS2Mu/kejgpgcpcUk=; b=d84EtypEw3YOZPEj14Cy+1tsV890PI9GGysEyTiDevcf/6y9pLX+Tmvpsgqu5AW/tD qUDeeVgSBoYVbPh3CWfZKO4OgqcjG6RSiUlTuTtOFVJebfp5S8fdU00WUVwe4fCg3Ifu wEGveVZ9GNW56fW9MSs1WoMboIXkYzOfeT7x9GvipJ/BzCWYqMU/4N5At7FcYUPpHd8a o8d16yIryXMO3irpgmqMxpnkThB+LVwWueaxLNt9aoFu/Iyr5IWOZOSfWN7HcxaT3/rz /y3hsp8rF/ZQc5PQanfKzP6fMdbhUfU9UwzQJ6+7v6x7CxIiphESGfclC17n8EoKEtM6 WbFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0sMFYlUXA/wvNrdz6yghIiBT/tIS2Mu/kejgpgcpcUk=; b=dXss3v6duawsqPSDb6gO2N262QmafNJjvioq01cIJmwGHJZpFxLD9NTimX0jGCPmNr Y+7dqf6LPw4pBKH4FKZYX1qRE9kr5LIm2VBBKadIUdVWSu0K6M/AyxsU1hVDWeZQYhaF Zlj4soDCVJLS/mk/RWN8JwpoVm7lMDD8LpDO8d62vrnxgw6TgjNzW3+pFabTPyEdMZDW f4WmhZDb6oE065/OMyaLFFwmIqbm753qewtNzEBSWhDd22FEeCPigq7063acb4hlvwoc hJWRd9OLvGyMHjns8HwkybljE47fDKu61jMBM/DbuLlb3iT/S6VdymhrkbTML+qAlbdV 8PTw== X-Gm-Message-State: AJIora83EqyNlPUPBBlTKxh1Wa2TYhCSs3z1TOqwu+1UOv0d9Ow0BFPK X3y+SFVMlpSEloRVw3J2jAg= X-Google-Smtp-Source: AGRyM1ua7TCIedMtFkEXgDnLA/rlwx26jhGhXrgdrpp2JJPABfQTwGlNH6fuH4j9MWpjRTpFMF5ESg== X-Received: by 2002:a63:5620:0:b0:41a:4c1e:a821 with SMTP id k32-20020a635620000000b0041a4c1ea821mr3591003pgb.611.1658251120965; Tue, 19 Jul 2022 10:18:40 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id w65-20020a626244000000b0050dc762816dsm11702733pfb.71.2022.07.19.10.18.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:40 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 01/13] drm/msm: Reorder lock vs submit alloc Date: Tue, 19 Jul 2022 10:18:45 -0700 Message-Id: <20220719171900.289265-2-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark This lets us drop the NORETRY. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index c9e4aeb14f4a..b7c61a99d274 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -36,7 +36,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, if (sz > SIZE_MAX) return ERR_PTR(-ENOMEM); - submit = kzalloc(sz, GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY); + submit = kzalloc(sz, GFP_KERNEL); if (!submit) return ERR_PTR(-ENOMEM); @@ -771,25 +771,21 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, trace_msm_gpu_submit(pid_nr(pid), ring->id, submitid, args->nr_bos, args->nr_cmds); - ret = mutex_lock_interruptible(&queue->lock); - if (ret) - goto out_post_unlock; - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) { out_fence_fd = get_unused_fd_flags(O_CLOEXEC); if (out_fence_fd < 0) { ret = out_fence_fd; - goto out_unlock; + return ret; } } - submit = submit_create(dev, gpu, queue, args->nr_bos, - args->nr_cmds); - if (IS_ERR(submit)) { - ret = PTR_ERR(submit); - submit = NULL; - goto out_unlock; - } + submit = submit_create(dev, gpu, queue, args->nr_bos, args->nr_cmds); + if (IS_ERR(submit)) + return PTR_ERR(submit); + + ret = mutex_lock_interruptible(&queue->lock); + if (ret) + goto out_post_unlock; submit->pid = pid; submit->ident = submitid; @@ -965,9 +961,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (ret && (out_fence_fd >= 0)) put_unused_fd(out_fence_fd); mutex_unlock(&queue->lock); +out_post_unlock: if (submit) msm_gem_submit_put(submit); -out_post_unlock: if (!IS_ERR_OR_NULL(post_deps)) { for (i = 0; i < args->nr_out_syncobjs; ++i) { kfree(post_deps[i].chain); From patchwork Tue Jul 19 17:18:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A273CC43334 for ; Tue, 19 Jul 2022 17:18:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236021AbiGSRSv (ORCPT ); Tue, 19 Jul 2022 13:18:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238628AbiGSRSp (ORCPT ); Tue, 19 Jul 2022 13:18:45 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61BD547BBA; Tue, 19 Jul 2022 10:18:44 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id 70so14187544pfx.1; Tue, 19 Jul 2022 10:18:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FlC/bVN5qW7CT1CMAFt2x0HFIxhESZPM94joMv7rafM=; b=Lbh9s2g13yWjWLtBl9Kt+y6e/wC7cxwUw0b8qgyetirgPr8MtFwaBiRdowAbjr4uJK kTO3y4+wcOAp7ZZF40dQLLcP6evHJT6rG3ScpXx4yRZeYjw2f/bUPOSQ9I4tlWq+FbUC eAXROc7sM1ARDls+UL+rhGeVLDi5bWRTAi9S3iyAN154lL7+dpHc8qBylOYj1+c7aCfz 5cL5ivG+GJKgSb2JtJlLqodYm5cjm/MJv+12se0IFlj+eQeGgORGY5UFbYHX3054Bbfy AzYCgxhb7DicKjHDnJbzHY3bRKQD+UEXgVF7goioHti+ZkZb6COpqupPMtDBeJTIYeUX RtCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FlC/bVN5qW7CT1CMAFt2x0HFIxhESZPM94joMv7rafM=; b=RwIhrw6oJCvNT4QmtB64j/ZMQTOcIflIEboUm4F25pp1tsYL8DqBNRvUTyjYYz2Voo RIyoiv5pd1zek4XuvLwC0eITVfiDDQP0Vgt/bcF0osKJ7aXIqpB1+R91L2aWadk8fnfM t8+zUs15Hp47wngO7m5yCOTLZ1Sq6TcXpPBEXbch7tiWHAPE4uJpFY4TL8BiN9qqtDAg xl4+h8o5Pxx3EY89oTcJbl82zb+yWIHdsc/nZXtMiOEBrkDVVgthpXncYLnQsb8x+CHk WfPHq6aiI57N8IVJ58Rp+FNKw19NRCnV5E2iyFYUrMEaBeqWR8tvw3kdOr9SJChp/vgV c94w== X-Gm-Message-State: AJIora9zcYojKDZ9xFg29aGAMztUjiUYFfDkPdXCQ5W9I2ABvShAvz3p PUFfA5kGZ5tBYNc6oy8ikydzrscMchM= X-Google-Smtp-Source: AGRyM1siWuYUVeXLva8EenGWnt/QaLz7KxaosG9gsqAREN8dt9Zd9ETzrsWVj+JjxqdhNtz0s5msDg== X-Received: by 2002:a05:6a00:158e:b0:52a:e628:8b3b with SMTP id u14-20020a056a00158e00b0052ae6288b3bmr35372045pfk.80.1658251123786; Tue, 19 Jul 2022 10:18:43 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id w13-20020a1709029a8d00b0016a4db13435sm11809538plp.191.2022.07.19.10.18.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:42 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 02/13] drm/msm: Small submit cleanup Date: Tue, 19 Jul 2022 10:18:46 -0700 Message-Id: <20220719171900.289265-3-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Move more initialization into submit_create(). Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index b7c61a99d274..c7819781879c 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -26,6 +26,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, struct msm_gpu_submitqueue *queue, uint32_t nr_bos, uint32_t nr_cmds) { + static atomic_t ident = ATOMIC_INIT(0); struct msm_gem_submit *submit; uint64_t sz; int ret; @@ -52,9 +53,13 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, submit->gpu = gpu; submit->cmd = (void *)&submit->bos[nr_bos]; submit->queue = queue; + submit->pid = get_pid(task_pid(current)); submit->ring = gpu->rb[queue->ring_nr]; submit->fault_dumped = false; + /* Get a unique identifier for the submission for logging purposes */ + submit->ident = atomic_inc_return(&ident) - 1; + INIT_LIST_HEAD(&submit->node); return submit; @@ -718,7 +723,6 @@ static void msm_process_post_deps(struct msm_submit_post_dep *post_deps, int msm_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file) { - static atomic_t ident = ATOMIC_INIT(0); struct msm_drm_private *priv = dev->dev_private; struct drm_msm_gem_submit *args = data; struct msm_file_private *ctx = file->driver_priv; @@ -729,10 +733,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, struct msm_submit_post_dep *post_deps = NULL; struct drm_syncobj **syncobjs_to_reset = NULL; int out_fence_fd = -1; - struct pid *pid = get_pid(task_pid(current)); bool has_ww_ticket = false; unsigned i; - int ret, submitid; + int ret; if (!gpu) return -ENXIO; @@ -764,12 +767,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (!queue) return -ENOENT; - /* Get a unique identifier for the submission for logging purposes */ - submitid = atomic_inc_return(&ident) - 1; - ring = gpu->rb[queue->ring_nr]; - trace_msm_gpu_submit(pid_nr(pid), ring->id, submitid, - args->nr_bos, args->nr_cmds); if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) { out_fence_fd = get_unused_fd_flags(O_CLOEXEC); @@ -783,13 +781,13 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (IS_ERR(submit)) return PTR_ERR(submit); + trace_msm_gpu_submit(pid_nr(submit->pid), ring->id, submit->ident, + args->nr_bos, args->nr_cmds); + ret = mutex_lock_interruptible(&queue->lock); if (ret) goto out_post_unlock; - submit->pid = pid; - submit->ident = submitid; - if (args->flags & MSM_SUBMIT_SUDO) submit->in_rb = true; From patchwork Tue Jul 19 17:18:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF88BC433EF for ; Tue, 19 Jul 2022 17:18:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238719AbiGSRSv (ORCPT ); Tue, 19 Jul 2022 13:18:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238677AbiGSRSt (ORCPT ); Tue, 19 Jul 2022 13:18:49 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EC7A481DC; Tue, 19 Jul 2022 10:18:46 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id q16so10976175pgq.6; Tue, 19 Jul 2022 10:18:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AwYCVdrGkHnM2D+4h4BxfmYuAccPj8wFXhKV37+GyLo=; b=ZSfDdk6YMZ9Xin72zYX3eWgoVjeyCsNqo4nzrWEtQerLnQeGj7wiD+XdydEuSNBGiM AixBzDQDz9V5PL5tQ4STW7Mw9fb7b7vq1IGT7puboqUIyK84PeEKE/gUO/7uaZFUQE5Y Z81tah/N3JXlJ30gtWFPqJaW1j/+1FEJG40ZmHhw/qGAYwZgTk9jdCM/f9D/S+ZOQxDl SKLBbIG1cqZOK2TRo3kbaOch1HlfqECN6PnborDuNCfKfmQxUpi1stONUoUSWtLYfR0h i61xaa3uO0VhOLWj1M1U2Y4cIBjPCU0Q0ioyR6M1lTvD6XbBR7gq0kMWijeLBuZHYwGj w/Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AwYCVdrGkHnM2D+4h4BxfmYuAccPj8wFXhKV37+GyLo=; b=GIpltOmuOeR8gRA153b/2aTMrtpF5j8ozy6P26j3cBQWkrvHMxQpKWXCm8TsUWb3Bf bl+wlpRR4pF1pwP3UhNw+jeEKFtBfwQoULY5OdCwvzHo5QIp7TA+pfk/gGBy+8KNzcLp ty7Gs0s6zzmg0qcruV0uK5vrIsEp0x0K4MTh8IV5Qx85YFoqd4eoY8SZMh2z9puIoE/V 8YPdSVN80RWDFHAzklflnXuSK+XWeZKbNqNH93SS6kWP82OPEvDaaiDWahIZbrgqOdwq +bpgpX/y2DZ6+tJVKNQm6422mvioj7RyCA1WfWeBt2gN1dqapObPQOrhbVINSBa0f8k6 mzvg== X-Gm-Message-State: AJIora/iGi4+z9yUmNBm2PAMEpkfQCmTTUzpW+jhxLlZ2GcWeGqX2z/a cq5dFRrdEJ1toPNvI4quBvo= X-Google-Smtp-Source: AGRyM1vrTXrLejBYmT+gXPQJMxY4GT3MYbQgcVB/BHfFc2lus4Q8+kzsdRFYebbfBQxkcNHsU67MRQ== X-Received: by 2002:a63:1246:0:b0:41a:58f:9fee with SMTP id 6-20020a631246000000b0041a058f9feemr14074197pgs.413.1658251126440; Tue, 19 Jul 2022 10:18:46 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id go18-20020a17090b03d200b001ef87123615sm1995924pjb.37.2022.07.19.10.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:45 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 03/13] drm/msm: Split out idr_lock Date: Tue, 19 Jul 2022 10:18:47 -0700 Message-Id: <20220719171900.289265-4-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Otherwise if we hit reclaim pinning objects in the submit path, we'll be blocking retire_worker trying to free a submit. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_drv.c | 4 ++-- drivers/gpu/drm/msm/msm_gem_submit.c | 10 ++++++++-- drivers/gpu/drm/msm/msm_gpu.h | 4 +++- drivers/gpu/drm/msm/msm_submitqueue.c | 1 + 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 1ed4cd09dbf8..d7ca025457b6 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -883,13 +883,13 @@ static int wait_fence(struct msm_gpu_submitqueue *queue, uint32_t fence_id, * retired, so if the fence is not found it means there is nothing * to wait for */ - ret = mutex_lock_interruptible(&queue->lock); + ret = mutex_lock_interruptible(&queue->idr_lock); if (ret) return ret; fence = idr_find(&queue->fence_idr, fence_id); if (fence) fence = dma_fence_get_rcu(fence); - mutex_unlock(&queue->lock); + mutex_unlock(&queue->idr_lock); if (!fence) return 0; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index c7819781879c..16c662808522 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -72,9 +72,9 @@ void __msm_gem_submit_destroy(struct kref *kref) unsigned i; if (submit->fence_id) { - mutex_lock(&submit->queue->lock); + mutex_lock(&submit->queue->idr_lock); idr_remove(&submit->queue->fence_idr, submit->fence_id); - mutex_unlock(&submit->queue->lock); + mutex_unlock(&submit->queue->idr_lock); } dma_fence_put(submit->user_fence); @@ -881,6 +881,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->nr_cmds = i; + mutex_lock(&queue->idr_lock); + /* * If using userspace provided seqno fence, validate that the id * is available before arming sched job. Since access to fence_idr @@ -889,6 +891,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, */ if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) && idr_find(&queue->fence_idr, args->fence)) { + mutex_unlock(&queue->idr_lock); ret = -EINVAL; goto out; } @@ -921,6 +924,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->user_fence, 1, INT_MAX, GFP_KERNEL); } + + mutex_unlock(&queue->idr_lock); + if (submit->fence_id < 0) { ret = submit->fence_id; submit->fence_id = 0; diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 4d935fedd2ac..962d2070bcdf 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -466,7 +466,8 @@ static inline int msm_gpu_convert_priority(struct msm_gpu *gpu, int prio, * @node: node in the context's list of submitqueues * @fence_idr: maps fence-id to dma_fence for userspace visible fence * seqno, protected by submitqueue lock - * @lock: submitqueue lock + * @idr_lock: for serializing access to fence_idr + * @lock: submitqueue lock for serializing submits on a queue * @ref: reference count * @entity: the submit job-queue */ @@ -479,6 +480,7 @@ struct msm_gpu_submitqueue { struct msm_file_private *ctx; struct list_head node; struct idr fence_idr; + struct mutex idr_lock; struct mutex lock; struct kref ref; struct drm_sched_entity *entity; diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c b/drivers/gpu/drm/msm/msm_submitqueue.c index f486a3cd4e55..c6929e205b51 100644 --- a/drivers/gpu/drm/msm/msm_submitqueue.c +++ b/drivers/gpu/drm/msm/msm_submitqueue.c @@ -200,6 +200,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx, *id = queue->id; idr_init(&queue->fence_idr); + mutex_init(&queue->idr_lock); mutex_init(&queue->lock); list_add_tail(&queue->node, &ctx->submitqueues); From patchwork Tue Jul 19 17:18:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAF48C43334 for ; Tue, 19 Jul 2022 17:18:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238625AbiGSRSx (ORCPT ); Tue, 19 Jul 2022 13:18:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238707AbiGSRSv (ORCPT ); Tue, 19 Jul 2022 13:18:51 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF70D47BA4; Tue, 19 Jul 2022 10:18:49 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id f11so12556802plr.4; Tue, 19 Jul 2022 10:18:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6E5wGlWgwtmRN55cq/vUaSJZuL1NyuRgvoYtJGWwhUY=; b=BuJsnc398rfPIFc8fRsJaLTnog5hsqN/+XgNHPJk9ZBqXssTwHL7CTuyZuzl92Krnj mN7ZPWDTK2qjPE4w/VlRi6wAPL9CLJ8cq+f60wcmRKLSdFGRjva0LwVRgqWXHljBAkZ1 ObtaN+pdx/IsNJMNBFQDlLBWdFE3l5noKaOWNrm12N91zorP5FARH3nEvSj8MvL/R3f0 paspiTNMF2vYlIzznZ/RSZ+2ia4ai7FiqRxMTLi2AeI3dP/B37DWmvJAQ1KUjGtsrS6h /tzeir28FbRXoA5PPto/pNtzOnRIGfwLgnOx1G5zIZYsyMecglSQc0ugQVdX6JPYu7wV pXuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6E5wGlWgwtmRN55cq/vUaSJZuL1NyuRgvoYtJGWwhUY=; b=MrvyvTnLU2O7Yx74lpkudAtNCYC+Wv2h/wnr1DYum44cf6xp62MCwL0IYaPQoL9/Gn lDu/qTgrQbTuARqo0jB7wB/vn1MdIP3k9Cu+e+z4IWmCywxlPcwbs+pqMq2UeyJhtXFr NV3M2Ubifx1puKNw4jcjUh+fjtCA6xOKW6cNDsIviAx9QRm6vZhg9gOtUYS1EJ+Vs+xx ++gfqiXj6liGKs3+4r2kYCsFwNVJ7s8gCvMT3lwDzXWF73mv5nQ79xaJWd50/eJG7C3T lRp2HoFvg3aGLC7Yz/UYCtw2I9UHKsp6YT2EnevacrurdO4RmqAqarLiYdJxg/FU1Pib LUkQ== X-Gm-Message-State: AJIora80LyM7uPHQWK3R2NLCWUDflsLh9c9TO9fM2+/IQxSNT8374X8Z hVJ4ScMx+D4S5APkaBsaZMM= X-Google-Smtp-Source: AGRyM1vQvvozAIxX6sWiAmFPrxMnWkv2PukPVsxPIawN7d+3XSkrBXeJkyuxtUxYCRCiD4niUkiXEA== X-Received: by 2002:a17:90b:33ce:b0:1ef:e5f4:f8e2 with SMTP id lk14-20020a17090b33ce00b001efe5f4f8e2mr451516pjb.70.1658251129300; Tue, 19 Jul 2022 10:18:49 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id nt14-20020a17090b248e00b001ef82e5f5aesm11802310pjb.47.2022.07.19.10.18.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:48 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 04/13] drm/msm/gem: Check for active in shrinker path Date: Tue, 19 Jul 2022 10:18:48 -0700 Message-Id: <20220719171900.289265-5-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Currently in our shrinker path we shouldn't be encountering anything that is active, but this will change in subsequent patches. So check if there are unsignaled fences. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 10 ++++++++++ drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gem_shrinker.c | 6 ++++++ 3 files changed, 17 insertions(+) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 8ddbd2e001d4..b55d252aef17 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -870,6 +870,16 @@ static void update_inactive(struct msm_gem_object *msm_obj) mutex_unlock(&priv->mm_lock); } +bool msm_gem_active(struct drm_gem_object *obj) +{ + GEM_WARN_ON(!msm_gem_is_locked(obj)); + + if (to_msm_bo(obj)->pin_count) + return true; + + return !dma_resv_test_signaled(obj->resv, dma_resv_usage_rw(true)); +} + int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout) { bool write = !!(op & MSM_PREP_WRITE); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 432032ad4aed..0ab0dc4f8c25 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -173,6 +173,7 @@ void msm_gem_put_vaddr(struct drm_gem_object *obj); int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv); void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu); void msm_gem_active_put(struct drm_gem_object *obj); +bool msm_gem_active(struct drm_gem_object *obj); int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout); int msm_gem_cpu_fini(struct drm_gem_object *obj); int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file, diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 6e39d959b9f0..ea8ed74982c1 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -43,6 +43,9 @@ purge(struct msm_gem_object *msm_obj) if (!is_purgeable(msm_obj)) return false; + if (msm_gem_active(&msm_obj->base)) + return false; + /* * This will move the obj out of still_in_list to * the purged list @@ -58,6 +61,9 @@ evict(struct msm_gem_object *msm_obj) if (is_unevictable(msm_obj)) return false; + if (msm_gem_active(&msm_obj->base)) + return false; + msm_gem_evict(&msm_obj->base); return true; From patchwork Tue Jul 19 17:18:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81363C433EF for ; Tue, 19 Jul 2022 17:19:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238760AbiGSRTV (ORCPT ); Tue, 19 Jul 2022 13:19:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238774AbiGSRS4 (ORCPT ); Tue, 19 Jul 2022 13:18:56 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99806481F2; Tue, 19 Jul 2022 10:18:52 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id l124so14146507pfl.8; Tue, 19 Jul 2022 10:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mxFT+vmlgH8OMaHZf0MGM1sQcdQOD7qQazzf52IZIdo=; b=pX4QvSF8ZvLQfdCSHWZFZOvchRVu8kUZ1QMPP7yZODkgH5UK+sel9ep8wsSSRKqBVu kkUzH0kUx+RsndmqlXKGHLDHa43r56upHNqdkktnGdEAXjsvQoF88U163YSB8mPZNsaA g72Z2BKhq4CZFUt88MnUCGEUdBGNjhVYHa2Ne3g/p4QmcMKP4RQ/myuRHO2+8yk0bd5i MXb4HV5FT4SsbaeAp8QAiL/4alU4jjJaoV2S1rKWp4Y5Qt5w5Rqe4wMcX/FB7JUcg/Rs 0iQd8Ty+/4fX9XJYndL0gpFROt0MtQoB4N2LhJIS8mp0lCFRJGsOQ9LfKxABQ74KcXaU ODEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mxFT+vmlgH8OMaHZf0MGM1sQcdQOD7qQazzf52IZIdo=; b=DI5ze0BvSGt1vJjUyXVrcKrcTUsgeM0XZG1qSSihKa+gAEFMNx38ZGfNimmH5RuIl4 OVlLVk5et0UQNdBNXrDLsU5XdQX9o/O/7yS5XH5v/C0ZG5BOhnHmW+3IUMcvxv6YD4gG +5VvvFYif6tibE4L/he94Mmtuvkoi9FKpc2aP+cG91mE1wQtdsTTO+gjtvET9EVQ0GJW MFRqh4DrJLDyyX8kz5HiKYKj9x02PA88ww9tD5h9p43Eyni+OTXQhkR2OgwPZkwJeAaT oPZn5ginVfz3OLG5R53OTPDQNsvZrhreT+2zgfos3MPsIMSj+K5TxPkfMk3F3mfrWgSR YoaA== X-Gm-Message-State: AJIora/UPOOyGZ3X5n+uf6RHv9w/rVZACawAqzULVPKBAO5FfM/+RlRM ZlcIqT35aLk6Hb6DOXj0N+g= X-Google-Smtp-Source: AGRyM1sH6WAjdZeZjLMvuCM6e9KbQu41Poi6tjCVuZpE2gcICGHekqRPSyGoItba8tKCDP/vrjKfgQ== X-Received: by 2002:a05:6a00:14c5:b0:52b:8877:8263 with SMTP id w5-20020a056a0014c500b0052b88778263mr6313813pfu.1.1658251132049; Tue, 19 Jul 2022 10:18:52 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id u10-20020a170902e5ca00b0016a4f3ca28bsm12070284plf.274.2022.07.19.10.18.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:50 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 05/13] drm/msm/gem: Rename update_inactive Date: Tue, 19 Jul 2022 10:18:49 -0700 Message-Id: <20220719171900.289265-6-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Really what this is doing is updating various LRU lists. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index b55d252aef17..97467364dc0a 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -19,7 +19,7 @@ #include "msm_gpu.h" #include "msm_mmu.h" -static void update_inactive(struct msm_gem_object *msm_obj); +static void update_lru(struct drm_gem_object *obj); static dma_addr_t physaddr(struct drm_gem_object *obj) { @@ -132,7 +132,7 @@ static struct page **get_pages(struct drm_gem_object *obj) if (msm_obj->flags & MSM_BO_WC) sync_for_device(msm_obj); - update_inactive(msm_obj); + update_lru(obj); } return msm_obj->pages; @@ -193,7 +193,7 @@ struct page **msm_gem_get_pages(struct drm_gem_object *obj) if (!IS_ERR(p)) { msm_obj->pin_count++; - update_inactive(msm_obj); + update_lru(obj); } msm_gem_unlock(obj); @@ -207,7 +207,7 @@ void msm_gem_put_pages(struct drm_gem_object *obj) msm_gem_lock(obj); msm_obj->pin_count--; GEM_WARN_ON(msm_obj->pin_count < 0); - update_inactive(msm_obj); + update_lru(obj); msm_gem_unlock(obj); } @@ -449,7 +449,7 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) msm_obj->pin_count--; GEM_WARN_ON(msm_obj->pin_count < 0); - update_inactive(msm_obj); + update_lru(obj); } struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj, @@ -658,7 +658,7 @@ static void *get_vaddr(struct drm_gem_object *obj, unsigned madv) goto fail; } - update_inactive(msm_obj); + update_lru(obj); } return msm_obj->vaddr; @@ -730,7 +730,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) * between inactive lists */ if (msm_obj->active_count == 0) - update_inactive(msm_obj); + update_lru(obj); msm_gem_unlock(obj); @@ -757,7 +757,7 @@ void msm_gem_purge(struct drm_gem_object *obj) put_iova_vmas(obj); msm_obj->madv = __MSM_MADV_PURGED; - update_inactive(msm_obj); + update_lru(obj); drm_gem_free_mmap_offset(obj); @@ -792,7 +792,7 @@ void msm_gem_evict(struct drm_gem_object *obj) put_pages(obj); - update_inactive(msm_obj); + update_lru(obj); } void msm_gem_vunmap(struct drm_gem_object *obj) @@ -835,13 +835,14 @@ void msm_gem_active_put(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); if (--msm_obj->active_count == 0) { - update_inactive(msm_obj); + update_lru(obj); } } -static void update_inactive(struct msm_gem_object *msm_obj) +static void update_lru(struct drm_gem_object *obj) { - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; + struct msm_drm_private *priv = obj->dev->dev_private; + struct msm_gem_object *msm_obj = to_msm_bo(obj); GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); From patchwork Tue Jul 19 17:18:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D5FCCA483 for ; Tue, 19 Jul 2022 17:19:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238705AbiGSRTV (ORCPT ); Tue, 19 Jul 2022 13:19:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238900AbiGSRTF (ORCPT ); Tue, 19 Jul 2022 13:19:05 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D97E50195; Tue, 19 Jul 2022 10:18:55 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id f65so14052987pgc.12; Tue, 19 Jul 2022 10:18:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4TbOfcqFkaHon89dsPDId7NNakdXJT5mDYYEM5ZbQu4=; b=RKXxxJNOMmrIvD2K9FHnDoHJDSySY9Pr9GW0j7hZHwCdyC1cQ2KC/Ri8GmirMDd6LO 1NNS432ZjfDA7GIbgLQc7kPDawJca/cA1jEe3+P2pLSL2T90P+1SlkehOHBH59nXuHak inaq8td1A5cswfD3meNz5QApSVnf2AQJqpvAuOextTZvRi3KqnsURwVO/+eNFgmL9/Um PufKk6i0NobNuVgehwUxehb9aVVuLKsvBpbbx/FYLTxCWVzE/qlt/7nkAXNt7yzU8j9J Z/O6kSHlE+nlC8f2xi0quvlLE7dNiU3+pU0L9DTLZQhh8Lo6BTujTd0IECTKqeXqFtzZ YYYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4TbOfcqFkaHon89dsPDId7NNakdXJT5mDYYEM5ZbQu4=; b=waDmYtI1ZJT+Tp8GRTNqvcsH0D9AfLBlqs+kLP/CboFnD0CE4/y76Rl5pZ2aChD42p T3gt0rIKROkyLR6/6tlRw+F141yGqG1C1bPvvSzBVHm1b00GPM2algNqQ/IZbQgrEAsf 1XPC/EhvK+224I+wwGmJxvRuWVZr6Kb4zZ5+qxEox23nD1pzuJIZQHS2ml6XoZ/ITrM3 PoXxHBT6iU96qXrUPl6o/feuVmnqSBkDU108kya04z4bLd3r5DqoYdnmUj3yQZqA42sf EXk3FWMVsaW6tW0etQWcJIcj/IVWRPAbxbhNyBR4c1QXJpMprMX7q6rbGIZGUsnI28sm k2mA== X-Gm-Message-State: AJIora/QZAMf+IeYbHp1gHYmlvbqhpLLlNVbUpV3DJJTtBKSWJjswx1m 5FDjAbWSMELo9vLjUwVtMho= X-Google-Smtp-Source: AGRyM1vSPo6luOBnOdh3fOvkjQdqW8BN2hrChpa04kg6m+Cj6xRnjknz/OdvxhwoLLfeQb2pN9c7dA== X-Received: by 2002:a63:6c42:0:b0:3fe:465:7a71 with SMTP id h63-20020a636c42000000b003fe04657a71mr29526258pgc.101.1658251134845; Tue, 19 Jul 2022 10:18:54 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id a201-20020a621ad2000000b0050dc76281e0sm11667317pfa.186.2022.07.19.10.18.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:53 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 06/13] drm/msm/gem: Rename to pin/unpin_pages Date: Tue, 19 Jul 2022 10:18:50 -0700 Message-Id: <20220719171900.289265-7-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Since that is what these fxns actually do.. they are getting *pinned* pages (as opposed to cases where we need pages, but don't need them pinned, like CPU mappings). Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 18 +++++++++++++----- drivers/gpu/drm/msm/msm_gem.h | 4 ++-- drivers/gpu/drm/msm/msm_gem_prime.c | 4 ++-- 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 97467364dc0a..3da64c7f65a2 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -177,30 +177,38 @@ static void put_pages(struct drm_gem_object *obj) } } -struct page **msm_gem_get_pages(struct drm_gem_object *obj) +static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); struct page **p; - msm_gem_lock(obj); + GEM_WARN_ON(!msm_gem_is_locked(obj)); if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) { - msm_gem_unlock(obj); return ERR_PTR(-EBUSY); } p = get_pages(obj); - if (!IS_ERR(p)) { msm_obj->pin_count++; update_lru(obj); } + return p; +} + +struct page **msm_gem_pin_pages(struct drm_gem_object *obj) +{ + struct page **p; + + msm_gem_lock(obj); + p = msm_gem_pin_pages_locked(obj); msm_gem_unlock(obj); + return p; } -void msm_gem_put_pages(struct drm_gem_object *obj) +void msm_gem_unpin_pages(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 0ab0dc4f8c25..6fe521ccda45 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -159,8 +159,8 @@ int msm_gem_get_and_pin_iova(struct drm_gem_object *obj, struct msm_gem_address_space *aspace, uint64_t *iova); void msm_gem_unpin_iova(struct drm_gem_object *obj, struct msm_gem_address_space *aspace); -struct page **msm_gem_get_pages(struct drm_gem_object *obj); -void msm_gem_put_pages(struct drm_gem_object *obj); +struct page **msm_gem_pin_pages(struct drm_gem_object *obj); +void msm_gem_unpin_pages(struct drm_gem_object *obj); int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev, struct drm_mode_create_dumb *args); int msm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev, diff --git a/drivers/gpu/drm/msm/msm_gem_prime.c b/drivers/gpu/drm/msm/msm_gem_prime.c index dcc8a573bc76..c1d91863df05 100644 --- a/drivers/gpu/drm/msm/msm_gem_prime.c +++ b/drivers/gpu/drm/msm/msm_gem_prime.c @@ -63,12 +63,12 @@ struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev, int msm_gem_prime_pin(struct drm_gem_object *obj) { if (!obj->import_attach) - msm_gem_get_pages(obj); + msm_gem_pin_pages(obj); return 0; } void msm_gem_prime_unpin(struct drm_gem_object *obj) { if (!obj->import_attach) - msm_gem_put_pages(obj); + msm_gem_unpin_pages(obj); } From patchwork Tue Jul 19 17:18:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D841CCA485 for ; Tue, 19 Jul 2022 17:19:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238780AbiGSRTW (ORCPT ); Tue, 19 Jul 2022 13:19:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238922AbiGSRTG (ORCPT ); Tue, 19 Jul 2022 13:19:06 -0400 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C583A57275; Tue, 19 Jul 2022 10:18:57 -0700 (PDT) Received: by mail-pf1-x42b.google.com with SMTP id c3so13255835pfb.13; Tue, 19 Jul 2022 10:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UVam+OtTXmVRJOOFcNCglGVq9lRc/LFnzlVeuJ/h90Y=; b=ZBhwD7ytYU8o+a2Bt2oJHeG9MY40h6GSVLZqXzGXTC0FFRkECyQUdR7BCabQaLBaYg 7d0/+Arfy1ACzhn0His7fTdu34dZxSC327lOlWQq8ssDW3XXLRVK/WpBrr98LHXPvw6z SK4isVMaL8HQiH6CB9T3e3aBK5BX0cbpBcDImxm30hhGsSgH24tVRlpqQFqOFFLBoA0/ hDchYdRDp5FC/mEwqjiWoEBUv83V+UHPKRsoihuk010s54MtfIITOBBLcXl3n8Wwhwyg 9pKmGGmB5biVLhfrOKUfj4M8ZqAJBxSnynwmbr2hCUQVaAW3lJmL6hymNjcX8ukUvPly ceHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UVam+OtTXmVRJOOFcNCglGVq9lRc/LFnzlVeuJ/h90Y=; b=kI6o2Q1qQeA3/Ji3I0IrWSHcBfbgDX2nVAbRRFuP4QTfvh8xC3S7YHmwgj/J2xKvdo 5J+GPbfgSLkqSEXiK81m7SHmQ5y9iPnvUKCk34sTfjlchEgolyD7D7POoCtpB9SbHsTt +7MxS3lab8x3dUw7uygM6EWx7QvZJ2iiCoK94RN7howchyOtLekOQvqKwkzbEquCe0ck N+L578RxEe/OqN/DzmgDdJVCvwRephN8YnIeBxN2SyFZYhw40X9tjdi0aaXI4hREWVeq d9wr0TzxGbic/uajVLf2bZdqdDCaoWEHD46RXlQ/tIMHN3azPKofRyWVP/rS1wxC4HGB EAuQ== X-Gm-Message-State: AJIora+AQTO8/NLA9tAV5ESxJRv3q/tO70tWQ7J6e/JcSFX8VMUx+nLb EQL/ACronZAzStIiQOi3SkhCOalBz/4= X-Google-Smtp-Source: AGRyM1ujYcxPDjQPBqvE6HN7KMwZ0j+SwqlhLBBk4FEbT8ggh686jbNbt4OEn2htkgDny2yEA6sthw== X-Received: by 2002:aa7:961d:0:b0:528:ce2b:d9ce with SMTP id q29-20020aa7961d000000b00528ce2bd9cemr35016532pfg.83.1658251137144; Tue, 19 Jul 2022 10:18:57 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id x23-20020a1709027c1700b001696751796asm11896202pll.139.2022.07.19.10.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:56 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 07/13] drm/msm/gem: Consolidate pin/unpin paths Date: Tue, 19 Jul 2022 10:18:51 -0700 Message-Id: <20220719171900.289265-8-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Avoid having multiple spots where we increment/decrement pin_count (and associated LRU updating) Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 3da64c7f65a2..407b18a24dc4 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -190,7 +190,7 @@ static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) p = get_pages(obj); if (!IS_ERR(p)) { - msm_obj->pin_count++; + to_msm_bo(obj)->pin_count++; update_lru(obj); } @@ -213,9 +213,7 @@ void msm_gem_unpin_pages(struct drm_gem_object *obj) struct msm_gem_object *msm_obj = to_msm_bo(obj); msm_gem_lock(obj); - msm_obj->pin_count--; - GEM_WARN_ON(msm_obj->pin_count < 0); - update_lru(obj); + msm_gem_unpin_locked(obj); msm_gem_unlock(obj); } @@ -436,14 +434,13 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma) if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) return -EBUSY; - pages = get_pages(obj); + pages = msm_gem_pin_pages_locked(obj); if (IS_ERR(pages)) return PTR_ERR(pages); ret = msm_gem_map_vma(vma->aspace, vma, prot, msm_obj->sgt, obj->size); - - if (!ret) - msm_obj->pin_count++; + if (ret) + msm_gem_unpin_locked(obj); return ret; } From patchwork Tue Jul 19 17:18:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E765C43334 for ; Tue, 19 Jul 2022 17:19:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238690AbiGSRTU (ORCPT ); Tue, 19 Jul 2022 13:19:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238971AbiGSRTH (ORCPT ); Tue, 19 Jul 2022 13:19:07 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B80495A2DC; Tue, 19 Jul 2022 10:19:00 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id f11so14075978pgj.7; Tue, 19 Jul 2022 10:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0Nc8381KvCAKtcJgWAO7GB/O/Xk1BB4fgkOydTh0/AI=; b=gt2tQwC5bzf/qcbScBJr6Zme3YEkkMDqsRBB+9qiT87SxE5UpI/cw4fDADwqQLzBCp QQlOUoFGo4A3nKbM38qu0s02N2bd+nKkcd1Iv5dkuSnaMyT+syetA474yDliMMIqDsC6 l6cNNIxg1cJ+F/sz8VlS6et/UrMCF+KuTiMvh5s9yZHhv11pobWJBDygnVsHEi47RD8y 3OxBGnlb5rSUt4tCSyJG2v8wkrz6FUsLubi1rSD1yC/Q1F6voRsJwmpa5bUsRT0x/H9T 4deUrPoyKP3XbihbYYAMq2jqpAPPJOZMLbEeBjMf7Tch/fFt6PEQ68/51yQRhRZvlMor cndg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Nc8381KvCAKtcJgWAO7GB/O/Xk1BB4fgkOydTh0/AI=; b=iJWu6m34wDkrJa75GWJqqHqL6EQRF1bOIwnlal4H7EIhd26ay8yRwxaILIy/IOe9Vt M+sDaxoJCZr1EzUp0s9OcQoXune3otZKjFbAO8PMty+Wug7WufOqYVjnvr17bu6Haj86 Jg4jpWUWtOthJIHVrMnPKomVyu4kV7awQHlRcuvZC0/8q/S3DwvmsAtyHytS8kwlzV4j Jh6CxsngtCYgVIDgS1hoZkbaQYDmz14O5gqX3CCEE3qvu/8QSleViftE+Ki/MzmfR+x2 7luVJ4QhrJJwL454sv8UJhNcRUxy1J5Kr3JGFHtTzS3Johcw8Qwgzk5zOj0Jn/mUvgS+ +kKQ== X-Gm-Message-State: AJIora9tg3UVKcJY9xFNAgY2PcR4bE9XRZEyRNFhC7agYG/T1sSkh2S7 xPIyyaPHUFdG2etygXlchmw= X-Google-Smtp-Source: AGRyM1skM6TaEVXDz/2XoGPGh1uhTOwfqQaMiefDkdOg1gRJTJTybhtVV3/vZmJoSbyP/Jot8vJ1+g== X-Received: by 2002:a63:8741:0:b0:419:f886:4f9d with SMTP id i62-20020a638741000000b00419f8864f9dmr16938377pge.612.1658251139946; Tue, 19 Jul 2022 10:18:59 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id w20-20020a627b14000000b0052ab912b0fasm11734482pfc.2.2022.07.19.10.18.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:18:59 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 08/13] drm/msm/gem: Remove active refcnt Date: Tue, 19 Jul 2022 10:18:52 -0700 Message-Id: <20220719171900.289265-9-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark At this point the pinned refcnt is sufficient, and the shrinker is already prepared to encounter objects which are still active according to fences attached to the resv. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 45 ++-------------------------- drivers/gpu/drm/msm/msm_gem.h | 14 ++------- drivers/gpu/drm/msm/msm_gem_submit.c | 22 ++------------ 3 files changed, 8 insertions(+), 73 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 407b18a24dc4..209438744bab 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -734,8 +734,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) /* If the obj is inactive, we might need to move it * between inactive lists */ - if (msm_obj->active_count == 0) - update_lru(obj); + update_lru(obj); msm_gem_unlock(obj); @@ -788,7 +787,6 @@ void msm_gem_evict(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); GEM_WARN_ON(is_unevictable(msm_obj)); GEM_WARN_ON(!msm_obj->evictable); - GEM_WARN_ON(msm_obj->active_count); /* Get rid of any iommu mapping(s): */ put_iova_spaces(obj, false); @@ -813,37 +811,6 @@ void msm_gem_vunmap(struct drm_gem_object *obj) msm_obj->vaddr = NULL; } -void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu) -{ - struct msm_gem_object *msm_obj = to_msm_bo(obj); - struct msm_drm_private *priv = obj->dev->dev_private; - - might_sleep(); - GEM_WARN_ON(!msm_gem_is_locked(obj)); - GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED); - GEM_WARN_ON(msm_obj->dontneed); - - if (msm_obj->active_count++ == 0) { - mutex_lock(&priv->mm_lock); - if (msm_obj->evictable) - mark_unevictable(msm_obj); - list_move_tail(&msm_obj->mm_list, &gpu->active_list); - mutex_unlock(&priv->mm_lock); - } -} - -void msm_gem_active_put(struct drm_gem_object *obj) -{ - struct msm_gem_object *msm_obj = to_msm_bo(obj); - - might_sleep(); - GEM_WARN_ON(!msm_gem_is_locked(obj)); - - if (--msm_obj->active_count == 0) { - update_lru(obj); - } -} - static void update_lru(struct drm_gem_object *obj) { struct msm_drm_private *priv = obj->dev->dev_private; @@ -851,9 +818,6 @@ static void update_lru(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - if (msm_obj->active_count != 0) - return; - mutex_lock(&priv->mm_lock); if (msm_obj->dontneed) @@ -926,7 +890,7 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, stats->all.count++; stats->all.size += obj->size; - if (is_active(msm_obj)) { + if (msm_gem_active(obj)) { stats->active.count++; stats->active.size += obj->size; } @@ -954,7 +918,7 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, } seq_printf(m, "%08x: %c %2d (%2d) %08llx %p", - msm_obj->flags, is_active(msm_obj) ? 'A' : 'I', + msm_obj->flags, msm_gem_active(obj) ? 'A' : 'I', obj->name, kref_read(&obj->refcount), off, msm_obj->vaddr); @@ -1037,9 +1001,6 @@ static void msm_gem_free_object(struct drm_gem_object *obj) list_del(&msm_obj->mm_list); mutex_unlock(&priv->mm_lock); - /* object should not be on active list: */ - GEM_WARN_ON(is_active(msm_obj)); - put_iova_spaces(obj, true); if (obj->import_attach) { diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 6fe521ccda45..420ba49bf21a 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -138,7 +138,6 @@ struct msm_gem_object { char name[32]; /* Identifier to print for the debugfs files */ - int active_count; int pin_count; }; #define to_msm_bo(x) container_of(x, struct msm_gem_object, base) @@ -171,8 +170,6 @@ void *msm_gem_get_vaddr_active(struct drm_gem_object *obj); void msm_gem_put_vaddr_locked(struct drm_gem_object *obj); void msm_gem_put_vaddr(struct drm_gem_object *obj); int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv); -void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu); -void msm_gem_active_put(struct drm_gem_object *obj); bool msm_gem_active(struct drm_gem_object *obj); int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout); int msm_gem_cpu_fini(struct drm_gem_object *obj); @@ -245,12 +242,6 @@ msm_gem_is_locked(struct drm_gem_object *obj) return dma_resv_is_locked(obj->resv) || (kref_read(&obj->refcount) == 0); } -static inline bool is_active(struct msm_gem_object *msm_obj) -{ - GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - return msm_obj->active_count; -} - /* imported/exported objects are not purgeable: */ static inline bool is_unpurgeable(struct msm_gem_object *msm_obj) { @@ -391,9 +382,8 @@ struct msm_gem_submit { /* make sure these don't conflict w/ MSM_SUBMIT_BO_x */ #define BO_VALID 0x8000 /* is current addr in cmdstream correct/valid? */ #define BO_LOCKED 0x4000 /* obj lock is held */ -#define BO_ACTIVE 0x2000 /* active refcnt is held */ -#define BO_OBJ_PINNED 0x1000 /* obj (pages) is pinned and on active list */ -#define BO_VMA_PINNED 0x0800 /* vma (virtual address) is pinned */ +#define BO_OBJ_PINNED 0x2000 /* obj (pages) is pinned and on active list */ +#define BO_VMA_PINNED 0x1000 /* vma (virtual address) is pinned */ uint32_t flags; union { struct msm_gem_object *obj; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 16c662808522..adf358fb8e9d 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -243,17 +243,13 @@ static void submit_cleanup_bo(struct msm_gem_submit *submit, int i, if (flags & BO_OBJ_PINNED) msm_gem_unpin_locked(obj); - if (flags & BO_ACTIVE) - msm_gem_active_put(obj); - if (flags & BO_LOCKED) dma_resv_unlock(obj->resv); } static void submit_unlock_unpin_bo(struct msm_gem_submit *submit, int i) { - unsigned cleanup_flags = BO_VMA_PINNED | BO_OBJ_PINNED | - BO_ACTIVE | BO_LOCKED; + unsigned cleanup_flags = BO_VMA_PINNED | BO_OBJ_PINNED | BO_LOCKED; submit_cleanup_bo(submit, i, cleanup_flags); if (!(submit->bos[i].flags & BO_VALID)) @@ -358,18 +354,6 @@ static int submit_pin_objects(struct msm_gem_submit *submit) submit->valid = true; - /* - * Increment active_count first, so if under memory pressure, we - * don't inadvertently evict a bo needed by the submit in order - * to pin an earlier bo in the same submit. - */ - for (i = 0; i < submit->nr_bos; i++) { - struct drm_gem_object *obj = &submit->bos[i].obj->base; - - msm_gem_active_get(obj, submit->gpu); - submit->bos[i].flags |= BO_ACTIVE; - } - for (i = 0; i < submit->nr_bos; i++) { struct drm_gem_object *obj = &submit->bos[i].obj->base; struct msm_gem_vma *vma; @@ -521,7 +505,7 @@ static void submit_cleanup(struct msm_gem_submit *submit, bool error) unsigned i; if (error) - cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED | BO_ACTIVE; + cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED; for (i = 0; i < submit->nr_bos; i++) { struct msm_gem_object *msm_obj = submit->bos[i].obj; @@ -540,7 +524,7 @@ void msm_submit_retire(struct msm_gem_submit *submit) msm_gem_lock(obj); /* Note, VMA already fence-unpinned before submit: */ - submit_cleanup_bo(submit, i, BO_OBJ_PINNED | BO_ACTIVE); + submit_cleanup_bo(submit, i, BO_OBJ_PINNED); msm_gem_unlock(obj); drm_gem_object_put(obj); } From patchwork Tue Jul 19 17:18:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8764C433EF for ; Tue, 19 Jul 2022 17:25:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234779AbiGSRZL (ORCPT ); Tue, 19 Jul 2022 13:25:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239028AbiGSRTI (ORCPT ); Tue, 19 Jul 2022 13:19:08 -0400 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 542395A449; Tue, 19 Jul 2022 10:19:03 -0700 (PDT) Received: by mail-pf1-x42b.google.com with SMTP id e16so14151796pfm.11; Tue, 19 Jul 2022 10:19:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AeMYgqPNjsyamzteMz4JJUbia+3HcpDgXI4+3iGYjqA=; b=KrduwNUj06YxyiDX8jfp0iyYX8UquvesSQ6YvQjU13SatKc9yZvpGP5UupllavDK3U spj9bptWkVYl7o/d+ewOb8nbBdozp7VoxhF04v3nZ6xW+A+vAn95zjq2p4gygfq0l2BN O/odBB+vr8kW44AlPOKWuHHLb7O4+PSxca6zP1nSp9bHHwpgdqe5cMTj+qZfAT/4plsG nriN23TFjGG3+z+uplYebmVefFJTKHtC/kfu/zMzdgqKcYsUaQpui8FQqHF63X+dpanr bzkc2vgD7UjPqS2fgOMH4CXr2Hw7KqZ87Y0NSgIPgwnR4u8i9gxyeJN4fPJv5h64rt68 SC5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AeMYgqPNjsyamzteMz4JJUbia+3HcpDgXI4+3iGYjqA=; b=XVW7v4i4q3T9C3Wv9FuEREC/iHajZneyWiYVdSc0EIA7WpBS8lm46LJv2L97M5jRVE IzZf5OTiVhJ4MyyYIUx7kA0tbtaeDefiPedVG/vCrkEUAFVn6dDlaZkICG5/V0PD44Zo 27KODvdY5uHAf4JcUWQ3xUI1D+3VDsD3z8wnXax+dhhwIrMHs0PLWxcqhwc9Vz825Qpw qpfCOGEWhN9pDsN8gH5zlaK763z/0AxUySH4HKlPI1hnWaLq9Guq1oIwTkcfFgjpA4rW DzLJw0KVcrj85igUd+C9kaxfMr7o+zRsU+rnqx69gePGIXmBowJCOebxn+tnT0NdTVEQ vXTw== X-Gm-Message-State: AJIora+mAI4IcqBfhNGjcwbnjXnmpK+BFmkwGBWjl/r+I++s+HauM1lZ EvsUgpdCcnPdEP2Js3iGqhFbjQHn0fg= X-Google-Smtp-Source: AGRyM1viV+awj6YiX4bbXnqfWo4P1A9SySSXXs1nfmQszF9VewVJ+XloRh3enPgiWaRphLp6+9beWg== X-Received: by 2002:a62:140e:0:b0:52b:780d:fb9d with SMTP id 14-20020a62140e000000b0052b780dfb9dmr9413148pfu.65.1658251142447; Tue, 19 Jul 2022 10:19:02 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id x186-20020a6263c3000000b0052b798ff60esm2803601pfb.177.2022.07.19.10.19.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:19:01 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Daniel Vetter , Thomas Zimmermann , Dmitry Osipenko , Maarten Lankhorst , Maxime Ripard , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 09/13] drm/gem: Add LRU/shrinker helper Date: Tue, 19 Jul 2022 10:18:53 -0700 Message-Id: <20220719171900.289265-10-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Add a simple LRU helper to assist with driver's shrinker implementation. It handles tracking the number of backing pages associated with a given LRU, and provides a helper to implement shrinker_scan. A driver can use multiple LRU instances to track objects in various states, for example a dontneed LRU for purgeable objects, a willneed LRU for evictable objects, and an unpinned LRU for objects without backing pages. All LRUs that the object can be moved between must share a single lock. Cc: Daniel Vetter Cc: Thomas Zimmermann Cc: Dmitry Osipenko Signed-off-by: Rob Clark Reviewed-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem.c | 183 ++++++++++++++++++++++++++++++++++++++ include/drm/drm_gem.h | 56 ++++++++++++ 2 files changed, 239 insertions(+) diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c index eb0c2d041f13..684db28cc71c 100644 --- a/drivers/gpu/drm/drm_gem.c +++ b/drivers/gpu/drm/drm_gem.c @@ -165,6 +165,7 @@ void drm_gem_private_object_init(struct drm_device *dev, obj->resv = &obj->_resv; drm_vma_node_reset(&obj->vma_node); + INIT_LIST_HEAD(&obj->lru_node); } EXPORT_SYMBOL(drm_gem_private_object_init); @@ -951,6 +952,7 @@ drm_gem_object_release(struct drm_gem_object *obj) dma_resv_fini(&obj->_resv); drm_gem_free_mmap_offset(obj); + drm_gem_lru_remove(obj); } EXPORT_SYMBOL(drm_gem_object_release); @@ -1274,3 +1276,184 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count, ww_acquire_fini(acquire_ctx); } EXPORT_SYMBOL(drm_gem_unlock_reservations); + +/** + * drm_gem_lru_init - initialize a LRU + * + * @lru: The LRU to initialize + * @lock: The lock protecting the LRU + */ +void +drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock) +{ + lru->lock = lock; + lru->count = 0; + INIT_LIST_HEAD(&lru->list); +} +EXPORT_SYMBOL(drm_gem_lru_init); + +static void +lru_remove(struct drm_gem_object *obj) +{ + obj->lru->count -= obj->size >> PAGE_SHIFT; + WARN_ON(obj->lru->count < 0); + list_del(&obj->lru_node); + obj->lru = NULL; +} + +/** + * drm_gem_lru_remove - remove object from whatever LRU it is in + * + * If the object is currently in any LRU, remove it. + * + * @obj: The GEM object to remove from current LRU + */ +void +drm_gem_lru_remove(struct drm_gem_object *obj) +{ + struct drm_gem_lru *lru = obj->lru; + + if (!lru) + return; + + mutex_lock(lru->lock); + lru_remove(obj); + mutex_unlock(lru->lock); +} +EXPORT_SYMBOL(drm_gem_lru_remove); + +/** + * drm_gem_lru_move_tail - move the object to the tail of the LRU + * + * If the object is already in this LRU it will be moved to the + * tail. Otherwise it will be removed from whichever other LRU + * it is in (if any) and moved into this LRU. + * + * @lru: The LRU to move the object into. + * @obj: The GEM object to move into this LRU + */ +void +drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj) +{ + mutex_lock(lru->lock); + drm_gem_lru_move_tail_locked(lru, obj); + mutex_unlock(lru->lock); +} +EXPORT_SYMBOL(drm_gem_lru_move_tail); + +/** + * drm_gem_lru_move_tail_locked - move the object to the tail of the LRU + * + * If the object is already in this LRU it will be moved to the + * tail. Otherwise it will be removed from whichever other LRU + * it is in (if any) and moved into this LRU. + * + * Call with LRU lock held. + * + * @lru: The LRU to move the object into. + * @obj: The GEM object to move into this LRU + */ +void +drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj) +{ + WARN_ON(!mutex_is_locked(lru->lock)); + + if (obj->lru) + lru_remove(obj); + + lru->count += obj->size >> PAGE_SHIFT; + list_add_tail(&obj->lru_node, &lru->list); + obj->lru = lru; +} +EXPORT_SYMBOL(drm_gem_lru_move_tail_locked); + +/** + * drm_gem_lru_scan - helper to implement shrinker.scan_objects + * + * If the shrink callback succeeds, it is expected that the driver + * move the object out of this LRU. + * + * If the LRU possibly contain active buffers, it is the responsibility + * of the shrink callback to check for this (ie. dma_resv_test_signaled()) + * or if necessary block until the buffer becomes idle. + * + * @lru: The LRU to scan + * @nr_to_scan: The number of pages to try to reclaim + * @shrink: Callback to try to shrink/reclaim the object. + */ +unsigned long +drm_gem_lru_scan(struct drm_gem_lru *lru, unsigned nr_to_scan, + bool (*shrink)(struct drm_gem_object *obj)) +{ + struct drm_gem_lru still_in_lru; + struct drm_gem_object *obj; + unsigned freed = 0; + + drm_gem_lru_init(&still_in_lru, lru->lock); + + mutex_lock(lru->lock); + + while (freed < nr_to_scan) { + obj = list_first_entry_or_null(&lru->list, typeof(*obj), lru_node); + + if (!obj) + break; + + drm_gem_lru_move_tail_locked(&still_in_lru, obj); + + /* + * If it's in the process of being freed, gem_object->free() + * may be blocked on lock waiting to remove it. So just + * skip it. + */ + if (!kref_get_unless_zero(&obj->refcount)) + continue; + + /* + * Now that we own a reference, we can drop the lock for the + * rest of the loop body, to reduce contention with other + * code paths that need the LRU lock + */ + mutex_unlock(lru->lock); + + /* + * Note that this still needs to be trylock, since we can + * hit shrinker in response to trying to get backing pages + * for this obj (ie. while it's lock is already held) + */ + if (!dma_resv_trylock(obj->resv)) + goto tail; + + if (shrink(obj)) { + freed += obj->size >> PAGE_SHIFT; + + /* + * If we succeeded in releasing the object's backing + * pages, we expect the driver to have moved the object + * out of this LRU + */ + WARN_ON(obj->lru == &still_in_lru); + WARN_ON(obj->lru == lru); + } + + dma_resv_unlock(obj->resv); + +tail: + drm_gem_object_put(obj); + mutex_lock(lru->lock); + } + + /* + * Move objects we've skipped over out of the temporary still_in_lru + * back into this LRU + */ + list_for_each_entry (obj, &still_in_lru.list, lru_node) + obj->lru = lru; + list_splice_tail(&still_in_lru.list, &lru->list); + lru->count += still_in_lru.count; + + mutex_unlock(lru->lock); + + return freed; +} +EXPORT_SYMBOL(drm_gem_lru_scan); diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h index 87cffc9efa85..f13a9080af37 100644 --- a/include/drm/drm_gem.h +++ b/include/drm/drm_gem.h @@ -174,6 +174,41 @@ struct drm_gem_object_funcs { const struct vm_operations_struct *vm_ops; }; +/** + * struct drm_gem_lru - A simple LRU helper + * + * A helper for tracking GEM objects in a given state, to aid in + * driver's shrinker implementation. Tracks the count of pages + * for lockless &shrinker.count_objects, and provides + * &drm_gem_lru_scan for driver's &shrinker.scan_objects + * implementation. + */ +struct drm_gem_lru { + /** + * @lock: + * + * Lock protecting movement of GEM objects between LRUs. All + * LRUs that the object can move between should be protected + * by the same lock. + */ + struct mutex *lock; + + /** + * @count: + * + * The total number of backing pages of the GEM objects in + * this LRU. + */ + long count; + + /** + * @list: + * + * The LRU list. + */ + struct list_head list; +}; + /** * struct drm_gem_object - GEM buffer object * @@ -312,6 +347,20 @@ struct drm_gem_object { * */ const struct drm_gem_object_funcs *funcs; + + /** + * @lru_node: + * + * List node in a &drm_gem_lru. + */ + struct list_head lru_node; + + /** + * @lru: + * + * The current LRU list that the GEM object is on. + */ + struct drm_gem_lru *lru; }; /** @@ -420,4 +469,11 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count, int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev, u32 handle, u64 *offset); +void drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock); +void drm_gem_lru_remove(struct drm_gem_object *obj); +void drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj); +void drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj); +unsigned long drm_gem_lru_scan(struct drm_gem_lru *lru, unsigned nr_to_scan, + bool (*shrink)(struct drm_gem_object *obj)); + #endif /* __DRM_GEM_H__ */ From patchwork Tue Jul 19 17:18:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922810 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F11BC43334 for ; Tue, 19 Jul 2022 17:25:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238995AbiGSRZM (ORCPT ); Tue, 19 Jul 2022 13:25:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239222AbiGSRTN (ORCPT ); Tue, 19 Jul 2022 13:19:13 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69D935A14B; Tue, 19 Jul 2022 10:19:06 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id q5so12553475plr.11; Tue, 19 Jul 2022 10:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zh/8/t0ZilHENXdeKCjYIuWBV3SQcdEuA5JHSYImukA=; b=ncclU9QfEOJ6JtEkY5yqZvvRfsk5PPOK+e0A+DXFO1yVloDfkIrTZocupiBweXbg14 CbYzw5GrjqjtWEgAqnmtGk0GOfZ/HRLfDyn15raynPfLOzvQ8E6wuOrvSV8xqU47/Je2 R143QpuyaB3c3vRqHt1txS7T6xVUNba3ocjrn5Ji9eLaOldk4VR9nycbKCbVXr6n75jX HuIsTbbX38KlAoPZE7m/HgpaaVCUH7hQHELmbaCVii1Xp7kAWYA7CxGgdE95LUI+xbKb zL4w/zIFerbWLhGVMfAd6r/rTFnEMvPmnYLrDPn6v6UMirk8jhKRfHyn61g2bwPLDecG oprg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zh/8/t0ZilHENXdeKCjYIuWBV3SQcdEuA5JHSYImukA=; b=N4cFKzQezVxDcHE7EXJjrlMiCJlpDkCC300V1Dm3m3hjWHOnkA2yN7SiBHCgw6nS1W rNbWOoXDws4Wf1C5T6YlYTVP8MITGZqm7sqQGx8wP0/0fWKrx3p+IWo4oawbiGpSULNq ThNmktp1ch4mVo23rsh4EYyiVHV47PK/pBB4QO8CRwH0OsAp3C3aBdu8Pmil0INUa9A4 5lLD2XJeeWHXioG0XrNfMsrdHdSIOFncj815bm2wnCvlwahxuzxpVlGZ0D/RXYMnryjT Wq0FkEUlcJy9SV+SZe4bRJ0a4laaNUkWgYeRm+Cjpz43Fjg3zso0BdQ4TcVUOhlvzK7r nfGQ== X-Gm-Message-State: AJIora8kKXAdk7MMPeXmo+iCihrHnkQZfFJQi2JH1xhEN5lTq3X6k7c2 2IGE9eIEEeb5Iu1J4BZaRzY= X-Google-Smtp-Source: AGRyM1uHQ/fKVGFG2PDXLwwL1uOu1ZsyNMpbUJdEQI4e6m66a3QYFp7oPL0bIsWIQuVHeFi/t2EM2g== X-Received: by 2002:a17:902:e885:b0:16c:4a26:3895 with SMTP id w5-20020a170902e88500b0016c4a263895mr36132211plg.131.1658251145510; Tue, 19 Jul 2022 10:19:05 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id x14-20020aa79ace000000b0052ad49292f0sm11661673pfp.48.2022.07.19.10.19.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:19:04 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 10/13] drm/msm/gem: Convert to using drm_gem_lru Date: Tue, 19 Jul 2022 10:18:54 -0700 Message-Id: <20220719171900.289265-11-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark This converts over to use the shared GEM LRU/shrinker helpers. Note that it means we are no longer tracking purgeable or willneed buffers that are active separately. But the most recently pinned buffers should be at the tail of the various LRUs, and the shrinker is already prepared to encounter objects which are still active. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_drv.c | 14 +-- drivers/gpu/drm/msm/msm_drv.h | 70 +++++++++++---- drivers/gpu/drm/msm/msm_gem.c | 58 ++++-------- drivers/gpu/drm/msm/msm_gem.h | 93 -------------------- drivers/gpu/drm/msm/msm_gem_shrinker.c | 117 ++++++------------------- drivers/gpu/drm/msm/msm_gpu.c | 3 - drivers/gpu/drm/msm/msm_gpu.h | 6 -- 7 files changed, 104 insertions(+), 257 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index d7ca025457b6..1ca4a92ba96e 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -418,14 +418,18 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv) INIT_LIST_HEAD(&priv->objects); mutex_init(&priv->obj_lock); - INIT_LIST_HEAD(&priv->inactive_willneed); - INIT_LIST_HEAD(&priv->inactive_dontneed); - INIT_LIST_HEAD(&priv->inactive_unpinned); - mutex_init(&priv->mm_lock); + /* + * Initialize the LRUs: + */ + mutex_init(&priv->lru.lock); + drm_gem_lru_init(&priv->lru.unbacked, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.pinned, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.willneed, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.dontneed, &priv->lru.lock); /* Teach lockdep about lock ordering wrt. shrinker: */ fs_reclaim_acquire(GFP_KERNEL); - might_lock(&priv->mm_lock); + might_lock(&priv->lru.lock); fs_reclaim_release(GFP_KERNEL); drm_mode_config_init(ddev); diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index b3689a2d27d7..208ae5bc5e6b 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -142,28 +142,60 @@ struct msm_drm_private { struct mutex obj_lock; /** - * LRUs of inactive GEM objects. Every bo is either in one of the - * inactive lists (depending on whether or not it is shrinkable) or - * gpu->active_list (for the gpu it is active on[1]), or transiently - * on a temporary list as the shrinker is running. + * lru: * - * Note that inactive_willneed also contains pinned and vmap'd bos, - * but the number of pinned-but-not-active objects is small (scanout - * buffers, ringbuffer, etc). + * The various LRU's that a GEM object is in at various stages of + * it's lifetime. Objects start out in the unbacked LRU. When + * pinned (for scannout or permanently mapped GPU buffers, like + * ringbuffer, memptr, fw, etc) it moves to the pinned LRU. When + * unpinned, it moves into willneed or dontneed LRU depending on + * madvise state. When backing pages are evicted (willneed) or + * purged (dontneed) it moves back into the unbacked LRU. * - * These lists are protected by mm_lock (which should be acquired - * before per GEM object lock). One should *not* hold mm_lock in - * get_pages()/vmap()/etc paths, as they can trigger the shrinker. - * - * [1] if someone ever added support for the old 2d cores, there could be - * more than one gpu object + * The dontneed LRU is considered by the shrinker for objects + * that are candidate for purging, and the willneed LRU is + * considered for objects that could be evicted. */ - struct list_head inactive_willneed; /* inactive + potentially unpin/evictable */ - struct list_head inactive_dontneed; /* inactive + shrinkable */ - struct list_head inactive_unpinned; /* inactive + purged or unpinned */ - long shrinkable_count; /* write access under mm_lock */ - long evictable_count; /* write access under mm_lock */ - struct mutex mm_lock; + struct { + /** + * unbacked: + * + * The LRU for GEM objects without backing pages allocated. + * This mostly exists so that objects are always is one + * LRU. + */ + struct drm_gem_lru unbacked; + + /** + * pinned: + * + * The LRU for pinned GEM objects + */ + struct drm_gem_lru pinned; + + /** + * willneed: + * + * The LRU for unpinned GEM objects which are in madvise + * WILLNEED state (ie. can be evicted) + */ + struct drm_gem_lru willneed; + + /** + * dontneed: + * + * The LRU for unpinned GEM objects which are in madvise + * DONTNEED state (ie. can be purged) + */ + struct drm_gem_lru dontneed; + + /** + * lock: + * + * Protects manipulation of all of the LRUs. + */ + struct mutex lock; + } lru; struct workqueue_struct *wq; diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 209438744bab..d4e8af46f4ef 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -174,6 +174,7 @@ static void put_pages(struct drm_gem_object *obj) put_pages_vram(obj); msm_obj->pages = NULL; + update_lru(obj); } } @@ -210,8 +211,6 @@ struct page **msm_gem_pin_pages(struct drm_gem_object *obj) void msm_gem_unpin_pages(struct drm_gem_object *obj) { - struct msm_gem_object *msm_obj = to_msm_bo(obj); - msm_gem_lock(obj); msm_gem_unpin_locked(obj); msm_gem_unlock(obj); @@ -761,7 +760,6 @@ void msm_gem_purge(struct drm_gem_object *obj) put_iova_vmas(obj); msm_obj->madv = __MSM_MADV_PURGED; - update_lru(obj); drm_gem_free_mmap_offset(obj); @@ -786,7 +784,6 @@ void msm_gem_evict(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); GEM_WARN_ON(is_unevictable(msm_obj)); - GEM_WARN_ON(!msm_obj->evictable); /* Get rid of any iommu mapping(s): */ put_iova_spaces(obj, false); @@ -794,8 +791,6 @@ void msm_gem_evict(struct drm_gem_object *obj) drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping); put_pages(obj); - - update_lru(obj); } void msm_gem_vunmap(struct drm_gem_object *obj) @@ -818,26 +813,20 @@ static void update_lru(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - mutex_lock(&priv->mm_lock); - - if (msm_obj->dontneed) - mark_unpurgeable(msm_obj); - if (msm_obj->evictable) - mark_unevictable(msm_obj); - - list_del(&msm_obj->mm_list); - if ((msm_obj->madv == MSM_MADV_WILLNEED) && msm_obj->sgt) { - list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed); - mark_evictable(msm_obj); - } else if (msm_obj->madv == MSM_MADV_DONTNEED) { - list_add_tail(&msm_obj->mm_list, &priv->inactive_dontneed); - mark_purgeable(msm_obj); + if (!msm_obj->pages) { + GEM_WARN_ON(msm_obj->pin_count); + GEM_WARN_ON(msm_obj->vmap_count); + + drm_gem_lru_move_tail(&priv->lru.unbacked, obj); + } else if (msm_obj->pin_count || msm_obj->vmap_count) { + drm_gem_lru_move_tail(&priv->lru.pinned, obj); + } else if (msm_obj->madv == MSM_MADV_WILLNEED) { + drm_gem_lru_move_tail(&priv->lru.willneed, obj); } else { - GEM_WARN_ON((msm_obj->madv != __MSM_MADV_PURGED) && msm_obj->sgt); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - } + GEM_WARN_ON(msm_obj->madv != MSM_MADV_DONTNEED); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.dontneed, obj); + } } bool msm_gem_active(struct drm_gem_object *obj) @@ -995,12 +984,6 @@ static void msm_gem_free_object(struct drm_gem_object *obj) list_del(&msm_obj->node); mutex_unlock(&priv->obj_lock); - mutex_lock(&priv->mm_lock); - if (msm_obj->dontneed) - mark_unpurgeable(msm_obj); - list_del(&msm_obj->mm_list); - mutex_unlock(&priv->mm_lock); - put_iova_spaces(obj, true); if (obj->import_attach) { @@ -1160,13 +1143,6 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32 to_msm_bo(obj)->vram_node = &vma->node; - /* Call chain get_pages() -> update_inactive() tries to - * access msm_obj->mm_list, but it is not initialized yet. - * To avoid NULL pointer dereference error, initialize - * mm_list to be empty. - */ - INIT_LIST_HEAD(&msm_obj->mm_list); - msm_gem_lock(obj); pages = get_pages(obj); msm_gem_unlock(obj); @@ -1189,9 +1165,7 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32 mapping_set_gfp_mask(obj->filp->f_mapping, GFP_HIGHUSER); } - mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.unbacked, obj); mutex_lock(&priv->obj_lock); list_add_tail(&msm_obj->node, &priv->objects); @@ -1247,9 +1221,7 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev, msm_gem_unlock(obj); - mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.pinned, obj); mutex_lock(&priv->obj_lock); list_add_tail(&msm_obj->node, &priv->objects); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 420ba49bf21a..0403b27ff779 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -93,16 +93,6 @@ struct msm_gem_object { */ uint8_t madv; - /** - * Is object on inactive_dontneed list (ie. counted in priv->shrinkable_count)? - */ - bool dontneed : 1; - - /** - * Is object evictable (ie. counted in priv->evictable_count)? - */ - bool evictable : 1; - /** * count of active vmap'ing */ @@ -114,17 +104,6 @@ struct msm_gem_object { */ struct list_head node; - /** - * An object is either: - * inactive - on priv->inactive_dontneed or priv->inactive_willneed - * (depending on purgeability status) - * active - on one one of the gpu's active_list.. well, at - * least for now we don't have (I don't think) hw sync between - * 2d and 3d one devices which have both, meaning we need to - * block on submit if a bo is already on other ring - */ - struct list_head mm_list; - struct page **pages; struct sg_table *sgt; void *vaddr; @@ -206,12 +185,6 @@ msm_gem_lock(struct drm_gem_object *obj) dma_resv_lock(obj->resv, NULL); } -static inline bool __must_check -msm_gem_trylock(struct drm_gem_object *obj) -{ - return dma_resv_trylock(obj->resv); -} - static inline int msm_gem_lock_interruptible(struct drm_gem_object *obj) { @@ -260,77 +233,11 @@ static inline bool is_vunmapable(struct msm_gem_object *msm_obj) return (msm_obj->vmap_count == 0) && msm_obj->vaddr; } -static inline void mark_purgeable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unpurgeable(msm_obj)) - return; - - if (GEM_WARN_ON(msm_obj->dontneed)) - return; - - priv->shrinkable_count += msm_obj->base.size >> PAGE_SHIFT; - msm_obj->dontneed = true; -} - -static inline void mark_unpurgeable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unpurgeable(msm_obj)) - return; - - if (GEM_WARN_ON(!msm_obj->dontneed)) - return; - - priv->shrinkable_count -= msm_obj->base.size >> PAGE_SHIFT; - GEM_WARN_ON(priv->shrinkable_count < 0); - msm_obj->dontneed = false; -} - static inline bool is_unevictable(struct msm_gem_object *msm_obj) { return is_unpurgeable(msm_obj) || msm_obj->vaddr; } -static inline void mark_evictable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unevictable(msm_obj)) - return; - - if (WARN_ON(msm_obj->evictable)) - return; - - priv->evictable_count += msm_obj->base.size >> PAGE_SHIFT; - msm_obj->evictable = true; -} - -static inline void mark_unevictable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unevictable(msm_obj)) - return; - - if (WARN_ON(!msm_obj->evictable)) - return; - - priv->evictable_count -= msm_obj->base.size >> PAGE_SHIFT; - WARN_ON(priv->evictable_count < 0); - msm_obj->evictable = false; -} - void msm_gem_purge(struct drm_gem_object *obj); void msm_gem_evict(struct drm_gem_object *obj); void msm_gem_vunmap(struct drm_gem_object *obj); diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index ea8ed74982c1..530b1102b46d 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -29,121 +29,61 @@ msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); - unsigned count = priv->shrinkable_count; + unsigned count = priv->lru.dontneed.count; if (can_swap()) - count += priv->evictable_count; + count += priv->lru.willneed.count; return count; } static bool -purge(struct msm_gem_object *msm_obj) +purge(struct drm_gem_object *obj) { - if (!is_purgeable(msm_obj)) + if (!is_purgeable(to_msm_bo(obj))) return false; - if (msm_gem_active(&msm_obj->base)) + if (msm_gem_active(obj)) return false; - /* - * This will move the obj out of still_in_list to - * the purged list - */ - msm_gem_purge(&msm_obj->base); + msm_gem_purge(obj); return true; } static bool -evict(struct msm_gem_object *msm_obj) +evict(struct drm_gem_object *obj) { - if (is_unevictable(msm_obj)) + if (is_unevictable(to_msm_bo(obj))) return false; - if (msm_gem_active(&msm_obj->base)) + if (msm_gem_active(obj)) return false; - msm_gem_evict(&msm_obj->base); + msm_gem_evict(obj); return true; } -static unsigned long -scan(struct msm_drm_private *priv, unsigned nr_to_scan, struct list_head *list, - bool (*shrink)(struct msm_gem_object *msm_obj)) -{ - unsigned freed = 0; - struct list_head still_in_list; - - INIT_LIST_HEAD(&still_in_list); - - mutex_lock(&priv->mm_lock); - - while (freed < nr_to_scan) { - struct msm_gem_object *msm_obj = list_first_entry_or_null( - list, typeof(*msm_obj), mm_list); - - if (!msm_obj) - break; - - list_move_tail(&msm_obj->mm_list, &still_in_list); - - /* - * If it is in the process of being freed, msm_gem_free_object - * can be blocked on mm_lock waiting to remove it. So just - * skip it. - */ - if (!kref_get_unless_zero(&msm_obj->base.refcount)) - continue; - - /* - * Now that we own a reference, we can drop mm_lock for the - * rest of the loop body, to reduce contention with the - * retire_submit path (which could make more objects purgeable) - */ - - mutex_unlock(&priv->mm_lock); - - /* - * Note that this still needs to be trylock, since we can - * hit shrinker in response to trying to get backing pages - * for this obj (ie. while it's lock is already held) - */ - if (!msm_gem_trylock(&msm_obj->base)) - goto tail; - - if (shrink(msm_obj)) - freed += msm_obj->base.size >> PAGE_SHIFT; - - msm_gem_unlock(&msm_obj->base); - -tail: - drm_gem_object_put(&msm_obj->base); - mutex_lock(&priv->mm_lock); - } - - list_splice_tail(&still_in_list, list); - mutex_unlock(&priv->mm_lock); - - return freed; -} - static unsigned long msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); + long nr = sc->nr_to_scan; unsigned long freed; - freed = scan(priv, sc->nr_to_scan, &priv->inactive_dontneed, purge); + freed = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); + nr -= freed; if (freed > 0) trace_msm_gem_purge(freed << PAGE_SHIFT); - if (can_swap() && freed < sc->nr_to_scan) { - int evicted = scan(priv, sc->nr_to_scan - freed, - &priv->inactive_willneed, evict); + if (can_swap() && nr > 0) { + unsigned long evicted; + + evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); + nr -= evicted; if (evicted > 0) trace_msm_gem_evict(evicted << PAGE_SHIFT); @@ -179,12 +119,12 @@ msm_gem_shrinker_shrink(struct drm_device *dev, unsigned long nr_to_scan) static const int vmap_shrink_limit = 15; static bool -vmap_shrink(struct msm_gem_object *msm_obj) +vmap_shrink(struct drm_gem_object *obj) { - if (!is_vunmapable(msm_obj)) + if (!is_vunmapable(to_msm_bo(obj))) return false; - msm_gem_vunmap(&msm_obj->base); + msm_gem_vunmap(obj); return true; } @@ -194,17 +134,18 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) { struct msm_drm_private *priv = container_of(nb, struct msm_drm_private, vmap_notifier); - struct list_head *mm_lists[] = { - &priv->inactive_dontneed, - &priv->inactive_willneed, - priv->gpu ? &priv->gpu->active_list : NULL, + struct drm_gem_lru *lrus[] = { + &priv->lru.dontneed, + &priv->lru.willneed, + &priv->lru.pinned, NULL, }; unsigned idx, unmapped = 0; - for (idx = 0; mm_lists[idx] && unmapped < vmap_shrink_limit; idx++) { - unmapped += scan(priv, vmap_shrink_limit - unmapped, - mm_lists[idx], vmap_shrink); + for (idx = 0; lrus[idx] && unmapped < vmap_shrink_limit; idx++) { + unmapped += drm_gem_lru_scan(lrus[idx], + vmap_shrink_limit - unmapped, + vmap_shrink); } *(unsigned long *)ptr += unmapped; diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index c2bfcf3f1f40..fba85f894314 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -846,7 +846,6 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, sched_set_fifo_low(gpu->worker->task); - INIT_LIST_HEAD(&gpu->active_list); mutex_init(&gpu->active_lock); mutex_init(&gpu->lock); init_waitqueue_head(&gpu->retire_event); @@ -974,8 +973,6 @@ void msm_gpu_cleanup(struct msm_gpu *gpu) DBG("%s", gpu->name); - WARN_ON(!list_empty(&gpu->active_list)); - for (i = 0; i < ARRAY_SIZE(gpu->rb); i++) { msm_ringbuffer_destroy(gpu->rb[i]); gpu->rb[i] = NULL; diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 962d2070bcdf..a0885ff99e5b 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -187,12 +187,6 @@ struct msm_gpu { */ int cur_ctx_seqno; - /* - * List of GEM active objects on this gpu. Protected by - * msm_drm_private::mm_lock - */ - struct list_head active_list; - /** * lock: * From patchwork Tue Jul 19 17:18:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E9ABC43334 for ; Tue, 19 Jul 2022 17:19:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238793AbiGSRTW (ORCPT ); Tue, 19 Jul 2022 13:19:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239298AbiGSRTP (ORCPT ); Tue, 19 Jul 2022 13:19:15 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 930444A816; Tue, 19 Jul 2022 10:19:08 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id c6so12553262pla.6; Tue, 19 Jul 2022 10:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FzvQD1BdX2RVx+fHXlbnajcgEgfcaBeelT02bqIAfkE=; b=JMLFaDiOl3fTF1vQ8KHqql+oNThjBy7zWx/y8DQ9Z/0c7uVTm8yACi7Z2w6/5G66/Q 0/gthVxtHxEJoGxvMNkZA2OiexiIsS/h9L+UhaFIlsCGbSWvk5bjbugJPd5v1KD75WPT Lvr4foB1JtxEKojRY5LTQtiTmcLz/2hbrt2eDW9YBIy8gf1ZEsQNCYCJ866+c1VWoDYi jZUJHp9AukkB96Ynu/yZq6y6cUtf8U0EsOE/kfBRWOFn9zPtasaCuPV7EysePzx9c+Bw wuSz13pddVCYbbL6gj/OUIm/wDJVGmDeBosyBGsNqJY9rvVBRkxWz9FOrvDalIOCofGi gZkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FzvQD1BdX2RVx+fHXlbnajcgEgfcaBeelT02bqIAfkE=; b=3UfvsDohFFGNUWa6TWGvE+6FAss7OiyHFHzfaxj3iaxoUzb8/S08nO8bpxTxNplhWm atojdHRlA3kN8o+PuqH0s13QWmSF3XJAun16jqoYIx4x9tYo+tjXzfxhho/cuDPPLT4F tnhcFsDg4Dh6+ByLNfw/72qIru+MYc0uDfxyYtb385jEGo9P7Vp6t/eG4ZKyahBJeVxi 5ae2+1/nOBntPGBPzPmcMo5mg2YTtU0Fa7QwR0JEmeXZxSKDMPTaUac94WcMBs8gIZk+ 4yErSlJAJ2/ZNqc8C8kRW30Lf1WDXo9UODQYz2/Ij8N5GszZmbV+uvGI8rtL/KRAy/S/ 0A7A== X-Gm-Message-State: AJIora/pQxV+CNJalnndw4QYG6ZtvVfmfA9bPoA4pMhIMc3q+hRuZKb+ 8oq9JAgQwo6uXNtul6Fe3jU= X-Google-Smtp-Source: AGRyM1t1cQXyO0F3MTKjBuxm2SfHhx+7oXShSQf6dGqX09a/GWI1vg9zGWJQKN+pDaC5UENRewuDqA== X-Received: by 2002:a17:90a:df18:b0:1f2:12ba:9092 with SMTP id gp24-20020a17090adf1800b001f212ba9092mr406971pjb.204.1658251148083; Tue, 19 Jul 2022 10:19:08 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id n8-20020a056a00212800b0052877d6f66bsm11801040pfj.50.2022.07.19.10.19.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:19:07 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 11/13] drm/msm/gem: Unpin buffers earlier Date: Tue, 19 Jul 2022 10:18:55 -0700 Message-Id: <20220719171900.289265-12-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark We've already attached the fences, so obj->resv (which shrinker checks) tells us whether they are still active. So we can unpin sooner, before we drop the queue lock. This also avoids the need to grab the obj lock in the retire path, avoiding potential for lock contention between submit and retire. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index adf358fb8e9d..5599d93ec0d2 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -501,11 +501,11 @@ static int submit_reloc(struct msm_gem_submit *submit, struct msm_gem_object *ob */ static void submit_cleanup(struct msm_gem_submit *submit, bool error) { - unsigned cleanup_flags = BO_LOCKED; + unsigned cleanup_flags = BO_LOCKED | BO_OBJ_PINNED; unsigned i; if (error) - cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED; + cleanup_flags |= BO_VMA_PINNED; for (i = 0; i < submit->nr_bos; i++) { struct msm_gem_object *msm_obj = submit->bos[i].obj; @@ -522,10 +522,6 @@ void msm_submit_retire(struct msm_gem_submit *submit) for (i = 0; i < submit->nr_bos; i++) { struct drm_gem_object *obj = &submit->bos[i].obj->base; - msm_gem_lock(obj); - /* Note, VMA already fence-unpinned before submit: */ - submit_cleanup_bo(submit, i, BO_OBJ_PINNED); - msm_gem_unlock(obj); drm_gem_object_put(obj); } } From patchwork Tue Jul 19 17:18:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D886DC43334 for ; Tue, 19 Jul 2022 17:19:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238809AbiGSRTX (ORCPT ); Tue, 19 Jul 2022 13:19:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239324AbiGSRTP (ORCPT ); Tue, 19 Jul 2022 13:19:15 -0400 Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 381824F679; Tue, 19 Jul 2022 10:19:11 -0700 (PDT) Received: by mail-pg1-x52a.google.com with SMTP id bf13so14063160pgb.11; Tue, 19 Jul 2022 10:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CHnA8EA1zYWAltmKrCqgpnviThEUlN7Pg3SwIn6d3WM=; b=HxFXlJDplzmQ2h/eoq6VJ/iQz8QcdeHyWrbrzwhVTrHkT+aZsj4LrYOJI4xQGBFmU6 RouW14uwgHYnEdzj6Y4Pz3AC8GzDNnlJhfzkMrRYxR/xMMEM7+XDnZxXBnbo6V+dPBsx NasDy4uAilmDAEIf6ESpgA7njGtQ08CI+Vp5V6otgcfCJtBeFYnjFcT5iFbwKIP6P5u5 el2UCwnKnDyfJVZk/2Ym01QDp4ehgPlEG+YZ9NXbTQQgUQSikrILZ5KsEftO2kQEFBhm 2YujfFtgO3zo0xGUFyCTWpUQkzhD2V6ZLCLe2+CNa5KPB3sRPMu8pdb4Etys5QiJoHdS 8LwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CHnA8EA1zYWAltmKrCqgpnviThEUlN7Pg3SwIn6d3WM=; b=qIUCVWvnoBvPPYLrEoYci4L06en/ZSjZ2hPuNPerm4BXrfBxJ53zHhXAwsmSKMy1lg kBXATHPUoqWXDDiEWqYtnEmZLzX+H54fxXKNwuB0BIWzdSKUUTdsyn+0vazb2puj1Xyt J7BhaOrkcCdJnSy8FCQHopQ88GcgJIvc7HnEikjVlJK0D36yzk/QdZaUL7i8SeoouPgq qDs0T41qx5tZrz6DnvgM3cs2FSMGMi6d+TvkuglEOyloelBuEt6NpzR+azavE2XYquJK 8G/jPL7uoGPtIUsKIHuHU4qSLlA0C4qLjVfQeRPe7T4mD/d+Y9+zIw7Xg8Q2gme8dTBQ tcWQ== X-Gm-Message-State: AJIora9DcmN6MSfKPF0udVLLHzeIi9edusbDtFY2HOko3Gfbd7iz1E7H igDCZ6vt52S2YlJEU/GfNNA= X-Google-Smtp-Source: AGRyM1ubRokMAdpxe40b/813XmRiTeKXFGokevdqztUpJ9LNB9bYSUN0s9MvgeWiCmxv4t/tGZlG4A== X-Received: by 2002:a63:ff66:0:b0:412:6f4c:1e11 with SMTP id s38-20020a63ff66000000b004126f4c1e11mr30096810pgk.396.1658251150723; Tue, 19 Jul 2022 10:19:10 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id w11-20020a62820b000000b005252433bdbdsm12175346pfd.95.2022.07.19.10.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:19:09 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 12/13] drm/msm/gem: Consolidate shrinker trace Date: Tue, 19 Jul 2022 10:18:56 -0700 Message-Id: <20220719171900.289265-13-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Combine separate trace events for purge vs evict into one. When we add support for purging/evicting active buffers we'll just add more info into this one trace event, rather than adding a bunch more events. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_shrinker.c | 19 ++++++--------- drivers/gpu/drm/msm/msm_gpu_trace.h | 32 +++++++++++--------------- 2 files changed, 20 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 530b1102b46d..5cc05d669a08 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -71,25 +71,20 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); long nr = sc->nr_to_scan; - unsigned long freed; + unsigned long freed, purged, evicted = 0; - freed = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); - nr -= freed; - - if (freed > 0) - trace_msm_gem_purge(freed << PAGE_SHIFT); + purged = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); + nr -= purged; if (can_swap() && nr > 0) { - unsigned long evicted; - evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); nr -= evicted; + } - if (evicted > 0) - trace_msm_gem_evict(evicted << PAGE_SHIFT); + freed = purged + evicted; - freed += evicted; - } + if (freed) + trace_msm_gem_shrink(sc->nr_to_scan, purged, evicted); return (freed > 0) ? freed : SHRINK_STOP; } diff --git a/drivers/gpu/drm/msm/msm_gpu_trace.h b/drivers/gpu/drm/msm/msm_gpu_trace.h index ca0b08d7875b..8867fa0a0306 100644 --- a/drivers/gpu/drm/msm/msm_gpu_trace.h +++ b/drivers/gpu/drm/msm/msm_gpu_trace.h @@ -115,29 +115,23 @@ TRACE_EVENT(msm_gmu_freq_change, ); -TRACE_EVENT(msm_gem_purge, - TP_PROTO(u32 bytes), - TP_ARGS(bytes), +TRACE_EVENT(msm_gem_shrink, + TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted), + TP_ARGS(nr_to_scan, purged, evicted), TP_STRUCT__entry( - __field(u32, bytes) + __field(u32, nr_to_scan) + __field(u32, purged) + __field(u32, evicted) ), TP_fast_assign( - __entry->bytes = bytes; + __entry->nr_to_scan = nr_to_scan; + __entry->purged = purged; + __entry->evicted = evicted; ), - TP_printk("Purging %u bytes", __entry->bytes) -); - - -TRACE_EVENT(msm_gem_evict, - TP_PROTO(u32 bytes), - TP_ARGS(bytes), - TP_STRUCT__entry( - __field(u32, bytes) - ), - TP_fast_assign( - __entry->bytes = bytes; - ), - TP_printk("Evicting %u bytes", __entry->bytes) + TP_printk("nr_to_scan=%u pages, purged=%u pages, evicted=%u pages", + __entry->nr_to_scan, + __entry->purged, + __entry->evicted) ); From patchwork Tue Jul 19 17:18:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 12922779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18325CCA487 for ; Tue, 19 Jul 2022 17:19:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238818AbiGSRTX (ORCPT ); Tue, 19 Jul 2022 13:19:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239345AbiGSRTP (ORCPT ); Tue, 19 Jul 2022 13:19:15 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5ADE50195; Tue, 19 Jul 2022 10:19:13 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id a15so15434784pjs.0; Tue, 19 Jul 2022 10:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kFWCPQ026Vm0ssc92OEIjelADHl+en2PNRs5WLmvYxM=; b=d5p1DKgFJK5ocgl5j7a/xuyGPSpf9br86Y47dsdKhDlhDTxJ+/teIhBdG43BMWGbnr 4MrZgmjt95TKM8bURo82WnHISJvEM4B9eQbSfNlPl/GfozUUo5MRiOOML08CowZOaH0j rWSnZZ0m6XOzoMgj/Yg+5Dmz/8VdraFVcm87xTBXSmsH/mI5g+LTiU9+1pYmn2MX3y3A eAzSi+bQ1l7uQ9AVJMYgDhjsm6A71rrDCKWohDrKthoLBYNQXH8spVWZ52ZXN0WIt/l0 z2FqSfjcbydBSE50zTJLLqqNSapRUDj3wtmGkJWWbAtnhaInSgL77i4qeWK++tDkmK6L l4eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kFWCPQ026Vm0ssc92OEIjelADHl+en2PNRs5WLmvYxM=; b=ktNqmmNRCY+SzilLrEjYwriIUJLJ+FROIXHiMeaMSuDs//FXzkQrePVwfDmxx6kKqL f93tnhZQV/5FPfOVQsFarvO7WC2tkDajppiqvsJy4GZvkrGRdWSmvpKwa1MEY7c+YP0n QLyJVtR/Fhm4QCe9BnMkn3PNuRv3AleNhKPe1XYoA4ybPuMeMXcjJUjKD+3l29TpCRDi kcyzJfhNfMOM+D0K1Wiy42hsVdN7UHJZVtIXdmhJh9bcvLZvxoS3d0rRFu1aDQIxZSbw f37KbmQBHhAOmXxI6e+R7v3BpwlA/wy4b1JKFwx0wr73QvibS/B+1b7TXvPlFY2HnYGa Vc5Q== X-Gm-Message-State: AJIora9SYov21MVYD5rHjB4XldoSlv39Fg/6BicssW9KySAfIddyl38a vrHmu/62cUXXRsbEPysRz6Y= X-Google-Smtp-Source: AGRyM1vMj72JXulUYuZJFzA60brwT7i+vZCtisHt/9lNHjAaA9i6+14x189DXlhP36gp/RDvXviHKg== X-Received: by 2002:a17:90a:e7cd:b0:1f0:2304:f579 with SMTP id kb13-20020a17090ae7cd00b001f02304f579mr417553pjb.212.1658251153211; Tue, 19 Jul 2022 10:19:13 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id y20-20020aa79af4000000b00518764d09cdsm11816762pfp.164.2022.07.19.10.19.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 10:19:12 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 13/13] drm/msm/gem: Evict active GEM objects when necessary Date: Tue, 19 Jul 2022 10:18:57 -0700 Message-Id: <20220719171900.289265-14-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220719171900.289265-1-robdclark@gmail.com> References: <20220719171900.289265-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark If we are under enough memory pressure, we should stall waiting for active buffers to become idle in order to evict. v2: Check for __GFP_ATOMIC before blocking Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_shrinker.c | 70 +++++++++++++++++++++----- drivers/gpu/drm/msm/msm_gpu_trace.h | 16 +++--- 2 files changed, 68 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 5cc05d669a08..f31054d25314 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -24,6 +24,13 @@ static bool can_swap(void) return enable_eviction && get_nr_swap_pages() > 0; } +static bool can_block(struct shrink_control *sc) +{ + if (sc->gfp_mask & __GFP_ATOMIC) + return false; + return current_is_kswapd() || (sc->gfp_mask & __GFP_RECLAIM); +} + static unsigned long msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) { @@ -65,26 +72,65 @@ evict(struct drm_gem_object *obj) return true; } +static bool +wait_for_idle(struct drm_gem_object *obj) +{ + enum dma_resv_usage usage = dma_resv_usage_rw(true); + return dma_resv_wait_timeout(obj->resv, usage, false, 1000) > 0; +} + +static bool +active_purge(struct drm_gem_object *obj) +{ + if (!wait_for_idle(obj)) + return false; + + return purge(obj); +} + +static bool +active_evict(struct drm_gem_object *obj) +{ + if (!wait_for_idle(obj)) + return false; + + return evict(obj); +} + static unsigned long msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); + struct { + struct drm_gem_lru *lru; + bool (*shrink)(struct drm_gem_object *obj); + bool cond; + unsigned long freed; + } stages[] = { + /* Stages of progressively more aggressive/expensive reclaim: */ + { &priv->lru.dontneed, purge, true }, + { &priv->lru.willneed, evict, can_swap() }, + { &priv->lru.dontneed, active_purge, can_block(sc) }, + { &priv->lru.willneed, active_evict, can_swap() && can_block(sc) }, + }; long nr = sc->nr_to_scan; - unsigned long freed, purged, evicted = 0; - - purged = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); - nr -= purged; - - if (can_swap() && nr > 0) { - evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); - nr -= evicted; + unsigned long freed = 0; + + for (unsigned i = 0; (nr > 0) && (i < ARRAY_SIZE(stages)); i++) { + if (!stages[i].cond) + continue; + stages[i].freed = + drm_gem_lru_scan(stages[i].lru, nr, stages[i].shrink); + nr -= stages[i].freed; + freed += stages[i].freed; } - freed = purged + evicted; - - if (freed) - trace_msm_gem_shrink(sc->nr_to_scan, purged, evicted); + if (freed) { + trace_msm_gem_shrink(sc->nr_to_scan, stages[0].freed, + stages[1].freed, stages[2].freed, + stages[3].freed); + } return (freed > 0) ? freed : SHRINK_STOP; } diff --git a/drivers/gpu/drm/msm/msm_gpu_trace.h b/drivers/gpu/drm/msm/msm_gpu_trace.h index 8867fa0a0306..ac40d857bc45 100644 --- a/drivers/gpu/drm/msm/msm_gpu_trace.h +++ b/drivers/gpu/drm/msm/msm_gpu_trace.h @@ -116,22 +116,26 @@ TRACE_EVENT(msm_gmu_freq_change, TRACE_EVENT(msm_gem_shrink, - TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted), - TP_ARGS(nr_to_scan, purged, evicted), + TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted, + u32 active_purged, u32 active_evicted), + TP_ARGS(nr_to_scan, purged, evicted, active_purged, active_evicted), TP_STRUCT__entry( __field(u32, nr_to_scan) __field(u32, purged) __field(u32, evicted) + __field(u32, active_purged) + __field(u32, active_evicted) ), TP_fast_assign( __entry->nr_to_scan = nr_to_scan; __entry->purged = purged; __entry->evicted = evicted; + __entry->active_purged = active_purged; + __entry->active_evicted = active_evicted; ), - TP_printk("nr_to_scan=%u pages, purged=%u pages, evicted=%u pages", - __entry->nr_to_scan, - __entry->purged, - __entry->evicted) + TP_printk("nr_to_scan=%u pg, purged=%u pg, evicted=%u pg, active_purged=%u pg, active_evicted=%u pg", + __entry->nr_to_scan, __entry->purged, __entry->evicted, + __entry->active_purged, __entry->active_evicted) );