From patchwork Mon Aug 5 14:33:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rob Herring (Arm)" X-Patchwork-Id: 11077151 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B206113B1 for ; Mon, 5 Aug 2019 14:34:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB7228764 for ; Mon, 5 Aug 2019 14:34:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93C4D28789; Mon, 5 Aug 2019 14:34:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C93A828757 for ; Mon, 5 Aug 2019 14:34:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4AF5A89E43; Mon, 5 Aug 2019 14:34:06 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0424B89E43 for ; Mon, 5 Aug 2019 14:34:03 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id j5so163633341ioj.8 for ; Mon, 05 Aug 2019 07:34:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=djYV0KIn02cpLdU/vfvsLTVJ2q5zHPzlC2MLNfcEfRE=; b=AYyev5LUxLeDj8rkwFR/lj7urgr0epH4JtFRmVlAYOofxMYxBTgl+xuWCBvoGlzzxo wx1t41QX5POGPzZKZPgtcwZqAiBGiBuH1IybgT6FKptUOUhZsTNsZgvScIckGRc4xVi6 eCbkMoNyKdIh1t0CyI6HubxdDC2QsTxC8BjDCzuTqpLgeWARRsOajoNf11ybZ+d5Z3la NCPc9D2eNSuqR4EIhtf50s7KtBJwz9LxaUejzocUvD9bNgC5zHmhWzH+wcV2M1lFElxg 0gdc3e3aVrygiRPBoeL/3+1lJJOyozlyj8V7SDiTXZ5XZdGJx2+jl6yFN6mlgIF88Dd3 2B9A== X-Gm-Message-State: APjAAAXI7DuMTYKw4cAn3e/KlkrkGAZ1xkNfhIBT8loTIOO1k5Y8JodO u/oE+xCNpf1GZI/F8oxeuCalP44= X-Google-Smtp-Source: APXvYqy7z9bsjY1f1K+phdvYfrM4FQw4Sa5yVq01B/ysj2kT6YkQEbHxz7qMloHeOTb2pPIBiZHUVA== X-Received: by 2002:a6b:cf17:: with SMTP id o23mr15773536ioa.176.1565015641784; Mon, 05 Aug 2019 07:34:01 -0700 (PDT) Received: from xps15.herring.priv ([64.188.179.254]) by smtp.googlemail.com with ESMTPSA id y5sm87918337ioc.86.2019.08.05.07.34.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 05 Aug 2019 07:34:01 -0700 (PDT) From: Rob Herring To: dri-devel@lists.freedesktop.org Subject: [PATCH 2/2] drm/panfrost: Add madvise and shrinker support Date: Mon, 5 Aug 2019 08:33:58 -0600 Message-Id: <20190805143358.21245-2-robh@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190805143358.21245-1-robh@kernel.org> References: <20190805143358.21245-1-robh@kernel.org> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , Tomeu Vizoso , David Airlie , Steven Price , Alyssa Rosenzweig Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add support for madvise and a shrinker similar to other drivers. This allows userspace to mark BOs which can be freed when there is memory pressure. Unlike other implementations, we don't depend on struct_mutex. The driver maintains a list of BOs which can be freed when the shrinker is called. Access to the list is serialized with the shrinker_lock. Cc: Tomeu Vizoso Cc: David Airlie Cc: Daniel Vetter Signed-off-by: Rob Herring --- drivers/gpu/drm/panfrost/Makefile | 1 + drivers/gpu/drm/panfrost/TODO | 3 - drivers/gpu/drm/panfrost/panfrost_device.h | 4 + drivers/gpu/drm/panfrost/panfrost_drv.c | 38 +++++++ drivers/gpu/drm/panfrost/panfrost_gem.c | 5 + drivers/gpu/drm/panfrost/panfrost_gem.h | 3 + .../gpu/drm/panfrost/panfrost_gem_shrinker.c | 107 ++++++++++++++++++ include/uapi/drm/panfrost_drm.h | 22 ++++ 8 files changed, 180 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile index ecf0864cb515..b71935862417 100644 --- a/drivers/gpu/drm/panfrost/Makefile +++ b/drivers/gpu/drm/panfrost/Makefile @@ -5,6 +5,7 @@ panfrost-y := \ panfrost_device.o \ panfrost_devfreq.o \ panfrost_gem.o \ + panfrost_gem_shrinker.o \ panfrost_gpu.o \ panfrost_job.o \ panfrost_mmu.o \ diff --git a/drivers/gpu/drm/panfrost/TODO b/drivers/gpu/drm/panfrost/TODO index c2e44add37d8..f3beca8d4ce9 100644 --- a/drivers/gpu/drm/panfrost/TODO +++ b/drivers/gpu/drm/panfrost/TODO @@ -18,10 +18,7 @@ - Support userspace controlled GPU virtual addresses. Needed for Vulkan. (Tomeu) -- Support for madvise and a shrinker. - - Compute job support. So called 'compute only' jobs need to be plumbed up to userspace. - Performance counter support. (Boris) - diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h index 83cc01cafde1..9751e8dc9ec6 100644 --- a/drivers/gpu/drm/panfrost/panfrost_device.h +++ b/drivers/gpu/drm/panfrost/panfrost_device.h @@ -84,6 +84,10 @@ struct panfrost_device { struct mutex sched_lock; struct mutex reset_lock; + struct mutex shrinker_lock; + struct list_head shrinker_list; + struct shrinker shrinker; + struct { struct devfreq *devfreq; struct thermal_cooling_device *cooling; diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c index cb43ff4ebf4a..399007d306ea 100644 --- a/drivers/gpu/drm/panfrost/panfrost_drv.c +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c @@ -301,6 +301,38 @@ static int panfrost_ioctl_get_bo_offset(struct drm_device *dev, void *data, return 0; } +static int panfrost_ioctl_madvise(struct drm_device *dev, void *data, + struct drm_file *file_priv) +{ + struct drm_panfrost_madvise *args = data; + struct panfrost_device *pfdev = dev->dev_private; + struct drm_gem_object *gem_obj; + + gem_obj = drm_gem_object_lookup(file_priv, args->handle); + if (!gem_obj) { + DRM_DEBUG("Failed to look up GEM BO %d\n", args->handle); + return -ENOENT; + } + + args->retained = drm_gem_shmem_madvise(gem_obj, args->madv); + + if (args->retained) { + struct panfrost_gem_object *bo = to_panfrost_bo(gem_obj); + + mutex_lock(&pfdev->shrinker_lock); + + if (args->madv == PANFROST_MADV_DONTNEED) + list_add_tail(&bo->base.madv_list, &pfdev->shrinker_list); + else if (args->madv == PANFROST_MADV_WILLNEED) + list_del_init(&bo->base.madv_list); + + mutex_unlock(&pfdev->shrinker_lock); + } + + drm_gem_object_put_unlocked(gem_obj); + return 0; +} + int panfrost_unstable_ioctl_check(void) { if (!unstable_ioctls) @@ -352,6 +384,7 @@ static const struct drm_ioctl_desc panfrost_drm_driver_ioctls[] = { PANFROST_IOCTL(GET_BO_OFFSET, get_bo_offset, DRM_RENDER_ALLOW), PANFROST_IOCTL(PERFCNT_ENABLE, perfcnt_enable, DRM_RENDER_ALLOW), PANFROST_IOCTL(PERFCNT_DUMP, perfcnt_dump, DRM_RENDER_ALLOW), + PANFROST_IOCTL(MADVISE, madvise, DRM_RENDER_ALLOW), }; DEFINE_DRM_GEM_SHMEM_FOPS(panfrost_drm_driver_fops); @@ -400,6 +433,8 @@ static int panfrost_probe(struct platform_device *pdev) pfdev->ddev = ddev; spin_lock_init(&pfdev->mm_lock); + mutex_init(&pfdev->shrinker_lock); + INIT_LIST_HEAD(&pfdev->shrinker_list); /* 4G enough for now. can be 48-bit */ drm_mm_init(&pfdev->mm, SZ_32M >> PAGE_SHIFT, (SZ_4G - SZ_32M) >> PAGE_SHIFT); @@ -430,6 +465,8 @@ static int panfrost_probe(struct platform_device *pdev) if (err < 0) goto err_out1; + panfrost_gem_shrinker_init(ddev); + return 0; err_out1: @@ -446,6 +483,7 @@ static int panfrost_remove(struct platform_device *pdev) struct drm_device *ddev = pfdev->ddev; drm_dev_unregister(ddev); + panfrost_gem_shrinker_cleanup(ddev); pm_runtime_get_sync(pfdev->dev); pm_runtime_put_sync_autosuspend(pfdev->dev); pm_runtime_disable(pfdev->dev); diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c index 543ab1b81bd5..67d374184340 100644 --- a/drivers/gpu/drm/panfrost/panfrost_gem.c +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c @@ -26,6 +26,11 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj) drm_mm_remove_node(&bo->node); spin_unlock(&pfdev->mm_lock); + mutex_lock(&pfdev->shrinker_lock); + if (!list_empty(&bo->base.madv_list)) + list_del(&bo->base.madv_list); + mutex_unlock(&pfdev->shrinker_lock); + drm_gem_shmem_free_object(obj); } diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h index 6dbcaba020fc..5f51f881ea3f 100644 --- a/drivers/gpu/drm/panfrost/panfrost_gem.h +++ b/drivers/gpu/drm/panfrost/panfrost_gem.h @@ -27,4 +27,7 @@ panfrost_gem_prime_import_sg_table(struct drm_device *dev, struct dma_buf_attachment *attach, struct sg_table *sgt); +void panfrost_gem_shrinker_init(struct drm_device *dev); +void panfrost_gem_shrinker_cleanup(struct drm_device *dev); + #endif /* __PANFROST_GEM_H__ */ diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c new file mode 100644 index 000000000000..d191632b6197 --- /dev/null +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2019 Arm Ltd. + * + * Based on msm_gem_freedreno.c: + * Copyright (C) 2016 Red Hat + * Author: Rob Clark + */ + +#include + +#include +#include + +#include "panfrost_device.h" +#include "panfrost_gem.h" +#include "panfrost_mmu.h" + +static unsigned long +panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) +{ + struct panfrost_device *pfdev = + container_of(shrinker, struct panfrost_device, shrinker); + struct drm_gem_shmem_object *shmem; + unsigned long count = 0; + + if (!mutex_trylock(&pfdev->shrinker_lock)) + return 0; + + list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) { + if (drm_gem_shmem_is_purgeable(shmem)) + count += shmem->base.size >> PAGE_SHIFT; + } + + mutex_unlock(&pfdev->shrinker_lock); + + return count; +} + +static void panfrost_gem_purge(struct drm_gem_object *obj) +{ + struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj); + mutex_lock(&shmem->pages_lock); + + panfrost_mmu_unmap(to_panfrost_bo(obj)); + drm_gem_shmem_purge_locked(obj); + + mutex_unlock(&shmem->pages_lock); +} + +static unsigned long +panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) +{ + struct panfrost_device *pfdev = + container_of(shrinker, struct panfrost_device, shrinker); + struct drm_gem_shmem_object *shmem, *tmp; + unsigned long freed = 0; + + if (!mutex_trylock(&pfdev->shrinker_lock)) + return SHRINK_STOP; + + list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) { + if (freed >= sc->nr_to_scan) + break; + if (drm_gem_shmem_is_purgeable(shmem)) { + panfrost_gem_purge(&shmem->base); + freed += shmem->base.size >> PAGE_SHIFT; + list_del_init(&shmem->madv_list); + } + } + + mutex_unlock(&pfdev->shrinker_lock); + + if (freed > 0) + pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT); + + return freed; +} + +/** + * panfrost_gem_shrinker_init - Initialize panfrost shrinker + * @dev: DRM device + * + * This function registers and sets up the panfrost shrinker. + */ +void panfrost_gem_shrinker_init(struct drm_device *dev) +{ + struct panfrost_device *pfdev = dev->dev_private; + pfdev->shrinker.count_objects = panfrost_gem_shrinker_count; + pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan; + pfdev->shrinker.seeks = DEFAULT_SEEKS; + WARN_ON(register_shrinker(&pfdev->shrinker)); +} + +/** + * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker + * @dev: DRM device + * + * This function unregisters the panfrost shrinker. + */ +void panfrost_gem_shrinker_cleanup(struct drm_device *dev) +{ + struct panfrost_device *pfdev = dev->dev_private; + + if (pfdev->shrinker.nr_deferred) { + unregister_shrinker(&pfdev->shrinker); + } +} diff --git a/include/uapi/drm/panfrost_drm.h b/include/uapi/drm/panfrost_drm.h index b5d370638846..1b3ff910ebe7 100644 --- a/include/uapi/drm/panfrost_drm.h +++ b/include/uapi/drm/panfrost_drm.h @@ -20,6 +20,7 @@ extern "C" { #define DRM_PANFROST_GET_BO_OFFSET 0x05 #define DRM_PANFROST_PERFCNT_ENABLE 0x06 #define DRM_PANFROST_PERFCNT_DUMP 0x07 +#define DRM_PANFROST_MADVISE 0x08 #define DRM_IOCTL_PANFROST_SUBMIT DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_SUBMIT, struct drm_panfrost_submit) #define DRM_IOCTL_PANFROST_WAIT_BO DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_WAIT_BO, struct drm_panfrost_wait_bo) @@ -27,6 +28,7 @@ extern "C" { #define DRM_IOCTL_PANFROST_MMAP_BO DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_MMAP_BO, struct drm_panfrost_mmap_bo) #define DRM_IOCTL_PANFROST_GET_PARAM DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_GET_PARAM, struct drm_panfrost_get_param) #define DRM_IOCTL_PANFROST_GET_BO_OFFSET DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_GET_BO_OFFSET, struct drm_panfrost_get_bo_offset) +#define DRM_IOCTL_PANFROST_MADVISE DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_MADVISE, struct drm_panfrost_madvise) /* * Unstable ioctl(s): only exposed when the unsafe unstable_ioctls module @@ -159,6 +161,26 @@ struct drm_panfrost_perfcnt_dump { __u64 buf_ptr; }; +/* madvise provides a way to tell the kernel in case a buffers contents + * can be discarded under memory pressure, which is useful for userspace + * bo cache where we want to optimistically hold on to buffer allocate + * and potential mmap, but allow the pages to be discarded under memory + * pressure. + * + * Typical usage would involve madvise(DONTNEED) when buffer enters BO + * cache, and madvise(WILLNEED) if trying to recycle buffer from BO cache. + * In the WILLNEED case, 'retained' indicates to userspace whether the + * backing pages still exist. + */ +#define PANFROST_MADV_WILLNEED 0 /* backing pages are needed, status returned in 'retained' */ +#define PANFROST_MADV_DONTNEED 1 /* backing pages not needed */ + +struct drm_panfrost_madvise { + __u32 handle; /* in, GEM handle */ + __u32 madv; /* in, PANFROST_MADV_x */ + __u32 retained; /* out, whether backing store still exists */ +}; + #if defined(__cplusplus) } #endif