From patchwork Thu Apr 20 18:21:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Francois Dugast X-Patchwork-Id: 13219063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2F86C77B73 for ; Thu, 20 Apr 2023 18:21:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BA8BB10E0C6; Thu, 20 Apr 2023 18:21:20 +0000 (UTC) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by gabe.freedesktop.org (Postfix) with ESMTPS id F3A1010E086; Thu, 20 Apr 2023 18:21:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682014879; x=1713550879; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=kjs+YPM6ZHCJBO7dr18LBrY5kfon/DIUG9l7/gtPX/s=; b=McFbk9kC2YMquC65dwLBqKPBjxrs5sf8+5NJGvsAncUwiGl2EZxFpX97 fBIQNx5+TB9+2oSdGXLV1+cn92GHwAYwVZbGNbaOIlStAfW/4jc5KN76D G51sdZb6vl3cFWtHUb9cPjx5g5gH0ZkRds/z0TcBfI4MwVeV3IuN6pCv/ dg2Rf040XbWL2x8WQi++rmkShMuE0zrxOBEI6Ld5XFOLt1uLr6hcRElSh K2emspi7ygRPx+twwGZZFab4kKsboPCgvW3z2d1r0dHDv4giK+DuNfGsH nGvYNXUOx8HbQWyh0ICs+pHdiVXut7ob9xriEpnnZNocVl0XHFxUPrdef Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="330014331" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="330014331" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 11:21:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816132845" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="816132845" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga004.jf.intel.com with ESMTP; 20 Apr 2023 11:21:17 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:17 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Thu, 20 Apr 2023 11:21:17 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.172) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NvL2U418ij3rVRqW41lx+C56sKaajjoL6uc18wKKirGeF7hzW8KFHqcHPgsYMXDf2GEZQf96vKTliag1AQc+AXPPJ1OqF2fomKai1L5uIKivJYcjKYyvojdgDTlVdvPEotLIQVx8qbDjvw60q/5aFaeBaSniOrH20PSVyLOIpJkE6FDBn4cjPAZ1OVRkAb+6zCW7/5xLqT41mEbq+TNskWxDgDbtnmqz76IRtihSmbng488Sl2tTjBOKU0FiddUswR2hpcgNFPQ8Q0WxSkq4zYBVq+SVLuJedHhfM47G7HqcLeCTGOlGgbARSR/489HQ2ONGdbbH2oaJjnZfH2HIyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GC1gqq3amCVPemQsxq8sWVzW7BDefnpjpcIdBT47STs=; b=f6rbur3+RYT5bFrcfLD4nhDIvZpLyF9t6bIXtu45H/p6I9GN2y01vBXnuKrc3KDGDY0De5brZ0Lj7KfQ3mF3Gl5xhd0Te9xUZIh+5O3WElyPmMlklEF0ROwocEW0JBpDlgLlJbAXeUgZLN3jzcg6wrsch4A/IRYLbFyiriZoHtOJEv4iL9jYWZEdmLg3TFJFLu99Txw7c9ntCNpBHQXAOGm2i06naH9GdcdFATSreb5sxdMHMcOFz2hdvu37NbGwGKe1hrT9+rlgFKele+qEpuGRpAT4E/QiDEJAFE26AoetVjQ5EYn2npkKk2SiwBpCr9UA/9JlECW5SO6Nl4O31A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) by DM4PR11MB7184.namprd11.prod.outlook.com (2603:10b6:8:110::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Thu, 20 Apr 2023 18:21:15 +0000 Received: from DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458]) by DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458%2]) with mapi id 15.20.6319.022; Thu, 20 Apr 2023 18:21:15 +0000 From: Francois Dugast To: Subject: [PATCH v4 1/3] drm: execution context for GEM buffers v3 Date: Thu, 20 Apr 2023 20:21:00 +0200 Message-ID: <20230420182102.3302646-2-francois.dugast@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230420182102.3302646-1-francois.dugast@intel.com> References: <20230420182102.3302646-1-francois.dugast@intel.com> X-ClientProxiedBy: LO4P123CA0348.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::11) To DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB5389:EE_|DM4PR11MB7184:EE_ X-MS-Office365-Filtering-Correlation-Id: ba52401a-eb05-469b-e660-08db41cc0740 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VTB0hg5EP3lJB2mDvrweJc70aenL8jxaHxIFWPfOzqfJ2bX9M3jTynzJFN7vPuPV6w1uHijxsqO+sbPpfpTGbqXbDYeaGNGHhKXU5vwD/+SjmJwmwX9z4NSiWOUloTuvfmOytf9RoDLGtTgZXlt9jIpJG4nqzVpaHeoLaNut8PhGkyJ+B7RDMATItpOVGRJL1UDNtbkT16qMfIk+pqT+ML+U/8OTCTtwGMtOWVZ+LH3kvLCOS6hlLXiLdj0ZMZ85uYpd9Eqf5aNZzNuxuGFyyyn6oToGcyzNa4u2Dwt+CHbiDWnrPA1OKP0tZ36I3cXj5wsoS07PrUSBXfSUuK3dVJy6Vp8oYQpV11KnIUEqmF6JjZXy74usTnN9A2xNASRfQZcWlySjMGzfL6RQtg5G4ur8q84qEnmB5ugE2bIIf5sokKrEkChrPp1itrknfjCRJnJQmd5CKJLbvRvPtpVoR1dYlEl5F5/UdiufFZIPuiRYn19op8CHbVBc6pdUobEnq2qeF9tnRVGTWXdwaWaT44UflBmVhUZ6vi91klliJYv/NIDMyfVEOY+bcJnBwxvP0c1b4HDQioKszmFmZZ4J5aXUPGmrcOR6ApWyUSsXid1PsxLPO7ma9wKYffEqEVZJ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5389.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(39860400002)(376002)(396003)(366004)(346002)(136003)(451199021)(38100700002)(41300700001)(5660300002)(44832011)(8676002)(2906002)(30864003)(8936002)(36756003)(86362001)(6486002)(6666004)(6512007)(1076003)(26005)(6506007)(478600001)(2616005)(83380400001)(66574015)(186003)(66556008)(66476007)(66946007)(316002)(6916009)(4326008)(82960400001)(309714004); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?+Q0hft7YmEbsk/4n0hQeHohjpzsY?= =?utf-8?q?W96gSE0Hw/lm6gvC5d9aiEL1ROrmfWtgD8r6G8u8wM+XD/6QxPHHL5NK4KzhytNY0?= =?utf-8?q?5hVgniGsw+9kLC9mmgLaNU89Fs2HwELf9BNtvG536knjnAlEtSIDyS3rPXmjRoqMg?= =?utf-8?q?RgqK9V2l4NBhFAQdB0JZT61UCp5ot6MFFKqHJfXUoDs4S+J6PLpuvne9mS0kamEP+?= =?utf-8?q?iFTBYy/v0v6dQsItRU1Gurd+rCboU9E5oYPgf25rEuzb24OyaOwsiXi6JOra2hCfe?= =?utf-8?q?L7EFBz3LTnhPEG4ukO58HDyTaD98kRoJWeYAJ11ul8t8wPG/KPvQv6KszetD7ku95?= =?utf-8?q?W/tfIgOkWzigaZl19PrxsFm93PwK6emuf1B3P0q7HYb5evao1bkB3FOkJ/IjgokkW?= =?utf-8?q?FAcHlHuJ0a7Os2CYMDzbI5fD331l3F/MYk2+E+OqUD1ZwD8hCD3pfLIfvjCqXyGFQ?= =?utf-8?q?t9JcTyipn1LO5iU8Ri7GMEtLGmK4Nqapyl9OKRIVVwKmWv/6EXaoOcPO/0EwzYI0N?= =?utf-8?q?91mAfVwoY9rO66oz3Wr7Go1eN6fppqdgwLdNWyQgCUaHsAtbX2PIibFSpg+wAdP5m?= =?utf-8?q?oDrj8YzMFxhIYWP/w/8qb1fX42TqgsBVBfi4zKzYvzw+dBPsFkvL+uw7t5aZJXHna?= =?utf-8?q?jzZTWN5/i4qYrMjCTVtXCh5DPBnrbIHlAnn/IXh8VQcrkcnKHBxZQTSGIGEg15J+n?= =?utf-8?q?cAARvJVjzvKWISukhDf4qJkNeKfSrS+l0hl+Xilwyafun8H+o/9/mBxIW1l/aD5up?= =?utf-8?q?1HYFZLiQU3pKTj/WopAOKR55OIE4enlB0PoPu6E+NbS1xaI9aybk/llt87ltpexb4?= =?utf-8?q?SrEqxD2uh9FvjpAzm7Ibik6vFNawFYdvVih7P97qDM2Xy4kzUomM3EQ4NDBSV1ghJ?= =?utf-8?q?vK8rVFjsd6kpFhHe5jvsTWdFos91ytBHYDhMENBj4w+9sE8oNiwJiG6Z++bBmzsxB?= =?utf-8?q?TAT1KsIIbFrBoF/VWkNxr8zeIVDTJsLBLKYpPOCFRpK8oWtFGAOn/hEnoRE98j0xG?= =?utf-8?q?UB4aRAdDkkCOP9nPXU7ACwb8b0AoJyKH36fusPvSpwPlypQ5ydFSyigvBuEf9GVfy?= =?utf-8?q?CMqePDrQu5EQrYHaPOHt4Vm5GLZ6Tkof9zayLe6P2yxBHxW2sZCwdH98/5rr2Iq4v?= =?utf-8?q?GJtl4xlmOS+NrXn5SESOggl1MK73J8DjFnD4wPdIMwBzP+JySYRmfAlKyHPvpOGFn?= =?utf-8?q?7xlp7rUixxI8eF+23muPBXdbukOvPmvp16BbMR//rS3SDlU8NqH3O64WXYRpvdfXU?= =?utf-8?q?3SUZKlsTrF3dvlTONMwFfVXncFfalBUbzEjrlHFEZzH047kD1ddy0AxbiVKQIAnH5?= =?utf-8?q?HaqOlocXnkriT5TYzK2fgqIuiJBqNkMKKDCTJYAfOZdJliDP0DpawlyI5G7z2ysQg?= =?utf-8?q?r5JD0+ddAI+7kW6LrZNMRFFcy+SwmQE2RPTj5FQb6GyhEfxGFhQCh4TduUE7nAD3u?= =?utf-8?q?syqDnsgB1ypt3LZvrJfQy+OaPSy2UX/w+1E5FGzrFvURD0RVKeVSBeeBmx9mvSSzG?= =?utf-8?q?UqF4wz5ldhCkkAvNxIqm1eqUcyOfBJU75w=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: ba52401a-eb05-469b-e660-08db41cc0740 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5389.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2023 18:21:15.6886 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UIef3zHgstVokCwdUeOPmG3a6GlNVFiKRET5dFYEHU1/GXDxJa7yV62gvG8dM8NDMixpch/kr7fT6meQWssjtSm844O7SPatp3lTZsgABDA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB7184 X-OriginatorOrg: intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, =?utf-8?q?Christian_K=C3=B6nig?= , lucas.demarchi@intel.com, dri-devel@lists.freedesktop.org, dakr@redhat.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Christian König This adds the infrastructure for an execution context for GEM buffers which is similar to the existinc TTMs execbuf util and intended to replace it in the long term. The basic functionality is that we abstracts the necessary loop to lock many different GEM buffers with automated deadlock and duplicate handling. v2: drop xarray and use dynamic resized array instead, the locking overhead is unecessary and measureable. v3: drop duplicate tracking, radeon is really the only one needing that. Signed-off-by: Christian König --- Documentation/gpu/drm-mm.rst | 12 ++ drivers/gpu/drm/Kconfig | 6 + drivers/gpu/drm/Makefile | 2 + drivers/gpu/drm/drm_exec.c | 249 +++++++++++++++++++++++++++++++++++ include/drm/drm_exec.h | 115 ++++++++++++++++ 5 files changed, 384 insertions(+) create mode 100644 drivers/gpu/drm/drm_exec.c create mode 100644 include/drm/drm_exec.h diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst index a79fd3549ff8..a52e6f4117d6 100644 --- a/Documentation/gpu/drm-mm.rst +++ b/Documentation/gpu/drm-mm.rst @@ -493,6 +493,18 @@ DRM Sync Objects .. kernel-doc:: drivers/gpu/drm/drm_syncobj.c :export: +DRM Execution context +===================== + +.. kernel-doc:: drivers/gpu/drm/drm_exec.c + :doc: Overview + +.. kernel-doc:: include/drm/drm_exec.h + :internal: + +.. kernel-doc:: drivers/gpu/drm/drm_exec.c + :export: + GPU Scheduler ============= diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index e928284b4357..39c9d079d52a 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -201,6 +201,12 @@ config DRM_TTM GPU memory types. Will be enabled automatically if a device driver uses it. +config DRM_EXEC + tristate + depends on DRM + help + Execution context for command submissions + config DRM_BUDDY tristate depends on DRM diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index 66dd2c48944a..9c4461f0a665 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -79,6 +79,8 @@ obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o # # Memory-management helpers # +# +obj-$(CONFIG_DRM_EXEC) += drm_exec.o obj-$(CONFIG_DRM_BUDDY) += drm_buddy.o diff --git a/drivers/gpu/drm/drm_exec.c b/drivers/gpu/drm/drm_exec.c new file mode 100644 index 000000000000..df546cc5a227 --- /dev/null +++ b/drivers/gpu/drm/drm_exec.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ + +#include +#include +#include + +/** + * DOC: Overview + * + * This component mainly abstracts the retry loop necessary for locking + * multiple GEM objects while preparing hardware operations (e.g. command + * submissions, page table updates etc..). + * + * If a contention is detected while locking a GEM object the cleanup procedure + * unlocks all previously locked GEM objects and locks the contended one first + * before locking any further objects. + * + * After an object is locked fences slots can optionally be reserved on the + * dma_resv object inside the GEM object. + * + * A typical usage pattern should look like this:: + * + * struct drm_gem_object *obj; + * struct drm_exec exec; + * unsigned long index; + * int ret; + * + * drm_exec_init(&exec, true); + * drm_exec_while_not_all_locked(&exec) { + * ret = drm_exec_prepare_obj(&exec, boA, 1); + * drm_exec_continue_on_contention(&exec); + * if (ret) + * goto error; + * + * ret = drm_exec_lock(&exec, boB, 1); + * drm_exec_continue_on_contention(&exec); + * if (ret) + * goto error; + * } + * + * drm_exec_for_each_locked_object(&exec, index, obj) { + * dma_resv_add_fence(obj->resv, fence, DMA_RESV_USAGE_READ); + * ... + * } + * drm_exec_fini(&exec); + * + * See struct dma_exec for more details. + */ + +/* Dummy value used to initially enter the retry loop */ +#define DRM_EXEC_DUMMY (void*)~0 + +/* Unlock all objects and drop references */ +static void drm_exec_unlock_all(struct drm_exec *exec) +{ + struct drm_gem_object *obj; + unsigned long index; + + drm_exec_for_each_locked_object(exec, index, obj) { + dma_resv_unlock(obj->resv); + drm_gem_object_put(obj); + } + + if (exec->prelocked) { + dma_resv_unlock(exec->prelocked->resv); + drm_gem_object_put(exec->prelocked); + exec->prelocked = NULL; + } +} + +/** + * drm_exec_init - initialize a drm_exec object + * @exec: the drm_exec object to initialize + * @interruptible: if locks should be acquired interruptible + * + * Initialize the object and make sure that we can track locked and duplicate + * objects. + */ +void drm_exec_init(struct drm_exec *exec, bool interruptible) +{ + exec->interruptible = interruptible; + exec->objects = kmalloc(PAGE_SIZE, GFP_KERNEL); + + /* If allocation here fails, just delay that till the first use */ + exec->max_objects = exec->objects ? PAGE_SIZE / sizeof(void *) : 0; + exec->num_objects = 0; + exec->contended = DRM_EXEC_DUMMY; + exec->prelocked = NULL; +} +EXPORT_SYMBOL(drm_exec_init); + +/** + * drm_exec_fini - finalize a drm_exec object + * @exec: the drm_exec object to finilize + * + * Unlock all locked objects, drop the references to objects and free all memory + * used for tracking the state. + */ +void drm_exec_fini(struct drm_exec *exec) +{ + drm_exec_unlock_all(exec); + kvfree(exec->objects); + if (exec->contended != DRM_EXEC_DUMMY) { + drm_gem_object_put(exec->contended); + ww_acquire_fini(&exec->ticket); + } +} +EXPORT_SYMBOL(drm_exec_fini); + +/** + * drm_exec_cleanup - cleanup when contention is detected + * @exec: the drm_exec object to cleanup + * + * Cleanup the current state and return true if we should stay inside the retry + * loop, false if there wasn't any contention detected and we can keep the + * objects locked. + */ +bool drm_exec_cleanup(struct drm_exec *exec) +{ + if (likely(!exec->contended)) { + ww_acquire_done(&exec->ticket); + return false; + } + + if (likely(exec->contended == DRM_EXEC_DUMMY)) { + exec->contended = NULL; + ww_acquire_init(&exec->ticket, &reservation_ww_class); + return true; + } + + drm_exec_unlock_all(exec); + exec->num_objects = 0; + return true; +} +EXPORT_SYMBOL(drm_exec_cleanup); + +/* Track the locked object in the xa and reserve fences */ +static int drm_exec_obj_locked(struct drm_exec *exec, + struct drm_gem_object *obj) +{ + if (unlikely(exec->num_objects == exec->max_objects)) { + size_t size = exec->max_objects * sizeof(void *); + void *tmp; + + tmp = kvrealloc(exec->objects, size, size + PAGE_SIZE, + GFP_KERNEL); + if (!tmp) + return -ENOMEM; + + exec->objects = tmp; + exec->max_objects += PAGE_SIZE / sizeof(void *); + } + drm_gem_object_get(obj); + exec->objects[exec->num_objects++] = obj; + + return 0; +} + +/* Make sure the contended object is locked first */ +static int drm_exec_lock_contended(struct drm_exec *exec) +{ + struct drm_gem_object *obj = exec->contended; + int ret; + + if (likely(!obj)) + return 0; + + if (exec->interruptible) { + ret = dma_resv_lock_slow_interruptible(obj->resv, + &exec->ticket); + if (unlikely(ret)) + goto error_dropref; + } else { + dma_resv_lock_slow(obj->resv, &exec->ticket); + } + + ret = drm_exec_obj_locked(exec, obj); + if (unlikely(ret)) { + dma_resv_unlock(obj->resv); + goto error_dropref; + } + + swap(exec->prelocked, obj); + +error_dropref: + /* Always cleanup the contention so that error handling can kick in */ + drm_gem_object_put(obj); + exec->contended = NULL; + return ret; +} + +/** + * drm_exec_prepare_obj - prepare a GEM object for use + * @exec: the drm_exec object with the state + * @obj: the GEM object to prepare + * @num_fences: how many fences to reserve + * + * Prepare a GEM object for use by locking it and reserving fence slots. All + * succesfully locked objects are put into the locked container. Duplicates + * detected as well and automatically moved into the duplicates container. + * + * Returns: -EDEADLK if a contention is detected, -ENOMEM when memory + * allocation failed and zero for success. + */ +int drm_exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj, + unsigned int num_fences) +{ + int ret; + + ret = drm_exec_lock_contended(exec); + if (unlikely(ret)) + return ret; + + if (exec->prelocked == obj) { + drm_gem_object_put(exec->prelocked); + exec->prelocked = NULL; + + return dma_resv_reserve_fences(obj->resv, num_fences); + } + + if (exec->interruptible) + ret = dma_resv_lock_interruptible(obj->resv, &exec->ticket); + else + ret = dma_resv_lock(obj->resv, &exec->ticket); + + if (unlikely(ret == -EDEADLK)) { + drm_gem_object_get(obj); + exec->contended = obj; + return -EDEADLK; + } + + if (unlikely(ret)) + return ret; + + ret = drm_exec_obj_locked(exec, obj); + if (ret) + goto error_unlock; + + /* Keep locked when reserving fences fails */ + return dma_resv_reserve_fences(obj->resv, num_fences); + +error_unlock: + dma_resv_unlock(obj->resv); + return ret; +} +EXPORT_SYMBOL(drm_exec_prepare_obj); + +MODULE_DESCRIPTION("DRM execution context"); +MODULE_LICENSE("Dual MIT/GPL"); diff --git a/include/drm/drm_exec.h b/include/drm/drm_exec.h new file mode 100644 index 000000000000..65e518c01db3 --- /dev/null +++ b/include/drm/drm_exec.h @@ -0,0 +1,115 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ + +#ifndef __DRM_EXEC_H__ +#define __DRM_EXEC_H__ + +#include + +struct drm_gem_object; + +/** + * struct drm_exec - Execution context + */ +struct drm_exec { + /** + * @interruptible: If locks should be taken interruptible + */ + bool interruptible; + + /** + * @ticket: WW ticket used for acquiring locks + */ + struct ww_acquire_ctx ticket; + + /** + * @num_objects: number of objects locked + */ + unsigned int num_objects; + + /** + * @max_objects: maximum objects in array + */ + unsigned int max_objects; + + /** + * @objects: array of the locked objects + */ + struct drm_gem_object **objects; + + /** + * @contended: contended GEM object we backet of for + */ + struct drm_gem_object *contended; + + /** + * @prelocked: already locked GEM object because of contention + */ + struct drm_gem_object *prelocked; +}; + +/** + * drm_exec_for_each_locked_object - iterate over all the locked objects + * @exec: drm_exec object + * @index: unsigned long index for the iteration + * @obj: the current GEM object + * + * Iterate over all the locked GEM objects inside the drm_exec object. + */ +#define drm_exec_for_each_locked_object(exec, index, obj) \ + for (index = 0, obj = (exec)->objects[0]; \ + index < (exec)->num_objects; \ + ++index, obj = (exec)->objects[index]) + +/** + * drm_exec_while_not_all_locked - loop until all GEM objects are prepared + * @exec: drm_exec object + * + * Core functionality of the drm_exec object. Loops until all GEM objects are + * prepared and no more contention exists. + * + * At the beginning of the loop it is guaranteed that no GEM object is locked. + */ +#define drm_exec_while_not_all_locked(exec) \ + while (drm_exec_cleanup(exec)) + +/** + * drm_exec_continue_on_contention - continue the loop when we need to cleanup + * @exec: drm_exec object + * + * Control flow helper to continue when a contention was detected and we need to + * clean up and re-start the loop to prepare all GEM objects. + */ +#define drm_exec_continue_on_contention(exec) \ + if (unlikely(drm_exec_is_contended(exec))) \ + continue + +/** + * drm_exec_break_on_contention - break a subordinal loop on contention + * @exec: drm_exec object + * + * Control flow helper to break a subordinal loop when a contention was detected + * and we need to clean up and re-start the loop to prepare all GEM objects. + */ +#define drm_exec_break_on_contention(exec) \ + if (unlikely(drm_exec_is_contended(exec))) \ + break + +/** + * drm_exec_is_contended - check for contention + * @exec: drm_exec object + * + * Returns true if the drm_exec object has run into some contention while + * locking a GEM object and needs to clean up. + */ +static inline bool drm_exec_is_contended(struct drm_exec *exec) +{ + return !!exec->contended; +} + +void drm_exec_init(struct drm_exec *exec, bool interruptible); +void drm_exec_fini(struct drm_exec *exec); +bool drm_exec_cleanup(struct drm_exec *exec); +int drm_exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj, + unsigned int num_fences); + +#endif From patchwork Thu Apr 20 18:21:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francois Dugast X-Patchwork-Id: 13219064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1D3BC77B73 for ; Thu, 20 Apr 2023 18:21:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A07BD10E104; Thu, 20 Apr 2023 18:21:25 +0000 (UTC) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by gabe.freedesktop.org (Postfix) with ESMTPS id 44C3110E0C6; Thu, 20 Apr 2023 18:21:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682014879; x=1713550879; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=ICFwj3LSsnneqRC8pBvtL0nGndzJnsGniMtGw1uWFDc=; b=UNOOti/UQuvNIMcG5LpazZeV0fIZh7MHtXD+Zh7fJkadwY3bkKQPN/pT SbVXDkhVlDlv8aP0VbfKukMP+Sn3X9a6bCfzRP1zOEvW+MH/9FQ8qHggS f5E6K1rjlQqVRqA3K6ET95ONm1vPGf22kPeOzXMK+jR996fiOd8A5QrRR doBJr8gTXDkByc0jsPiuu+BnH/3/xyxbql5/0YHccp5JkPbI5yR2LzGKs IUoEuFwD2HETELZoOyqocd6TLmz3K/2DQoBlUMTRlCXQyR1KSwdNnHVER z0rwOsKe8yZUCb3I/QM1m35r7RAIOoVm/0ecQVQp7cwWJDVlbCseWtic9 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="330014337" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="330014337" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 11:21:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816132848" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="816132848" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga004.jf.intel.com with ESMTP; 20 Apr 2023 11:21:18 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:18 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Thu, 20 Apr 2023 11:21:18 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.176) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:18 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N8/X1NAze3XZgotqUb2XmR031ngPe8vA8w02tAgtF8ZimmKsQUcUV8kaxlzMD1tJCqDwWGSjP9xtxFzaVKciCUcgGeWy/mS308CJ047FyECS40ho50WpqIR2bhI0aV/Q47FUPQFMatvlgPAlZYyCkaZOAz/saepiX/NnoS0Vdb6V/65raUeQ3u/fnsvyEFp5kDMS3Z+yHpvixFkDtOOG3Zfmxq9A9pNoYxTGG/9Rw4L3GhSeFRj7q1ScZD2ic2W+v7DPStjvYR8dgHJO8iiBNCi32Hbe9j6iJplazX+vZ5i6t7zt619sj5J70bAg3+0kRLMdOAol/orKEAqG020BXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OReqnnuDOxM/MPo2msDlv9YaNzhSmj+JzVI+Dr2b9y0=; b=Vd7EtMt4sfyYJPJBWyay351jTr/65zhmWoh/vryxf1PcIYDqYC6ZCZ7jrIUVJlRPW9aO9RZWd4wdnha0NIT76lQwaBj6vyccQnyxy76lzbkeJ1KzjixWK14NUXstYkThFRMwfTdJ3sP3gVsyRxiDwkLdxF+KlS4k0LPZk3iHpWAFgMnEj0JLREOhGF6hCkzR5i4pxB4Jxgkqp6P7lNjEjP04DX51oPHw4RaU+x4Tpa/38lLWF2AXMoEjY000LJ6IQsRlgtkU6PVoy1UygkhR1hrUZKq6+WKponE6wlxu7v8CsGXzyuWkZBueyoZBR0PYdIB4tUb8jP6b7nYHGw0zHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) by DM4PR11MB7184.namprd11.prod.outlook.com (2603:10b6:8:110::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Thu, 20 Apr 2023 18:21:17 +0000 Received: from DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458]) by DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458%2]) with mapi id 15.20.6319.022; Thu, 20 Apr 2023 18:21:17 +0000 From: Francois Dugast To: Subject: [PATCH v4 2/3] drm_exec: fix double dma_resv unlock Date: Thu, 20 Apr 2023 20:21:01 +0200 Message-ID: <20230420182102.3302646-3-francois.dugast@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230420182102.3302646-1-francois.dugast@intel.com> References: <20230420182102.3302646-1-francois.dugast@intel.com> X-ClientProxiedBy: LO4P123CA0348.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::11) To DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB5389:EE_|DM4PR11MB7184:EE_ X-MS-Office365-Filtering-Correlation-Id: 296cbce8-c6db-409a-a8d7-08db41cc0831 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7KEPXV6NY46+75XCDUw/tYwB/JaSfMFcCWVbrc10zePWk9UhgGSAvjh2ElQkM31Q5bmDpus27eoGJvdHoMyMAW7TP8EzWqxkqX48TawtjoZm/vLnPUPn+ucQdaB6V/TA5mwvGaljBrEHZlBGn95r+avmM2O4XHTGzZGOVeALsl8NsLYKAbXzsT1Dl+C+R+DOLK9BPtRQa+Nwvy8eKQe7DnQrYP8vhYcvC3lFWjCojDirrC6PfcmooSHnhgfTSRCWP+qSVPBW4+gv/7/obzeFwr4cNOpxyuBaKsUqFq7vEhqE+IgNxl3qh0LXMZ6ZkPliqTyHKd/TPv2t7qt1zslh2eW2TBkumHAvAu50PCd7/ADpMElaCQkCHm8JnEGhjqLWDQJ/dr1zGtW2ZEys2nYs2UHYu3ZRJziLSmf3rQe4UloADd84HQw6T0XvqXjr2HTBBH4YV+jbmQIT6PVFMU5u7bKbhZnLQpRWT/QQdorGrqvMaVWtzytyr9cQ49rjdl3gj/9tJyOJVaIIBliuxrmcJlUR4LqSVbFlvqXXwP/q/+PUiDXhk8bf3HsVkrcqh4V0 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5389.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(39860400002)(376002)(396003)(366004)(346002)(136003)(451199021)(38100700002)(41300700001)(5660300002)(44832011)(8676002)(2906002)(8936002)(4744005)(36756003)(86362001)(6486002)(6666004)(6512007)(1076003)(26005)(6506007)(107886003)(478600001)(2616005)(83380400001)(186003)(66556008)(66476007)(66946007)(316002)(6916009)(4326008)(82960400001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PxtZHxILVtLE95U827XzEoXQdWB7PNKl8XVMJCRKw+MBIUfrs+auyFYDvrXIgyjpj0ZPoHH2agGRWDMPV03Khe6H50VNI08OA2Kg+baoqVH9nOegUGnk+usa1o/Q09tZuV8ERklEzD2nJc42zw068FLLaXABujlS7Yu6yxUmwJVYy7xOWJQKhcTUK/mkdeWIgqMECE9//9oE2+qk0GbYa7Il6Dw+tzi2igMqODocjwAmzHNCkIeUzOlLKgY0m2mE0jzEzNhFN8C3IoB+RlfUv5q7TjGTj0flRx7wk8nC4VeiStsxcTST0CRHPUgG7SmY5e4DnpwNBdvrYG9HhtWR5i+aYYwP5QEIZLTRuTUPV8zTTOb+CV44tKPY1nx9GpFUOOxBSeoO4FA5/Yeae++LMw7ZemsnoA/GPKvGk3FGSjOi0xo/0cTRbpAAlOeeapgxwHZFu53+/ZjbMPo/bGT/iuC97RIFCSBPsFYNf/dRUQTIYcBcRSlsVYI05tthl2CcGMzLjT7Mhhg07rpdbVcrZxW+J+0KF8auoQElQA8gxNfGAgQUmYiJB8aq+XDShYx3IE0F2f9VBKs6yNZMf1Q+Q8467zcB0dO2bHHIuIG4Fo/YMxDCd6MF+fE4IrjzlOAut2jkWtWV53He7Uh0GZrJkX2ispUbO6rHooDvw3TkYDCHKXPZKfbe8UMXMxLmWn+1gFI0l6mFlD405TXtKm/PC2f9EHycjY5a2iJv5PXr9XDvwzZUwthcyPc0awnePusD0KkBK7IXEfxj7jFgSp+0Ay6IAVV2mBJc1ViPGEEXxkU6Kb4cHsukogiZu0HE5dPdlNEB24Fc2cv8mmSCRc0s8WEPSOaQmRo00NJna3pU/w9tMKrsJVoncxt05g3PtblGDavfXPkliwX/VQ1YH/u9kE5zIkNZkBv5GYGiklTVQ6U6IPl0RgC5/dkmThRq8fo54ajsSQsMYkBGdpEpc5AfbpHBngf/zPkw0UrSBwaDTjqixEqXlJvTWEi/2hVYheRcdtHjCyd51QTiiljhZnMvqezGzcUMuAr/yg+EwTmfW54h3o+v4RZmCLY1S7VRQdVvrajr2epfqPZbPbOkyAU6ThZoSUXU44QWfYZgk4IyWHP7IFHBOsgmlIHWSYGHkC60O7GLXeTKSDZF6Y2tcA+pwXpDkduqgRfk0k0zsOegzrsnszs+MtrAGfLEQ8RGD4q7m6r/vjJH/rx1dgVAjAtyZe1MNOCCjItEf5/fMpZHpfj8kzPT5j+a2TVlpEsHjYZO+dKFNDDo0VVE18SB0PyEPDXH1La+W7uLLi/6tUgotg/hy52KYOJ1wnjc5sN+UV6+tPG7RNQ8OoDRiZW/7BBHTVBw/SNOXMmGc2lJiiLZv09WbL+pgL/JlWbX7R3bC7xcIZoCknNvGbCmFK0ogA89O9DY1qa2RiPoOzmOLyLGTvReV+xliG2uiBS4/4g29JaTCYGoBjcy13WCeOrsf9EW50587zbcXL8kHHge3swr5VnuVYup0tgZm8/nyYYm968B4003ZLTLPkbGlN0gGcxodLi+Bd1zncopcUz/q7kKC7YCmnnjmSId1tZkn5No1Wf17gFTQXgaXKnhKu5wsrOShQ== X-MS-Exchange-CrossTenant-Network-Message-Id: 296cbce8-c6db-409a-a8d7-08db41cc0831 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5389.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2023 18:21:17.2333 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Counren5WUdD1WmoLFwjdaUbqaF/pxR6GVr81aTlITkIDohmjwmMaPMuKrlqyPP8QDnf08O1EIZYjQOG+nvf1v0On70HnHN5e/Js3dkQu6Q= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB7184 X-OriginatorOrg: intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, lucas.demarchi@intel.com, dakr@redhat.com, christian.koenig@amd.com, dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Danilo Krummrich Signed-off-by: Danilo Krummrich --- drivers/gpu/drm/drm_exec.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/drm_exec.c b/drivers/gpu/drm/drm_exec.c index df546cc5a227..f645d22a0863 100644 --- a/drivers/gpu/drm/drm_exec.c +++ b/drivers/gpu/drm/drm_exec.c @@ -62,7 +62,6 @@ static void drm_exec_unlock_all(struct drm_exec *exec) } if (exec->prelocked) { - dma_resv_unlock(exec->prelocked->resv); drm_gem_object_put(exec->prelocked); exec->prelocked = NULL; } From patchwork Thu Apr 20 18:21:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Francois Dugast X-Patchwork-Id: 13219065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C5C0C77B7A for ; Thu, 20 Apr 2023 18:21:31 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C767710E429; Thu, 20 Apr 2023 18:21:30 +0000 (UTC) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by gabe.freedesktop.org (Postfix) with ESMTPS id F3F1D10E086; Thu, 20 Apr 2023 18:21:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682014886; x=1713550886; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=eVjZlVC++/aTWwLm/7ebeorh6lTeUT53J5zju8PiwIQ=; b=avs2imPjYBR99+hFDDNhVCkLILbFA142vXQUqoC0EfVUds4VkHt2PEw1 OKlZ0KJIEMrP7IvDg60XqsIzIvGW1xVco4ebpbI4S06ByX4fkKgwgfyv4 gwGNTl+W7OjrVVntGKngj2mFNiUo1sVEPAJ3HqdofoPuSpVHtSXiFddOg ZEeciWTwDK4oGOc2nakM9Ytzg3PoIQpm1bjkfKbA4TbOQAwoeP++nvVNu rJyL0nXFZW337jjUockYBRoqN/IlIbGiNluFCMVutFZSa1stM1fQvgGZG yIZSK8RFLqAEil6UkgqjHEp2EGuIbeNGT95cKJHkzDoLDZPA8OHPtLr5I g==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="325422273" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="325422273" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 11:21:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="866436797" X-IronPort-AV: E=Sophos;i="5.99,213,1677571200"; d="scan'208";a="866436797" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga005.jf.intel.com with ESMTP; 20 Apr 2023 11:21:22 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:21 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Thu, 20 Apr 2023 11:21:21 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.108) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Thu, 20 Apr 2023 11:21:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RpSHmVvyVEin5uT91e4ejBWL9XzF5/25cnHhfycCdXBx0zKveil4fBfSs/16bWiIXNe+9CcvMbJOEP6MBxH+G3y/ZsbEGJ70u+70KWGgnGuO+JRjFDn9prlZD9HF1t4/5y9T4OvcnD+HmO9dzENMhg9FyyFLv8/4ZfdkS03vH18KEPZuDB5kzamyMJftOHqQ9zBZtslDZAbbTnxdUrdaEG24L1IQg8BN+IL9TcehhpGDbgWfTLR2AdupdJIwMer+37cnH+dAgg2tVR9noG9MLbDKgpl8ros7sWJiXeylu7XrjpJ+Sf4Ie8kro+QbpwVnkuOn+NIj3jqNBqktwQnfOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EvdNuHxVJYtUm0+fNiQ8KDyjswg/Ia9sAuZL0oMSPR0=; b=T5FWaXvhcl08if6HMr3xKPXDaxK2OilV61dMFmqrwh1egqWQFLzSUptSdeBAQ8gNJVy2kqIAC7pneAnwALfTt/vhFoxCrvpOx9ahASl9sMYI6aVgUPBzuom17KkrJxTLjZbBcqSv3vt/pzBRRFWBDh4XFlQdhc4h5ePLPZwzy4LtqIPN+FAKd6W+0zP6W5v2dr7CRZXHtQZ2cs9OyGYRrzH6GeQSvhmuxO240u6hLSng7huRFUPYqACki2FG1nMAf+riaY53qvr+Yv6PByXVFXETCeDFYkJV0yPmCT24qzoy4XGTmFFxh/FTsaPUvYbLqrKf9kkEFF+Q6G7qogtdfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) by DM4PR11MB7184.namprd11.prod.outlook.com (2603:10b6:8:110::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Thu, 20 Apr 2023 18:21:19 +0000 Received: from DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458]) by DM4PR11MB5389.namprd11.prod.outlook.com ([fe80::31f7:5096:35b0:b458%2]) with mapi id 15.20.6319.022; Thu, 20 Apr 2023 18:21:19 +0000 From: Francois Dugast To: Subject: [PATCH v4 3/3] drm/xe: switch to using drm_exec Date: Thu, 20 Apr 2023 20:21:02 +0200 Message-ID: <20230420182102.3302646-4-francois.dugast@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230420182102.3302646-1-francois.dugast@intel.com> References: <20230420182102.3302646-1-francois.dugast@intel.com> X-ClientProxiedBy: LO4P123CA0348.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::11) To DM4PR11MB5389.namprd11.prod.outlook.com (2603:10b6:5:394::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB5389:EE_|DM4PR11MB7184:EE_ X-MS-Office365-Filtering-Correlation-Id: 02751370-7cac-4fce-566d-08db41cc092e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9b0Zrs61g4OtvVr5BrRnlHG5osQ/+OJmUeBXT3vQLQ4uQyLfueJrcwWVUHzCWTgO610xYD7hZpAuKrRmNrwn5Q9dFXCj+1ratH9+Qe56xnVJRH8YBkScwm/jSGbmOcEHnMwaXHuWCuPUATo1UTembYxnba7fNlXF/uzxmUhWgFPENciAPQYXHHE1m67XCot3rg4Q1wO29HqDcBj3dvUjBYX+k2mwY6cS4y2/bk9cFzT4X1yUlEGJEq/bqYstIhcE96htiDxf6SXqs0C1GOxATq/R3IfKk8/CLM/rrBHT9T013eFIKS5LMF7sw/3J3ABcSsLL59CqL81uDAYw0dqCgAjkbXPNafmL+xvFwboq1tfxLc5gdax17Za6chjgf48VrwlOuiGen1b6itReraB5tKV2FqX1XbbBmUGj4zIO79RR7/ifOGI0DhkDPJKPcbAkEDUHnqr3swjK/Z2nAV8CVmyMexcDee9q14jm4MVupu+URFg4I8KSACl8ubzx+njfyUcR/ulQQVzPlAR+JrleD+JIBnkGe3/e52JlQC1O3ygoQbWfIIgYN300A2GHHo56rZlp0F3P4Qw+BARDOURglTC/awykFMWc3+CvN47coGBsN11u3N2i3tjnGl4Ed0kh X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5389.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(39860400002)(376002)(396003)(366004)(346002)(136003)(451199021)(38100700002)(66899021)(41300700001)(5660300002)(44832011)(8676002)(2906002)(30864003)(8936002)(36756003)(86362001)(6486002)(6666004)(6512007)(1076003)(26005)(6506007)(107886003)(478600001)(2616005)(83380400001)(186003)(66556008)(66476007)(66946007)(316002)(6916009)(4326008)(82960400001)(21314003)(579004)(309714004); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?c0FjlHmsIV3Albr97dr73ydEFB6L?= =?utf-8?q?Fv3nH/qq58tacWSVPrClzDTmWfOOXplgPAWsanKSxFh3Wwv+qpWvcyFBakei4pkDD?= =?utf-8?q?qk/+IlqpJ8ELSOq1xtBJjHGiS2CsOd8buZM7ZVqR05UUvs2l5+x4t07Z5LxXAEdkF?= =?utf-8?q?EF/KRfvFQJol+NL21rmHS7Ll11EVr4tjl2ddWYdQQv9W+FLO/d9eRsuCme48zmc6I?= =?utf-8?q?OlHJyKPmlmqRRIeZnaYLfEFbP+6Dld721h5O1osms3Ncw/52mc0EqBTPMSa/Zc2Fz?= =?utf-8?q?pTe3WgK/4LIVnx88T8wsOim7cXtPWETxJSjgV6o2ePAZ/rDCpF3AUtzIgK5ex79Nv?= =?utf-8?q?QoNZnTcOaPBh9pJdaw0kt6Q49URsGZ5caeOSAiqZToHb2opAdK1vkWJQSMDmUWFnv?= =?utf-8?q?MrOuGaF+G0mzkj6z1JSIFbfq51CcOFmVJLeyWn1m/eLX+KOwFqhQ/Q4L4AqmPkRpx?= =?utf-8?q?RFFKgVfEGEZVobtM7/ZLJsD807AO7D3Q1qSQtjCK/OFf3rCGbCR3GX+1VCdiiTvWo?= =?utf-8?q?w1Nqp1l3MWTPqZjgxFN034Vsu/PPjcRJaiPHego2h/qGmMZx01wnYXWhen9uqUAs+?= =?utf-8?q?pYqyo9dahep/YN6wDWyJOb++Vb32CTS4+3/6fM0+G382LHxuvhhDCMDW0avhOeaIT?= =?utf-8?q?2CkeEo3kqshoawTgdpF3WNV8v58eeZVcF2KErbb+sMpwaDzqCk+NiSesK6Rrv2Kms?= =?utf-8?q?fIDe87qzgpfMoEnpAM8dfrLgnVvJrT3KN/f7ONnIMy0aFU4RFlwY6cmsVlOJY3eW0?= =?utf-8?q?Hf0PZVF1u5Ay9HdAkqSeg8U/jiqyJ7B3aYlG0E1OGQ00bub4xB7ylF2M1k/A34QDP?= =?utf-8?q?1DNGNj2NbTTqrZZzO5j/Iyn7D7ShV0aZYflcJ/lJMdcECANh5CXCEEmtpUj9nq+BA?= =?utf-8?q?CBsODx6+lNTfIMTgxgMCioXoi3Ox/mlQ9JwsJWKZTifWPAmy/6o86ji4/a9j+voGU?= =?utf-8?q?+oHKjXNoi47iCf8uO218MRmtXGaDFKuMLy/ZJkIk//3i2bP/7OQsqsSiQN93D1Xd9?= =?utf-8?q?nbNvyCxThSDjjjeBkQZyUxxXRHunO/+ge3RDVulkR597V1aZTsKYJzlfqSV/F4gSl?= =?utf-8?q?afXmLzAzC48GrGVo1Z9nRUx5sdfNBXvb6T9+XA6tC2m27O2BxxTRfjpwf3TwaRqNM?= =?utf-8?q?nIk8HCyXFfIE9AhYOZ9AZX+eI0B9ohQODoRBKy4o099OVXdM6xLegjojAh3ous6RP?= =?utf-8?q?EvhVZmMCApmYY941vAKsISas/DiBxxj7fcLiyZX+Kb1J5ZOakB2lvw8Fbj6jrw4Lt?= =?utf-8?q?poy9eeaqQlGUL1jwdKNgNrx/+hBlA+LIHcMV1UVPmLPWpmw9eYeNU4MAVg59f2iAs?= =?utf-8?q?suugZjKSPNNZwz49HcLe3aQDs28+ucBz9UrxybZkzp2bmKW9d83NArbesuz5/E7BL?= =?utf-8?q?JzIaHwYd9I9AKXKsohxwJK4HzRjNEYbcU1QSZgRQnowS4pnSMAFuUjK/DLT7+RW8w?= =?utf-8?q?4ZGWBWiqvtSR0YQM60wLjYbLEc3pvCZAzgYQbZb2dDdz62Hb1cKJKfgtGyiA+9hrs?= =?utf-8?q?z953z7BaiEAaNGnt0Fg5LRRUtYJVdKHo8A=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 02751370-7cac-4fce-566d-08db41cc092e X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5389.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2023 18:21:18.9285 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0AaLOv1KrZuZZoETv9ykHRzr3YM8c1m1ZXGMCKSYCb6vtw+06/oLckO/K6gW4F5xV3pxKRWSYjPM83O3QoptmJ4WoErAMHvIhargchWdy9U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB7184 X-OriginatorOrg: intel.com X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, Francois Dugast , lucas.demarchi@intel.com, dri-devel@lists.freedesktop.org, dakr@redhat.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Replace the use of ttm_execbuf_util helpers with the drm_exec helpers. Signed-off-by: Francois Dugast Signed-off-by: Matthew Brost --- drivers/gpu/drm/i915/display/intel_display.c | 7 +- drivers/gpu/drm/xe/Kconfig | 1 + drivers/gpu/drm/xe/tests/xe_bo.c | 26 +- drivers/gpu/drm/xe/tests/xe_migrate.c | 6 +- drivers/gpu/drm/xe/xe_bo.c | 35 +- drivers/gpu/drm/xe/xe_bo.h | 6 +- drivers/gpu/drm/xe/xe_bo_evict.c | 24 +- drivers/gpu/drm/xe/xe_bo_types.h | 1 - drivers/gpu/drm/xe/xe_engine.c | 6 +- drivers/gpu/drm/xe/xe_exec.c | 30 +- drivers/gpu/drm/xe/xe_gt_pagefault.c | 56 +--- drivers/gpu/drm/xe/xe_lrc.c | 6 +- drivers/gpu/drm/xe/xe_migrate.c | 12 +- drivers/gpu/drm/xe/xe_vm.c | 321 +++++++++---------- drivers/gpu/drm/xe/xe_vm.h | 33 +- drivers/gpu/drm/xe/xe_vm_madvise.c | 36 +-- 16 files changed, 276 insertions(+), 330 deletions(-) diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c index 6d9abc95dbcd..4f317f984e57 100644 --- a/drivers/gpu/drm/i915/display/intel_display.c +++ b/drivers/gpu/drm/i915/display/intel_display.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -7340,11 +7341,11 @@ static int i915_gem_object_read_from_page(struct xe_bo *bo, void *virtual; bool is_iomem; int ret; - struct ww_acquire_ctx ww; + struct drm_exec exec; XE_BUG_ON(size != 8); - ret = xe_bo_lock(bo, &ww, 0, true); + ret = xe_bo_lock(bo, &exec, 0, true); if (ret) return ret; @@ -7361,7 +7362,7 @@ static int i915_gem_object_read_from_page(struct xe_bo *bo, ttm_bo_kunmap(&map); out_unlock: - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); return ret; } #endif diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig index f6f3b491d162..bbcc9b64b776 100644 --- a/drivers/gpu/drm/xe/Kconfig +++ b/drivers/gpu/drm/xe/Kconfig @@ -8,6 +8,7 @@ config DRM_XE select SHMEM select TMPFS select DRM_BUDDY + select DRM_EXEC select DRM_KMS_HELPER select DRM_PANEL select DRM_SUBALLOC_HELPER diff --git a/drivers/gpu/drm/xe/tests/xe_bo.c b/drivers/gpu/drm/xe/tests/xe_bo.c index 9bd381e5b7a6..316c6cf2bb86 100644 --- a/drivers/gpu/drm/xe/tests/xe_bo.c +++ b/drivers/gpu/drm/xe/tests/xe_bo.c @@ -175,17 +175,17 @@ static int evict_test_run_gt(struct xe_device *xe, struct xe_gt *gt, struct kuni unsigned int bo_flags = XE_BO_CREATE_USER_BIT | XE_BO_CREATE_VRAM_IF_DGFX(gt); struct xe_vm *vm = xe_migrate_get_vm(xe->gt[0].migrate); - struct ww_acquire_ctx ww; + struct drm_exec exec; int err, i; kunit_info(test, "Testing device %s gt id %u vram id %u\n", dev_name(xe->drm.dev), gt->info.id, gt->info.vram_id); for (i = 0; i < 2; ++i) { - xe_vm_lock(vm, &ww, 0, false); + xe_vm_lock(vm, &exec, 0, false); bo = xe_bo_create(xe, NULL, vm, 0x10000, ttm_bo_type_device, bo_flags); - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); if (IS_ERR(bo)) { KUNIT_FAIL(test, "bo create err=%pe\n", bo); break; @@ -198,9 +198,9 @@ static int evict_test_run_gt(struct xe_device *xe, struct xe_gt *gt, struct kuni goto cleanup_bo; } - xe_bo_lock(external, &ww, 0, false); + xe_bo_lock(external, &exec, 0, false); err = xe_bo_pin_external(external); - xe_bo_unlock(external, &ww); + xe_bo_unlock(external, &exec); if (err) { KUNIT_FAIL(test, "external bo pin err=%pe\n", ERR_PTR(err)); @@ -240,18 +240,18 @@ static int evict_test_run_gt(struct xe_device *xe, struct xe_gt *gt, struct kuni if (i) { down_read(&vm->lock); - xe_vm_lock(vm, &ww, 0, false); + xe_vm_lock(vm, &exec, 0, false); err = xe_bo_validate(bo, bo->vm, false); - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); up_read(&vm->lock); if (err) { KUNIT_FAIL(test, "bo valid err=%pe\n", ERR_PTR(err)); goto cleanup_all; } - xe_bo_lock(external, &ww, 0, false); + xe_bo_lock(external, &exec, 0, false); err = xe_bo_validate(external, NULL, false); - xe_bo_unlock(external, &ww); + xe_bo_unlock(external, &exec); if (err) { KUNIT_FAIL(test, "external bo valid err=%pe\n", ERR_PTR(err)); @@ -259,18 +259,18 @@ static int evict_test_run_gt(struct xe_device *xe, struct xe_gt *gt, struct kuni } } - xe_bo_lock(external, &ww, 0, false); + xe_bo_lock(external, &exec, 0, false); xe_bo_unpin_external(external); - xe_bo_unlock(external, &ww); + xe_bo_unlock(external, &exec); xe_bo_put(external); xe_bo_put(bo); continue; cleanup_all: - xe_bo_lock(external, &ww, 0, false); + xe_bo_lock(external, &exec, 0, false); xe_bo_unpin_external(external); - xe_bo_unlock(external, &ww); + xe_bo_unlock(external, &exec); cleanup_external: xe_bo_put(external); cleanup_bo: diff --git a/drivers/gpu/drm/xe/tests/xe_migrate.c b/drivers/gpu/drm/xe/tests/xe_migrate.c index cdcecf8d5eef..7b0c76956146 100644 --- a/drivers/gpu/drm/xe/tests/xe_migrate.c +++ b/drivers/gpu/drm/xe/tests/xe_migrate.c @@ -393,14 +393,14 @@ static int migrate_test_run_device(struct xe_device *xe) for_each_gt(gt, xe, id) { struct xe_migrate *m = gt->migrate; - struct ww_acquire_ctx ww; + struct drm_exec exec; kunit_info(test, "Testing gt id %d.\n", id); - xe_vm_lock(m->eng->vm, &ww, 0, true); + xe_vm_lock(m->eng->vm, &exec, 0, true); xe_device_mem_access_get(xe); xe_migrate_sanity_test(m, test); xe_device_mem_access_put(xe); - xe_vm_unlock(m->eng->vm, &ww); + xe_vm_unlock(m->eng->vm, &exec); } return 0; diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 3ab404e33fae..3a1c1b5e3284 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -8,6 +8,7 @@ #include #include +#include #include #include #include @@ -1626,7 +1627,7 @@ int xe_gem_create_ioctl(struct drm_device *dev, void *data, struct xe_device *xe = to_xe_device(dev); struct xe_file *xef = to_xe_file(file); struct drm_xe_gem_create *args = data; - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_vm *vm = NULL; struct xe_bo *bo; unsigned bo_flags = XE_BO_CREATE_USER_BIT; @@ -1662,7 +1663,7 @@ int xe_gem_create_ioctl(struct drm_device *dev, void *data, vm = xe_vm_lookup(xef, args->vm_id); if (XE_IOCTL_ERR(xe, !vm)) return -ENOENT; - err = xe_vm_lock(vm, &ww, 0, true); + err = xe_vm_lock(vm, &exec, 0, true); if (err) { xe_vm_put(vm); return err; @@ -1679,7 +1680,7 @@ int xe_gem_create_ioctl(struct drm_device *dev, void *data, bo = xe_bo_create(xe, NULL, vm, args->size, ttm_bo_type_device, bo_flags); if (vm) { - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); xe_vm_put(vm); } @@ -1720,26 +1721,30 @@ int xe_gem_mmap_offset_ioctl(struct drm_device *dev, void *data, return 0; } -int xe_bo_lock(struct xe_bo *bo, struct ww_acquire_ctx *ww, +int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec, int num_resv, bool intr) { - struct ttm_validate_buffer tv_bo; - LIST_HEAD(objs); - LIST_HEAD(dups); + int err; - XE_BUG_ON(!ww); + drm_exec_init(exec, intr); + drm_exec_while_not_all_locked(exec) { + err = drm_exec_prepare_obj(exec, &bo->ttm.base, + num_resv); + drm_exec_continue_on_contention(exec); + if (err && err != -EALREADY) + goto out_err; + } - tv_bo.num_shared = num_resv; - tv_bo.bo = &bo->ttm;; - list_add_tail(&tv_bo.head, &objs); + return 0; - return ttm_eu_reserve_buffers(ww, &objs, intr, &dups); +out_err: + drm_exec_fini(exec); + return err; } -void xe_bo_unlock(struct xe_bo *bo, struct ww_acquire_ctx *ww) +void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec) { - dma_resv_unlock(bo->ttm.base.resv); - ww_acquire_fini(ww); + drm_exec_fini(exec); } /** diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h index effa9d0cf0f6..553d9270fffb 100644 --- a/drivers/gpu/drm/xe/xe_bo.h +++ b/drivers/gpu/drm/xe/xe_bo.h @@ -74,6 +74,7 @@ #define XE_BO_PROPS_INVALID (-1) +struct drm_exec; struct sg_table; struct xe_bo *xe_bo_alloc(void); @@ -141,10 +142,9 @@ static inline void xe_bo_assert_held(struct xe_bo *bo) dma_resv_assert_held((bo)->ttm.base.resv); } -int xe_bo_lock(struct xe_bo *bo, struct ww_acquire_ctx *ww, +int xe_bo_lock(struct xe_bo *bo, struct drm_exec *exec, int num_resv, bool intr); - -void xe_bo_unlock(struct xe_bo *bo, struct ww_acquire_ctx *ww); +void xe_bo_unlock(struct xe_bo *bo, struct drm_exec *exec); static inline void xe_bo_unlock_vm_held(struct xe_bo *bo) { diff --git a/drivers/gpu/drm/xe/xe_bo_evict.c b/drivers/gpu/drm/xe/xe_bo_evict.c index 6642c5f52009..46d9d9eb110c 100644 --- a/drivers/gpu/drm/xe/xe_bo_evict.c +++ b/drivers/gpu/drm/xe/xe_bo_evict.c @@ -3,6 +3,8 @@ * Copyright © 2022 Intel Corporation */ +#include + #include "xe_bo_evict.h" #include "xe_bo.h" @@ -27,7 +29,7 @@ int xe_bo_evict_all(struct xe_device *xe) { struct ttm_device *bdev = &xe->ttm; - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_bo *bo; struct xe_gt *gt; struct list_head still_in_list; @@ -62,9 +64,9 @@ int xe_bo_evict_all(struct xe_device *xe) list_move_tail(&bo->pinned_link, &still_in_list); spin_unlock(&xe->pinned.lock); - xe_bo_lock(bo, &ww, 0, false); + xe_bo_lock(bo, &exec, 0, false); ret = xe_bo_evict_pinned(bo); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); xe_bo_put(bo); if (ret) { spin_lock(&xe->pinned.lock); @@ -96,9 +98,9 @@ int xe_bo_evict_all(struct xe_device *xe) list_move_tail(&bo->pinned_link, &xe->pinned.evicted); spin_unlock(&xe->pinned.lock); - xe_bo_lock(bo, &ww, 0, false); + xe_bo_lock(bo, &exec, 0, false); ret = xe_bo_evict_pinned(bo); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); xe_bo_put(bo); if (ret) return ret; @@ -123,7 +125,7 @@ int xe_bo_evict_all(struct xe_device *xe) */ int xe_bo_restore_kernel(struct xe_device *xe) { - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_bo *bo; int ret; @@ -140,9 +142,9 @@ int xe_bo_restore_kernel(struct xe_device *xe) list_move_tail(&bo->pinned_link, &xe->pinned.kernel_bo_present); spin_unlock(&xe->pinned.lock); - xe_bo_lock(bo, &ww, 0, false); + xe_bo_lock(bo, &exec, 0, false); ret = xe_bo_restore_pinned(bo); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); if (ret) { xe_bo_put(bo); return ret; @@ -182,7 +184,7 @@ int xe_bo_restore_kernel(struct xe_device *xe) */ int xe_bo_restore_user(struct xe_device *xe) { - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_bo *bo; struct xe_gt *gt; struct list_head still_in_list; @@ -204,9 +206,9 @@ int xe_bo_restore_user(struct xe_device *xe) xe_bo_get(bo); spin_unlock(&xe->pinned.lock); - xe_bo_lock(bo, &ww, 0, false); + xe_bo_lock(bo, &exec, 0, false); ret = xe_bo_restore_pinned(bo); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); xe_bo_put(bo); if (ret) { spin_lock(&xe->pinned.lock); diff --git a/drivers/gpu/drm/xe/xe_bo_types.h b/drivers/gpu/drm/xe/xe_bo_types.h index 06de3330211d..2ba34a8c9b66 100644 --- a/drivers/gpu/drm/xe/xe_bo_types.h +++ b/drivers/gpu/drm/xe/xe_bo_types.h @@ -11,7 +11,6 @@ #include #include #include -#include #include struct xe_device; diff --git a/drivers/gpu/drm/xe/xe_engine.c b/drivers/gpu/drm/xe/xe_engine.c index 094ec17d3004..0840006f644d 100644 --- a/drivers/gpu/drm/xe/xe_engine.c +++ b/drivers/gpu/drm/xe/xe_engine.c @@ -88,18 +88,18 @@ struct xe_engine *xe_engine_create(struct xe_device *xe, struct xe_vm *vm, u32 logical_mask, u16 width, struct xe_hw_engine *hwe, u32 flags) { - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_engine *e; int err; if (vm) { - err = xe_vm_lock(vm, &ww, 0, true); + err = xe_vm_lock(vm, &exec, 0, true); if (err) return ERR_PTR(err); } e = __xe_engine_create(xe, vm, logical_mask, width, hwe, flags); if (vm) - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); return e; } diff --git a/drivers/gpu/drm/xe/xe_exec.c b/drivers/gpu/drm/xe/xe_exec.c index ea869f2452ef..b7f0a2f551a6 100644 --- a/drivers/gpu/drm/xe/xe_exec.c +++ b/drivers/gpu/drm/xe/xe_exec.c @@ -6,6 +6,7 @@ #include "xe_exec.h" #include +#include #include #include @@ -91,21 +92,16 @@ * Unlock all */ -static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww, - struct ttm_validate_buffer tv_onstack[], - struct ttm_validate_buffer **tv, - struct list_head *objs) +static int xe_exec_begin(struct xe_engine *e, struct drm_exec *exec) { struct xe_vm *vm = e->vm; struct xe_vma *vma; - LIST_HEAD(dups); int err; - *tv = NULL; if (xe_vm_no_dma_fences(e->vm)) return 0; - err = xe_vm_lock_dma_resv(vm, ww, tv_onstack, tv, objs, true, 1); + err = xe_vm_lock_dma_resv(vm, exec, true, 1); if (err) return err; @@ -120,8 +116,7 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww, err = xe_bo_validate(vma->bo, vm, false); if (err) { - xe_vm_unlock_dma_resv(vm, tv_onstack, *tv, ww, objs); - *tv = NULL; + xe_vm_unlock_dma_resv(vm, exec); return err; } } @@ -129,14 +124,10 @@ static int xe_exec_begin(struct xe_engine *e, struct ww_acquire_ctx *ww, return 0; } -static void xe_exec_end(struct xe_engine *e, - struct ttm_validate_buffer *tv_onstack, - struct ttm_validate_buffer *tv, - struct ww_acquire_ctx *ww, - struct list_head *objs) +static void xe_exec_end(struct xe_engine *e, struct drm_exec *exec) { if (!xe_vm_no_dma_fences(e->vm)) - xe_vm_unlock_dma_resv(e->vm, tv_onstack, tv, ww, objs); + xe_vm_unlock_dma_resv(e->vm, exec); } int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) @@ -149,14 +140,11 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) struct xe_engine *engine; struct xe_sync_entry *syncs = NULL; u64 addresses[XE_HW_ENGINE_MAX_INSTANCE]; - struct ttm_validate_buffer tv_onstack[XE_ONSTACK_TV]; - struct ttm_validate_buffer *tv = NULL; u32 i, num_syncs = 0; struct xe_sched_job *job; struct dma_fence *rebind_fence; struct xe_vm *vm; - struct ww_acquire_ctx ww; - struct list_head objs; + struct drm_exec exec; bool write_locked; int err = 0; @@ -267,7 +255,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_unlock_list; } - err = xe_exec_begin(engine, &ww, tv_onstack, &tv, &objs); + err = xe_exec_begin(engine, &exec); if (err) goto err_unlock_list; @@ -373,7 +361,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) if (err) xe_sched_job_put(job); err_engine_end: - xe_exec_end(engine, tv_onstack, tv, &ww, &objs); + xe_exec_end(engine, &exec); err_unlock_list: if (write_locked) up_write(&vm->lock); diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c index 1677640e1075..365a675f3663 100644 --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c @@ -8,8 +8,8 @@ #include #include +#include #include -#include #include "xe_bo.h" #include "xe_gt.h" @@ -83,11 +83,6 @@ static bool vma_matches(struct xe_vma *vma, struct xe_vma *lookup) return true; } -static bool only_needs_bo_lock(struct xe_bo *bo) -{ - return bo && bo->vm; -} - static struct xe_vma *lookup_vma(struct xe_vm *vm, u64 page_addr) { struct xe_vma *vma = NULL, lookup; @@ -110,10 +105,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) struct xe_vm *vm; struct xe_vma *vma = NULL; struct xe_bo *bo; - LIST_HEAD(objs); - LIST_HEAD(dups); - struct ttm_validate_buffer tv_bo, tv_vm; - struct ww_acquire_ctx ww; + struct drm_exec exec; struct dma_fence *fence; bool write_locked; int ret = 0; @@ -171,20 +163,8 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) /* Lock VM and BOs dma-resv */ bo = vma->bo; - if (only_needs_bo_lock(bo)) { - /* This path ensures the BO's LRU is updated */ - ret = xe_bo_lock(bo, &ww, xe->info.tile_count, false); - } else { - tv_vm.num_shared = xe->info.tile_count; - tv_vm.bo = xe_vm_ttm_bo(vm); - list_add(&tv_vm.head, &objs); - if (bo) { - tv_bo.bo = &bo->ttm; - tv_bo.num_shared = xe->info.tile_count; - list_add(&tv_bo.head, &objs); - } - ret = ttm_eu_reserve_buffers(&ww, &objs, false, &dups); - } + ret = xe_vm_bo_lock(vm, bo, &exec, xe->info.tile_count, false); + if (ret) goto unlock_vm; @@ -227,10 +207,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) vma->usm.gt_invalidated &= ~BIT(gt->info.id); unlock_dma_resv: - if (only_needs_bo_lock(bo)) - xe_bo_unlock(bo, &ww); - else - ttm_eu_backoff_reservation(&ww, &objs); + xe_vm_bo_unlock(vm, bo, &exec, true); unlock_vm: if (!ret) vm->usm.last_fault_vma = vma; @@ -501,10 +478,7 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc) struct xe_vm *vm; struct xe_vma *vma; struct xe_bo *bo; - LIST_HEAD(objs); - LIST_HEAD(dups); - struct ttm_validate_buffer tv_bo, tv_vm; - struct ww_acquire_ctx ww; + struct drm_exec exec; int ret = 0; /* We only support ACC_TRIGGER at the moment */ @@ -537,28 +511,14 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc) /* Lock VM and BOs dma-resv */ bo = vma->bo; - if (only_needs_bo_lock(bo)) { - /* This path ensures the BO's LRU is updated */ - ret = xe_bo_lock(bo, &ww, xe->info.tile_count, false); - } else { - tv_vm.num_shared = xe->info.tile_count; - tv_vm.bo = xe_vm_ttm_bo(vm); - list_add(&tv_vm.head, &objs); - tv_bo.bo = &bo->ttm; - tv_bo.num_shared = xe->info.tile_count; - list_add(&tv_bo.head, &objs); - ret = ttm_eu_reserve_buffers(&ww, &objs, false, &dups); - } + ret = xe_vm_bo_lock(vm, bo, &exec, xe->info.tile_count, false); if (ret) goto unlock_vm; /* Migrate to VRAM, move should invalidate the VMA first */ ret = xe_bo_migrate(bo, XE_PL_VRAM0 + gt->info.vram_id); - if (only_needs_bo_lock(bo)) - xe_bo_unlock(bo, &ww); - else - ttm_eu_backoff_reservation(&ww, &objs); + xe_vm_bo_unlock(vm, bo, &exec, true); unlock_vm: up_read(&vm->lock); xe_vm_put(vm); diff --git a/drivers/gpu/drm/xe/xe_lrc.c b/drivers/gpu/drm/xe/xe_lrc.c index fb8c6f7d6528..bc07a7c0718d 100644 --- a/drivers/gpu/drm/xe/xe_lrc.c +++ b/drivers/gpu/drm/xe/xe_lrc.c @@ -709,16 +709,16 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, void xe_lrc_finish(struct xe_lrc *lrc) { - struct ww_acquire_ctx ww; + struct drm_exec exec; xe_hw_fence_ctx_finish(&lrc->fence_ctx); if (lrc->bo->vm) - xe_vm_lock(lrc->bo->vm, &ww, 0, false); + xe_vm_lock(lrc->bo->vm, &exec, 0, false); else xe_bo_lock_no_vm(lrc->bo, NULL); xe_bo_unpin(lrc->bo); if (lrc->bo->vm) - xe_vm_unlock(lrc->bo->vm, &ww); + xe_vm_unlock(lrc->bo->vm, &exec); else xe_bo_unlock_no_vm(lrc->bo); xe_bo_put(lrc->bo); diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index 8686a2a7b035..97238971458d 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -86,13 +86,13 @@ struct xe_engine *xe_gt_migrate_engine(struct xe_gt *gt) static void xe_migrate_fini(struct drm_device *dev, void *arg) { struct xe_migrate *m = arg; - struct ww_acquire_ctx ww; + struct drm_exec exec; - xe_vm_lock(m->eng->vm, &ww, 0, false); + xe_vm_lock(m->eng->vm, &exec, 0, false); xe_bo_unpin(m->pt_bo); if (m->cleared_bo) xe_bo_unpin(m->cleared_bo); - xe_vm_unlock(m->eng->vm, &ww); + xe_vm_unlock(m->eng->vm, &exec); dma_fence_put(m->fence); if (m->cleared_bo) @@ -315,7 +315,7 @@ struct xe_migrate *xe_migrate_init(struct xe_gt *gt) struct xe_device *xe = gt_to_xe(gt); struct xe_migrate *m; struct xe_vm *vm; - struct ww_acquire_ctx ww; + struct drm_exec exec; int err; XE_BUG_ON(xe_gt_is_media_type(gt)); @@ -332,9 +332,9 @@ struct xe_migrate *xe_migrate_init(struct xe_gt *gt) if (IS_ERR(vm)) return ERR_CAST(vm); - xe_vm_lock(vm, &ww, 0, false); + xe_vm_lock(vm, &exec, 0, false); err = xe_migrate_prepare_vm(gt, m, vm); - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); if (err) { xe_vm_close_and_put(vm); return ERR_PTR(err); diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index bdf82d34eb66..3a53a2d65853 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -7,7 +7,7 @@ #include -#include +#include #include #include #include @@ -261,10 +261,10 @@ static void arm_preempt_fences(struct xe_vm *vm, struct list_head *list) static int add_preempt_fences(struct xe_vm *vm, struct xe_bo *bo) { struct xe_engine *e; - struct ww_acquire_ctx ww; + struct drm_exec exec; int err; - err = xe_bo_lock(bo, &ww, vm->preempt.num_engines, true); + err = xe_bo_lock(bo, &exec, vm->preempt.num_engines, true); if (err) return err; @@ -275,7 +275,7 @@ static int add_preempt_fences(struct xe_vm *vm, struct xe_bo *bo) DMA_RESV_USAGE_BOOKKEEP); } - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); return 0; } @@ -317,11 +317,8 @@ static void resume_and_reinstall_preempt_fences(struct xe_vm *vm) int xe_vm_add_compute_engine(struct xe_vm *vm, struct xe_engine *e) { - struct ttm_validate_buffer tv_onstack[XE_ONSTACK_TV]; - struct ttm_validate_buffer *tv; - struct ww_acquire_ctx ww; - struct list_head objs; struct dma_fence *pfence; + struct drm_exec exec; int err; bool wait; @@ -329,7 +326,7 @@ int xe_vm_add_compute_engine(struct xe_vm *vm, struct xe_engine *e) down_write(&vm->lock); - err = xe_vm_lock_dma_resv(vm, &ww, tv_onstack, &tv, &objs, true, 1); + err = xe_vm_lock_dma_resv(vm, &exec, true, 1); if (err) goto out_unlock_outer; @@ -363,7 +360,7 @@ int xe_vm_add_compute_engine(struct xe_vm *vm, struct xe_engine *e) up_read(&vm->userptr.notifier_lock); out_unlock: - xe_vm_unlock_dma_resv(vm, tv_onstack, tv, &ww, &objs); + xe_vm_unlock_dma_resv(vm, &exec); out_unlock_outer: up_write(&vm->lock); @@ -389,72 +386,57 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm) list_empty(&vm->userptr.invalidated)) ? 0 : -EAGAIN; } +static struct drm_gem_object *xe_vm_gem(struct xe_vm *vm) +{ + int idx = vm->flags & XE_VM_FLAG_MIGRATION ? + XE_VM_FLAG_GT_ID(vm->flags) : 0; + + /* Safe to use index 0 as all BO in the VM share a single dma-resv lock */ + return &vm->pt_root[idx]->bo->ttm.base; +} + + /** * xe_vm_lock_dma_resv() - Lock the vm dma_resv object and the dma_resv * objects of the vm's external buffer objects. * @vm: The vm. - * @ww: Pointer to a struct ww_acquire_ctx locking context. - * @tv_onstack: Array size XE_ONSTACK_TV of storage for the struct - * ttm_validate_buffers used for locking. - * @tv: Pointer to a pointer that on output contains the actual storage used. - * @objs: List head for the buffer objects locked. + * @exec: Pointer to a struct drm_exec execution context. * @intr: Whether to lock interruptible. * @num_shared: Number of dma-fence slots to reserve in the locked objects. * * Locks the vm dma-resv objects and all the dma-resv objects of the - * buffer objects on the vm external object list. The TTM utilities require - * a list of struct ttm_validate_buffers pointing to the actual buffer - * objects to lock. Storage for those struct ttm_validate_buffers should - * be provided in @tv_onstack, and is typically reserved on the stack - * of the caller. If the size of @tv_onstack isn't sufficient, then - * storage will be allocated internally using kvmalloc(). + * buffer objects on the vm external object list using helpers provided + * by drm_exec. * * The function performs deadlock handling internally, and after a * successful return the ww locking transaction should be considered * sealed. * - * Return: 0 on success, Negative error code on error. In particular if - * @intr is set to true, -EINTR or -ERESTARTSYS may be returned. In case - * of error, any locking performed has been reverted. + * Return: 0 on success, Negative error code on error. */ -int xe_vm_lock_dma_resv(struct xe_vm *vm, struct ww_acquire_ctx *ww, - struct ttm_validate_buffer *tv_onstack, - struct ttm_validate_buffer **tv, - struct list_head *objs, - bool intr, - unsigned int num_shared) -{ - struct ttm_validate_buffer *tv_vm, *tv_bo; +int xe_vm_lock_dma_resv(struct xe_vm *vm, struct drm_exec *exec, + bool intr, unsigned int num_shared) +{ struct xe_vma *vma, *next; - LIST_HEAD(dups); + struct drm_gem_object *obj; int err; lockdep_assert_held(&vm->lock); - if (vm->extobj.entries < XE_ONSTACK_TV) { - tv_vm = tv_onstack; - } else { - tv_vm = kvmalloc_array(vm->extobj.entries + 1, sizeof(*tv_vm), - GFP_KERNEL); - if (!tv_vm) - return -ENOMEM; - } - tv_bo = tv_vm + 1; - - INIT_LIST_HEAD(objs); - list_for_each_entry(vma, &vm->extobj.list, extobj.link) { - tv_bo->num_shared = num_shared; - tv_bo->bo = &vma->bo->ttm; - - list_add_tail(&tv_bo->head, objs); - tv_bo++; + drm_exec_init(exec, intr); + drm_exec_while_not_all_locked(exec) { + err = drm_exec_prepare_obj(exec, xe_vm_gem(vm), num_shared); + drm_exec_continue_on_contention(exec); + if (unlikely(err) && err != -EALREADY) + goto out_err; + list_for_each_entry(vma, &vm->extobj.list, extobj.link) { + obj = &vma->bo->ttm.base; + err = drm_exec_prepare_obj(exec, obj, num_shared); + drm_exec_break_on_contention(exec); + if (unlikely(err) && err != -EALREADY) + goto out_err; + } } - tv_vm->num_shared = num_shared; - tv_vm->bo = xe_vm_ttm_bo(vm); - list_add_tail(&tv_vm->head, objs); - err = ttm_eu_reserve_buffers(ww, objs, intr, &dups); - if (err) - goto out_err; spin_lock(&vm->notifier.list_lock); list_for_each_entry_safe(vma, next, &vm->notifier.rebind_list, @@ -466,14 +448,10 @@ int xe_vm_lock_dma_resv(struct xe_vm *vm, struct ww_acquire_ctx *ww, list_move_tail(&vma->rebind_link, &vm->rebind_list); } spin_unlock(&vm->notifier.list_lock); - - *tv = tv_vm; return 0; out_err: - if (tv_vm != tv_onstack) - kvfree(tv_vm); - + drm_exec_fini(exec); return err; } @@ -481,20 +459,16 @@ int xe_vm_lock_dma_resv(struct xe_vm *vm, struct ww_acquire_ctx *ww, * xe_vm_unlock_dma_resv() - Unlock reservation objects locked by * xe_vm_lock_dma_resv() * @vm: The vm. - * @tv_onstack: The @tv_onstack array given to xe_vm_lock_dma_resv(). - * @tv: The value of *@tv given by xe_vm_lock_dma_resv(). - * @ww: The ww_acquire_context used for locking. - * @objs: The list returned from xe_vm_lock_dma_resv(). + * @exec: The @drm_exec given to xe_vm_lock_dma_resv(). * * Unlocks the reservation objects and frees any memory allocated by * xe_vm_lock_dma_resv(). */ -void xe_vm_unlock_dma_resv(struct xe_vm *vm, - struct ttm_validate_buffer *tv_onstack, - struct ttm_validate_buffer *tv, - struct ww_acquire_ctx *ww, - struct list_head *objs) +void xe_vm_unlock_dma_resv(struct xe_vm *vm, struct drm_exec *exec) { + struct drm_gem_object *obj, *skip = xe_vm_gem(vm); + unsigned long index; + /* * Nothing should've been able to enter the list while we were locked, * since we've held the dma-resvs of all the vm's external objects, @@ -503,20 +477,22 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm, */ XE_WARN_ON(!list_empty(&vm->notifier.rebind_list)); - ttm_eu_backoff_reservation(ww, objs); - if (tv && tv != tv_onstack) - kvfree(tv); + drm_exec_for_each_locked_object(exec, index, obj) { + struct xe_bo *bo = gem_to_xe_bo(obj); + + if (obj != skip) + ttm_bo_move_to_lru_tail_unlocked(&bo->ttm); + } + drm_exec_fini(exec); } + static void preempt_rebind_work_func(struct work_struct *w) { struct xe_vm *vm = container_of(w, struct xe_vm, preempt.rebind_work); struct xe_vma *vma; - struct ttm_validate_buffer tv_onstack[XE_ONSTACK_TV]; - struct ttm_validate_buffer *tv; - struct ww_acquire_ctx ww; - struct list_head objs; struct dma_fence *rebind_fence; + struct drm_exec exec; unsigned int fence_count = 0; LIST_HEAD(preempt_fences); int err; @@ -556,8 +532,7 @@ static void preempt_rebind_work_func(struct work_struct *w) goto out_unlock_outer; } - err = xe_vm_lock_dma_resv(vm, &ww, tv_onstack, &tv, &objs, - false, vm->preempt.num_engines); + err = xe_vm_lock_dma_resv(vm, &exec, false, vm->preempt.num_engines); if (err) goto out_unlock_outer; @@ -631,7 +606,7 @@ static void preempt_rebind_work_func(struct work_struct *w) up_read(&vm->userptr.notifier_lock); out_unlock: - xe_vm_unlock_dma_resv(vm, tv_onstack, tv, &ww, &objs); + xe_vm_unlock_dma_resv(vm, &exec); out_unlock_outer: if (err == -EAGAIN) { trace_xe_vm_rebind_worker_retry(vm); @@ -979,27 +954,16 @@ static void xe_vma_destroy(struct xe_vma *vma, struct dma_fence *fence) static void xe_vma_destroy_unlocked(struct xe_vma *vma) { - struct ttm_validate_buffer tv[2]; - struct ww_acquire_ctx ww; + struct xe_vm *vm = vma->vm; struct xe_bo *bo = vma->bo; - LIST_HEAD(objs); - LIST_HEAD(dups); + struct drm_exec exec; int err; - memset(tv, 0, sizeof(tv)); - tv[0].bo = xe_vm_ttm_bo(vma->vm); - list_add(&tv[0].head, &objs); - - if (bo) { - tv[1].bo = &xe_bo_get(bo)->ttm; - list_add(&tv[1].head, &objs); - } - err = ttm_eu_reserve_buffers(&ww, &objs, false, &dups); + err = xe_vm_bo_lock(vm, xe_bo_get(bo), &exec, 0, false); XE_WARN_ON(err); - xe_vma_destroy(vma, NULL); + xe_vm_bo_unlock(vm, bo, &exec, false); - ttm_eu_backoff_reservation(&ww, &objs); if (bo) xe_bo_put(bo); } @@ -1285,7 +1249,7 @@ static void vm_error_capture(struct xe_vm *vm, int err, void xe_vm_close_and_put(struct xe_vm *vm) { struct rb_root contested = RB_ROOT; - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_device *xe = vm->xe; struct xe_gt *gt; u8 id; @@ -1307,7 +1271,7 @@ void xe_vm_close_and_put(struct xe_vm *vm) } down_write(&vm->lock); - xe_vm_lock(vm, &ww, 0, false); + xe_vm_lock(vm, &exec, 0, false); while (vm->vmas.rb_node) { struct xe_vma *vma = to_xe_vma(vm->vmas.rb_node); @@ -1346,7 +1310,7 @@ void xe_vm_close_and_put(struct xe_vm *vm) NULL); } } - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); if (contested.rb_node) { @@ -1383,7 +1347,7 @@ static void vm_destroy_work_func(struct work_struct *w) { struct xe_vm *vm = container_of(w, struct xe_vm, destroy_work); - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_device *xe = vm->xe; struct xe_gt *gt; u8 id; @@ -1409,14 +1373,14 @@ static void vm_destroy_work_func(struct work_struct *w) * is needed for xe_vm_lock to work. If we remove that dependency this * can be moved to xe_vm_close_and_put. */ - xe_vm_lock(vm, &ww, 0, false); + xe_vm_lock(vm, &exec, 0, false); for_each_gt(gt, xe, id) { if (vm->pt_root[id]) { xe_pt_destroy(vm->pt_root[id], vm->flags, NULL); vm->pt_root[id] = NULL; } } - xe_vm_unlock(vm, &ww); + xe_vm_unlock(vm, &exec); trace_xe_vm_free(vm); dma_fence_put(vm->rebind_fence); @@ -1999,21 +1963,6 @@ static int __vm_bind_ioctl(struct xe_vm *vm, struct xe_vma *vma, } } -struct ttm_buffer_object *xe_vm_ttm_bo(struct xe_vm *vm) -{ - int idx = vm->flags & XE_VM_FLAG_MIGRATION ? - XE_VM_FLAG_GT_ID(vm->flags) : 0; - - /* Safe to use index 0 as all BO in the VM share a single dma-resv lock */ - return &vm->pt_root[idx]->bo->ttm; -} - -static void xe_vm_tv_populate(struct xe_vm *vm, struct ttm_validate_buffer *tv) -{ - tv->num_shared = 1; - tv->bo = xe_vm_ttm_bo(vm); -} - static bool is_map_op(u32 op) { return VM_BIND_OP(op) == XE_VM_BIND_OP_MAP || @@ -2032,11 +1981,8 @@ static int vm_bind_ioctl(struct xe_vm *vm, struct xe_vma *vma, struct xe_sync_entry *syncs, u32 num_syncs, struct async_op_fence *afence) { - LIST_HEAD(objs); - LIST_HEAD(dups); - struct ttm_validate_buffer tv_bo, tv_vm; - struct ww_acquire_ctx ww; struct xe_bo *vbo; + struct drm_exec exec; int err, i; lockdep_assert_held(&vm->lock); @@ -2053,8 +1999,6 @@ static int vm_bind_ioctl(struct xe_vm *vm, struct xe_vma *vma, return 0; } - xe_vm_tv_populate(vm, &tv_vm); - list_add_tail(&tv_vm.head, &objs); vbo = vma->bo; if (vbo) { /* @@ -2063,29 +2007,29 @@ static int vm_bind_ioctl(struct xe_vm *vm, struct xe_vma *vma, * take a reference here. */ xe_bo_get(vbo); - - tv_bo.bo = &vbo->ttm; - tv_bo.num_shared = 1; - list_add(&tv_bo.head, &objs); } again: - err = ttm_eu_reserve_buffers(&ww, &objs, true, &dups); - if (!err) { - err = __vm_bind_ioctl(vm, vma, e, bo, - bind_op->op, bind_op->region, syncs, - num_syncs, afence); - ttm_eu_backoff_reservation(&ww, &objs); - if (err == -EAGAIN && xe_vma_is_userptr(vma)) { - lockdep_assert_held_write(&vm->lock); - err = xe_vma_userptr_pin_pages(vma); - if (!err) - goto again; - } + err = xe_vm_bo_lock(vm, vbo, &exec, 1, true); + if (err) + goto error; + err = __vm_bind_ioctl(vm, vma, e, bo, + bind_op->op, bind_op->region, syncs, + num_syncs, afence); + xe_vm_bo_unlock(vm, vbo, &exec, false); + if (err == -EAGAIN && xe_vma_is_userptr(vma)) { + lockdep_assert_held_write(&vm->lock); + err = xe_vma_userptr_pin_pages(vma); + if (!err) + goto again; } xe_bo_put(vbo); return err; + +error: + xe_bo_put(vbo); + return err; } struct async_op { @@ -2450,18 +2394,18 @@ static int vm_bind_ioctl_async(struct xe_vm *vm, struct xe_vma *vma, static bool bo_has_vm_references(struct xe_bo *bo, struct xe_vm *vm, struct xe_vma *ignore) { - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_vma *vma; bool ret = false; - xe_bo_lock(bo, &ww, 0, false); + xe_bo_lock(bo, &exec, 0, false); list_for_each_entry(vma, &bo->vmas, bo_link) { if (vma != ignore && vma->vm == vm && !vma->destroyed) { ret = true; break; } } - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); return ret; } @@ -2582,10 +2526,10 @@ static struct xe_vma *vm_unbind_lookup_vmas(struct xe_vm *vm, } if (first->start != lookup->start) { - struct ww_acquire_ctx ww; + struct drm_exec exec; if (first->bo) - err = xe_bo_lock(first->bo, &ww, 0, true); + err = xe_bo_lock(first->bo, &exec, 0, true); if (err) goto unwind; new_first = xe_vma_create(first->vm, first->bo, @@ -2596,7 +2540,7 @@ static struct xe_vma *vm_unbind_lookup_vmas(struct xe_vm *vm, (first->pte_flags & PTE_READ_ONLY), first->gt_mask); if (first->bo) - xe_bo_unlock(first->bo, &ww); + xe_bo_unlock(first->bo, &exec); if (!new_first) { err = -ENOMEM; goto unwind; @@ -2612,11 +2556,11 @@ static struct xe_vma *vm_unbind_lookup_vmas(struct xe_vm *vm, } if (last->end != lookup->end) { - struct ww_acquire_ctx ww; + struct drm_exec exec; u64 chunk = lookup->end + 1 - last->start; if (last->bo) - err = xe_bo_lock(last->bo, &ww, 0, true); + err = xe_bo_lock(last->bo, &exec, 0, true); if (err) goto unwind; new_last = xe_vma_create(last->vm, last->bo, @@ -2627,7 +2571,7 @@ static struct xe_vma *vm_unbind_lookup_vmas(struct xe_vm *vm, (last->pte_flags & PTE_READ_ONLY), last->gt_mask); if (last->bo) - xe_bo_unlock(last->bo, &ww); + xe_bo_unlock(last->bo, &exec); if (!new_last) { err = -ENOMEM; goto unwind; @@ -2763,7 +2707,7 @@ static struct xe_vma *vm_bind_ioctl_lookup_vma(struct xe_vm *vm, u64 addr, u64 range, u32 op, u64 gt_mask, u32 region) { - struct ww_acquire_ctx ww; + struct drm_exec exec; struct xe_vma *vma, lookup; int err; @@ -2776,14 +2720,14 @@ static struct xe_vma *vm_bind_ioctl_lookup_vma(struct xe_vm *vm, case XE_VM_BIND_OP_MAP: XE_BUG_ON(!bo); - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return ERR_PTR(err); vma = xe_vma_create(vm, bo, bo_offset_or_userptr, addr, addr + range - 1, op & XE_VM_BIND_FLAG_READONLY, gt_mask); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); if (!vma) return ERR_PTR(-ENOMEM); @@ -2808,13 +2752,13 @@ static struct xe_vma *vm_bind_ioctl_lookup_vma(struct xe_vm *vm, case XE_VM_BIND_OP_UNMAP_ALL: XE_BUG_ON(!bo); - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return ERR_PTR(err); vma = vm_unbind_all_lookup_vmas(vm, bo); if (!vma) vma = ERR_PTR(-EINVAL); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); break; case XE_VM_BIND_OP_MAP_USERPTR: XE_BUG_ON(bo); @@ -3288,28 +3232,83 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) * XXX: Using the TTM wrappers for now, likely can call into dma-resv code * directly to optimize. Also this likely should be an inline function. */ -int xe_vm_lock(struct xe_vm *vm, struct ww_acquire_ctx *ww, +int xe_vm_lock(struct xe_vm *vm, struct drm_exec *exec, int num_resv, bool intr) { - struct ttm_validate_buffer tv_vm; - LIST_HEAD(objs); - LIST_HEAD(dups); + struct dma_resv *obj; + struct ww_acquire_ctx *ww = &exec->ticket; + int err; XE_BUG_ON(!ww); - tv_vm.num_shared = num_resv; - tv_vm.bo = xe_vm_ttm_bo(vm);; - list_add_tail(&tv_vm.head, &objs); + obj = xe_vm_gem(vm)->resv; + ww_acquire_init(ww, &reservation_ww_class); + + if (intr) + err = dma_resv_lock_interruptible(obj, ww); + else + err = dma_resv_lock(obj, ww); - return ttm_eu_reserve_buffers(ww, &objs, intr, &dups); + if (unlikely(err)) + return err; + + num_resv = max(num_resv, 1); + err = dma_resv_reserve_fences(obj, num_resv); + if (err) + goto out_err; + + return 0; + +out_err: + dma_resv_unlock(&vm->resv); + return err; } -void xe_vm_unlock(struct xe_vm *vm, struct ww_acquire_ctx *ww) +void xe_vm_unlock(struct xe_vm *vm, struct drm_exec *exec) { + struct ww_acquire_ctx *ww = &exec->ticket; + dma_resv_unlock(&vm->resv); ww_acquire_fini(ww); } +int xe_vm_bo_lock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec, + int num_resv, bool intr) +{ + int err; + + drm_exec_init(exec, intr); + drm_exec_while_not_all_locked(exec) { + err = drm_exec_prepare_obj(exec, xe_vm_gem(vm), + num_resv); + drm_exec_continue_on_contention(exec); + if (err && err != -EALREADY) + goto out_err; + + if (bo && !bo->vm) { + err = drm_exec_prepare_obj(exec, &bo->ttm.base, + num_resv); + drm_exec_continue_on_contention(exec); + if (err && err != -EALREADY) + goto out_err; + } + } + + return 0; + +out_err: + drm_exec_fini(exec); + return err; +} + +void xe_vm_bo_unlock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec, + bool lru_update) +{ + if (lru_update && bo && (!bo->vm || xe_vm_no_dma_fences(vm))) + ttm_bo_move_to_lru_tail_unlocked(&bo->ttm); + drm_exec_fini(exec); +} + /** * xe_vm_invalidate_vma - invalidate GPU mappings for VMA without a lock * @vma: VMA to invalidate diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h index 748dc16ebed9..b2b7ff5c12fb 100644 --- a/drivers/gpu/drm/xe/xe_vm.h +++ b/drivers/gpu/drm/xe/xe_vm.h @@ -6,6 +6,8 @@ #ifndef _XE_VM_H_ #define _XE_VM_H_ +#include + #include "xe_macros.h" #include "xe_map.h" #include "xe_vm_types.h" @@ -38,10 +40,14 @@ static inline void xe_vm_put(struct xe_vm *vm) kref_put(&vm->refcount, xe_vm_free); } -int xe_vm_lock(struct xe_vm *vm, struct ww_acquire_ctx *ww, +int xe_vm_lock(struct xe_vm *vm, struct drm_exec *exec, int num_resv, bool intr); +void xe_vm_unlock(struct xe_vm *vm, struct drm_exec *exec); -void xe_vm_unlock(struct xe_vm *vm, struct ww_acquire_ctx *ww); +int xe_vm_bo_lock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec, + int num_resv, bool intr); +void xe_vm_bo_unlock(struct xe_vm *vm, struct xe_bo *bo, struct drm_exec *exec, + bool lru_update); static inline bool xe_vm_is_closed(struct xe_vm *vm) { @@ -96,8 +102,6 @@ int xe_vm_async_fence_wait_start(struct dma_fence *fence); extern struct ttm_device_funcs xe_ttm_funcs; -struct ttm_buffer_object *xe_vm_ttm_bo(struct xe_vm *vm); - /** * xe_vm_reactivate_rebind() - Reactivate the rebind functionality on compute * vms. @@ -124,23 +128,10 @@ int xe_vma_userptr_pin_pages(struct xe_vma *vma); int xe_vma_userptr_check_repin(struct xe_vma *vma); -/* - * XE_ONSTACK_TV is used to size the tv_onstack array that is input - * to xe_vm_lock_dma_resv() and xe_vm_unlock_dma_resv(). - */ -#define XE_ONSTACK_TV 20 -int xe_vm_lock_dma_resv(struct xe_vm *vm, struct ww_acquire_ctx *ww, - struct ttm_validate_buffer *tv_onstack, - struct ttm_validate_buffer **tv, - struct list_head *objs, - bool intr, - unsigned int num_shared); - -void xe_vm_unlock_dma_resv(struct xe_vm *vm, - struct ttm_validate_buffer *tv_onstack, - struct ttm_validate_buffer *tv, - struct ww_acquire_ctx *ww, - struct list_head *objs); +int xe_vm_lock_dma_resv(struct xe_vm *vm, struct drm_exec *exec, + bool intr, unsigned int num_shared); + +void xe_vm_unlock_dma_resv(struct xe_vm *vm, struct drm_exec *exec); void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence, enum dma_resv_usage usage); diff --git a/drivers/gpu/drm/xe/xe_vm_madvise.c b/drivers/gpu/drm/xe/xe_vm_madvise.c index 29815852985a..6fe1316ea229 100644 --- a/drivers/gpu/drm/xe/xe_vm_madvise.c +++ b/drivers/gpu/drm/xe/xe_vm_madvise.c @@ -28,16 +28,16 @@ static int madvise_preferred_mem_class(struct xe_device *xe, struct xe_vm *vm, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->props.preferred_mem_class = value; xe_bo_placement_for_flags(xe, bo, bo->flags); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0; @@ -53,16 +53,16 @@ static int madvise_preferred_gt(struct xe_device *xe, struct xe_vm *vm, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->props.preferred_gt = value; xe_bo_placement_for_flags(xe, bo, bo->flags); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0; @@ -89,17 +89,17 @@ static int madvise_preferred_mem_class_gt(struct xe_device *xe, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->props.preferred_mem_class = mem_class; bo->props.preferred_gt = gt_id; xe_bo_placement_for_flags(xe, bo, bo->flags); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0; @@ -112,13 +112,13 @@ static int madvise_cpu_atomic(struct xe_device *xe, struct xe_vm *vm, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; if (XE_IOCTL_ERR(xe, !(bo->flags & XE_BO_CREATE_SYSTEM_BIT))) return -EINVAL; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->props.cpu_atomic = !!value; @@ -130,7 +130,7 @@ static int madvise_cpu_atomic(struct xe_device *xe, struct xe_vm *vm, */ if (bo->props.cpu_atomic) ttm_bo_unmap_virtual(&bo->ttm); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0; @@ -143,18 +143,18 @@ static int madvise_device_atomic(struct xe_device *xe, struct xe_vm *vm, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; if (XE_IOCTL_ERR(xe, !(bo->flags & XE_BO_CREATE_VRAM0_BIT) && !(bo->flags & XE_BO_CREATE_VRAM1_BIT))) return -EINVAL; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->props.device_atomic = !!value; - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0; @@ -174,16 +174,16 @@ static int madvise_priority(struct xe_device *xe, struct xe_vm *vm, for (i = 0; i < num_vmas; ++i) { struct xe_bo *bo; - struct ww_acquire_ctx ww; + struct drm_exec exec; bo = vmas[i]->bo; - err = xe_bo_lock(bo, &ww, 0, true); + err = xe_bo_lock(bo, &exec, 0, true); if (err) return err; bo->ttm.priority = value; ttm_bo_move_to_lru_tail(&bo->ttm); - xe_bo_unlock(bo, &ww); + xe_bo_unlock(bo, &exec); } return 0;