From patchwork Thu May 27 23:08:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12285589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A326EC4708A for ; Thu, 27 May 2021 23:09:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 43AF161176 for ; Thu, 27 May 2021 23:09:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43AF161176 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DB5B36B006E; Thu, 27 May 2021 19:09:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8C786B0070; Thu, 27 May 2021 19:09:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2EC26B0071; Thu, 27 May 2021 19:09:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 8F8036B006E for ; Thu, 27 May 2021 19:09:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2B61DB9E3 for ; Thu, 27 May 2021 23:09:42 +0000 (UTC) X-FDA: 78188555004.10.512F519 Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) by imf17.hostedemail.com (Postfix) with ESMTP id 5C6214142C6C for ; Thu, 27 May 2021 23:09:36 +0000 (UTC) Received: by mail-il1-f181.google.com with SMTP id o9so1762397ilh.6 for ; Thu, 27 May 2021 16:09:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1vTFumJbcjlC9H58za7civP9oS0BxFYnQlOIDDd8vSU=; b=HAIg1uSz4RBzsZCtp4d4QdB9b8SdCfGZGGkYyjQJ1gpqJOsXAvP98PnBrf2z4CMtNU QzSbafdmDgr4kszEaSkqeJcgdmzAvYEqqA5a70bmnambeDgQ0u4GM/6R/nmp3dJKVRo3 0bYjKH8I1+oipkfzOoA8chQFIQBF4MCZYCzKAekcY98QcSsEjbLAaoqZmOtHzPgcAvih KhwHdE9VI6bNbwHYvNtiWHV/Fk4zWs4Rv9xiflD+lQBLlEgjrCBraoaU//SEG3Z84rEv uUoYXnEnBbhbG9dgYvBhC8qb51RDUrreN+AAApT6iYh9ivrPx97e76JBAO9+apO1gsEa F0ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1vTFumJbcjlC9H58za7civP9oS0BxFYnQlOIDDd8vSU=; b=pvla0biSbqvHX0Gx6iOdwyxiQJBDIDXJfiSindc1sShTGwsemEHyO/otLQ9iF5qcMk e5fxuUXVvCtBqUTB4BISuyZ8IqViTCm/Ehw3Kaem9E4bdIe7A3FUXPkHc/4dhVxMS6pG WYH7CP3Bn4aTs5Oru/bQucYJsG/NaL181AeIWTC3vNqHIkKp7C6RrfXY1g/P5KR3Kf1P 8RwDB7DOrQTuK+sSx+FmwDvNgOLLHqr++KvsZUx3XBaAtxC83eMaRF5iQF1q7YiaFrds ibv6h/Tm2gxWf8Xg72SCutC9wKF+m6wo3yzBgyeyerlbRF0OuBfMuqMiHka2Dt/pkQ5B Q0JA== X-Gm-Message-State: AOAM530eVywCCEZ63kw17s8T6DnxA/v1Ez88FknMdzbXP1F4JP0dqn5W TAIe1bYDEQN6rxR0yU+PPzM= X-Google-Smtp-Source: ABdhPJxGyk+/fitlH3T/Ywo0Dq9sm0GcM01SIGfE1kwmfZ4Tql9lUnIKyyNl5J+eQtri+AKvK9GjAg== X-Received: by 2002:a92:3302:: with SMTP id a2mr4869468ilf.62.1622156981342; Thu, 27 May 2021 16:09:41 -0700 (PDT) Received: from Harpoon.amd.com ([165.204.55.251]) by smtp.gmail.com with ESMTPSA id r5sm1860014ilb.1.2021.05.27.16.09.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 16:09:32 -0700 (PDT) From: Felix Kuehling X-Google-Original-From: Felix Kuehling To: felix.kuehling@amd.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hch@lst.de, jglisse@redhat.com, jgg@nvidia.com, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Sierra Subject: [RFC PATCH 1/5] drm/amdkfd: add SPM support for SVM Date: Thu, 27 May 2021 19:08:05 -0400 Message-Id: <20210527230809.3701-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210527230809.3701-1-Felix.Kuehling@amd.com> References: <20210527230809.3701-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5C6214142C6C Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=HAIg1uSz; spf=pass (imf17.hostedemail.com: domain of felixkuehling@gmail.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=felixkuehling@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Stat-Signature: by8uj5i4mwehs7kkn5x6ym15chh3bm3t X-HE-Tag: 1622156976-264893 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra When CPU is connected throug XGMI, it has coherent access to VRAM resource. In this case that resource is taken from a table in the device gmc aperture base. This resource is used along with the device type, which could be DEVICE_PRIVATE or DEVICE_GENERIC to create the device page map region. Signed-off-by: Alex Sierra --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 12 +++++++++--- drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 1 - kernel/resource.c | 2 +- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index c8ca3252cbc2..f5939449a99f 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -895,6 +895,7 @@ int svm_migrate_init(struct amdgpu_device *adev) struct resource *res; unsigned long size; void *r; + bool xgmi_connected_to_cpu = adev->gmc.xgmi.connected_to_cpu; /* Page migration works on Vega10 or newer */ if (kfddev->device_info->asic_family < CHIP_VEGA10) @@ -907,17 +908,22 @@ int svm_migrate_init(struct amdgpu_device *adev) * should remove reserved size */ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); - res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (xgmi_connected_to_cpu) + res = lookup_resource(&iomem_resource, adev->gmc.aper_base); + else + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) return -ENOMEM; - pgmap->type = MEMORY_DEVICE_PRIVATE; pgmap->nr_range = 1; pgmap->range.start = res->start; pgmap->range.end = res->end; + pgmap->type = xgmi_connected_to_cpu ? + MEMORY_DEVICE_GENERIC : MEMORY_DEVICE_PRIVATE; pgmap->ops = &svm_migrate_pgmap_ops; pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev); - pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; + pgmap->flags = 0; r = devm_memremap_pages(adev->dev, pgmap); if (IS_ERR(r)) { pr_err("failed to register HMM device memory\n"); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h index 21f693767a0d..3881a93192ed 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h @@ -38,7 +38,6 @@ #define SVM_RANGE_VRAM_DOMAIN (1UL << 0) #define SVM_ADEV_PGMAP_OWNER(adev)\ ((adev)->hive ? (void *)(adev)->hive : (void *)(adev)) - struct svm_range_bo { struct amdgpu_bo *bo; struct kref kref; diff --git a/kernel/resource.c b/kernel/resource.c index 627e61b0c124..da137553b83e 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -783,7 +783,7 @@ struct resource *lookup_resource(struct resource *root, resource_size_t start) return res; } - +EXPORT_SYMBOL(lookup_resource); /* * Insert a resource into the resource tree. If successful, return NULL, * otherwise return the conflicting resource (compare to __request_resource()) From patchwork Thu May 27 23:08:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12285591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9551FC4707F for ; Thu, 27 May 2021 23:10:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 27EE6613D8 for ; Thu, 27 May 2021 23:10:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27EE6613D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9AAA66B0070; Thu, 27 May 2021 19:09:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 953106B0071; Thu, 27 May 2021 19:09:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44AB06B0072; Thu, 27 May 2021 19:09:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 07BE56B0070 for ; Thu, 27 May 2021 19:09:58 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A0F73824999B for ; Thu, 27 May 2021 23:09:58 +0000 (UTC) X-FDA: 78188555676.30.F8C72BA Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) by imf15.hostedemail.com (Postfix) with ESMTP id 7D0C9A0001F8 for ; Thu, 27 May 2021 23:09:53 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id e17so2256885iol.7 for ; Thu, 27 May 2021 16:09:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rXMEJXZjUbtBJkh2+REg2+RPpXku92/vdnGkFUqmhuc=; b=Ay93GmLCUIM7OteWbUa90Gw5fkekkkBc+XOVdI8G84UMUZ/kuiX/h2Ao2PW5vHiFB6 hQZPvmdGX7vzVDQMkXNpkSEkc/zmFtCmCRxDII/7InR6IyjNW/4f/uZ5OZ7xZqyO4a89 stoaqe9D38iucOA8h8bF8IbwTqqroGwp4KG5inSRTrKAvQa2KQjMiKzzKmk+xIFqRIKX daGZ65vuyKaYYpcK5mJB9vWsr9S8x2sXLjTXqTFQjQkajyxjMBNZuToZ7/0S4T/4dsXI FG9HLXMIthbY0L9GMsDBQWF3QNGmGO6su0wuZlxkjAPLI/ZxjKUxstdPY0bwKHceGX2x SAmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rXMEJXZjUbtBJkh2+REg2+RPpXku92/vdnGkFUqmhuc=; b=QHPbWGEmFJb5fZ3oOdBGIb9qB8xwPp4j8BrxB9olf1luoUNWLK9LVEKvJboLBuZg1+ n1OfZarINGmpozT/GaCcy7e8zCiFMoqIszMUjM806IDEeePogOjJhM2qbxOBXFOk6Sg9 xJSxYZ6i6Y5EGi9QrnpPRfi6Bub8kVxZrTHVo3W28EswfQ2NTTT5LjN6udS2PUlhNN7t 2AB7y6IYU0iA1cr+F0IjcA8Ebylz7JkKS0JL3ZLHS4M83HvQJFt1MRkfh8QQIafjWSM6 l8vMSTULvI+Wyflg1h1vbT5EYP+eZX4uwmmuCy5aFltmJk5C7K3PpnSWWsCAhbKjojFU ad6w== X-Gm-Message-State: AOAM531izSk6Hgy5EjPIxfczDqrrQedeXU5gGsr8/l9bjMrsQ5IPuxaV J1adcGTu0w6GqcSOH/l1YsQ= X-Google-Smtp-Source: ABdhPJyzp92hDnF8efLXJm+0tvdMSeJf8uS6fJ+jMitfdarllFRWiq1kbTXZrJAJlWHy2hx5t+p1uw== X-Received: by 2002:a6b:5015:: with SMTP id e21mr4817176iob.104.1622156997727; Thu, 27 May 2021 16:09:57 -0700 (PDT) Received: from Harpoon.amd.com ([165.204.55.251]) by smtp.gmail.com with ESMTPSA id r5sm1860014ilb.1.2021.05.27.16.09.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 16:09:48 -0700 (PDT) From: Felix Kuehling X-Google-Original-From: Felix Kuehling To: felix.kuehling@amd.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hch@lst.de, jglisse@redhat.com, jgg@nvidia.com, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Sierra Subject: [RFC PATCH 2/5] drm/amdkfd: generic type as sys mem on migration to ram Date: Thu, 27 May 2021 19:08:06 -0400 Message-Id: <20210527230809.3701-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210527230809.3701-1-Felix.Kuehling@amd.com> References: <20210527230809.3701-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7D0C9A0001F8 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Ay93GmLC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of felixkuehling@gmail.com designates 209.85.166.42 as permitted sender) smtp.mailfrom=felixkuehling@gmail.com X-Rspamd-Server: rspam03 X-Stat-Signature: 1haabpjo3cjhsrqxb8fycma4xjrqxpr3 X-HE-Tag: 1622156993-842571 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra Generic device type memory on VRAM to RAM migration, has similar access as System RAM from the CPU. This flag sets the source from the sender. Which in Generic type case, should be set as SYSTEM. Signed-off-by: Alex Sierra --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index f5939449a99f..7b41006c1164 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -653,8 +653,9 @@ svm_migrate_vma_to_ram(struct amdgpu_device *adev, struct svm_range *prange, migrate.vma = vma; migrate.start = start; migrate.end = end; - migrate.flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; migrate.pgmap_owner = SVM_ADEV_PGMAP_OWNER(adev); + migrate.flags = adev->gmc.xgmi.connected_to_cpu ? + MIGRATE_VMA_SELECT_SYSTEM : MIGRATE_VMA_SELECT_DEVICE_PRIVATE; size = 2 * sizeof(*migrate.src) + sizeof(uint64_t) + sizeof(dma_addr_t); size *= npages; From patchwork Thu May 27 23:08:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12285593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05F55C4708A for ; Thu, 27 May 2021 23:10:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A86C861176 for ; Thu, 27 May 2021 23:10:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A86C861176 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C41A26B0071; Thu, 27 May 2021 19:10:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE97D6B0072; Thu, 27 May 2021 19:10:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70A1A6B0073; Thu, 27 May 2021 19:10:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0160.hostedemail.com [216.40.44.160]) by kanga.kvack.org (Postfix) with ESMTP id 33C086B0071 for ; Thu, 27 May 2021 19:10:28 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id CC4BEA748 for ; Thu, 27 May 2021 23:10:27 +0000 (UTC) X-FDA: 78188556894.40.424D17B Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by imf05.hostedemail.com (Postfix) with ESMTP id B8A88E007A65 for ; Thu, 27 May 2021 23:10:16 +0000 (UTC) Received: by mail-io1-f52.google.com with SMTP id d25so2286501ioe.1 for ; Thu, 27 May 2021 16:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Nfy2IfPBB1SzRRj1M8FEyKWIiZqz+B359r93DqSZMBU=; b=EIxTA24bC6jvrwGt7a6INUPZdEOCMUS/1wV3tWyvWq7RG8odSBOoO6bRwkqtNUdWkX imZoSeaVdxkqMuqGRxZalIEm0dSVhabr+IeAIyu3ZpQ6zus++vi7+EAc+7CeRWpuOKm+ kvlVhSxyRLH/GuxmK3Wh1mnfZGdDtshlwPG/ghWkxX8tsazVmTq9Mh4iYS8mF8Kukklk l0339XlDI0e6JLDK394lOtRN5gCMkKOqkh4p0ZlptgEYfF2IPy3w0or3HoTGZxh+8Sh0 tW1bGrkTw3IDy6KROQzGYQPxxPRlLMc+ewwddj+Nk+XbhLtjHgr3IjdTW3CFnAy/3vHN CsbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nfy2IfPBB1SzRRj1M8FEyKWIiZqz+B359r93DqSZMBU=; b=q7auaYK9Nat6irSKFJDrAi6SRY0DlKNiJzY4q1L0/KTecelYNkM4T9hMaxAMgqF9A7 BMJa/nRU14vpAdxdOF5/9RLqMpI9Lz00ZWrXZdL8J/sVFlDvhDVCZI5rCOYXG63tM5Hb 5RwSf5bD3yKFe05sFADgR2NG1l6zdHSajMmL1S0Pan3Za60KJEE0vxBrN5H4UF3Vbt6P 5V0tveSTbe1lZC5mA6n6kmQUxhzhzzALf7SIT2TBS/lhZdNWnZH29CNASUzyuUWk7NYs li4ybfLyyvFFUeu3EkBDN4weDd4ywr6h0KJODjTPVyLtobYCJU1nQLCORP01DTWxWkSL rxpQ== X-Gm-Message-State: AOAM5335pQ2catGtzriXj94cyjdya6Ej0UCD5bsF3cuJKut+uyaVlaHY 9kyq3xugpTCcnHSwQUADWrU= X-Google-Smtp-Source: ABdhPJzzDrfBqzx1tvMKp22i200oopXJtvSuhCpPS4llgK9p8ZzFsOXk94lGGj2reSJ7a36ZyYRM1g== X-Received: by 2002:a5e:c744:: with SMTP id g4mr4783689iop.26.1622157027022; Thu, 27 May 2021 16:10:27 -0700 (PDT) Received: from Harpoon.amd.com ([165.204.55.251]) by smtp.gmail.com with ESMTPSA id r5sm1860014ilb.1.2021.05.27.16.09.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 16:10:04 -0700 (PDT) From: Felix Kuehling X-Google-Original-From: Felix Kuehling To: felix.kuehling@amd.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hch@lst.de, jglisse@redhat.com, jgg@nvidia.com, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Sierra Subject: [RFC PATCH 3/5] include/linux/mm.h: helper to check zone device generic type Date: Thu, 27 May 2021 19:08:07 -0400 Message-Id: <20210527230809.3701-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210527230809.3701-1-Felix.Kuehling@amd.com> References: <20210527230809.3701-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B8A88E007A65 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=EIxTA24b; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of felixkuehling@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=felixkuehling@gmail.com X-Rspamd-Server: rspam03 X-Stat-Signature: auzb5t7jt6qeipsz9cp3spb51hii3nem X-HE-Tag: 1622157016-457408 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra Helper to check if zone device page is generic type. Signed-off-by: Alex Sierra --- include/linux/mm.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c9900aedc195..1af7b9b76948 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1158,6 +1158,13 @@ static inline bool is_device_private_page(const struct page *page) page->pgmap->type == MEMORY_DEVICE_PRIVATE; } +static inline bool is_device_generic_page(const struct page *page) +{ + return IS_ENABLED(CONFIG_DEV_PAGEMAP_OPS) && + is_zone_device_page(page) && + page->pgmap->type == MEMORY_DEVICE_GENERIC; +} + static inline bool is_pci_p2pdma_page(const struct page *page) { return IS_ENABLED(CONFIG_DEV_PAGEMAP_OPS) && From patchwork Thu May 27 23:08:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12285595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFBE0C4707F for ; Thu, 27 May 2021 23:11:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 506CE613D4 for ; Thu, 27 May 2021 23:11:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 506CE613D4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E78176B006C; Thu, 27 May 2021 19:10:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E27AC6B006E; Thu, 27 May 2021 19:10:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D16546B0072; Thu, 27 May 2021 19:10:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id A024D6B006C for ; Thu, 27 May 2021 19:10:59 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 30277A76F for ; Thu, 27 May 2021 23:10:59 +0000 (UTC) X-FDA: 78188558238.24.2AFA080 Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) by imf09.hostedemail.com (Postfix) with ESMTP id 0D3B56000ECD for ; Thu, 27 May 2021 23:10:53 +0000 (UTC) Received: by mail-io1-f45.google.com with SMTP id n10so2250749ion.8 for ; Thu, 27 May 2021 16:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hyMnVQT3uILhsayUiB51yYlscbJXei219nOcfMZbk0g=; b=ndasi/1WhYCH93Zprui/GY40/sn19ao3HLj2XT2v9EbazWcigmdEugKPhYF0n+yT6/ r90yLFnoxKRv4zHSeUCR6aRAK2G4JXu9lCBMk2OC7M3y5j41ApJyGGWLSEiZvYr+/Iw7 EoWG/0tcEi1Za1aImy3BIlTABk7lIRYgfs1f5Vf5EbQamG8Ga/Z7fqRphGxadI9PWZtZ j9ZmGl+5QTm3SB8QXpLXxAnznbsX8v9BJ8T9BFTvTdbYVGnTZhs6qn32xzJQfOOa6+sj zR36KQWCvD8Tz/MU1YxGj7IYqhLJRSPt7aoffgsDK8WEG5sLd/t4whHwulfZG+3RNdbM ldYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hyMnVQT3uILhsayUiB51yYlscbJXei219nOcfMZbk0g=; b=cJr4fjP4wdX5EY5mwRr/MyhWnn2nnnknLhU25oViX0J+M6Sj0ASw23tQ3OtPlmJHhd UoV44MpuS7Vq7xFzsHlDkM3qlNVa9yv4i+U/P/nxemWv/GB120NW0Duib13B2VOifkc5 WPN7EcMH4q6JlwCmDgBMb5vsyCI++CdoUAkXLOEMNwCz8pHDRgSbfZjVqaQ8pSu3az7h BMhKiV8lYkcDP+RPnjHhrCcJgucLb2MenLFKjAm2GiicfF1/ljgbn2NpFFtJ37MXruGl +JWf15uczSNou09hFTGQ/7uDNq51KOsdU6NozMDYSF8ts7RJXfT8BKHyHTVYoi/YsLXA +JMg== X-Gm-Message-State: AOAM531lbgf/9SUJQ3wHXeGGLl6VhvT14D+h2w5WlCOrWUiTi04IoE0U 1s10/W9/lkVbQ8ZrqzR9X+nnp98D8H4aiDtP X-Google-Smtp-Source: ABdhPJy41x9vuWRCqnDRM6SkBRLwhShGggcTKxx9ubB3k12ssllC99aPqznxBEX133FPxW+i4upInQ== X-Received: by 2002:a05:6602:2ccf:: with SMTP id j15mr4777023iow.56.1622157058325; Thu, 27 May 2021 16:10:58 -0700 (PDT) Received: from Harpoon.amd.com ([165.204.55.251]) by smtp.gmail.com with ESMTPSA id r5sm1860014ilb.1.2021.05.27.16.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 16:10:40 -0700 (PDT) From: Felix Kuehling X-Google-Original-From: Felix Kuehling To: felix.kuehling@amd.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hch@lst.de, jglisse@redhat.com, jgg@nvidia.com, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Sierra Subject: [RFC PATCH 4/5] mm: add generic type support for device zone page migration Date: Thu, 27 May 2021 19:08:08 -0400 Message-Id: <20210527230809.3701-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210527230809.3701-1-Felix.Kuehling@amd.com> References: <20210527230809.3701-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="ndasi/1W"; spf=pass (imf09.hostedemail.com: domain of felixkuehling@gmail.com designates 209.85.166.45 as permitted sender) smtp.mailfrom=felixkuehling@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 0D3B56000ECD X-Stat-Signature: yzcka5ua4x6phmen8bddx6b78b8x5osm X-HE-Tag: 1622157053-280745 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra This support is only for generic type anonymous memory. Generic type with zone device pages require to take an extra reference, as it's done with device private type. Also, support added to migrate pages meta-data for generic device type. Signed-off-by: Alex Sierra --- mm/migrate.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 20ca887ea769..33e573a992e5 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -380,7 +380,8 @@ static int expected_page_refs(struct address_space *mapping, struct page *page) * Device private pages have an extra refcount as they are * ZONE_DEVICE pages. */ - expected_count += is_device_private_page(page); + expected_count += + (is_device_private_page(page) || is_device_generic_page(page)); if (mapping) expected_count += thp_nr_pages(page) + page_has_private(page); @@ -2607,7 +2608,7 @@ static bool migrate_vma_check_page(struct page *page) * FIXME proper solution is to rework migration_entry_wait() so * it does not need to take a reference on page. */ - return is_device_private_page(page); + return is_device_private_page(page) | is_device_generic_page(page); } /* For file back page */ @@ -3069,10 +3070,12 @@ void migrate_vma_pages(struct migrate_vma *migrate) mapping = page_mapping(page); if (is_zone_device_page(newpage)) { - if (is_device_private_page(newpage)) { + if (is_device_private_page(newpage) || + is_device_generic_page(newpage)) { /* - * For now only support private anonymous when - * migrating to un-addressable device memory. + * For now only support private and devdax/generic + * anonymous when migrating to un-addressable + * device memory. */ if (mapping) { migrate->src[i] &= ~MIGRATE_PFN_MIGRATE; From patchwork Thu May 27 23:08:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12285597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96C7AC4708B for ; Thu, 27 May 2021 23:11:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B5B961176 for ; Thu, 27 May 2021 23:11:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B5B961176 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9D0556B006E; Thu, 27 May 2021 19:11:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97ABF6B0070; Thu, 27 May 2021 19:11:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4496B6B0072; Thu, 27 May 2021 19:11:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id 03CAE6B006E for ; Thu, 27 May 2021 19:11:12 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8340D181AF5C6 for ; Thu, 27 May 2021 23:11:12 +0000 (UTC) X-FDA: 78188558784.31.AD21BF8 Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) by imf17.hostedemail.com (Postfix) with ESMTP id C3BA04142C73 for ; Thu, 27 May 2021 23:11:06 +0000 (UTC) Received: by mail-io1-f50.google.com with SMTP id b81so2284507iof.2 for ; Thu, 27 May 2021 16:11:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+fnXZ/6nvmXOv34c4RRw63958oJli0u/lIVwH76TfVs=; b=hfsWP/9+JPpVyYC9O3ojjoSJgCVa8bg3ZxS8juUlu3nvrnQa1FDwqPHFiW7Wq1FFmL ILWXJk8rzcTa4KRswm4pXmJB+49rU98TN4ZkYCTQXtDevKsG0VUzsfAZ+VgXi5v2V2z6 br8VB8OP9GF+MpfgShzjNvZF+o4rtD8nnnIkPLsY9T2PeLX7b8qySpRzCRn4yrGrJbQN n49lox+UZw6fKMZgz5uCdzEvhyO7JY/CVfdfc67IRcEvH8E6dkt6lcuC4SMqOrj1a1nb 9XnyOq+IBtpMPCU0nMcRhEn/d+SYQkcG/Q1L3J6P7bj0eq09etW3Ka4Hw+8doA5no8zw eipQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+fnXZ/6nvmXOv34c4RRw63958oJli0u/lIVwH76TfVs=; b=gZ0k/+a7Vu1+MJtScC6gzTizPOZ/1Hi8Wfs4XgEri42/OS1C+58oWJdXKzIQD9rcdG RgIwmiZO7fI7qKMlfaOdyrbtKmmreNb7VufCRUs8LdovAgc4OVoS+iOpunSt9nJZcqJQ dZsM3+PuB+tKYUmoCpbEg0zCjLsgyohiGZw0XedoFHYe1I3t6CGkoISqHMQJWV9AiTS+ sq5BH0JSA3PqOnUtYewAb4lbX3Vb8CAL9EYdomCW1hrWx/36hcjmV+MHzeTS1WWhvkBH vZmEd2NcBKh5kKkFCT225sILJMvWBYw0AZY99ALRRe1Wix9IShp/tYGsJOZd0hJyKaIP d9xQ== X-Gm-Message-State: AOAM530+krDSdr7RzV9WTo/yvGUXoKue1Ica0w90vJBK99jqFg1ERZj7 IKJtRe0dgSTUTeebYojSI/w= X-Google-Smtp-Source: ABdhPJxhTAtS0KfJXNAGDVbVTIZAdKf4Y0VwXKHDCbpaIaFnpP8Peruryw244ygyruPJzPvmeqvEhQ== X-Received: by 2002:a5d:8d87:: with SMTP id b7mr4546232ioj.46.1622157071809; Thu, 27 May 2021 16:11:11 -0700 (PDT) Received: from Harpoon.amd.com ([165.204.55.251]) by smtp.gmail.com with ESMTPSA id r5sm1860014ilb.1.2021.05.27.16.10.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 16:11:03 -0700 (PDT) From: Felix Kuehling X-Google-Original-From: Felix Kuehling To: felix.kuehling@amd.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hch@lst.de, jglisse@redhat.com, jgg@nvidia.com, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Sierra Subject: [RFC PATCH 5/5] mm: changes to unref pages with Generic type Date: Thu, 27 May 2021 19:08:09 -0400 Message-Id: <20210527230809.3701-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210527230809.3701-1-Felix.Kuehling@amd.com> References: <20210527230809.3701-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="hfsWP/9+"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of felixkuehling@gmail.com designates 209.85.166.50 as permitted sender) smtp.mailfrom=felixkuehling@gmail.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C3BA04142C73 X-Stat-Signature: j6qeqg6dug9x3kddxtqg561rzwkzgm5o X-HE-Tag: 1622157066-918795 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra pages in device mapping refcounts are 1-based, instead of 0-based. If refcount 1, means it can be freed. This logic is not set for Generic memory type. Therefore, its release is threated as a normal page, instead of the callback device driver release it. Signed-off-by: Alex Sierra --- include/linux/mm.h | 1 + mm/memremap.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1af7b9b76948..83bd2f3e111b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1130,6 +1130,7 @@ static inline bool page_is_devmap_managed(struct page *page) switch (page->pgmap->type) { case MEMORY_DEVICE_PRIVATE: case MEMORY_DEVICE_FS_DAX: + case MEMORY_DEVICE_GENERIC: return true; default: break; diff --git a/mm/memremap.c b/mm/memremap.c index 16b2fb482da1..d2563fbcf987 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -44,6 +44,7 @@ EXPORT_SYMBOL(devmap_managed_key); static void devmap_managed_enable_put(struct dev_pagemap *pgmap) { if (pgmap->type == MEMORY_DEVICE_PRIVATE || + pgmap->type == MEMORY_DEVICE_GENERIC || pgmap->type == MEMORY_DEVICE_FS_DAX) static_branch_dec(&devmap_managed_key); } @@ -51,6 +52,7 @@ static void devmap_managed_enable_put(struct dev_pagemap *pgmap) static void devmap_managed_enable_get(struct dev_pagemap *pgmap) { if (pgmap->type == MEMORY_DEVICE_PRIVATE || + pgmap->type == MEMORY_DEVICE_GENERIC || pgmap->type == MEMORY_DEVICE_FS_DAX) static_branch_inc(&devmap_managed_key); } @@ -480,7 +482,8 @@ EXPORT_SYMBOL_GPL(get_dev_pagemap); void free_devmap_managed_page(struct page *page) { /* notify page idle for dax */ - if (!is_device_private_page(page)) { + if (!(is_device_private_page(page) || + is_device_generic_page(page))) { wake_up_var(&page->_refcount); return; }