From patchwork Fri Feb 2 07:22:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "He, Hongbo" X-Patchwork-Id: 10196331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B52EA603D7 for ; Fri, 2 Feb 2018 07:23:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D59028D75 for ; Fri, 2 Feb 2018 07:23:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9050528E08; Fri, 2 Feb 2018 07:23:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C7A3328D75 for ; Fri, 2 Feb 2018 07:23:28 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1256F6EFA9; Fri, 2 Feb 2018 07:23:28 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0081.outbound.protection.outlook.com [104.47.40.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7E7796EFA9 for ; Fri, 2 Feb 2018 07:23:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=r1CNBE+GyucFnx5GAbOCUVOArUasQFV0z1v0njIAyRM=; b=GBqmzaBbj79vfanJGUNssyUTP99BxwBfqaq1lokajeAu0fzdX3I0McNEZBN3WXH9LzormTY1D8d3/Ta4MNkHz5uHkE1jfyd880ecEuSXJ2Q8F810Oc/Cufjdd0vKPuU60bkubjX35+xZd2n121jhCPnAUB+tPtDMrd0QA6NmQi4= Received: from BN6PR1201CA0019.namprd12.prod.outlook.com (10.174.238.29) by DM5PR1201MB0057.namprd12.prod.outlook.com (10.174.106.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.444.14; Fri, 2 Feb 2018 07:23:25 +0000 Received: from DM3NAM03FT041.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::204) by BN6PR1201CA0019.outlook.office365.com (2603:10b6:405:4c::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.464.11 via Frontend Transport; Fri, 2 Feb 2018 07:23:25 +0000 Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV02.amd.com (165.204.84.17) by DM3NAM03FT041.mail.protection.outlook.com (10.152.83.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.464.13 via Frontend Transport; Fri, 2 Feb 2018 07:23:24 +0000 Received: from roger-build-server.amd.com (10.34.1.3) by SATLEXCHOV02.amd.com (10.181.40.72) with Microsoft SMTP Server id 14.3.361.1; Fri, 2 Feb 2018 01:23:23 -0600 From: Roger He To: Subject: [PATCH 3/5] drm/ttm: add ttm page_flags TTM_PAGE_FLAG_PAGEFAULT Date: Fri, 2 Feb 2018 15:22:23 +0800 Message-ID: <1517556145-23505-3-git-send-email-Hongbo.He@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517556145-23505-1-git-send-email-Hongbo.He@amd.com> References: <1517556145-23505-1-git-send-email-Hongbo.He@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(376002)(346002)(39380400002)(396003)(39860400002)(2980300002)(428003)(199004)(189003)(81166006)(2906002)(8676002)(81156014)(4326008)(478600001)(2351001)(72206003)(50226002)(104016004)(48376002)(50466002)(36756003)(97736004)(106466001)(305945005)(8936002)(105586002)(6916009)(356003)(86362001)(7696005)(51416003)(59450400001)(186003)(336011)(77096007)(76176011)(26005)(47776003)(316002)(16586007)(2950100002)(54906003)(5660300001)(68736007)(53936002)(53416004); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR1201MB0057; H:SATLEXCHOV02.amd.com; FPR:; SPF:None; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT041; 1:J13vhGl0wNR0rScCoEnwPmBf85dBef+sdH/afcSLmSpLGF1cqz/Uuzbzq7gzLGSxkcMmPpvv9GUrxGsAff5R+DEPjaF/L26PB2Ffi1sg/oyltOpuozzk+5MbvywBrljo X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bee12595-084f-4102-f41e-08d56a0dd8d7 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060); SRVR:DM5PR1201MB0057; X-Microsoft-Exchange-Diagnostics: 1; DM5PR1201MB0057; 3:dEDD7gKNwQNk+McgWChigi4SokT77MSQikWwqB/l0qtFQqCv6WIrmbDTDhKHACw4mPVgbIEsHXxz6fTMJPuQbJA8OCkOi7cRd4btFv9+jPMjj6PzbrK0cPKhI4rL5kkAwHI0EbtK7Dklm56+ufTNjCME5KmrvzkY1FdnOnRzuPSsqxXFcFRi2iKZG+VX9pKg2x5lP2vGPJCAAU+PJav8rynp0IKadR6CciB4IGBcBJvgnCxn8f2m7jyT/u1HO37HYHnupk6a+QjezSgQmEKjsdiEe+jESrVE8dU27+EX5G+cSqpWxreO8ecHpQbf/kks2CAQZ+Q8cfrGQHYw8dr9SV+EMDWphpdPpcRG8cuGuxM=; 25:N57WuNOaDQHmrN/I9ONal/ad5/c3wWjicfoXG/rmIsBNHDVF5fOW0ElvjLeqvZtNrSbTyau+sqW49GYVjb4SSqsDtfxc/GaQOLQSEVybf5SVz18IzA9F8egMLRryJy7yJGeOUvjiUfNzFhCCNkd9tKKvEMXtjCCT0837MLolhXMkG6leEmImQqcCRdypFHhK+EJg/nNh/kB8366cIyFQRssrNbs6pBELtGfkvpC5BBLoKBhKsJrNw5q/MQZRFEksj4lbtOsgXFDOW/1v8504n1j8HSLRimXGrzZ/3KuJJVZsima3E2egQenOQW1oTokZAOOKNuR6pkARqKu+/4d16g== X-MS-TrafficTypeDiagnostic: DM5PR1201MB0057: X-Microsoft-Exchange-Diagnostics: 1; DM5PR1201MB0057; 31:KmAYQ6Eb/cBFkUjZSeoxIGqEML5dn9T1s+zlzUo/8mgYBG0jGAv2R9QuBbkM8Sqs8nJDjasSam8qXSqOaIgGbQY1o2RqPbCSqZ+ZoE7Ptm+niW6cnHl6WP5lzuhq6uFLWpdcKSJHzrW0LdWh/EfPwdGEibRBFXHodGCMOtQIUdT5ue/0SovW2715y1TXGPK611C9ZBoPERnJrQ/gmp1+C3/6D7n3lCUQ7uAhzrAquFo=; 20:27Ax4zFURm+kzzIsyXihSMBL+q4mvV/N8DY5ObWdvOUrdX7ZU2lQL48uyNd7SGvY2+iG2nfSUqzd7aJ0NAO3HIfYvefqs7GFCwRQLW6umIyso5uI3l6o9wXn+mYZ5Tbmt005PBwoLTXL9Iz8TUuGUSXsKbLDHyvjZ0HZlsqdA7qfKDVqD+XiGB0dI9l7WtAeFUTwon15ZNhAUdZvVQ4e8JOEZ5vdnK4lG5Rc8JtsoXSWLGu00vI4FV8y+/teHjZoPW7otRyCIbtU9KNgNLweEzFxq/SJwgRuwTMtfIA0bU+xPkJ77NU8jswv8MiFcbskKT/HskszyTjUiusSqcabOEHqEjrnI/OjgByzhQ2ZO55QXqq6Lc2lLPjHcNrq5JHYNVGkeS6mUhopk7rdQq7vP1wSRO/kxFw7NORwLZABPk7tK9PeBYVazSLbaLqwVsJomMjhaeO3sMXrsDkG+DsU7w4eWtCHspyftB8W2jhCCAaEKmHIRZhOeyP/EDaeO/rJ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(10201501046)(3002001)(3231101)(2400082)(944501161)(93006095)(93003095)(6055026)(6041288)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(6072148)(201708071742011); SRVR:DM5PR1201MB0057; BCL:0; PCL:0; RULEID:; SRVR:DM5PR1201MB0057; X-Microsoft-Exchange-Diagnostics: 1; DM5PR1201MB0057; 4:aS1m3x0xTkbRW/hm2mVBp//cHSxth5d8ABg+V1mVNgyubfzOhDbWgshOrf62ZGjanp8gGcWYlrPm7NZ+6AtN5p5i6Kcjfdmw7AMw45VgphDotY8vobw9hnCNH71Bg+WaEBxnXm3jSqvWLX9nRjJmR1lLdZ6agNw75cNpP9O3S3Csh55UWJRZE0oHRiA9YEOcn2stu3Jzcb4AosFBcpAiKNhzc9RVDqXeNRECOLx7jaXNt52aFsih13c1aXAf3hatyL5o7W5tXLbVcyXjDpUHAjReX7nojaIS/d0yKzOwajdV2KwB+7zTT1WhQ+s0YMJa X-Forefront-PRVS: 05715BE7FD X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR1201MB0057; 23:fquI9giAoJjip5SZQMW7AOOcQU9aHmbsaOv6YeY?= =?us-ascii?Q?iD0hj/E07JXX02PAgH9XoTslGyqZ2uPO0Qhcg572a6C1mEBEgr9DevycZYJZ?= =?us-ascii?Q?APIDL38R9pQeTszK8S2t7jCIP+OXwJ0gsjDka5bCVt37z/2DkCnlZRveOWeI?= =?us-ascii?Q?psz73dxHmiY6I3WDpCbGbWiKQiwjIJig4wcBe0NKBDFuIcxHLeXUSNj6M0sc?= =?us-ascii?Q?a84734cuaycagJYUUCAUdKuvMo1gVKD2bpr/fCqj+sBGMo4/n/35E3lFHpmx?= =?us-ascii?Q?XBlQy7eYHMR/5iLQ47UfDVsVhc6uIWB3Dr89gNWQ23EMUwwjQfXDvhmQPPz0?= =?us-ascii?Q?NsR8BR0A/OZsfaoyOxQVsmrTo8Q2DKk/4Ev4COgCDDfrzT2tZbg1cN3T0k0O?= =?us-ascii?Q?UiALPB0VqLsKw81FDYYd/y6TT2YgKwkbgTfuW8XpTc5gQbsvIu0IHnzuUBtE?= =?us-ascii?Q?WO9mLGSe6egAWxGspdtDrdr+EfHqrGh37XhF6dDPJXkmR5sQxzwJdmjWmm8W?= =?us-ascii?Q?VClFIGo1VfZ4ctNGlWnCJQplsn/cVdyL2U2OHE+hkmrJG4G1f80vcSUyAaQT?= =?us-ascii?Q?9NIf7fMj2CKcqTmvYu7OCHQOLhDTp5kwpn0SfbEK6qZ8bfwCyTAXlTTBHyzN?= =?us-ascii?Q?MT+eqGVJfRIvw1OTXYkZYejaD2RX9uM3g5USek60/ewakgZqEyOVxrp79jnH?= =?us-ascii?Q?Ee3dmARFvGwD0qRqne94TSboVFmRgHDV11w9j9m9mdqIU8cZ/a2n0TIjqAzb?= =?us-ascii?Q?IzfzAMfF7PjgffbfP4ZqlMNmCtbVFxWyKY4YkbnCwb3apOyM3xPnMI9pEqcu?= =?us-ascii?Q?DzGVXFJ+ZVzUAqhvg224aS06EVpTwaD5MT6nvNeXxr0LUai2pg9uJ93hUuNY?= =?us-ascii?Q?H6BlzDBwJs+mnsV8x6tjW0a2NCqhT3ulDn/9NEJ23wHrfCuX7wF3tcG80Yl5?= =?us-ascii?Q?7onoh1m703O2CsAjDad6P1pgSpYYd/mFtNFxsEVgKS+X85bqqeP72XyOdRyQ?= =?us-ascii?Q?qywQAl+AX6KkUkv9x1ve87K5mk4Ao1CKKLG9TF7tT169lQ6KM4/SQIEWzdnm?= =?us-ascii?Q?mKv31cWWZJu2iKHD8uFIflKQq5upmzd6wt4yacygO4KVgAmLKKw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR1201MB0057; 6:uFUsL3h8Vhz1ReAatJ/l+fyVHMPTlAh+JStBf6SKSnh+lG/nLkdXo7jZr/JduxhKKKXUvE2Cv4nr1f3VO84hjS24jA5kwtN78Cznt4r9J9JiYlkbVOVF9oNvEfUWA5dTIlohmitDCNZcbMXS8qFFqgHWmvMTTmbjH9USOk5/NiAonx5cP5Hq/8IsDz4CmgSdQWIAUunHPfiCjxFKsTbzWZs/4x07azJTeA7ZQimPcHkqdUyvRD7s8KTR2Jbm7LH3ZLptiZylM0z8fPwzHg+Y4YnGEd6bcqBQhFVP/Ihkj1UxlGdFCGWl7AhlhuhP88fTRijKSVO9COKY9ehSkj6aKIi40xfNwdKWohqGbivcrSw=; 5:M8Bjjuxm+fOD73uONk1rpvgXjCnvHTVdN7IPj/klSd9qvIOe4MrBlU4WGnh2bRGEIlz5XEaE1W4DimGlUJI8AujhPi7IxqVe+cWOCrBsPu54ueRyi30rKpG3XvsD+EYjHMrsc0KgqWwJLw142o+XrwT3s0h1H0nyY3gqOKlrWnE=; 24:wuwX4eLz/EhpkScLTmqTqsU6ID57fbscxhuccz24W0KiDkadhpUfBDEmwIufsFlnY+ct3g6QqLF2Ep5g8AkIVVZAsR4qZwZldjUZFWKHTmM=; 7:FSV/6ZoKRvNg9N/s94cvUvvhsNBzXQm9v1oWAnm1ooBC2RJXhZOkW1pABs6YStjL6Gj8MWbDBrGObU06bAfkaThsoHFsRePx0185CnGsliXKVGaVxK/qbANhlYvim+qko3LB3GLF8yltawbH8f05W5sKzqDqd4SaK2MPMNypgOxCch5hjVKsJOwjoYg/MGnXCzkzHnG4KD/ch7HiOJgXe9aNYzpPSP++PLs3unLknJKCIJF9RmhfTseNPND05Wq3 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR1201MB0057; 20:I9MaNm1yzD5h2rV8rkK+ORN8o+BlHFgRpV52q2aJIhiSxIailOjNtGv1cfdE0zsDvb4ErU8BeHrfsTYfzMiAQcK+YAeeRhVpyN9gKoHTE4Jc51JvM38ouTmeNyeic0AcTJkx4tVoPBhtmGGrYivpiQnkYPLkagB7i48Z+YSwROgS3ougT7Z0Xd2jiEN5lMx7amBE9QlZNjOUhGNNHBNGAG4NRydhCWZGkdD+8WEqMX2QiqafgiqkiOnP9tk/IYDR X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2018 07:23:24.6693 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bee12595-084f-4102-f41e-08d56a0dd8d7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0057 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Roger He , Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP to indicate whether we are servicing for page fault routine in ttm_mem_global_reserve. for ttm_mem_global_reserve if in page fault routine, allow success always. because page fault routing already grabbed system memory successfully and allow this exception is harmless. Otherwise, it will trigger OOM killer. v2: fix minor typo v3: keep original behavior except ttm page with flag NO_RETRY Signed-off-by: Roger He --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 6 +++++- drivers/gpu/drm/ttm/ttm_memory.c | 27 +++++++++++++++++++-------- drivers/gpu/drm/ttm/ttm_page_alloc.c | 5 ++++- drivers/gpu/drm/ttm/ttm_page_alloc_dma.c | 9 ++++++--- drivers/gpu/drm/ttm/ttm_tt.c | 2 +- include/drm/ttm/ttm_bo_driver.h | 1 + include/drm/ttm/ttm_memory.h | 3 ++- 7 files changed, 38 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 716e724..f03dd94 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -233,8 +233,12 @@ static int ttm_bo_vm_fault(struct vm_fault *vmf) cvma.vm_page_prot = ttm_io_prot(bo->mem.placement, cvma.vm_page_prot); + if (ttm->page_flags & TTM_PAGE_FLAG_NO_RETRY) + ttm->page_flags |= TTM_PAGE_FLAG_PAGEFAULT; /* Allocate all page at once, most common usage */ - if (ttm->bdev->driver->ttm_tt_populate(ttm, &ctx)) { + ret = ttm->bdev->driver->ttm_tt_populate(ttm, &ctx); + ttm->page_flags &= ~TTM_PAGE_FLAG_PAGEFAULT; + if (ret) { ret = VM_FAULT_OOM; goto out_io_unlock; } diff --git a/drivers/gpu/drm/ttm/ttm_memory.c b/drivers/gpu/drm/ttm/ttm_memory.c index 598b14b..f5227a5 100644 --- a/drivers/gpu/drm/ttm/ttm_memory.c +++ b/drivers/gpu/drm/ttm/ttm_memory.c @@ -472,7 +472,8 @@ EXPORT_SYMBOL(ttm_mem_global_free); static int ttm_mem_global_reserve(struct ttm_mem_global *glob, struct ttm_mem_zone *single_zone, - uint64_t amount, bool reserve) + uint64_t amount, bool reserve, + bool in_pagefault) { uint64_t free_swap_mem = get_nr_swap_pages() << (PAGE_SHIFT - 1); uint64_t swap_glob_mem, limit, total_used_mem; @@ -483,6 +484,13 @@ static int ttm_mem_global_reserve(struct ttm_mem_global *glob, swap_glob_mem = atomic64_read(&glob->swap_glob_mem); spin_lock(&glob->lock); + /* + * if in page_fault allow reserve successfully anyway since it + * already allocated system pages. Otherwise it will trigger OOM + */ + if (in_pagefault) + goto reserve_direct; + /* adjust the max_swap_mem to cover the new inserted swap space */ if (glob->max_swap_mem < free_swap_mem) glob->max_swap_mem = free_swap_mem; @@ -501,6 +509,7 @@ static int ttm_mem_global_reserve(struct ttm_mem_global *glob, goto out_unlock; } +reserve_direct: if (reserve) { for (i = 0; i < glob->num_zones; ++i) { zone = glob->zones[i]; @@ -522,13 +531,13 @@ static int ttm_mem_global_reserve(struct ttm_mem_global *glob, static int ttm_mem_global_alloc_zone(struct ttm_mem_global *glob, struct ttm_mem_zone *single_zone, uint64_t memory, - struct ttm_operation_ctx *ctx) + struct ttm_operation_ctx *ctx, + bool in_pagefault) { int count = TTM_MEMORY_ALLOC_RETRIES; - while (unlikely(ttm_mem_global_reserve(glob, - single_zone, - memory, true) + while (unlikely(ttm_mem_global_reserve(glob, single_zone, memory, + true, in_pagefault) != 0)) { if (ctx->no_wait_gpu) return -ENOMEM; @@ -548,13 +557,14 @@ int ttm_mem_global_alloc(struct ttm_mem_global *glob, uint64_t memory, * all zones. */ - return ttm_mem_global_alloc_zone(glob, NULL, memory, ctx); + return ttm_mem_global_alloc_zone(glob, NULL, memory, ctx, false); } EXPORT_SYMBOL(ttm_mem_global_alloc); int ttm_mem_global_alloc_page(struct ttm_mem_global *glob, struct page *page, uint64_t size, - struct ttm_operation_ctx *ctx) + struct ttm_operation_ctx *ctx, + bool in_pagefault) { struct ttm_mem_zone *zone = NULL; @@ -570,7 +580,8 @@ int ttm_mem_global_alloc_page(struct ttm_mem_global *glob, if (glob->zone_dma32 && page_to_pfn(page) > 0x00100000UL) zone = glob->zone_kernel; #endif - return ttm_mem_global_alloc_zone(glob, zone, size, ctx); + return ttm_mem_global_alloc_zone(glob, zone, size, + ctx, in_pagefault); } void ttm_mem_global_free_page(struct ttm_mem_global *glob, struct page *page, diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc.c b/drivers/gpu/drm/ttm/ttm_page_alloc.c index 5edcd89..15bf691 100644 --- a/drivers/gpu/drm/ttm/ttm_page_alloc.c +++ b/drivers/gpu/drm/ttm/ttm_page_alloc.c @@ -1094,6 +1094,7 @@ ttm_pool_unpopulate_helper(struct ttm_tt *ttm, unsigned mem_count_update) int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) { struct ttm_mem_global *mem_glob = ttm->glob->mem_glob; + bool in_pagefault; unsigned i; int ret; @@ -1107,9 +1108,11 @@ int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) return ret; } + in_pagefault = ttm->page_flags & TTM_PAGE_FLAG_PAGEFAULT; for (i = 0; i < ttm->num_pages; ++i) { ret = ttm_mem_global_alloc_page(mem_glob, ttm->pages[i], - PAGE_SIZE, ctx); + PAGE_SIZE, ctx, + in_pagefault); if (unlikely(ret != 0)) { ttm_pool_unpopulate_helper(ttm, i); return -ENOMEM; diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c b/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c index b122f6e..b8adf37 100644 --- a/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c +++ b/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c @@ -934,6 +934,7 @@ int ttm_dma_populate(struct ttm_dma_tt *ttm_dma, struct device *dev, struct dma_pool *pool; struct dma_page *d_page; enum pool_type type; + bool in_pagefault; unsigned i; int ret; @@ -943,8 +944,8 @@ int ttm_dma_populate(struct ttm_dma_tt *ttm_dma, struct device *dev, INIT_LIST_HEAD(&ttm_dma->pages_list); i = 0; + in_pagefault = ttm_dma->ttm.page_flags & TTM_PAGE_FLAG_PAGEFAULT; type = ttm_to_type(ttm->page_flags, ttm->caching_state); - #ifdef CONFIG_TRANSPARENT_HUGEPAGE if (ttm->page_flags & TTM_PAGE_FLAG_DMA32) goto skip_huge; @@ -966,7 +967,8 @@ int ttm_dma_populate(struct ttm_dma_tt *ttm_dma, struct device *dev, break; ret = ttm_mem_global_alloc_page(mem_glob, ttm->pages[i], - pool->size, ctx); + pool->size, ctx, + in_pagefault); if (unlikely(ret != 0)) { ttm_dma_unpopulate(ttm_dma, dev); return -ENOMEM; @@ -1003,7 +1005,8 @@ int ttm_dma_populate(struct ttm_dma_tt *ttm_dma, struct device *dev, } ret = ttm_mem_global_alloc_page(mem_glob, ttm->pages[i], - pool->size, ctx); + pool->size, ctx, + in_pagefault); if (unlikely(ret != 0)) { ttm_dma_unpopulate(ttm_dma, dev); return -ENOMEM; diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 5d441e0..a730cd6 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -391,7 +391,7 @@ int ttm_tt_swapout(struct ttm_tt *ttm, struct file *persistent_swap_storage) ttm->page_flags |= TTM_PAGE_FLAG_PERSISTENT_SWAP; else atomic64_add_return(ttm->num_pages << PAGE_SHIFT, - &ttm->glob->mem_glob->swap_glob_mem); + &ttm->glob->mem_glob->swap_glob_mem); return 0; out_err: diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index 9b417eb..8f68781 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -87,6 +87,7 @@ struct ttm_backend_func { #define TTM_PAGE_FLAG_DMA32 (1 << 7) #define TTM_PAGE_FLAG_SG (1 << 8) #define TTM_PAGE_FLAG_NO_RETRY (1 << 9) +#define TTM_PAGE_FLAG_PAGEFAULT (1 << 10) enum ttm_caching_state { tt_uncached, diff --git a/include/drm/ttm/ttm_memory.h b/include/drm/ttm/ttm_memory.h index 6282b50..6c29b5c 100644 --- a/include/drm/ttm/ttm_memory.h +++ b/include/drm/ttm/ttm_memory.h @@ -89,7 +89,8 @@ extern void ttm_mem_global_free(struct ttm_mem_global *glob, uint64_t amount); extern int ttm_mem_global_alloc_page(struct ttm_mem_global *glob, struct page *page, uint64_t size, - struct ttm_operation_ctx *ctx); + struct ttm_operation_ctx *ctx, + bool in_pagefault); extern void ttm_mem_global_free_page(struct ttm_mem_global *glob, struct page *page, uint64_t size); extern size_t ttm_round_pot(size_t size);