From patchwork Mon Feb 6 07:47:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC49C05027 for ; Mon, 6 Feb 2023 07:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229890AbjBFHsa (ORCPT ); Mon, 6 Feb 2023 02:48:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229861AbjBFHs1 (ORCPT ); Mon, 6 Feb 2023 02:48:27 -0500 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4175C18A9B; Sun, 5 Feb 2023 23:48:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aLsp8OkfnvgUX+LaCm1Dc+hdKUsiL3axpftCKUHDm1OQkZpSVWKd41QJpSxq9ko55snTctoS/hMmNjRr+itI3WbB3Q3AzmA6538HBTiXXlDO0S3AaJVzGqI3eu5++vIHZ28QtTlNa+k+dSpLAd+wLCU2EFbS1oHBSxKOk5zueaVmKcpYV50EsgXkylZrJGkkEePSOvrV9I/Bsj2oa8vxDvjsc+kGnGm5e5xycHnK4lkQOFyzYKvS14dWfW/63g0BN0S08F1XDTOjdIwd3UKL6vwTI7+T6eKerf0GqbZqLxKFY5wI1baz3a/2JfDv/WRas+jgQG8iAAdpSy/yR16sQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xC0ascId9hC0FQ0mqA9gGlsD4Tec21nIaiiKW1ANuos=; b=e2vefkFqyA3sW3pHTN46sHlZlBtRrXMyAf4In3Tgp7gdMSg10ziiTG5iwAIpdN+Nv9Oog6rQy/dMDY2dO6w3grxgRGp9gRCGkURF3SWuxdZ5pDPWPjWfF80aA0GfSO+a7RMpi+RETUh04airuQKTuoiT7NdVEbit7kL7o+ESiApp+a+LNO8scmfZwfOWs/wTgyVxb6qqRBkFNcpqOIFrxMM0d+LT8KWcaV3OeF5iSjB1ePBi+f8eg+llIHnurirOEwoKyueOezpUAMsA1l275ngrS9mwcE1qemLfX/zE+eYf570QvCXAfD87djcKO8wu77Flv39WV5CzVvqq2LSVCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xC0ascId9hC0FQ0mqA9gGlsD4Tec21nIaiiKW1ANuos=; b=A+FFKqJYzlIAVESoxZN19dbMh3tOUmAXg71BZGbiuV4HgT2WwNhXvo9H+EqVd1hYY7W1iy9x2eJ4H3XwaGr2jO5d9GIETU1TZlbpbaiAHeuE5em9pnJ4MNG1/6OKDbNRjYa1RYLSWzUTRajUam0hS8av8ruJv6K2TuFY+X4QQbLJGSDzZBmJnFND8gFdvARudQbQbxKxMrYh7Rhm7JbL033fzE2OtsmZD+IYJ1sO5/fcSIk019OYzNFb21GL+KvbDlEwCq+4mkJl2w71KvdFVvoO4dgzK/vrr1yASUdmbDEzRitZTTs59gTXhfNc79Z05lPOjSUH1lVBxsi65AXCxA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by CY8PR12MB7097.namprd12.prod.outlook.com (2603:10b6:930:63::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.34; Mon, 6 Feb 2023 07:48:21 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:48:21 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , linuxppc-dev@lists.ozlabs.org, linux-fpga@vger.kernel.org, linux-rdma@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, netdev@vger.kernel.org, io-uring@vger.kernel.org, bpf@vger.kernel.org, rds-devel@oss.oracle.com, linux-kselftest@vger.kernel.org Subject: [PATCH 01/19] mm: Introduce vm_account Date: Mon, 6 Feb 2023 18:47:38 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SY6PR01CA0010.ausprd01.prod.outlook.com (2603:10c6:10:e8::15) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|CY8PR12MB7097:EE_ X-MS-Office365-Filtering-Correlation-Id: f1f32d26-9f86-4df0-cce9-08db081684ae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d9R20AXDYdOn6o7fqjRKlbn88gfGEhrLB0K3l/C7lo/hVvC76CZp4upyM7QvG93ysjuMVTpq/EYzK3rKe+gwBlyHxY43+Hqd/7szzD+TETPh4YIz49dsl8Pco7UcHmIYR1jDb462M2A51jpFSyZ6+sGx3FpEttvQxlIDeCWQn6283qmfDF4QeL0MGLkLR28DA19JymNK1RtyPkmfKZ09MLSCaFGJ3h12WGdvnJ5w7VGFIKFvz5stxIwwdIt0+gYERQCSlt2L26GRq9KV2GcYX8YE/OcLGZSrecKGGipuwqQm8Z7Ynh+TybkKzd+Y+ZdZKAXFYvSW5vAWUTT6n5GHEu+1RV85ersB37hzKHCRBbBjQk2dt2lN4HS7GS390bwhNfyVDFRZjE/rWx/Sf6l81vQ0i636fN5go+YWW9m/Cs2GeJ2kUXo2o5oo3aZ6hLhWLaDc2cMg8f+dO2hQNOm31Wmf2fNIgwK112xrMKjJwP7uMzzPMRVJHnTwtouclvVkulpeSj9ZswUbyFtJMziahx/BcMaIlFXt789djlRAEZlvA5Dolz1bNDtuZdc6P5ZWA0RBVH2YzDo/Bl5+FviW2ege3AA0jp0C20IRSY3uXiO/9yQxMIEjpPmWqNjE32CPYcudf9GcWGjjbN/MTX2aSQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(39860400002)(396003)(136003)(346002)(376002)(451199018)(38100700002)(36756003)(6506007)(6666004)(6512007)(26005)(186003)(478600001)(86362001)(6486002)(8676002)(66476007)(83380400001)(66946007)(2616005)(66556008)(4326008)(316002)(54906003)(7416002)(2906002)(8936002)(5660300002)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +Q21fH/s4Vq+Fkqt0pEjMYKv/M0hPKFwEmzpwjecwZ8dEKYu6PTd6B1USsF/VIewWV1Iy7hIjEqgKdvNy9XCHb6UCEjDlG2t6luAcyANcpffEfiNXj358C55PHbMpnatLbnb82LSxL2sxcLHQRbRVzZM4h6g/3499Op4qz4eIG6rIq3sP9GRrHx6Tqds5OHj2oy53ycVE3dECU0kComR0bp0U7UTdqpK3MVIK9uKGtcSo4EkCbqps8ws+uJxEI5RpRZ5p7iBSlY1ReOPfXul3U6G9MPEBkweNR5Wo4HcAc+BjX1x0G6BMm8BJAXdvBORkAl4CFmowC8Z7E5ZKBPUT5ioGLu+OjJVqhEa8hveZC2M2WJRwoyMFTyjy2k0f0mQRnvj+PbEIX9nx0SvfTjUFABqKwU2Z/1Fm7Xv9WusQn3HTFnsyEhVU+QqZxtf77029QaAn7uZJvEiblZXE7TUUdcHfTdyjgSVOCgszweWtFLTMBCKcBoMx6yuvAiU7de3NbVmKBBV8LRfJqjM7RFqUHnRfyvtK1pKsfXe4zb2OgYq63aCbk2+0hF6r+nw/5hVzf3oLt+yiapdrUwdK204DQoAk7KTXc0CUaKI+YNGo7bKPIG5cPGui7kSYT8ZHQawvSD5trq1ErBFkeBBXADoptjHtDfi3Pg7Ff9bjSqv6u42noCshGmM1eBj496JFjjjXcJhMESM0UuKS0F2TKDR4mzK8GH/muRsjSoL0JfMn014g6VRrKHh+a22/xsmKrh3u4HljrqaMeDB4He5JHwM8s6+kyeU3CkKgOVeyN4xgfwHQ6JVUOEPzp4X3pKkge5V1WIX3TkAs/DwB8n2QfUb6ruFRychXQVgfdrAn7N5YcN5N2pPqmatRWTthyyJoTWcZlP35J8yKRhYIQwQbIQ2MaYqf5KLWk134Bzhxzo196snfUec2RW4rFit/7Kw9A1emZZT6nrbdKxVMb2x4rLRIxM6jc3snl4WQrocd8r+Jex/vzsh4zSwhhQK/ZeitOfxrewOqeLpG4hAG6rV+1VYTwlTQSKpvx/dthFLHcaIWOWtpB7h22PdCdxl4ctObXm+w4mkFr6WPaoITf8lxJbTwuYTIvy/Mq7HbwmfN07KQI8tkrtcRQ3iewm7r0HFtWQlYXGN5XBXsek/Aegzb5gF66fpZ5YIsbXbHelBwOnZo3v5sLifuZO8AKQCMFaIOK1Twc761o7A+q72PinuISoun30BXs8bsrh8ysRQpqLPQZ+xf48dz4zs6W4s9UdVoaLqG6poioJSXQUKEZX9xL5SLtn6VQrm9k6GtYtOTt9BqPFEyEwU4aQb7bAr9tjtFNFQyul65a2UajRWmTk8oOjmPQjG+0Na8AIoKRBXDTYolD25NtClc2GWUtxdF/r0g6WMambOrmoQN6PDvnNKiH2Y3LfxvCe/hoG96VD9Gr2SfaI7ut/DvIDGvS6cWJREA9GsI7fQpkHJD5us1nkC88mmR73jv99T+Zys5Jd8GStZTQDKhn9QFylG+VJvDSWAbh1nlVL/o3r8ujdUkBp+ttAnY5R83qnC7BMvidrwhW6C5R1H2kqzZKGGZ+24X44EGeMu X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f1f32d26-9f86-4df0-cce9-08db081684ae X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:48:21.2970 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VKLsBrolhjVwe4eRftYA1HkxxQ/SlJh07YGyjddLzMudWGCcOJdVx+UWim/wuIqJ+Bp6tmOMG+wJxI7NtMvsPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Kernel drivers that pin pages should account these pages against either user->locked_vm and/or mm->pinned_vm and fail the pinning if RLIMIT_MEMLOCK is exceeded and CAP_IPC_LOCK isn't held. Currently drivers open-code this accounting and use various methods to update the atomic variables and check against the limits leading to various bugs and inconsistencies. To fix this introduce a standard interface for charging pinned and locked memory. As this involves taking references on kernel objects such as mm_struct or user_struct we introduce a new vm_account struct to hold these references. Several helper functions are then introduced to grab references and check limits. As the way these limits are charged and enforced is visible to userspace we need to be careful not to break existing applications by charging to different counters. As a result the vm_account functions support accounting to different counters as required. A future change will extend this to also account against a cgroup for pinned pages. Signed-off-by: Alistair Popple Cc: linux-kernel@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-fpga@vger.kernel.org Cc: linux-rdma@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: kvm@vger.kernel.org Cc: netdev@vger.kernel.org Cc: cgroups@vger.kernel.org Cc: io-uring@vger.kernel.org Cc: linux-mm@kvack.org Cc: bpf@vger.kernel.org Cc: rds-devel@oss.oracle.com Cc: linux-kselftest@vger.kernel.org --- include/linux/vm_account.h | 56 +++++++++++++++++- mm/util.c | 127 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 183 insertions(+) create mode 100644 include/linux/vm_account.h diff --git a/include/linux/vm_account.h b/include/linux/vm_account.h new file mode 100644 index 0000000..b4b2e90 --- /dev/null +++ b/include/linux/vm_account.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_VM_ACCOUNT_H +#define _LINUX_VM_ACCOUNT_H + +/** + * enum vm_account_flags - Determine how pinned/locked memory is accounted. + * @VM_ACCOUNT_TASK: Account pinned memory to mm->pinned_vm. + * @VM_ACCOUNT_BYPASS: Don't enforce rlimit on any charges. + * @VM_ACCOUNT_USER: Account locked memory to user->locked_vm. + * + * Determines which statistic pinned/locked memory is accounted + * against. All limits will be enforced against RLIMIT_MEMLOCK and the + * pins cgroup if CONFIG_CGROUP_PINS is enabled. + * + * New drivers should use VM_ACCOUNT_USER. VM_ACCOUNT_TASK is used by + * pre-existing drivers to maintain existing accounting against + * mm->pinned_mm rather than user->locked_mm. + * + * VM_ACCOUNT_BYPASS may also be specified to bypass rlimit + * checks. Typically this is used to cache CAP_IPC_LOCK from when a + * driver is first initialised. Note that this does not bypass cgroup + * limit checks. + */ +enum vm_account_flags { + VM_ACCOUNT_USER = 0, + VM_ACCOUNT_BYPASS = 1, + VM_ACCOUNT_TASK = 1, +}; + +struct vm_account { + struct task_struct *task; + struct mm_struct *mm; + struct user_struct *user; + enum vm_account_flags flags; +}; + +void vm_account_init(struct vm_account *vm_account, struct task_struct *task, + struct user_struct *user, enum vm_account_flags flags); + +/** + * vm_account_init_current - Initialise a new struct vm_account. + * @vm_account: pointer to uninitialised vm_account. + * + * Helper to initialise a vm_account for the common case of charging + * with VM_ACCOUNT_TASK against current. + */ +static inline void vm_account_init_current(struct vm_account *vm_account) +{ + vm_account_init(vm_account, current, NULL, VM_ACCOUNT_TASK); +} + +void vm_account_release(struct vm_account *vm_account); +int vm_account_pinned(struct vm_account *vm_account, unsigned long npages); +void vm_unaccount_pinned(struct vm_account *vm_account, unsigned long npages); + +#endif /* _LINUX_VM_ACCOUNT_H */ diff --git a/mm/util.c b/mm/util.c index b56c92f..d8c19f8 100644 --- a/mm/util.c +++ b/mm/util.c @@ -23,6 +23,7 @@ #include #include #include +#include #include @@ -431,6 +432,132 @@ void arch_pick_mmap_layout(struct mm_struct *mm, struct rlimit *rlim_stack) #endif /** + * vm_account_init - Initialise a new struct vm_account. + * @vm_account: pointer to uninitialised vm_account. + * @task: task to charge against. + * @user: user to charge against. Must be non-NULL for VM_ACCOUNT_USER. + * @flags: flags to use when charging to vm_account. + * + * Initialise a new uninitialised struct vm_account. Takes references + * on the task/mm/user/cgroup as required although callers must ensure + * any references passed in remain valid for the duration of this + * call. + */ +void vm_account_init(struct vm_account *vm_account, struct task_struct *task, + struct user_struct *user, enum vm_account_flags flags) +{ + vm_account->task = get_task_struct(task); + + if (flags & VM_ACCOUNT_USER) + vm_account->user = get_uid(user); + + mmgrab(task->mm); + vm_account->mm = task->mm; + vm_account->flags = flags; +} +EXPORT_SYMBOL_GPL(vm_account_init); + +/** + * vm_account_release - Initialise a new struct vm_account. + * @vm_account: pointer to initialised vm_account. + * + * Drop any object references obtained by vm_account_init(). The + * vm_account must not be used after calling this unless reinitialised + * with vm_account_init(). + */ +void vm_account_release(struct vm_account *vm_account) +{ + put_task_struct(vm_account->task); + if (vm_account->flags & VM_ACCOUNT_USER) + free_uid(vm_account->user); + + mmdrop(vm_account->mm); +} +EXPORT_SYMBOL_GPL(vm_account_release); + +/* + * Charge pages with an atomic compare and swap. Returns -ENOMEM on + * failure, 1 on success and 0 for retry. + */ +static int vm_account_cmpxchg(struct vm_account *vm_account, + unsigned long npages, unsigned long lock_limit) +{ + u64 cur_pages, new_pages; + + if (vm_account->flags & VM_ACCOUNT_USER) + cur_pages = atomic_long_read(&vm_account->user->locked_vm); + else + cur_pages = atomic64_read(&vm_account->mm->pinned_vm); + + new_pages = cur_pages + npages; + if (lock_limit != RLIM_INFINITY && new_pages > lock_limit) + return -ENOMEM; + + if (vm_account->flags & VM_ACCOUNT_USER) { + return atomic_long_cmpxchg(&vm_account->user->locked_vm, + cur_pages, new_pages) == cur_pages; + } else { + return atomic64_cmpxchg(&vm_account->mm->pinned_vm, + cur_pages, new_pages) == cur_pages; + } +} + +/** + * vm_account_pinned - Charge pinned or locked memory to the vm_account. + * @vm_account: pointer to an initialised vm_account. + * @npages: number of pages to charge. + * + * Return: 0 on success, -ENOMEM if a limit would be exceeded. + * + * Note: All pages must be explicitly uncharged with + * vm_unaccount_pinned() prior to releasing the vm_account with + * vm_account_release(). + */ +int vm_account_pinned(struct vm_account *vm_account, unsigned long npages) +{ + unsigned long lock_limit = RLIM_INFINITY; + int ret; + + if (!(vm_account->flags & VM_ACCOUNT_BYPASS) && !capable(CAP_IPC_LOCK)) + lock_limit = task_rlimit(vm_account->task, + RLIMIT_MEMLOCK) >> PAGE_SHIFT; + + while (true) { + ret = vm_account_cmpxchg(vm_account, npages, lock_limit); + if (ret > 0) + break; + else if (ret < 0) + return ret; + } + + /* + * Always add pinned pages to mm->pinned_vm even when we're + * not enforcing the limit against that. + */ + if (vm_account->flags & VM_ACCOUNT_USER) + atomic64_add(npages, &vm_account->mm->pinned_vm); + + return 0; +} +EXPORT_SYMBOL_GPL(vm_account_pinned); + +/** + * vm_unaccount_pinned - Uncharge pinned or locked memory to the vm_account. + * @vm_account: pointer to an initialised vm_account. + * @npages: number of pages to uncharge. + */ +void vm_unaccount_pinned(struct vm_account *vm_account, unsigned long npages) +{ + if (vm_account->flags & VM_ACCOUNT_USER) { + atomic_long_sub(npages, &vm_account->user->locked_vm); + atomic64_sub(npages, &vm_account->mm->pinned_vm); + } else { + atomic64_sub(npages, &vm_account->mm->pinned_vm); + } +} +EXPORT_SYMBOL_GPL(vm_unaccount_pinned); + +/** * __account_locked_vm - account locked pages to an mm's locked_vm * @mm: mm to account against * @pages: number of pages to account From patchwork Mon Feb 6 07:47:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E565C05027 for ; Mon, 6 Feb 2023 07:49:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229965AbjBFHtF (ORCPT ); Mon, 6 Feb 2023 02:49:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229879AbjBFHsh (ORCPT ); Mon, 6 Feb 2023 02:48:37 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2051.outbound.protection.outlook.com [40.107.220.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2F6618AA0; Sun, 5 Feb 2023 23:48:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g1HGJAAb6x79B/2uO5UNl7Wjc2N7fO6FI2Dh1hbWDDlv+NO5uTH6mAdcm+er4k8nzu8P0Kaskg3FjHIudIEyRFfVc9Xm0Vk4e0HCDKF0MnkAmGYeoOz1TxF9jUHoP5rWgtbNqzdKjT/nhBnQVpGpsOmo11SmN+OhWMVQc8a3pxFwCVuSBKUFpmuYEYL9tW9J4OqExG8v9wf3KR1LoFdsLRxTuFo9mrXN5sxOPEk9l5jiCTs2QJNp66zeZI9ZeDpGsJCib950WH5sOGgsBpMI6sdq/39XLEmNcRQEo0wrXgYinIoFrAcUd1SZ9Tkb0u7ydaqjIuPUMKsr0WAVQ/xlwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=acjFfLRivjpU9Lg3kvi0rHomIWvWZtcIDwMkhY2Ky00=; b=WvQHQQY6ns3n0KefoFCSMggyX/x7tUCFBPrbp9NdXBsKIpZia9IjWTqt3eSONUAX+yLUVf/f7jSzoAsBeMeFbcbxdyGcuCNe8fhii7LnJQwbEVhoQP9ez9OILgeKvxr7XE6uG1UX4f3wtfuyx7uueogMthclboyNaqd3qFFXp9gdFuU9z5OQmVn7sX41V3f9gtlgjEpjlE4qtYykdA3BQ0qzRDi9puFRt2fttZpYdTSlTHSwl2yUwb/g78fOqsXXPtwz0BfZPXlM7XilI1VnZOxEpgjccCLr2WPMYZSyjgS55En9q1eQ/dhi5wL5ZcrgPrKpLkHNz+1GhTQZnn1PQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=acjFfLRivjpU9Lg3kvi0rHomIWvWZtcIDwMkhY2Ky00=; b=fHCzouLaE5kk62Ua+XJqCu+1MTMXe9QNNft5c35YdS0i50bvUAWqQwwDrn6WVqaAYfc5Aidg/xzaQ5SFBBhZ4gA+1ACieJwCaudY6l1BzZjCaxKVgn4qTaBx9hGKlo3/y6J/vkzzjo2G9GpCRfcCOkH2sQmIVJC57BwZg+W0DjiRsab/SpX6BqJr5kYCky0e9uZliBjJYNKp4n5PYxO1SsvltINtqi+iaoiKq/hCDojR1uoXMTeQmZEY3+t1wjV97EZ0mdNWs5YOSNhOa1mvSuEUpJttEznQpB3KvsQ9L9ynFNXmo9SFjZFg14jddI/QyfyTAlVenv/bKHF6vnVsrw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by CY8PR12MB7097.namprd12.prod.outlook.com (2603:10b6:930:63::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.34; Mon, 6 Feb 2023 07:48:29 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:48:28 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , "Michael S. Tsirkin" , Jason Wang , kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: [PATCH 02/19] drivers/vhost: Convert to use vm_account Date: Mon, 6 Feb 2023 18:47:39 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYYP282CA0007.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:b4::17) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|CY8PR12MB7097:EE_ X-MS-Office365-Filtering-Correlation-Id: d3c58a87-391d-4ca5-0a89-08db08168933 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qrNBIKLnIaAVk8pRj+5fZdPHiWQLbiCyBMI2zKg2cn/Fb5SIPPTq4lnCgIO9odXsb8ca5CT7XeYfek6V6/WonuEcy5JdMn6sLVlCiVnJ8+Ov1rIIBf9gLZjZuBCjnvH/Wd6xnfauUrOnS+56ZmvqmsGiKMU6TZbflY5PyeUFiyx3IRDXixQZZIT5ym6ZQ7MKh8raEJ/LH1o+UP7NJoPzzuQ48Jb0ifrzKOsxekho/unsBaIl2VV/xLiygpn8AGWQxju5/jRKJnca49WShesGkRCwuOwffP4oE2yAHv1f8cozt4/Yv0H5Uz2ztFJoql8uXWgUwQRX6IUgRTwCE7tQQuLG+Qz4yuo5eO9UsiTTacmJ8Km9Z6QVPwNo0KCrtfKt5095/INsZiz3W/Z6LobWgqDJ54JjdrO5q7VngPFH4X8T9mvvUz8lrVAsBxtlxyWg+EpHxRy7IUxEZP5qoAKo6gPJP0O2ia04BzC700tnSx/zt72yOijmqvpn2vGzxW2oWQqmePxjQduwtPn9uh9Dk2DH6zAoNnmfSHc6IIHnLxiJZaS8qc/la4R7HFKCn5HbPPtBif8eAJAqs5LEH2cvuBbe9xD5r6x0dls19uIBKIJ8FQRsdEQfuF6JagQheOknCfPj5fxaLNnCmWp0X7T88g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(39860400002)(396003)(136003)(346002)(376002)(451199018)(38100700002)(36756003)(6506007)(6666004)(6512007)(26005)(186003)(478600001)(86362001)(6486002)(8676002)(66476007)(83380400001)(66946007)(2616005)(66556008)(4326008)(316002)(54906003)(7416002)(2906002)(8936002)(5660300002)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BcZpvnNwT9Gb780Y2kPdCwKodPcm4oSD6aoJb8it9MH7scy9VMdJ8eQmHUo/tYk3av3s2dxZOcg3ogvWJjgYJU4CmqrjLA6eULGYWsluLBw3iUNDcrusaQMgU/JBCYKX4CFq4sbi4garbnQxS/dBlAc5UWaBAEJv37P0BnLLCYvW/Dwx9ye4gQxLpwkc2n5jCYJeEc6WoZby6mWJO568dMYKg2KqFAI8FtEb6Bo9JoSvaYrJXHnMENDbbxrwkfTQtV8qXbCNBsDq1bG4y4mUzAcpT6ltL5rHPqvzBcfp9a0GvjXvPnCh1LX1l8HQUHGtYt64+hq7tVHXRKdpIcExFvOwAmgADX31MwxqDpZ1EsigM4KMtzPfDVx/vfmJvaGRTwbpGx0EbqDCfRpQ/NiwpLIUmNj2TLoK9Fgqx52KVlNmvkfN/l+nVudtM3k3zgo7LK0zFvA4Opm+lkj5nJUUHw5HsDR07JdM7STkTeJ6+JFp8GUKNp2S3lSzvg+LD0UHH+9NTqzGYQR1OwQmv9X9qGGWuHfFfRshcD81zL7CSri0B6r0PZn2Skm99ITXb4U7QwWYgs8Mvnu0nltCKavJuxcKRHUid7Kc7yBRK6mvMSgyp1Vs1mvQ+0wMs/inEdtR24LXqGfQ2aWUSH4B4rhEUuKmK5VFvCbXiIbnoW07w8zy+rE9MQ4V2KxbUqGETOP/tnKo7LAqgyPnAZ86Evukt1YLLGqhWC9r7V7i+mv4n/P7mgmDR9Jk1VLcWKqTWA7TOBQHOcEGIUkPABm1MrZhjL02MNgERX18/btiG1nGe6WpG4Hb/+zY3+a16GI9j88p3JSFcs5xMYdMUgaDQElzTqvfLZRqq8dtaxudsmCkpMPccM85x3bsyKBzyUhOP5MgnZ7/qkaCt9tFdWlBIzyV6B0DN/8vtjiFBwpFdpR4rj0IU8zBuus0sVu51D/E4eGnljmwG46lyuzyj3hbjrpiJxL9cB+tZiZKndr5GDKLeOr5sV1FGNUfePKzE7Zb3cLL2zqzvs68vmMFuSwXCjEGF0hiTUqmUeAceR2buVQ29OrYUgQL5hgOPrjdgQnhr0AOdQyUVu+ZxlHmB9rtkkghCCZvOqN+HZQ5L5Qfu1tumWUtGXKur+We/kQfOlcujaDorNtRdjDJ57GxX1e5QaE153VWR9tbzgD3JE9w4nww7MTRGrthmSwRkuN2D6gN6680FZRM0Px/D0U8S1b+pXvjU36m3/F7jAHDOiewSiudjzHu2kmF/NEChDPlDSFg2JjWnmtwXC4I2NBznXKXqG09yDjvXfhsuspBJRQA5AXnbiEbStPOeCzR5Cmb9xR7X17C9fQEW3gmKHacB9XA1q9yijtgrp3HSRHGuDWlC8M7HjTVWQR8WM/jbVCBXk7zyCwt3eX/Ij0h6a/HZhYTBUCfUWznODflv9lgUyMH4VvEt9Ic7Ks70kksherMhreAowx2RXIO+QQfJcDbFi90lmLWQnUx5vZMdmzU90pOXc8Mjtz2U3x5Ed8urI5Ni3+MqGIBprHYtdNtEDAXVR+dp3tWVhl0URpCAcBafVezXDC9KtVY3LizG/14x+HYNs6/KqUA X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d3c58a87-391d-4ca5-0a89-08db08168933 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:48:28.8589 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XPSgdGbBaHcZDUXMyU+Av5M6xE1fuCzjJ6iTymGC1b4edw5OZwM9c0xmYlltTHFtDUBV871cdAahWmlacV+qeg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Convert vhost to use the new vm_account structure and associated account_pinned_vm() functions. This means vhost will start enforcing RLIMIT_MEMLOCK when a user does not have CAP_IPC_LOCK and fail the mapping request. Signed-off-by: Alistair Popple Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: kvm@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- drivers/vhost/vdpa.c | 17 ++++++++++------- drivers/vhost/vhost.c | 2 ++ drivers/vhost/vhost.h | 2 ++ 3 files changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index ec32f78..d970fcc 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -716,7 +716,7 @@ static void vhost_vdpa_pa_unmap(struct vhost_vdpa *v, struct vhost_iotlb *iotlb, set_page_dirty_lock(page); unpin_user_page(page); } - atomic64_sub(PFN_DOWN(map->size), &dev->mm->pinned_vm); + vm_unaccount_pinned(&dev->vm_account, PFN_DOWN(map->size)); vhost_vdpa_general_unmap(v, map, asid); vhost_iotlb_map_free(iotlb, map); } @@ -780,10 +780,14 @@ static int vhost_vdpa_map(struct vhost_vdpa *v, struct vhost_iotlb *iotlb, u32 asid = iotlb_to_asid(iotlb); int r = 0; + if (!vdpa->use_va) + if (vm_account_pinned(&dev->vm_account, PFN_DOWN(size))) + return -ENOMEM; + r = vhost_iotlb_add_range_ctx(iotlb, iova, iova + size - 1, pa, perm, opaque); if (r) - return r; + goto out_unaccount; if (ops->dma_map) { r = ops->dma_map(vdpa, asid, iova, size, pa, perm, opaque); @@ -794,15 +798,14 @@ static int vhost_vdpa_map(struct vhost_vdpa *v, struct vhost_iotlb *iotlb, r = iommu_map(v->domain, iova, pa, size, perm_to_iommu_flags(perm)); } - if (r) { + if (r) vhost_iotlb_del_range(iotlb, iova, iova + size - 1); - return r; - } +out_unaccount: if (!vdpa->use_va) - atomic64_add(PFN_DOWN(size), &dev->mm->pinned_vm); + vm_unaccount_pinned(&dev->vm_account, PFN_DOWN(size)); - return 0; + return r; } static void vhost_vdpa_unmap(struct vhost_vdpa *v, diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index cbe72bf..5645c26 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -556,6 +556,7 @@ static void vhost_attach_mm(struct vhost_dev *dev) dev->mm = current->mm; mmgrab(dev->mm); } + vm_account_init_current(&dev->vm_account); } static void vhost_detach_mm(struct vhost_dev *dev) @@ -569,6 +570,7 @@ static void vhost_detach_mm(struct vhost_dev *dev) mmdrop(dev->mm); dev->mm = NULL; + vm_account_release(&dev->vm_account); } /* Caller should have device mutex */ diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index d910910..b2434dd 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -14,6 +14,7 @@ #include #include #include +#include struct vhost_work; typedef void (*vhost_work_fn_t)(struct vhost_work *work); @@ -144,6 +145,7 @@ struct vhost_msg_node { struct vhost_dev { struct mm_struct *mm; struct mutex mutex; + struct vm_account vm_account; struct vhost_virtqueue **vqs; int nvqs; struct eventfd_ctx *log_ctx; From patchwork Mon Feb 6 07:47:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1E73C05027 for ; Mon, 6 Feb 2023 07:50:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229928AbjBFHuC (ORCPT ); Mon, 6 Feb 2023 02:50:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229921AbjBFHty (ORCPT ); Mon, 6 Feb 2023 02:49:54 -0500 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2083.outbound.protection.outlook.com [40.107.95.83]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B96CE1CF75; Sun, 5 Feb 2023 23:49:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FTNiPCF3AmyP3MYgZuTvz3MSn9Uq9zP3GgybfDDeC3e3lCUyL0UGqIWzr+L5C71E7bgB6hMmGqYga3WpRF56apR1HE3eRZlWxASaqHKXKdrBokTb7Ozg/gO+l2MC4Ld5RljeihEUYBK9IM+kWk+geqtPeyJJkTgdOKsQKcPlNU2CJzCmfZD3HRuO5byjQ3bAfKjWgC2LTdbxt81fe8oUrE5q0F7fKIWmJ4GxjfDOgl2M5gLeYH+QNuhWmmTTyxMbTmy4wRn+SpAGi/WUyBWrFnLE3DX+YUX78/SNDZ+BGjfT8chJUx110ciY+Y5Djwrz6rWHr6YPWf6uMI4gQbl1Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9EJyBunn0zT5d/t0YMoiL/ukwvRArAw0I+071Y/Pm2k=; b=VDyB2Hv2n45OEPJ8dQ9xlV4KOt3ZfMypAsGfaul0is5oM+cezUOq6+OHZ0fUJovFJF1q4cWJJgOKxMxDGFhGamJggn1ODOBfp0mEGCjfNHnbqtTIbsc94O42ORC7WJJ7pMeTSnqikVAvJ7uTW+vnSfpNgvtObwjqcPrr7oR96GErwF89fjgxsXW3FI0ibBHxFTl55wvrb8NvlK1yP1sgcni4B+6jPZDnPel+17kjARloof10xce5QhXn5ojJwHKo2qYOUHXUIGykKRujWbmldQPbSl2WPmS87CwCt2Yj9P0m2AuSYfTMBu9F2iZ2T8mQ5lZxxGJTdwndM+Tm4KVQJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9EJyBunn0zT5d/t0YMoiL/ukwvRArAw0I+071Y/Pm2k=; b=HdDvwTlno/2TJFtEwBIiS5c+AD1oEMvYqSxurzNyHD01OoIabNHwyzh/mIkTTgM9RtXxdonmgc5LImTBneN3gImrJwxvCtO3prL0TKaAWcP2MQIz0mjULA2GezhsbkxUfHouo9nN2zMGi79arJIK2cKCP0k29UU3oY1Vmesh6XMNMyXzk6ZdTnewXJuiGT0l1P6WEaPvoOtgok2ByKpKE+ZvFtpqZ8PxHNcfPKVqlAxX/8758AmEMs0krkoG0pqz0I6A0XwdxrMuKnjldSim9waR2MPiOJ0ZDre6eK3KkIk9o3QjRm4x6CRM0eqLykse91XKNIZp1mbef0MmDS5IEA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DM4PR12MB5118.namprd12.prod.outlook.com (2603:10b6:5:391::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.31; Mon, 6 Feb 2023 07:49:00 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:49:00 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , Cornelia Huck , kvm@vger.kernel.org Subject: [PATCH 07/19] vfio/type1: Charge pinned pages to pinned_vm instead of locked_vm Date: Mon, 6 Feb 2023 18:47:44 +1100 Message-Id: <60d8812272042f497b8d661f031f2cb593370b84.1675669136.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYBPR01CA0191.ausprd01.prod.outlook.com (2603:10c6:10:52::35) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|DM4PR12MB5118:EE_ X-MS-Office365-Filtering-Correlation-Id: a122e17a-0e75-495d-a900-08db08169bd0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: D19c20F/3H5aBZUJtRuOisDUPLPCb3g6LkxaoePLUkkit594ylhxEln8lwG/d82+XWI05rsnfmCDpC1vsCuFjbdq0T+KCv+KHNH3Z1aAqkjU33XcFrBUQZbMEeFQgtvkBiGTeIJSKIQ2MddZlxG3YfOmSew5kvepMZq48suLfmTQwBEwo33I70s+Ja7PUUn3QNSE2FUThE0XFwbxgUaMIVe9GUF7NxVwc0qy2wj7pQXxrzDJgUHC0Q/JBTDXEsmthnznbrRHDiwJWaA+aWWrSoIXSx0VRQWGcVYheLkGBLu+xXeyIC2Gw18dBemMc1lxOq9jPC/DEAWZgqE7doc79lVQdyjo3KWLVWjm0JFbRAuXGheBNJAPf+5889LNCR5KX85ktUrTfOcg9QmjlEGFM6ixS39AqYugLMq7WkqF9RollRnLkbbzSyyHvE7kVodm3SJcWO9ptdE4zFYUWLipDugvSD+eqj9yI9v3B3/ox962XFxUFNzLjsjNCmQByB6eqtA1ZuSgg2tVdvGMkg9/RJlmsGzydh7de7yXSi9TUo5rfcq4YSgkGmaVEU3CzfVio1l9S/WGrRARhBlmNFsOVmZEcnOu6RLIM+iasIdbBDJLU4TfVxUJv/FRYQK+cf6Yk+LM9OBZtSt76bhD3O/kNw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(39860400002)(366004)(396003)(346002)(136003)(451199018)(66946007)(66476007)(4326008)(41300700001)(8676002)(38100700002)(8936002)(6506007)(6512007)(26005)(316002)(54906003)(7416002)(66556008)(6666004)(5660300002)(478600001)(6486002)(83380400001)(186003)(2616005)(86362001)(36756003)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7zc1D3qcVz4pASVKaNOAYrrUtV515FbAbdEHWZkkbDHYjdqtKw0XPemXYJSYwcfzLz2N9qbtXnZJfs0tBfDB3GSEH1wVwNPZ1X6UY1BJ5KrPEplmF0HpxUZPsWX/gkHfhPPhl/FvT4t/8RXEzjCW9utzJPt1FSXPpNFy0wJvExmYLYGZliTc1vfC87bBfJ8P/upzU6ZwIuW0LWEBZ0TU7hNJQUK2ux5fpkRiux5QqH1EpPELod6+8+qHoXHKoWqjCAOCNFCIiM5F3sMg3f35sg62LsokIe5xTlMQWHSYlIOtv5HNIHWZaLJrqiPqD5GHCUJMUoruk4gxNmPkdWy4YTA91t2WFJajEHXQchyNePxdNSsLESXHZEPRmIfHIWUa/FXfhK2k5oLJ7EUi/Hz/vg37L2ip9MzKCGP1f1MtrZem2u0dS/oMmRY8wcCUgiedpGDfPp6TAzB4l3RAwnGq+VoSbkO/OEB1qAO/IY6mkMicIxQsOK1U7b/1SMMw1k/31EwMPMRVYNpKy3Qi1si8fFkTaNjcfpUMamA2bQl8HojR4dbZvu+kj2aXstteEXKk7+phR0UYv7Y4isQsqxem2f9QEtruYJGvO9qghNvs1VaZ/0iBS+N0UEzwEN663iGblYAGHlYVrnOU8sUd6YsOSQuba1ohT3+BzDMfhKym0u36eIbfwLYO6tjopztiaoJg77KqGbYGTr1ClDpDZH6yI6h/bfpBWef3lfgUoXTqskTgc+xq90tObDDdTXwsIOy6avYGH7xDcfFJrWFQgsNt3x1CObnIpEO2yHE+n1kw7WXR/wR8svDv2R8SJ/ycXZTmlZ3WogEY3gW/hBiaso/oZdTSl76qJGT1pwqVQCFjZsuJyxXjned/0NkrGnTQy9RGjWMWa3Y7HoamZbppfx3D+cZNoVUH4A7mdhLi/of3bKSbp4gnekkEQAhLOi7cGLz2UbtrH0Ji1fAe984WujANhSml4Tl4OQjAGSUPKQwWBScmb0mebOu2Ze2Lc6wR1cbriRon0gWwJNCqCWiyLu+yxeCWaanKbvUwSI3xWAClIRq+F6Hel6j33cT617ZUEojAfuS5quvUQH+xd7teq9ytucZb4M52lvZAzsiCjuLODlYHuwwfsEKMhu7PsKg1viWrQGxHxBax5NhSPmSsFb73FBaBWlSOq4ulWZQo1M0Qs28g3IjtoENbr7Wdb2t9E3AFJoaeQYNUzwrSDwOE7ARLQrq7us7gDNOGQR62g+f88SHwPq/D/D/hbmBVJYGY0JqE44wM0Zshh2j5g4t1znhkLLxDJuFvfefxHF6Z94MZGwci8zh+bwxhHueDNR/jB1YC5/krCaVWho7fV4J4PEPyfHN9/HhCvdBhhdA9DbjnXenyy1WwDwj0rojUpYu/JmgUwY578KclsGklzVMkBXHquwuys7EGWL+DAgJYfIIWMop+ns4bOfJ1/nduNR+t2xGPB8cOeXF132uSMdDue+oaGLwGRzbK4ltbp+RExjC37GcxTHoXk5J2BXSYY7x4hlNeGz425lHvovqNcU2nJ4s8/HNl3flko8QKgwAAa8P2g/ug2lOCRA/95k2JHR2Kid1U X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a122e17a-0e75-495d-a900-08db08169bd0 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:49:00.0753 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wp+R7my8ypqqhhSOfLO+TVLqtpDWGatzGK9uJJC/Lw68ZvcGUlzUXp6UMcD/A7fO22cL5YxY86EhX2g6PtdhlQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5118 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This switches the charge to pinned_vm to be consistent with other drivers that pin pages with FOLL_LONGTERM. It also allows the use of the vm_account helper struct which makes a future change to implement cgroup accounting of pinned pages easier to implement as that requires a reference to the cgroup to be maintained. Signed-off-by: Alistair Popple Cc: Alex Williamson Cc: Cornelia Huck Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- drivers/vfio/vfio_iommu_type1.c | 60 +++++++++------------------------- 1 file changed, 17 insertions(+), 43 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 23c24fe..a3957b8 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -38,6 +38,7 @@ #include #include #include +#include #include "vfio.h" #define DRIVER_VERSION "0.2" @@ -95,11 +96,11 @@ struct vfio_dma { size_t size; /* Map size (bytes) */ int prot; /* IOMMU_READ/WRITE */ bool iommu_mapped; - bool lock_cap; /* capable(CAP_IPC_LOCK) */ bool vaddr_invalid; struct task_struct *task; struct rb_root pfn_list; /* Ex-user pinned pfn list */ unsigned long *bitmap; + struct vm_account vm_account; }; struct vfio_batch { @@ -412,31 +413,6 @@ static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn) return ret; } -static int vfio_lock_acct(struct vfio_dma *dma, long npage, bool async) -{ - struct mm_struct *mm; - int ret; - - if (!npage) - return 0; - - mm = async ? get_task_mm(dma->task) : dma->task->mm; - if (!mm) - return -ESRCH; /* process exited */ - - ret = mmap_write_lock_killable(mm); - if (!ret) { - ret = __account_locked_vm(mm, abs(npage), npage > 0, dma->task, - dma->lock_cap); - mmap_write_unlock(mm); - } - - if (async) - mmput(mm); - - return ret; -} - /* * Some mappings aren't backed by a struct page, for example an mmap'd * MMIO range for our own or another device. These use a different @@ -715,16 +691,8 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, * externally pinned pages are already counted against * the user. */ - if (!rsvd && !vfio_find_vpfn(dma, iova)) { - if (!dma->lock_cap && - mm->locked_vm + lock_acct + 1 > limit) { - pr_warn("%s: RLIMIT_MEMLOCK (%ld) exceeded\n", - __func__, limit << PAGE_SHIFT); - ret = -ENOMEM; - goto unpin_out; - } + if (!rsvd && !vfio_find_vpfn(dma, iova)) lock_acct++; - } pinned++; npage--; @@ -744,7 +712,11 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, } out: - ret = vfio_lock_acct(dma, lock_acct, false); + if (vm_account_pinned(&dma->vm_account, lock_acct)) { + ret = -ENOMEM; + lock_acct = 0; + pr_warn("%s: RLIMIT_MEMLOCK exceeded\n", __func__); + } unpin_out: if (batch->size == 1 && !batch->offset) { @@ -759,6 +731,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, put_pfn(pfn, dma->prot); } vfio_batch_unpin(batch, dma); + vm_unaccount_pinned(&dma->vm_account, lock_acct); return ret; } @@ -782,7 +755,7 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova, } if (do_accounting) - vfio_lock_acct(dma, locked - unlocked, true); + vm_unaccount_pinned(&dma->vm_account, locked - unlocked); return unlocked; } @@ -805,7 +778,7 @@ static int vfio_pin_page_external(struct vfio_dma *dma, unsigned long vaddr, ret = 0; if (do_accounting && !is_invalid_reserved_pfn(*pfn_base)) { - ret = vfio_lock_acct(dma, 1, true); + ret = vm_account_pinned(&dma->vm_account, 1); if (ret) { put_pfn(*pfn_base, dma->prot); if (ret == -ENOMEM) @@ -833,7 +806,7 @@ static int vfio_unpin_page_external(struct vfio_dma *dma, dma_addr_t iova, unlocked = vfio_iova_put_vfio_pfn(dma, vpfn); if (do_accounting) - vfio_lock_acct(dma, -unlocked, true); + vm_unaccount_pinned(&dma->vm_account, unlocked); return unlocked; } @@ -921,7 +894,7 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data, ret = vfio_add_to_pfn_list(dma, iova, phys_pfn); if (ret) { if (put_pfn(phys_pfn, dma->prot) && do_accounting) - vfio_lock_acct(dma, -1, true); + vm_unaccount_pinned(&dma->vm_account, 1); goto pin_unwind; } @@ -1162,7 +1135,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, } if (do_accounting) { - vfio_lock_acct(dma, -unlocked, true); + vm_unaccount_pinned(&dma->vm_account, unlocked); return 0; } return unlocked; @@ -1674,7 +1647,8 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, */ get_task_struct(current->group_leader); dma->task = current->group_leader; - dma->lock_cap = capable(CAP_IPC_LOCK); + vm_account_init(&dma->vm_account, dma->task, NULL, VM_ACCOUNT_TASK | + (capable(CAP_IPC_LOCK) ? VM_ACCOUNT_BYPASS : 0)); dma->pfn_list = RB_ROOT; @@ -2398,7 +2372,7 @@ static void vfio_iommu_unmap_unpin_reaccount(struct vfio_iommu *iommu) if (!is_invalid_reserved_pfn(vpfn->pfn)) locked++; } - vfio_lock_acct(dma, locked - unlocked, true); + vm_unaccount_pinned(&dma->vm_account, locked - unlocked); } } From patchwork Mon Feb 6 07:47:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C9BBC05027 for ; Mon, 6 Feb 2023 07:50:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229974AbjBFHu3 (ORCPT ); Mon, 6 Feb 2023 02:50:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229788AbjBFHu0 (ORCPT ); Mon, 6 Feb 2023 02:50:26 -0500 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2083.outbound.protection.outlook.com [40.107.95.83]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F67518B2C; Sun, 5 Feb 2023 23:49:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ScbFAXZwDSHDQYWfo+yBKa5EfS66fkePVN0KUmOunWodLhHCV4ULVvfwNauI0zQ2eBvnik0jEi90CMzWz5DR1c6Cz2v8NQaBE+tWBt7nr4z0CWa10xiznBdgp1Nm+k1z4xDUJ7ye12qJ7T+J4cEtfuMMrxPdgZ3XbBqw6GRxD/ZwtvmU5PvrZGd1TDzht3yybqzIY3WAfqqINM5HegVLwJ+xrvke7Cb//NFdGo4yyK1cll0HqzKuL5jQ3bDL7w+1MLnYchBhYmlkXkwOAb1VEKqP6k55wRdeIn1vd7jSMr0Otg5wbt/EgYcaeFtLjUUsCOnTXcifHQvh3wCKcJH/mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4aNGQ6CEFF/Iwo4DWZWCLfPBaTkYJe6nei2IWX2bVLk=; b=LqvQqbW1VJRiXVEeYE2GmDfVFnPTJiQFDnWiyaE2fKtllJZNL8ZrjRiGcS+1GgtroJtApXou7U1szAKFuscrS8aTGqqBt1H1R4fte3aXwMG2Fm8ZpDmj7WmM3I542Qd5Itnh10o4X0h/BC6C2Dke/IswXSCFqovNS6UHd3FJwaDpvMHShncjv/gUFDEsj+ZCtlv29xxAX+8YqNctu7D42Y8wWdhoKx2wFvKSnwhCzUNWDZgMSg5ZTDnhECuel5n0D2Z6CYfCro/c3uCQnvcaAwRFbg2s7k6sgVTccfZQlbemI34922wSM58l57WYsgNnbpIEbY1eC4nSpEToU8vadA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4aNGQ6CEFF/Iwo4DWZWCLfPBaTkYJe6nei2IWX2bVLk=; b=VT3h3BuuFakyBa1tv3zMLWgdy0gFVM4xRbhSGmYmbGmqzVcCtyci/LlF1QBXJUbYcfqQfRAGEzvCGyS97l5/O8gteE7GX0CdtPbs52p6BURnxvnjHJspqYOjLDaThTCu4IvpaqqjDxhkgNVXSsp9lkSnDOTmiGW5EtiOkLLfcHwq6/fZSwaW5ITif9x+5mFkec0S+WcCxKzehbDCIKbbn/BUXbT/NNndSJ4xwCA2g+piUzTPeaOveLcnKjI/epZ6yyBYT86+NrBi/q8u9ZzZfezDPbPUSCaxiY05/P+coJEKaaS0S864sYfyKsSPOyxFPjBpmlmG4Jslgofey51zyQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DM4PR12MB5118.namprd12.prod.outlook.com (2603:10b6:5:391::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.31; Mon, 6 Feb 2023 07:49:07 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:49:07 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Cornelia Huck , Alexey Kardashevskiy , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org Subject: [PATCH 08/19] vfio/spapr_tce: Convert accounting to pinned_vm Date: Mon, 6 Feb 2023 18:47:45 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYXPR01CA0088.ausprd01.prod.outlook.com (2603:10c6:0:2e::21) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|DM4PR12MB5118:EE_ X-MS-Office365-Filtering-Correlation-Id: d4a9ea4d-62d6-4d24-e89f-08db0816a035 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iWUpf7QiSNpZ/BcD+AET9d7eIgcFareDzNOWhk99Dr/Ree6FI7pWqkP+D6TDXFKgY8TZ5XQ+RaPznI/kT55B3NJLbNkZrC6XyIUQ+uS+KzEPaNXGt0SPCwRVLF/egr6P5SexkJX9JwUJcKB6DKsLauOBEqRBUrgsWkSdBZRL0xisfwyTYcfgT4aUxOD6PveYHjh05ByLg2KqTXLbEZegTendZQWWSRguwakP8Pu4pSc7oSAKtf47yRVRj7sZvUKxHjmIEpwt1UXuONjAYH/+WTs8eDQTKj0+Oc96d5UOKXKIDxeYnAgNSeyKs3qWuwVmGdPoEdfHE8Bc0z2JrhYs44Z+51tYalcIMwPp2JWpLdeLtopMhcyR7CxIoEn/6FV/6Jp1+MQ5ymTSs91yipbEPNSwq00AGa1/MCqsZ+UT+zC3FmhhkdG6TjvRt9Q5e3Uj6xOskk3CW2y20+iqlSdM8liDU7ATBa+RDMqKOuczBmU5xlRR+4UIF1MlbNhlq2d2zlO9AF0Xat+u3C++9xus7by4OHqgYNtrtBtz723yxLGYW/72hska2T12DgReKGk+zTVqtepdD92mhiMdFY6VpIiUoEQN5D6+OhZR/0iCP0csDkfh2HsLjhte2EcaA2JjFmeXrzRdyu3g3649c31Alw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(39860400002)(366004)(396003)(346002)(136003)(451199018)(66946007)(66476007)(4326008)(41300700001)(8676002)(38100700002)(8936002)(6506007)(6512007)(26005)(316002)(54906003)(7416002)(66556008)(6666004)(5660300002)(478600001)(6486002)(83380400001)(15650500001)(186003)(2616005)(86362001)(36756003)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: J1lKquI9iAaemJegDUiKMox/qBsp+Xu2+DY++QRpmFu2QoWhovc9ib2zZo1G0eBx99eI1o60+2qoVPxEZsYAur9+TbUbor5EWDFYAuP2q7YyF9B6C64YH/301qDJdftQOmSC9O/wYd2RBUvPyPAtIjIj+4zEkJoFQcRlDv2hyMunJFFSTcWWm2TSqAbQoCTBKJ4kD0c+LSHsF9HYlZfQlkh/8X1snQVDsd/zBgU7rZ3R1zTmWVnm6jTDddQm7I4gRRnTHtyTGNe6NCMsB6eMHU6qgPaSF/A5iwsn3NnN6qUwU0tRwF1twT38zBAO2NmATvUJHDJkW8PixdBPoO+gbsDzPdxnaZRdo+zpnOZhvVx6SBtGJlT4Y2dySOAyVWVsbsqWpcs/tl1cd/zRfsC53ujuYWZ+q++5HwybfvVAqsiviPHv7z1K1x2Vdda6TEeH6S2rMjBjkvZHTVJSeVuwfs4+tcrLpECxV3HeiAy7oTrEv7IGbFVzF+C7KcRSaAM2SUv6gsQ0a+iGwqBPcsLyqxwZ3dCH7CuC57njt7PrfshTdkcNSU5G2vDOW2buKTZxe6F9NA8BhDn+ALD379YY08kmNpy4kpS1ZSQcvorw1Kk38j2qw55ljmZ0NTHktplOQN0f4sKdOQ6Qudf8nkM4Ndjr9+CHNdCNQl/eN5Pqc6J76tiB9XKevtclW8DcWajvicHkxDvmfAV9oQ6/WykblpQINY8vH87IkUbowuOC+RVsWg2P75oMch5v2sQKse4G5uWvEgHg5oHoALR0nlWwzLO21bn9UK1xSQipB//zA8WKE4DL0JsNmHlR8KYRB34ytvK2+ls4w8ZZtNKSR0uSnvuG3VsPsiN0p4Wi/L56Lqt/GYJ6Bl58SO8A2OV8gL4hfQxFcTFbreeXUQ160Nt9x4dbwL+4QckFmppuSqIsZ3xtQuc63FiQ+NpwvWpIgw/np0Al11hPivOjhWvEFAYq6AcMDRaqLEaKN+FOREWiQ6CCHlCZ6HrY5flgoIQdqsjJAOm/9Q8JeueN2rctmsBL4GkuebsbN4eXwBijfAbXafq44Q4x9GyPdBxLHorrrYJ/tn5I0Tv8rf7gmFnF1Gwgoa9Z6U1qsBQN/Ys9PH7YQBWH4k0wLqEREtpcSiPuGCwTi/GkR6g69bcmlTyOOzzLUnkc9KXHwQImA2mNPANldwc3PIAs1ytmZjp0RQeEwDUswEAULDJudNK/+SvQEHw5pT1en25CnrWqJwf/RUfu5z0UNIbJWPXyIIP7CmHy6+EEtc2EwZCOIvNZU1Sw8bYkESR21uXdb6dbYY3bW9cVZmmyVgDY2Rz2mjCsDjTMf1LFth/trwHqzeyDGpj46YlT44Tm1EIl+x/8/b+yT4bvEi1owVKwMAgyHXUrIJSEWMReQAr/YxRNRwlPMQ+d/5kYPfdqtvRAPQQYkWmWrEhzYYWR9Zf8I/2LwYnjCWbIy+234kzcnayjyKBTeqaQSvoIrUEu5N/xxoYTRZC2RUNO56usZwjOJ3Xg6iwgmPyci3b7yOmY0q02UDrefLlnVT97xyfmbwrDsxQTRGpDnGXMdzRSo1mAb+1+GPViNtvykK01 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4a9ea4d-62d6-4d24-e89f-08db0816a035 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:49:07.4497 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 52T72HI+k1c4VKZrgbWoZZ7MGyhRNB9OFQRaWOU/PKhx3Et+QT3kFLjIAGrHRAdgJ7sUVRlsW6ZUl94FqMJZnA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5118 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Convert from accounting pages against locked_vm to accounting them to pinned_vm. This allows struct vm_account to be used to track the mm_struct used to charge the pages. A future change also uses this to track a cgroup for controlling pinned pages. Signed-off-by: Alistair Popple Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy Cc: Alex Williamson Cc: Cornelia Huck Cc: Alexey Kardashevskiy Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/powerpc/mm/book3s64/iommu_api.c | 30 ++++++++++++++++++----------- drivers/vfio/vfio_iommu_spapr_tce.c | 16 ++++++++++----- 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/arch/powerpc/mm/book3s64/iommu_api.c b/arch/powerpc/mm/book3s64/iommu_api.c index 7fcfba1..338b111 100644 --- a/arch/powerpc/mm/book3s64/iommu_api.c +++ b/arch/powerpc/mm/book3s64/iommu_api.c @@ -18,6 +18,7 @@ #include #include #include +#include static DEFINE_MUTEX(mem_list_mutex); @@ -30,6 +31,7 @@ struct mm_iommu_table_group_mem_t { unsigned long used; atomic64_t mapped; unsigned int pageshift; + struct vm_account vm_account; u64 ua; /* userspace address */ u64 entries; /* number of entries in hpas/hpages[] */ /* @@ -62,20 +64,24 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, unsigned int pageshift; unsigned long entry, chunk; - if (dev_hpa == MM_IOMMU_TABLE_INVALID_HPA) { - ret = account_locked_vm(mm, entries, true); - if (ret) - return ret; - - locked_entries = entries; - } - mem = kzalloc(sizeof(*mem), GFP_KERNEL); if (!mem) { ret = -ENOMEM; goto unlock_exit; } + vm_account_init_current(&mem->vm_account); + if (dev_hpa == MM_IOMMU_TABLE_INVALID_HPA) { + ret = vm_account_pinned(&mem->vm_account, entries); + if (ret) { + vm_account_release(&mem->vm_account); + kfree(mem); + return ret; + } + + locked_entries = entries; + } + if (dev_hpa != MM_IOMMU_TABLE_INVALID_HPA) { mem->pageshift = __ffs(dev_hpa | (entries << PAGE_SHIFT)); mem->dev_hpa = dev_hpa; @@ -175,10 +181,11 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, unpin_user_pages(mem->hpages, pinned); vfree(mem->hpas); - kfree(mem); unlock_exit: - account_locked_vm(mm, locked_entries, false); + vm_unaccount_pinned(&mem->vm_account, locked_entries); + vm_account_release(&mem->vm_account); + kfree(mem); return ret; } @@ -229,6 +236,7 @@ static void mm_iommu_do_free(struct mm_iommu_table_group_mem_t *mem) mm_iommu_unpin(mem); vfree(mem->hpas); + vm_account_release(&mem->vm_account); kfree(mem); } @@ -279,7 +287,7 @@ long mm_iommu_put(struct mm_struct *mm, struct mm_iommu_table_group_mem_t *mem) unlock_exit: mutex_unlock(&mem_list_mutex); - account_locked_vm(mm, unlock_entries, false); + vm_unaccount_pinned(&mem->vm_account, unlock_entries); return ret; } diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index 60a50ce..454ccc4 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "vfio.h" #include @@ -67,6 +68,7 @@ struct tce_container { bool def_window_pending; unsigned long locked_pages; struct mm_struct *mm; + struct vm_account vm_account; struct iommu_table *tables[IOMMU_TABLE_GROUP_MAX_TABLES]; struct list_head group_list; struct list_head prereg_list; @@ -82,6 +84,7 @@ static long tce_iommu_mm_set(struct tce_container *container) BUG_ON(!current->mm); container->mm = current->mm; mmgrab(container->mm); + vm_account_init_current(&container->vm_account); return 0; } @@ -291,7 +294,7 @@ static int tce_iommu_enable(struct tce_container *container) return ret; locked = table_group->tce32_size >> PAGE_SHIFT; - ret = account_locked_vm(container->mm, locked, true); + ret = vm_account_pinned(&container->vm_accounnt, locked); if (ret) return ret; @@ -310,7 +313,7 @@ static void tce_iommu_disable(struct tce_container *container) container->enabled = false; BUG_ON(!container->mm); - account_locked_vm(container->mm, container->locked_pages, false); + vm_account_pinned(&container->vm_account, container->locked_pages); } static void *tce_iommu_open(unsigned long arg) @@ -372,8 +375,10 @@ static void tce_iommu_release(void *iommu_data) WARN_ON(tce_iommu_prereg_free(container, tcemem)); tce_iommu_disable(container); - if (container->mm) + if (container->mm) { mmdrop(container->mm); + vm_account_release(&container->vm_account); + } mutex_destroy(&container->lock); kfree(container); @@ -619,7 +624,8 @@ static long tce_iommu_create_table(struct tce_container *container, if (!table_size) return -EINVAL; - ret = account_locked_vm(container->mm, table_size >> PAGE_SHIFT, true); + ret = vm_account_pinned(&container->vm_account, + table_size >> PAGE_SHIFT); if (ret) return ret; @@ -638,7 +644,7 @@ static void tce_iommu_free_table(struct tce_container *container, unsigned long pages = tbl->it_allocated_size >> PAGE_SHIFT; iommu_tce_table_put(tbl); - account_locked_vm(container->mm, pages, false); + vm_unaccount_pinned(&container->vm_account, pages); } static long tce_iommu_create_window(struct tce_container *container, From patchwork Mon Feb 6 07:47:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B319EC05027 for ; Mon, 6 Feb 2023 07:50:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229978AbjBFHug (ORCPT ); Mon, 6 Feb 2023 02:50:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229921AbjBFHuc (ORCPT ); Mon, 6 Feb 2023 02:50:32 -0500 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on20603.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eaa::603]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E950B18AA1; Sun, 5 Feb 2023 23:50:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=obie/PwJcOe2hDpi6+/p0mbSZE+iDhqWzJUUigICaut5IiNVmMZcMK0s1zPImn5mryTULm8oG6FX05Q+fIoIJ48WxWZLG8j0S6AYWXDjPrYtZmdOlO7ztoKDhgjInSqWe+pDAxICHsYnluMjoSz3U6wTTjmbmpBLuTOW7bnT7WwcXK8BaXp+ZAoAmghhz/eZDuTVhoSgq+Yub+eQmBp00p/B16DOhWTzesK9GBmTheTGJjSgpHFdMgQbvi3dH7PyuFOrwI3gdkOuY0Q4e8Z/k3JksCiPrU2s71luD/qYO2rlokmq8mLCMFwg7ekZxSXAu3c85YcujhyBylouCQBJ6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sABmSCNhqUaki37ROAvn5I2o7zlTWP9iNz0mlKPN/jY=; b=CFPPMeAcsBiiJ46UEfy5VNfnk4mAIN6Dr9ckIFEop+AVXMXunqipKYFibH3NfUpVVLybvP4oE7GZRlJH7uE57qY7G3+DUvMQ+X3WbITsattT5PXhPfZ5CrVtPRHeOJNDLO2oTZDuawODWrGJDJcOIIcHRpgi6MXri/KS6PEB8CmBZPXonbKPMwEBFBcQ+wfiEwYtKcGA99gATAvqib8pez3DEKJbGIJea+7+BrO5kC0YcQF+hd5Q7uDl65rGhKN4jFj1jwP1gtSF0kA6BOp+xuoSJas3Oqyi4kkDQsSmdccxxrq5XtsJdnf3MxW349vZAQmWTJzukswV4GNS2HNZVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sABmSCNhqUaki37ROAvn5I2o7zlTWP9iNz0mlKPN/jY=; b=CHQwI+iS0hEE2NUjjhh1a8duKr1QEY6gPXkrUEGnhAhiYZh+EwVnEAFuu6/3arCe0qEFPyeE2mBJS0Udgi5k66OavT3AMmplZcz+uqJ83WF5eUQ5sS442WqtwUFWuLpFxK/zyKakGOwypeQsFr1EktqHWHS5HGj9MTzIj/AKquxhLnOuUR71GyVwKhW56iQeZoCko6AgUbTs5rLuExRipjp4QI8mrKGzItovJHdZJzO2Fpz/Ba0JCxnBKnH4N5jmB2+0c3yQah1YL+DcG/bEQext5pLO3aDQcjy6sqSW9YyWytpFSiGFz0j/RXveAB7XaOc8DdrOQSGve00kTSX1UA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DS0PR12MB8573.namprd12.prod.outlook.com (2603:10b6:8:162::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.32; Mon, 6 Feb 2023 07:49:34 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:49:34 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paolo Bonzini , Alexey Kardashevskiy , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org Subject: [PATCH 12/19] kvm/book3s_64_vio: Convert account_locked_vm() to vm_account_pinned() Date: Mon, 6 Feb 2023 18:47:49 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SY0PR01CA0008.ausprd01.prod.outlook.com (2603:10c6:10:1bb::7) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|DS0PR12MB8573:EE_ X-MS-Office365-Filtering-Correlation-Id: 1711d04f-1228-49b9-6835-08db0816affc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ivvth6W4nqFwCtLaRy31c0XU0ftAT7d0a+ncwklm/9hY8CDYIaLq8S9Jt2xQjUHxCuE39zwggwD/XO53FRHcYW1qtRs2tTLOW7vRhqIrBHYMJ9323/945oJADFzxZKHDVs6lUFv50zoYf0bfEusrFpmVmPVJQLmXVgYDaqhXSlybJ54Nb+MoKlEYynZapcQ13LWsg2+98QqVzcvI9NiZH4u12UOdiXeL29R/tul5Gx5oHmlnrsYMHLVbYioDcqqURbUci5Mwsw5cdFsI2/jjQdQ1LGAlr4ca4tdToJy6ks9DEVIb0CoVjPrJ3mmvqOtileYU5KI091ItQnZZdNTO9G+Ew9R+MxfNlFWij0HejOfkyr+GwPn/bJvNDs0QZGHbV5zwtu/7CvLHGuEsBLQbtuPSyAyOaNtdkwsBgX4ILtycqzAUXa2SuJNL3KL+iQU8SK/635mU4gavA2nari5HHpXFjPKm3fPLJsBM6IGHUdYfGDyF9/tMbN1Wfm+HbYu2hhi1aZuach6sacGBh0z7TgaMd0iDiWPRD5BLqE8YTEFT/D16746WxEkx0Ok1H6xxQ245Udui6PaqaB+SHubEythSM2HiRTk3B8ltzXn/PygycjHCQxhMrJpallA2QZIeQxTKJiB/X78qMo62cUKse58eA40a69NMAtKUimwAxyrRhv7TbnRiI5nrrqZKWN/2vbvQs0zCJkwryFodHzS3dw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(396003)(136003)(39860400002)(366004)(346002)(451199018)(54906003)(316002)(86362001)(6666004)(6486002)(38100700002)(186003)(2616005)(6506007)(26005)(478600001)(6512007)(7416002)(5660300002)(83380400001)(15650500001)(2906002)(36756003)(66946007)(8936002)(4326008)(41300700001)(66476007)(8676002)(66556008)(473944003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ay8XosogCpkzfmHNeC+DwsG6+2Mjp57f5iWgheg8h1dEFMr2NETA+gBVc0bYAPJv9JWXE263bQhiw4FP0PqKc4qUtuFNRZZ59XV7v4yHQU1T4a56TNbxaxHMFwG7LGQTMNg/jT0xVXpb8rSm7/z7dV4NrDckvYimWl9aodag6vY3l2LHvTs4wNS/La/dsWq5nOsK6QeI0rJmbpDpFngl5D986CNO2Vi3y3d1YQPIrkEQw8q2B0XUo8ExB3gHezUReau3OlMkQ8U6f6yU5EvKvX7+wCtY+UvXwCL2DwmIBMu8TTQ2vYtdJuqD/v0puA44iR8zcR8xYqjhp79cFFAwKZHDRPxpCw7vl06vRlITj6+B+Kn3yEW4xKVRFUQvGgZEoY6C6wKFwswXF9+f6zLkWftBmPibS9cBFE37y2c9FxUgoZFqaWdFy1bSP0VE7FiNhE4ccnivmk48iA6C3wbl7Qmf2gdG35dTUHApoTY74Bn0Nl7vVjc34616QxCdbBPWZrE7YPBoDxircSESUzSXRw0odxAGtFL4G2y5AavytvlgQavkOiSQwUTNHQwlYZQkzRDGo/NUgalbMI1+U73/xiQ+s7hnTA9EEs5vF6jTORbBotq5iE3gbufrqf9FQr80pzeT8urtpuX6xnuC9gyt7p1cVUBgZymWci/gN/z/Yorhpy8f1AhrIl45kLi9Fg0bdaGdGc8+xbuNc4LYyPkOUC/rCCBBV05mygMS9I06XydHCtXv2qMkw0M9XR6w+jljsWYGN5WVFaY+QSMUPl/tz368jzEJsCMLf3K8OxGMhvwnRD9Axf3NLz2JPWUHxtQ/Z59Z6ohf/fCPFE3XHQqTxMwzXzRn/E1Vj/oQddE6shZNenfuh2H5K9CurOaIXwvNrW8U+ZKskJfZn1p1l0Xgs8F/9UNeZUYX+f4llgO1FBTx9rwTImOm4+ezUZGYiNU8XVOhsxX/ko44nf9j7/niHDddmKcoxXO1ce41cj1wl1HOAT+xzBOCGxyqVsp49WhnHdKJvUgWduOylyhXr5r/0lF3ZAx4qH3Qu3N1NEszkeytXWG9YQKW6zNTm3H2ebdLOHxd8XQP61f/fKG38eC9B4IOh4GsXCylh6gRszJ3WIZjDG35JNcQkoP1sfG/xAwFRniir2Ipt3GbDiPjS34t7ca5A4XOk9bmxL5hJBpNF3us5+InTE9rH7qgH86T4ojOv4GLqIu/DlX3+JSDYDDFNLDGUQ0ZM0+QPi4ZO3zLsuPmOkK9CH03s+TLaKhzmlzvq/xU6W57wj7uF4s5o91nJwe9mYjKpLa6BrVzProPwFqtJoNi7y2wgI/j9353o1+8FLEY/veLPTutass9w4mSPjMDX0kHwfJp3ew6+VYOfimqvzYs/4X4NMXUJR4u6qhEJOarofjVSmAWiD5oGmjxS2RbCujzBeCgsO0y/ixQfmGbY5RR5a+/JLYdWq1WQV3jZqcbxo+6AwmXmxVyNdKZeNq7vU6xc53fke44EmeHgk5LjnTiEKr5CyKIrY2ReJSxnjFxfcIUbO6W+E0U6fdlJB5GmcPDcggpLbUqX2wDPn9PQDGXxt6HTikcSBMkvLb+ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1711d04f-1228-49b9-6835-08db0816affc X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:49:33.9164 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c9J1tJaY4pyotV0qCSTqLu5myKDaf6kn5nhV5WxMGeFphKFkr68yrsw6s6yLnhpSy6sjXSjUlVCrpf09HaGxPA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8573 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org book3s_64_vio currently accounts for pinned pages with account_locked_vm() which charges the pages to mm->locked_vm. To make this consistent with other drivers switch to using vm_account_pinned(). Signed-off-by: Alistair Popple Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy Cc: Paolo Bonzini Cc: Alexey Kardashevskiy Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org --- arch/powerpc/kvm/book3s_64_vio.c | 10 +++++----- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 3 +++ 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c index 95e738e..ecd1deb 100644 --- a/arch/powerpc/kvm/book3s_64_vio.c +++ b/arch/powerpc/kvm/book3s_64_vio.c @@ -273,8 +273,8 @@ static int kvm_spapr_tce_release(struct inode *inode, struct file *filp) } } - account_locked_vm(kvm->mm, - kvmppc_stt_pages(kvmppc_tce_pages(stt->size)), false); + vm_unaccount_pinned(&kvm->vm_account, + kvmppc_stt_pages(kvmppc_tce_pages(stt->size))); kvm_put_kvm(stt->kvm); @@ -301,8 +301,8 @@ long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm, (args->offset + args->size > (ULLONG_MAX >> args->page_shift))) return -EINVAL; - npages = kvmppc_tce_pages(args->size); - ret = account_locked_vm(mm, kvmppc_stt_pages(npages), true); + npages = kvmppc_tce_pages(size); + ret = vm_account_pinned(&kvm->vm_account, kvmppc_stt_pages(npages)); if (ret) return ret; @@ -347,7 +347,7 @@ long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm, kfree(stt); fail_acct: - account_locked_vm(mm, kvmppc_stt_pages(npages), false); + vm_unaccount_pinned(&kvm->vm_account, kvmppc_stt_pages(npages)); return ret; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4f26b24..bd7a7be 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -717,6 +718,7 @@ struct kvm { */ struct mutex slots_arch_lock; struct mm_struct *mm; /* userspace tied to this vm */ + struct vm_account vm_account; unsigned long nr_memslot_pages; /* The two memslot sets - active and inactive (per address space) */ struct kvm_memslots __memslots[KVM_ADDRESS_SPACE_NUM][2]; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 9c60384..770d037 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1142,6 +1142,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname) KVM_MMU_LOCK_INIT(kvm); mmgrab(current->mm); kvm->mm = current->mm; + vm_account_init_current(&kvm->vm_account); kvm_eventfd_init(kvm); mutex_init(&kvm->lock); mutex_init(&kvm->irq_lock); @@ -1258,6 +1259,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname) out_err_no_srcu: kvm_arch_free_vm(kvm); mmdrop(current->mm); + vm_account_release(&kvm->vm_account); module_put(kvm_chardev_ops.owner); return ERR_PTR(r); } @@ -1327,6 +1329,7 @@ static void kvm_destroy_vm(struct kvm *kvm) preempt_notifier_dec(); hardware_disable_all(); mmdrop(mm); + vm_account_release(&kvm->vm_account); module_put(kvm_chardev_ops.owner); }