From patchwork Sat Sep 16 00:30:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13387939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63FAACD37B2 for ; Sat, 16 Sep 2023 00:33:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238195AbjIPAct (ORCPT ); Fri, 15 Sep 2023 20:32:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238415AbjIPAcU (ORCPT ); Fri, 15 Sep 2023 20:32:20 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D4DD2726 for ; Fri, 15 Sep 2023 17:31:31 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-56fb25fdf06so2037909a12.1 for ; Fri, 15 Sep 2023 17:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694824291; x=1695429091; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=C0sn2AnVcRbsYEvqFgh4BzWbc2uKAyyxP92cnWXv5Gc=; b=jvDBnkFi8COrquzeFvzZe1NGIcw2+MdT3qMRI4S7MM7TtszA/o4dFGVmucHUardQ0N nGh/1bkIjVTOZz4YECMI/AWeidonXnZiVATf/POPsl2fjOLRExjl3/OV0GV3vw5y5BjP 6YYSZ48XUlyCXQuzXpGRve5bffxA0OSzOhkTIJj8ObEZO9M3vcX8ub2TXheKDnfYpGDJ WW9O9i8K/UqxEYpWt+N+D4xO1KLq5GpE5gSkZQbgtA2XIpHKtfv5EX2VrBgQz1Hb2oK6 aDcZuaC1yLFZfkK0plhW0iiZN+cRhrYegjy5+73NURC3ACgUCig5UVuHKxhPx6BM7Ra4 PyYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694824291; x=1695429091; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=C0sn2AnVcRbsYEvqFgh4BzWbc2uKAyyxP92cnWXv5Gc=; b=M9XF0Zsu/gOGaNmLFdu2mssD7p8IN6U6cWXkGbMVet9kxG6HNw9HDLArQS8s4jIZHo Gxr6zgGSTFgL3Us/6pIEngUyLwEw2tOU/NyvyYop3C/trfqQK7bFnfvr6sedOZwoeKSw Cbt8XDjst6OfZayOJGXJRyBuUbNl9Z/+c2Yq0m0ew1GjNMfjHblwYbEK5kFexNM0YRUT z+b7oQveaEL2AwnlF/iVee911HOd6NRWKz6vMy/tyEIm8ovZ3X3EYdS1HYl6uowxHUPI kCmdb6wz42l/Z2Aq+LNjTg5ff3c8FCWXfRUvu4AVlKdyigC62kM1V/0b9j1eeCk6U3iC KDdw== X-Gm-Message-State: AOJu0Yzx9QQIUMoG+x5CBhvsUG6KyieT9xllNQ9eRLyoVK2kZkv3v+Gy tNssIE+zBOvRjMhL2fNn9Ot4sdxSGGo= X-Google-Smtp-Source: AGHT+IHtRyj8S1iXRa1AyuzUMEtuqAzbJZbP2iFNCt3KVQZXM+xbuQMnV6uSJ406DSy/keoiXYz83X05kPo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:41c2:b0:1c1:fbec:bc39 with SMTP id u2-20020a17090341c200b001c1fbecbc39mr73146ple.4.1694824290982; Fri, 15 Sep 2023 17:31:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Sep 2023 17:30:57 -0700 In-Reply-To: <20230916003118.2540661-1-seanjc@google.com> Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> X-Mailer: git-send-email 2.42.0.459.ge4e396fd5e-goog Message-ID: <20230916003118.2540661-6-seanjc@google.com> Subject: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup From: Sean Christopherson To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Peter Zijlstra , Arnaldo Carvalho de Melo , Sean Christopherson , Paolo Bonzini , Tony Krowiak , Halil Pasic , Jason Herne , Harald Freudenberger , Alex Williamson , Andy Lutomirski Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Anish Ghulati , Venkatesh Srinivas , Andrew Thornton Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to VFIO instead of having VFIO do a symbol lookup back into KVM. Having both KVM and VFIO do symbol lookups increases the overall complexity and places an unnecessary dependency on KVM (from VFIO) without adding any value. Signed-off-by: Sean Christopherson Reviewed-by: Alex Williamson --- drivers/vfio/vfio.h | 2 ++ drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- include/linux/vfio.h | 4 ++- virt/kvm/vfio.c | 9 +++-- 4 files changed, 47 insertions(+), 42 deletions(-) diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index a1f741365075..eec51c7ee822 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -19,6 +19,8 @@ struct vfio_container; struct vfio_kvm_reference { struct kvm *kvm; + bool (*get_kvm)(struct kvm *kvm); + void (*put_kvm)(struct kvm *kvm); spinlock_t lock; }; diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index e77e8c6aae2f..1f58ab6dbcd2 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -16,7 +16,6 @@ #include #include #include -#include #include #include #include @@ -1306,38 +1305,22 @@ EXPORT_SYMBOL_GPL(vfio_file_enforced_coherent); void vfio_device_get_kvm_safe(struct vfio_device *device, struct vfio_kvm_reference *ref) { - void (*pfn)(struct kvm *kvm); - bool (*fn)(struct kvm *kvm); - bool ret; - lockdep_assert_held(&device->dev_set->lock); + /* + * Note! The "kvm" and "put_kvm" pointers *must* be transferred to the + * device so that the device can put its reference to KVM. KVM can + * invoke vfio_device_set_kvm() to detach from VFIO, i.e. nullify all + * pointers in @ref, even if a device holds a reference to KVM! That + * also means that detaching KVM from VFIO only prevents "new" devices + * from using KVM, it doesn't invalidate KVM references in existing + * devices. + */ spin_lock(&ref->lock); - - if (!ref->kvm) - goto out; - - pfn = symbol_get(kvm_put_kvm); - if (WARN_ON(!pfn)) - goto out; - - fn = symbol_get(kvm_get_kvm_safe); - if (WARN_ON(!fn)) { - symbol_put(kvm_put_kvm); - goto out; + if (ref->kvm && ref->get_kvm(ref->kvm)) { + device->kvm = ref->kvm; + device->put_kvm = ref->put_kvm; } - - ret = fn(ref->kvm); - symbol_put(kvm_get_kvm_safe); - if (!ret) { - symbol_put(kvm_put_kvm); - goto out; - } - - device->put_kvm = pfn; - device->kvm = ref->kvm; - -out: spin_unlock(&ref->lock); } @@ -1353,28 +1336,37 @@ void vfio_device_put_kvm(struct vfio_device *device) device->put_kvm(device->kvm); device->put_kvm = NULL; - symbol_put(kvm_put_kvm); - clear: device->kvm = NULL; } static void vfio_device_set_kvm(struct vfio_kvm_reference *ref, - struct kvm *kvm) + struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)) { + if (WARN_ON_ONCE(kvm && (!get_kvm || !put_kvm))) + return; + spin_lock(&ref->lock); ref->kvm = kvm; + ref->get_kvm = get_kvm; + ref->put_kvm = put_kvm; spin_unlock(&ref->lock); } -static void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) +static void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)) { #if IS_ENABLED(CONFIG_VFIO_GROUP) - vfio_device_set_kvm(&group->kvm_ref, kvm); + vfio_device_set_kvm(&group->kvm_ref, kvm, get_kvm, put_kvm); #endif } -static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm) +static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)) { struct vfio_device_file *df = file->private_data; @@ -1383,27 +1375,31 @@ static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm) * be propagated to vfio_device::kvm when the file is bound to * iommufd successfully in the vfio device cdev path. */ - vfio_device_set_kvm(&df->kvm_ref, kvm); + vfio_device_set_kvm(&df->kvm_ref, kvm, get_kvm, put_kvm); } /** * vfio_file_set_kvm - Link a kvm with VFIO drivers * @file: VFIO group file or VFIO device file * @kvm: KVM to link + * @get_kvm: Callback to get a reference to @kvm + * @put_kvm: Callback to put a reference to @kvm * * When a VFIO device is first opened the KVM will be available in * device->kvm if one was associated with the file. */ -void vfio_file_set_kvm(struct file *file, struct kvm *kvm) +void vfio_file_set_kvm(struct file *file, struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)) { struct vfio_group *group; group = vfio_group_from_file(file); if (group) - vfio_group_set_kvm(group, kvm); + vfio_group_set_kvm(group, kvm, get_kvm, put_kvm); if (vfio_device_from_file(file)) - vfio_device_file_set_kvm(file, kvm); + vfio_device_file_set_kvm(file, kvm, get_kvm, put_kvm); } EXPORT_SYMBOL_GPL(vfio_file_set_kvm); #endif diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e80955de266c..35e970e3d3fb 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -312,7 +312,9 @@ static inline bool vfio_file_has_dev(struct file *file, struct vfio_device *devi bool vfio_file_is_valid(struct file *file); bool vfio_file_enforced_coherent(struct file *file); #if IS_ENABLED(CONFIG_KVM) -void vfio_file_set_kvm(struct file *file, struct kvm *kvm); +void vfio_file_set_kvm(struct file *file, struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)); #endif #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index ca24ce120906..f14fcbb34bc6 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -37,13 +37,18 @@ struct kvm_vfio { static void kvm_vfio_file_set_kvm(struct file *file, struct kvm *kvm) { - void (*fn)(struct file *file, struct kvm *kvm); + void (*fn)(struct file *file, struct kvm *kvm, + bool (*get_kvm)(struct kvm *kvm), + void (*put_kvm)(struct kvm *kvm)); fn = symbol_get(vfio_file_set_kvm); if (!fn) return; - fn(file, kvm); + if (kvm) + fn(file, kvm, kvm_get_kvm_safe, kvm_put_kvm); + else + fn(file, kvm, NULL, NULL); symbol_put(vfio_file_set_kvm); }