From patchwork Thu Jul 21 00:03:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12924631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64E3DC43334 for ; Thu, 21 Jul 2022 00:03:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ABF6E6B0072; Wed, 20 Jul 2022 20:03:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A6FC86B0073; Wed, 20 Jul 2022 20:03:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9389D6B0074; Wed, 20 Jul 2022 20:03:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 856EB6B0072 for ; Wed, 20 Jul 2022 20:03:23 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 57B311207B7 for ; Thu, 21 Jul 2022 00:03:23 +0000 (UTC) X-FDA: 79709157486.27.129A5FE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id AB0D818005F for ; Thu, 21 Jul 2022 00:03:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1658361802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0HOOHkAh3MZpUMeUIVAREqDy3wbpVTpA7LvIV0AcxA0=; b=BIoRVQsjx3OYlE/wF7hBGVvUbC8XnhTzPQ3LDtP/yqreRVnzoQiqVkoYRDArJBlGmDtq9M D29ESBGVzprwJe1Ep1uIC/e7LxtnLryY/jSVcZHV2WMrTafNmwI/6/1DDuCdMcIg37k3F5 SV1vUmLM+7zEfh7mRgD4ACma+Jiqjfg= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-434-H4QSDkHNMVGWTGDlxYoK9Q-1; Wed, 20 Jul 2022 20:03:21 -0400 X-MC-Unique: H4QSDkHNMVGWTGDlxYoK9Q-1 Received: by mail-qt1-f198.google.com with SMTP id w26-20020a05622a191a00b0031f0b4a561bso161995qtc.21 for ; Wed, 20 Jul 2022 17:03:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=0HOOHkAh3MZpUMeUIVAREqDy3wbpVTpA7LvIV0AcxA0=; b=VYRk5bUBFVdJloUlq/oKkA0QVdiGChdbuXn+IfOf1knjVwFCURoLPKpHhazqijtXnF p0mggAkVgAUlwzldy6btORQwLP0wOBYxdQE6gG5mNRSWasYcMAinpTazEJdgF5CTMUU6 LSwzHcGbDlxoYe1v1IFSJ9GUknf9nIM4ID/sKmjdcBf1OVj8wWGKvilIUs3oQOAFv9EV 0btT61/3j3XbiQ5A4jeGRGwmE1XKRY/1GX7XK5/X4cjl/9fA1GIoZLG5eHOCFjyCiPSW 1Nsw1+qnGF5FPC1UFLi7li23LNX0nDWEVaCWqk4Z6PELYekmnPf9x5dMiwYtrSDoe2f5 TqrQ== X-Gm-Message-State: AJIora+DcF44H8fyiVDJFldnE+6iUJFthuYnUs6ZrnYvBNzr2YAP/Iji 2Io+NnVoAH/Fj7a8dB7ND0VYfIZWZu06oWqGvfyv3YDWL9XvzvrPagd9i6QBzzM9KGiTiJ7I5Jo TrZIIRWKcxpY= X-Received: by 2002:a05:6214:194b:b0:474:69c:c21a with SMTP id q11-20020a056214194b00b00474069cc21amr3792529qvk.25.1658361800642; Wed, 20 Jul 2022 17:03:20 -0700 (PDT) X-Google-Smtp-Source: AGRyM1td9GAJoudrL5DhtFUEXZ0vWM5eEgh/b3Z8hNa2qWsTt+tRvoCnZ7VlRAHlZ06k2CHi33a+3g== X-Received: by 2002:a05:6214:194b:b0:474:69c:c21a with SMTP id q11-20020a056214194b00b00474069cc21amr3792502qvk.25.1658361800353; Wed, 20 Jul 2022 17:03:20 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-37-74-12-30-48.dsl.bell.ca. [74.12.30.48]) by smtp.gmail.com with ESMTPSA id g4-20020ac87f44000000b0031eb3af3ffesm418640qtk.52.2022.07.20.17.03.19 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 20 Jul 2022 17:03:19 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: David Hildenbrand , "Dr . David Alan Gilbert" , peterx@redhat.com, John Hubbard , Sean Christopherson , Linux MM Mailing List , Andrew Morton , Paolo Bonzini , Andrea Arcangeli Subject: [PATCH v2 0/3] kvm/mm: Allow GUP to respond to non fatal signals Date: Wed, 20 Jul 2022 20:03:15 -0400 Message-Id: <20220721000318.93522-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BIoRVQsj; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf06.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658361802; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=0HOOHkAh3MZpUMeUIVAREqDy3wbpVTpA7LvIV0AcxA0=; b=16hZGf3KiRb5M7NohWM8mSUEzYgaCICEUSjrs2TKb/Vq2dv00vtu2vYqT5/uUQLsQjBxGS e7h3m4fhEH0W87Hrjec2FFgT2VySnv+vHqeJgLs1J88WMeZTPRz2gAvKjVeb83W4UKCLwj 4wAGbAs7TOe2hcGCCkLYBApaXFwB5Zg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658361802; a=rsa-sha256; cv=none; b=3upuqOysCu/QjlZleS7aNc1f+3doaxyzFoQHFzMQuR8XlkMZrKPYweJPJHzByeJE5eoYrA /CToebigBu71n9a5kOXYeCcaufi6GURNNw2sOHReluNexWcGh8NHj60xa12cmKH6Xtyx3J gcBgdU7q/Vx4JVu2fdkXu1Uj7TIPc3Q= X-Rspamd-Queue-Id: AB0D818005F Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BIoRVQsj; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf06.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: xphf4zjtaxzsxn5ro7rh55ehmx9kax6o X-HE-Tag: 1658361802-689639 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: v2: - Added r-b - Rewrite the comment in faultin_page() for FOLL_INTERRUPTIBLE [John] - Dropped the controversial patch to introduce a flag for __gfn_to_pfn_memslot(), instead used a boolean for now [Sean] - Rename s/is_sigpending_pfn/KVM_PFN_ERR_SIGPENDING/ [Sean] - Change comment in kvm_faultin_pfn() mentioning fatal signals [Sean] rfc: https://lore.kernel.org/kvm/20220617014147.7299-1-peterx@redhat.com v1: https://lore.kernel.org/kvm/20220622213656.81546-1-peterx@redhat.com One issue was reported that libvirt won't be able to stop the virtual machine using QMP command "stop" during a paused postcopy migration [1]. It won't work because "stop the VM" operation requires the hypervisor to kick all the vcpu threads out using SIG_IPI in QEMU (which is translated to a SIGUSR1). However since during a paused postcopy, the vcpu threads are hang death at handle_userfault() so there're simply not responding to the kicks. Further, the "stop" command will further hang the QMP channel. The mm has facility to process generic signal (FAULT_FLAG_INTERRUPTIBLE), however it's only used in the PF handlers only, not in GUP. Unluckily, KVM is a heavy GUP user on guest page faults. It means we won't be able to interrupt a long page fault for KVM fetching guest pages with what we have right now. I think it's reasonable for GUP to only listen to fatal signals, as most of the GUP users are not really ready to handle such case. But actually KVM is not such an user, and KVM actually has rich infrastructure to handle even generic signals, and properly deliver the signal to the userspace. Then the page fault can be retried in the next KVM_RUN. This patchset added FOLL_INTERRUPTIBLE to enable FAULT_FLAG_INTERRUPTIBLE, and let KVM be the first one to use it. KVM and mm/gup can always be able to respond to fatal signals, but not non-fatal ones until this patchset. One thing to mention is that this is not allowing all KVM paths to be able to respond to non fatal signals, but only on x86 slow page faults. In the future when more code is ready for handling signal interruptions, we can explore possibility to have more gup callers using FOLL_INTERRUPTIBLE. Tests ===== I created a postcopy environment, pause the migration by shutting down the network to emulate a network failure (so the handle_userfault() will stuck for a long time), then I tried three things: (1) Sending QMP command "stop" to QEMU monitor, (2) Hitting Ctrl-C from QEMU cmdline, (3) GDB attach to the dest QEMU process. Before this patchset, all three use case hang. After the patchset, all work just like when there's not network failure at all. Please have a look, thanks. [1] https://gitlab.com/qemu-project/qemu/-/issues/1052 Peter Xu (3): mm/gup: Add FOLL_INTERRUPTIBLE kvm: Add new pfn error KVM_PFN_ERR_SIGPENDING kvm/x86: Allow to respond to generic signals during slow page faults arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 16 +++++++++++-- include/linux/kvm_host.h | 15 ++++++++++-- include/linux/mm.h | 1 + mm/gup.c | 33 ++++++++++++++++++++++---- virt/kvm/kvm_main.c | 30 ++++++++++++++--------- virt/kvm/kvm_mm.h | 4 ++-- virt/kvm/pfncache.c | 2 +- 10 files changed, 82 insertions(+), 25 deletions(-)