From patchwork Mon Feb 15 12:21:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 12088059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BF6AC433DB for ; Mon, 15 Feb 2021 12:22:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 82BD160249 for ; Mon, 15 Feb 2021 12:22:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 82BD160249 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E0C9A8D00F4; Mon, 15 Feb 2021 07:21:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DBB728D00E6; Mon, 15 Feb 2021 07:21:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C35FA8D00F4; Mon, 15 Feb 2021 07:21:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id A8E6D8D00E6 for ; Mon, 15 Feb 2021 07:21:59 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 74FAD824805A for ; Mon, 15 Feb 2021 12:21:59 +0000 (UTC) X-FDA: 77820413958.05.money38_0e0df292763b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 5C4981800E39E for ; Mon, 15 Feb 2021 12:21:59 +0000 (UTC) X-HE-Tag: money38_0e0df292763b X-Filterd-Recvd-Size: 4658 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Mon, 15 Feb 2021 12:21:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613391718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iREUH3qcwe+8CMwRTYNN3sl03Q+aaxvnMDnwGmHEACo=; b=apxhdhsHqwgNupCVtK2Hrx9z9ZAZmc9/X4MmmOuw9ey8kjMpZpUnc52GHMaIt8rMSwOEh+ 2t/mWKJHpKZsjzmiNm4fk3qzy3pgHTiOrhQnbS4M0aPJMl8u3Mp9OC6wBnTu7WO2YoNK6W oMroZP2dVFTfayj2SQPukpujfDvw/do= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-398-0zyx5x9gNme3xLrVJohpQA-1; Mon, 15 Feb 2021 07:21:56 -0500 X-MC-Unique: 0zyx5x9gNme3xLrVJohpQA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EBDE4192CC43; Mon, 15 Feb 2021 12:21:54 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-89.ams2.redhat.com [10.36.114.89]) by smtp.corp.redhat.com (Postfix) with ESMTP id D81C72BFEC; Mon, 15 Feb 2021 12:21:52 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Marek Kedzierski , Hui Zhu , Pankaj Gupta , Wei Yang Subject: [PATCH v1 1/2] virtio-mem: don't read big block size in SBM Date: Mon, 15 Feb 2021 13:21:42 +0100 Message-Id: <20210215122143.27608-2-david@redhat.com> In-Reply-To: <20210215122143.27608-1-david@redhat.com> References: <20210215122143.27608-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We are reading the a BBM (Big Block Mode) value while in SBM (Sub Block Mode) while initializing. Fortunately, vm->bbm.bb_size maps to some counter in the vm->sbm.mb_count array, which is 0 at that point in time. No harm done; still, this was unintended and is not future-proof. Fixes: 4ba50cd3355d ("virtio-mem: Big Block Mode (BBM) memory hotplug") Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Marek Kedzierski Cc: Hui Zhu Cc: Pankaj Gupta Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 9fc9ec4a25f5..6d4e01c4e2fa 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2409,6 +2409,10 @@ static int virtio_mem_init(struct virtio_mem *vm) dev_warn(&vm->vdev->dev, "Some memory is not addressable. This can make some memory unusable.\n"); + /* Prepare the offline threshold - make sure we can add two blocks. */ + vm->offline_threshold = max_t(uint64_t, 2 * memory_block_size_bytes(), + VIRTIO_MEM_DEFAULT_OFFLINE_THRESHOLD); + /* * We want subblocks to span at least MAX_ORDER_NR_PAGES and * pageblock_nr_pages pages. This: @@ -2453,14 +2457,11 @@ static int virtio_mem_init(struct virtio_mem *vm) addr = vm->addr + vm->bbm.bb_size - 1; vm->bbm.first_bb_id = virtio_mem_phys_to_bb_id(vm, addr); vm->bbm.next_bb_id = vm->bbm.first_bb_id; - } - /* Prepare the offline threshold - make sure we can add two blocks. */ - vm->offline_threshold = max_t(uint64_t, 2 * memory_block_size_bytes(), - VIRTIO_MEM_DEFAULT_OFFLINE_THRESHOLD); - /* In BBM, we also want at least two big blocks. */ - vm->offline_threshold = max_t(uint64_t, 2 * vm->bbm.bb_size, - vm->offline_threshold); + /* Make sure we can add two big blocks. */ + vm->offline_threshold = max_t(uint64_t, 2 * vm->bbm.bb_size, + vm->offline_threshold); + } dev_info(&vm->vdev->dev, "start address: 0x%llx", vm->addr); dev_info(&vm->vdev->dev, "region size: 0x%llx", vm->region_size); From patchwork Mon Feb 15 12:24:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 12088089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C73DC433E0 for ; Mon, 15 Feb 2021 12:24:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B525764D9D for ; Mon, 15 Feb 2021 12:24:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B525764D9D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 468FA8D00F6; Mon, 15 Feb 2021 07:24:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4182B8D00F5; Mon, 15 Feb 2021 07:24:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 306038D00F6; Mon, 15 Feb 2021 07:24:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0027.hostedemail.com [216.40.44.27]) by kanga.kvack.org (Postfix) with ESMTP id 1756E8D00F5 for ; Mon, 15 Feb 2021 07:24:36 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CEA1218152772 for ; Mon, 15 Feb 2021 12:24:35 +0000 (UTC) X-FDA: 77820420510.09.honey74_2d02aaa2763b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id B67231802DA66 for ; Mon, 15 Feb 2021 12:24:35 +0000 (UTC) X-HE-Tag: honey74_2d02aaa2763b X-Filterd-Recvd-Size: 6805 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Mon, 15 Feb 2021 12:24:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613391874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uEB/obzSHOtmTVEsdMgjSRNQj0mbOgdNvz1v5u4sJFQ=; b=abOOfXn3xuXJd6rCsrwwhDn4XnKn9G2hNip2KRSlb4JyUNkBr2GidAN5HkvTh4Ds7xeh1B xG/WBWzHLqkmyBu3wt3zRxtnzurLS8dDHOKXIYv/8ESIYDXVpHyzqM2jBou+BV+8XTM4XJ ZhWoCHDMKdu37Q2wp/H+9+fa11j1oVw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-435-4uUu5GQgMdqoALxrJqkUWw-1; Mon, 15 Feb 2021 07:24:32 -0500 X-MC-Unique: 4uUu5GQgMdqoALxrJqkUWw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F82B801962; Mon, 15 Feb 2021 12:24:31 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-89.ams2.redhat.com [10.36.114.89]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0E9D60C0F; Mon, 15 Feb 2021 12:24:22 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Marek Kedzierski , Hui Zhu , Sebastien Boeuf , Pankaj Gupta , Wei Yang Subject: [PATCH v1 2/2] virtio-mem: support VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE Date: Mon, 15 Feb 2021 13:24:21 +0100 Message-Id: <20210215122421.27964-1-david@redhat.com> In-Reply-To: <20210215122143.27608-1-david@redhat.com> References: <20210215122143.27608-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The spec currently states that while unplugged memory should not be read, the device has to allow for reading unplugged memory inside the usable region. The primary motivation for this default handling was that in some corner cases in Linux, unplugged memory inside added Linux memory blocks (and exposed via /proc/iomem as "System RAM (...)") might still be read. So to support SBM (Sub Block Mode) in Linux cleanly, the device has to support reading unplugged memory. One example is kdump(): when makedumpfile isn't used or when an older version is used, PG_offline is ignored and unplugged memory might still be read. Another corner-case example is /dev/mem: even with STRICT_DEVMEM, some unplugged memory might be read by tools automatically. For example, QEMU won't support reading unplugged memory with file-backed memory backends initially: there is no shared zero page, thus, special handling will be required in the future to allow for reading unplugged memory when using a file backing (tmpfs/shmem, hugetlbfs, ...). The device will indicate VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE and fail device initialization if the driver does not indicate support. The result is that Linux won't be able to make use of any memory without this change. With this change, Linux will at least be able to (un)plug in Linux memory block granularity. Print an info so this handling can be identified. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Marek Kedzierski Cc: Hui Zhu Cc: Sebastien Boeuf Cc: Pankaj Gupta Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 ++++++++++++ include/uapi/linux/virtio_mem.h | 10 +++++++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 6d4e01c4e2fa..58d8df528728 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2425,6 +2425,17 @@ static int virtio_mem_init(struct virtio_mem *vm) pageblock_nr_pages) * PAGE_SIZE; sb_size = max_t(uint64_t, vm->device_block_size, sb_size); + /* + * Unplugged memory might be read in corner cases, for example, via + * kdump. Fallback to adding/removing individual Linux memory blocks. + */ + if (sb_size < memory_block_size_bytes() && !force_bbm && + virtio_has_feature(vm->vdev, VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE)) { + sb_size = memory_block_size_bytes(); + dev_info(&vm->vdev->dev, + "The device does not support reading unplugged memory: using big block mode\n"); + } + if (sb_size < memory_block_size_bytes() && !force_bbm) { /* SBM: At least two subblocks per Linux memory block. */ vm->in_sbm = true; @@ -2711,6 +2722,7 @@ static unsigned int virtio_mem_features[] = { #if defined(CONFIG_NUMA) && defined(CONFIG_ACPI_NUMA) VIRTIO_MEM_F_ACPI_PXM, #endif + VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE, }; static const struct virtio_device_id virtio_mem_id_table[] = { diff --git a/include/uapi/linux/virtio_mem.h b/include/uapi/linux/virtio_mem.h index 70e01c687d5e..6ac77cfb8aca 100644 --- a/include/uapi/linux/virtio_mem.h +++ b/include/uapi/linux/virtio_mem.h @@ -68,9 +68,11 @@ * explicitly triggered (VIRTIO_MEM_REQ_UNPLUG). * * There are no guarantees what will happen if unplugged memory is - * read/written. Such memory should, in general, not be touched. E.g., - * even writing might succeed, but the values will simply be discarded at - * random points in time. + * read/written. Often, unplugged memory inside the usable region can + * be read, to simplify creation of memory dumps; however, some devices + * don't even support that. Unplugged memory should, in general, not be + * touched. E.g., even writing might succeed, but the values will simply be + * discarded at random points in time. * * It can happen that the device cannot process a request, because it is * busy. The device driver has to retry later. @@ -87,6 +89,8 @@ /* node_id is an ACPI PXM and is valid */ #define VIRTIO_MEM_F_ACPI_PXM 0 +/* any unplugged memory must never be accessed */ +#define VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE 1 /* --- virtio-mem: guest -> host requests --- */