From patchwork Wed Jul 17 19:33:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 11048181 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF0B26C5 for ; Wed, 17 Jul 2019 19:35:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B04F128784 for ; Wed, 17 Jul 2019 19:35:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A408428686; Wed, 17 Jul 2019 19:35:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 304FE28686 for ; Wed, 17 Jul 2019 19:35:28 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgf-0000Hd-01; Wed, 17 Jul 2019 19:34:00 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgd-0000HL-KS for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 19:33:59 +0000 X-Inumbo-ID: d0043070-a8c9-11e9-be02-ebb5fa9022dc Received: from mail-io1-f68.google.com (unknown [209.85.166.68]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d0043070-a8c9-11e9-be02-ebb5fa9022dc; Wed, 17 Jul 2019 19:33:54 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id k20so47594701ios.10 for ; Wed, 17 Jul 2019 12:33:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JBw7l33La8xVqfg8L17ATjz/BcMg+hI1A6l2yXVZeo0=; b=Y3zTD7jqvwRSnSm5ATNyLUuheKGYsi1AD5YmxIE/F4OQUc3Q4wcfTT6cmG3Dgr3o58 euNHSEUvjBEfKtQ+y7v02eFIH8YF9WE1H3KdjRMCSuQU0BWBOMkYt2H7v5HWv66s6xBn eNC1rJbvY82Bbx81fevk2vOu9TDayhssYJRIqDgYU+JNlHwpX+9nU6ubSx3BIANUMb5R J7xySNEulGo4+EBR++qrAnrV9YzAmcoQLhHMvEjN7pg0qBBs/8kMlC4XhNojR4H1HLx3 FBsOkXVwtOVk4dmpPG9/mFBQqueKq1qRxTbfEXiway20YIjwAMIJkVM3P7/AHO0pxcrx n/oA== X-Gm-Message-State: APjAAAVis7ZJD6SQiY0iK/X0JkX513j17YvQLIHY8YkrA6cg+lM9zx60 ULIt25LZZOfQOfbO73QYFOsUExKq X-Google-Smtp-Source: APXvYqxg5NYQFGxMFuUPXTARLsjdGTi74JswUYe2OqezGVQUMo2s4PVGcQioR4sbN17+sHslUc7/+g== X-Received: by 2002:a6b:b985:: with SMTP id j127mr38797426iof.186.1563392033796; Wed, 17 Jul 2019 12:33:53 -0700 (PDT) Received: from l1.lan (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id v13sm21743421ioq.13.2019.07.17.12.33.52 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 12:33:53 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 13:33:31 -0600 Message-Id: <20190717193335.11991-2-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717193335.11991-1-tamas@tklengyel.com> References: <20190717193335.11991-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 1/5] x86/mem_sharing: reorder when pages are unlocked and released X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , George Dunlap , Andrew Cooper , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Calling _put_page_type while also holding the page_lock for that page can cause a deadlock. The comment being dropped is incorrect since it's now out-of-date. Signed-off-by: Tamas K Lengyel Cc: Jan Beulich Cc: Andrew Cooper Cc: George Dunlap Cc: Wei Liu Cc: Roger Pau Monne --- v6: rebase on staging v5: BUG_ON early before releasing references --- xen/arch/x86/mm/mem_sharing.c | 40 +++++++++++------------------------ 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 58d9157fa8..6c033d1488 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -648,10 +648,6 @@ static int page_make_private(struct domain *d, struct page_info *page) return -EBUSY; } - /* We can only change the type if count is one */ - /* Because we are locking pages individually, we need to drop - * the lock here, while the page is typed. We cannot risk the - * race of page_unlock and then put_page_type. */ expected_type = (PGT_shared_page | PGT_validated | PGT_locked | 2); if ( page->u.inuse.type_info != expected_type ) { @@ -660,12 +656,11 @@ static int page_make_private(struct domain *d, struct page_info *page) return -EEXIST; } + mem_sharing_page_unlock(page); + /* Drop the final typecount */ put_page_and_type(page); - /* Now that we've dropped the type, we can unlock */ - mem_sharing_page_unlock(page); - /* Change the owner */ ASSERT(page_get_owner(page) == dom_cow); page_set_owner(page, d); @@ -900,6 +895,7 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, p2m_type_t smfn_type, cmfn_type; struct two_gfns tg; struct rmap_iterator ri; + unsigned long put_count = 0; get_two_gfns(sd, sgfn, &smfn_type, NULL, &smfn, cd, cgfn, &cmfn_type, NULL, &cmfn, 0, &tg); @@ -964,15 +960,6 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, goto err_out; } - /* Acquire an extra reference, for the freeing below to be safe. */ - if ( !get_page(cpage, dom_cow) ) - { - ret = -EOVERFLOW; - mem_sharing_page_unlock(secondpg); - mem_sharing_page_unlock(firstpg); - goto err_out; - } - /* Merge the lists together */ rmap_seed_iterator(cpage, &ri); while ( (gfn = rmap_iterate(cpage, &ri)) != NULL) @@ -984,13 +971,14 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, * Don't change the type of rmap for the client page. */ rmap_del(gfn, cpage, 0); rmap_add(gfn, spage); - put_page_and_type(cpage); + put_count++; d = get_domain_by_id(gfn->domain); BUG_ON(!d); BUG_ON(set_shared_p2m_entry(d, gfn->gfn, smfn)); put_domain(d); } ASSERT(list_empty(&cpage->sharing->gfns)); + BUG_ON(!put_count); /* Clear the rest of the shared state */ page_sharing_dispose(cpage); @@ -1001,7 +989,9 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, /* Free the client page */ put_page_alloc_ref(cpage); - put_page(cpage); + + while ( put_count-- ) + put_page_and_type(cpage); /* We managed to free a domain page. */ atomic_dec(&nr_shared_mfns); @@ -1165,19 +1155,13 @@ int __mem_sharing_unshare_page(struct domain *d, { if ( !last_gfn ) mem_sharing_gfn_destroy(page, d, gfn_info); - put_page_and_type(page); + mem_sharing_page_unlock(page); + if ( last_gfn ) - { - if ( !get_page(page, dom_cow) ) - { - put_gfn(d, gfn); - domain_crash(d); - return -EOVERFLOW; - } put_page_alloc_ref(page); - put_page(page); - } + + put_page_and_type(page); put_gfn(d, gfn); return 0; From patchwork Wed Jul 17 19:33:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 11048185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E7215746 for ; Wed, 17 Jul 2019 19:35:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D74EF204FB for ; Wed, 17 Jul 2019 19:35:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA06E285D6; Wed, 17 Jul 2019 19:35:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 44BDF204FB for ; Wed, 17 Jul 2019 19:35:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgf-0000Hl-9N; Wed, 17 Jul 2019 19:34:01 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgd-0000HM-L8 for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 19:33:59 +0000 X-Inumbo-ID: d0f0b3d2-a8c9-11e9-8afc-8b45d339bfd7 Received: from mail-io1-f65.google.com (unknown [209.85.166.65]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d0f0b3d2-a8c9-11e9-8afc-8b45d339bfd7; Wed, 17 Jul 2019 19:33:56 +0000 (UTC) Received: by mail-io1-f65.google.com with SMTP id j5so43432508ioj.8 for ; Wed, 17 Jul 2019 12:33:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cEs8uh0BztRP1QpWyft084vesYDg1M1kqUeeQXfN2QA=; b=ieA41QAFnlXUtVqAb+JM6NPJ4gWcqmHfiLDuuZ3iucw+LaWk7zKtSfmUw4iAoi3pBC jhGsaqYnfM+Kgn21fINAQd38j+mumJoF7CyWIehRF1F4tEgdVcghF2yYB8lvnxGkfVaT DnB7NoqS6doyPLU/QFpYrNbB8/ntQUUU0XwvcegkrYb2G37Cjg3LN6qPlLOxssv8Xf7l 5+qDz9MAiVI9yZfaiwAwZHAI9MOUaUeTVaZN9e0rpjcN+NGphbdibLHs5udUlnlfCoie kNG48U8xca1lS71GRtLYoF7K53YfnT6h9H+eeCNWdr6EkX7ROvdgXvchzPie5OYRY7aj KAEg== X-Gm-Message-State: APjAAAU6TJuvUIvZ08AtbumGIsLhe4CuGXk5KXYT2M7GoPGrtm4+CU4f qoci8nyeslbzRr6Dbt4aL9k7ImU/ X-Google-Smtp-Source: APXvYqw1wWU9j0YZB4Yq3mDafb2VrcuUm21sJeF1GN2DXWDhPfQoz1P2dGTt837ss9tBcNz4Ph5HSQ== X-Received: by 2002:a5e:de4d:: with SMTP id e13mr16737420ioq.272.1563392035325; Wed, 17 Jul 2019 12:33:55 -0700 (PDT) Received: from l1.lan (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id v13sm21743421ioq.13.2019.07.17.12.33.53 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 12:33:54 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 13:33:32 -0600 Message-Id: <20190717193335.11991-3-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717193335.11991-1-tamas@tklengyel.com> References: <20190717193335.11991-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 2/5] x86/mem_sharing: copy a page_lock version to be internal to memshr X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , George Dunlap , Andrew Cooper , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Patch cf4b30dca0a "Add debug code to detect illegal page_lock and put_page_type ordering" added extra sanity checking to page_lock/page_unlock for debug builds with the assumption that no hypervisor path ever locks two pages at once. This assumption doesn't hold during memory sharing so we copy a version of page_lock/unlock to be used exclusively in the memory sharing subsystem without the sanity checks. Signed-off-by: Tamas K Lengyel Acked-by: Jan Beulich Cc: George Dunlap Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne --- v6: adjust comment according to Jan's suggestion --- xen/arch/x86/mm/mem_sharing.c | 54 ++++++++++++++++++++++++++++++++--- xen/include/asm-x86/mm.h | 15 ++-------- 2 files changed, 53 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 6c033d1488..a5fe89e339 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -112,13 +112,59 @@ static inline void page_sharing_dispose(struct page_info *page) #endif /* MEM_SHARING_AUDIT */ -static inline int mem_sharing_page_lock(struct page_info *pg) +/* + * Private implementations of page_lock/unlock to bypass PV-only + * sanity checks not applicable to mem-sharing. + * + * _page_lock is used in memory sharing to protect addition (share) and removal + * (unshare) of (gfn,domain) tupples to a list of gfn's that the shared page is + * currently backing. + * Nesting may happen when sharing (and locking) two pages. + * Deadlock is avoided by locking pages in increasing order. + * All memory sharing code paths take the p2m lock of the affected gfn before + * taking the lock for the underlying page. We enforce ordering between page_lock + * and p2m_lock using an mm-locks.h construct. + * + * TODO: Investigate if PGT_validated is necessary. + */ +static inline bool _page_lock(struct page_info *page) { - int rc; + unsigned long x, nx; + + do { + while ( (x = page->u.inuse.type_info) & PGT_locked ) + cpu_relax(); + nx = x + (1 | PGT_locked); + if ( !(x & PGT_validated) || + !(x & PGT_count_mask) || + !(nx & PGT_count_mask) ) + return false; + } while ( cmpxchg(&page->u.inuse.type_info, x, nx) != x ); + + return true; +} + +static inline void _page_unlock(struct page_info *page) +{ + unsigned long x, nx, y = page->u.inuse.type_info; + + do { + x = y; + ASSERT((x & PGT_count_mask) && (x & PGT_locked)); + + nx = x - (1 | PGT_locked); + /* We must not drop the last reference here. */ + ASSERT(nx & PGT_count_mask); + } while ( (y = cmpxchg(&page->u.inuse.type_info, x, nx)) != x ); +} + +static inline bool mem_sharing_page_lock(struct page_info *pg) +{ + bool rc; pg_lock_data_t *pld = &(this_cpu(__pld)); page_sharing_mm_pre_lock(); - rc = page_lock(pg); + rc = _page_lock(pg); if ( rc ) { preempt_disable(); @@ -135,7 +181,7 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) page_sharing_mm_unlock(pld->mm_unlock_level, &pld->recurse_count); preempt_enable(); - page_unlock(pg); + _page_unlock(pg); } static inline shr_handle_t get_next_handle(void) diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 6c14635270..087ad97351 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -356,24 +356,15 @@ struct platform_bad_page { const struct platform_bad_page *get_platform_badpages(unsigned int *array_size); /* Per page locks: - * page_lock() is used for two purposes: pte serialization, and memory sharing. + * page_lock() is used for pte serialization. * * All users of page lock for pte serialization live in mm.c, use it * to lock a page table page during pte updates, do not take other locks within * the critical section delimited by page_lock/unlock, and perform no * nesting. * - * All users of page lock for memory sharing live in mm/mem_sharing.c. Page_lock - * is used in memory sharing to protect addition (share) and removal (unshare) - * of (gfn,domain) tupples to a list of gfn's that the shared page is currently - * backing. Nesting may happen when sharing (and locking) two pages -- deadlock - * is avoided by locking pages in increasing order. - * All memory sharing code paths take the p2m lock of the affected gfn before - * taking the lock for the underlying page. We enforce ordering between page_lock - * and p2m_lock using an mm-locks.h construct. - * - * These two users (pte serialization and memory sharing) do not collide, since - * sharing is only supported for hvm guests, which do not perform pv pte updates. + * The use of PGT_locked in mem_sharing does not collide, since mem_sharing is + * only supported for hvm guests, which do not have PV PTEs updated. */ int page_lock(struct page_info *page); void page_unlock(struct page_info *page); From patchwork Wed Jul 17 19:33:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 11048179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5439A6C5 for ; Wed, 17 Jul 2019 19:35:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41E2128686 for ; Wed, 17 Jul 2019 19:35:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 316F428789; Wed, 17 Jul 2019 19:35:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B409728686 for ; Wed, 17 Jul 2019 19:35:26 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgf-0000I6-NO; Wed, 17 Jul 2019 19:34:01 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpge-0000HY-S5 for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 19:34:00 +0000 X-Inumbo-ID: d1b9fad0-a8c9-11e9-b1fa-239cefdce2bf Received: from mail-io1-f46.google.com (unknown [209.85.166.46]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d1b9fad0-a8c9-11e9-b1fa-239cefdce2bf; Wed, 17 Jul 2019 19:33:57 +0000 (UTC) Received: by mail-io1-f46.google.com with SMTP id k20so47594915ios.10 for ; Wed, 17 Jul 2019 12:33:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QETwMaZ1/fKFNeTqiJ8Ul71lh14Z/VE7ZvRMcOw3JcI=; b=S68CrYm4zRlMrgSGQSukOdusbX3mHSqXCiufgoZfUoeTchRwkJ3lJlCVltIC2lzoFf y4Ic8TAtvfwvV5pIyBYJR7YlwMg7PvRL6hgcx2GjVDooyCUwF0pLqvdWDOntPkJx0B8z 4gKXloa+6VMYurG/XCGAbldfd78G2iiwiwk3cqSar1q5mRAwAB9Wt+BtTSnzDkYNuvMA zgwq+4Z6zhVwacFkVCuoHNE+1LgypqlIURNJS0tzWXHbm6kDpxROquwr+B15w/dP6vhb mYaLoUXsDNuGdczn4fQqQHehpvHJolnWOl6XwuUQZa3B25r5IkuL7k43x4lLYcz6eHcj iRjw== X-Gm-Message-State: APjAAAUf7YvhYMz1UodXHQFhMHLV83HQtUrNI3kIjsEUSrhXsLdyhXEu O940s65i6OZ8T3v92Iv8AWcxV93E X-Google-Smtp-Source: APXvYqzjVjorbwpqQfSra1qhunjoOrVEu+JGhAndJEAN19qgBuyUhs219TP6oJ81kLqgltJH7XQtJA== X-Received: by 2002:a5d:885a:: with SMTP id t26mr33729373ios.218.1563392036708; Wed, 17 Jul 2019 12:33:56 -0700 (PDT) Received: from l1.lan (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id v13sm21743421ioq.13.2019.07.17.12.33.55 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 12:33:56 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 13:33:33 -0600 Message-Id: <20190717193335.11991-4-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717193335.11991-1-tamas@tklengyel.com> References: <20190717193335.11991-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 3/5] x86/mem_sharing: enable mem_share audit mode only in debug builds X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Tamas K Lengyel , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Improves performance for release builds. Signed-off-by: Tamas K Lengyel Cc: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne --- v6: patch ping --- xen/include/asm-x86/mem_sharing.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/xen/include/asm-x86/mem_sharing.h b/xen/include/asm-x86/mem_sharing.h index 9f9f7e93e3..afd0c17292 100644 --- a/xen/include/asm-x86/mem_sharing.h +++ b/xen/include/asm-x86/mem_sharing.h @@ -25,7 +25,11 @@ #include /* Auditing of memory sharing code? */ +#ifndef NDEBUG #define MEM_SHARING_AUDIT 1 +#else +#define MEM_SHARING_AUDIT 0 +#endif typedef uint64_t shr_handle_t; From patchwork Wed Jul 17 19:33:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 11048189 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B5846C5 for ; Wed, 17 Jul 2019 19:36:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A092204FB for ; Wed, 17 Jul 2019 19:36:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D7D0285D6; Wed, 17 Jul 2019 19:36:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 19F95204FB for ; Wed, 17 Jul 2019 19:36:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgh-0000IV-13; Wed, 17 Jul 2019 19:34:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgg-0000IL-3M for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 19:34:02 +0000 X-Inumbo-ID: d2f84668-a8c9-11e9-b34d-ef1fc16ccd93 Received: from mail-io1-f66.google.com (unknown [209.85.166.66]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d2f84668-a8c9-11e9-b34d-ef1fc16ccd93; Wed, 17 Jul 2019 19:33:59 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id o9so47686592iom.3 for ; Wed, 17 Jul 2019 12:33:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bhWcJS8ddgdUot0n7cOWsB1nWNTTcu28PhAqbt9unzE=; b=rce9aksren0NUfpFpaHvSv5PuZS+YsziuobuSK+M36Qj5BJrEbILpuadtSAql1CuMQ M+qmg8y0mo8iZnKbRhM63PicD1dseSt/MLElvotlNyr4dwYuaeKZMzTJtZbOedAbGlHS yOK57++MAZgCKwhPIyKsPyWfUq8SBw6tpifHnx1Kq5KjNLwB8L5gMXltz6zVMNtH6lY0 vktqKeQD5TphJrGz61MCR58Doq/csFneYlOiDZoVpX3/RATuwyhIBmgD5ktJemvRmGt9 hToHHNxUyub5nFtpe1KDd5Bk5vE/l53swImeY7JTXkkId31kXjcrshJVJi0jFC0ciusv XVWA== X-Gm-Message-State: APjAAAUaoEJV2naFtcbiF3rYYgdDBxmDvqxqud7weV5kYdUbahHvxad6 1eCsk+618EyM6Kx1UONHqNFAHVn8hTQ= X-Google-Smtp-Source: APXvYqzrM8D7/bpoymNk7/JIn6lgsbg1TXAklNJxkyfJnaG4sM7EImy+PUeVAPjyK7Hwmcn2ZiWtGQ== X-Received: by 2002:a5d:8794:: with SMTP id f20mr37851018ion.128.1563392038669; Wed, 17 Jul 2019 12:33:58 -0700 (PDT) Received: from l1.lan (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id v13sm21743421ioq.13.2019.07.17.12.33.56 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 12:33:57 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 13:33:34 -0600 Message-Id: <20190717193335.11991-5-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717193335.11991-1-tamas@tklengyel.com> References: <20190717193335.11991-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 4/5] x86/mem_sharing: compile mem_sharing subsystem only when kconfig is enabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , Razvan Cojocaru , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Stefano Stabellini , Jan Beulich , Daniel De Graaf , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Disable it by default as it is only an experimental subsystem. Signed-off-by: Tamas K Lengyel Acked-by: Daniel De Graaf Acked-by: Razvan Cojocaru Acked-by: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne Cc: George Dunlap Cc: Ian Jackson Cc: Julien Grall Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini Cc: Tim Deegan Cc: George Dunlap ---- v6: rebase on staging --- xen/arch/x86/Kconfig | 6 +++++- xen/arch/x86/domain.c | 2 ++ xen/arch/x86/domctl.c | 2 ++ xen/arch/x86/mm/Makefile | 2 +- xen/arch/x86/x86_64/compat/mm.c | 2 ++ xen/arch/x86/x86_64/mm.c | 2 ++ xen/common/Kconfig | 3 --- xen/common/domain.c | 6 +++--- xen/common/grant_table.c | 2 +- xen/common/memory.c | 2 +- xen/common/vm_event.c | 6 +++--- xen/include/asm-x86/mem_sharing.h | 28 ++++++++++++++++++++++++++++ xen/include/asm-x86/mm.h | 3 +++ xen/include/xen/mm.h | 2 +- xen/include/xen/sched.h | 2 +- xen/include/xsm/dummy.h | 2 +- xen/include/xsm/xsm.h | 4 ++-- xen/xsm/dummy.c | 2 +- xen/xsm/flask/hooks.c | 4 ++-- 19 files changed, 61 insertions(+), 21 deletions(-) diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index f502d765ba..288dc6c042 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -18,7 +18,6 @@ config X86 select HAS_KEXEC select MEM_ACCESS_ALWAYS_ON select HAS_MEM_PAGING - select HAS_MEM_SHARING select HAS_NS16550 select HAS_PASSTHROUGH select HAS_PCI @@ -199,6 +198,11 @@ config PV_SHIM_EXCLUSIVE firmware, and will not function correctly in other scenarios. If unsure, say N. + +config MEM_SHARING + bool "Xen memory sharing support" if EXPERT = "y" + depends on HVM + endmenu source "common/Kconfig" diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index e791d86892..ea55160887 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2095,6 +2095,7 @@ int domain_relinquish_resources(struct domain *d) d->arch.auto_unmask = 0; } +#ifdef CONFIG_MEM_SHARING PROGRESS(shared): if ( is_hvm_domain(d) ) @@ -2105,6 +2106,7 @@ int domain_relinquish_resources(struct domain *d) if ( ret ) return ret; } +#endif spin_lock(&d->page_alloc_lock); page_list_splice(&d->arch.relmem_list, &d->page_list); diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index c827790202..2d45e5b8a8 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1236,9 +1236,11 @@ long arch_do_domctl( break; } +#ifdef CONFIG_MEM_SHARING case XEN_DOMCTL_mem_sharing_op: ret = mem_sharing_domctl(d, &domctl->u.mem_sharing_op); break; +#endif #if P2M_AUDIT && defined(CONFIG_HVM) case XEN_DOMCTL_audit_p2m: diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile index 5a17646f98..5010a29d6c 100644 --- a/xen/arch/x86/mm/Makefile +++ b/xen/arch/x86/mm/Makefile @@ -6,7 +6,7 @@ obj-$(CONFIG_HVM) += guest_walk_2.o guest_walk_3.o guest_walk_4.o obj-$(CONFIG_SHADOW_PAGING) += guest_walk_2.o guest_walk_3.o guest_walk_4.o obj-$(CONFIG_MEM_ACCESS) += mem_access.o obj-y += mem_paging.o -obj-y += mem_sharing.o +obj-$(CONFIG_MEM_SHARING) += mem_sharing.o obj-y += p2m.o p2m-pt.o obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o obj-y += paging.o diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c index 32410ed273..d4c6be3032 100644 --- a/xen/arch/x86/x86_64/compat/mm.c +++ b/xen/arch/x86/x86_64/compat/mm.c @@ -152,8 +152,10 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case XENMEM_paging_op: return mem_paging_memop(guest_handle_cast(arg, xen_mem_paging_op_t)); +#ifdef CONFIG_MEM_SHARING case XENMEM_sharing_op: return mem_sharing_memop(guest_handle_cast(arg, xen_mem_sharing_op_t)); +#endif default: rc = -ENOSYS; diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 899b883b2d..1919cae18b 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -993,8 +993,10 @@ long subarch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case XENMEM_paging_op: return mem_paging_memop(guest_handle_cast(arg, xen_mem_paging_op_t)); +#ifdef CONFIG_MEM_SHARING case XENMEM_sharing_op: return mem_sharing_memop(guest_handle_cast(arg, xen_mem_sharing_op_t)); +#endif default: rc = -ENOSYS; diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 912630a4fb..16829f6274 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -48,9 +48,6 @@ config MEM_ACCESS config HAS_MEM_PAGING bool -config HAS_MEM_SHARING - bool - config HAS_PDX bool diff --git a/xen/common/domain.c b/xen/common/domain.c index 8a5fa8662b..55aa759b75 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -74,7 +74,7 @@ integer_param("hardware_dom", hardware_domid); /* Private domain structs for DOMID_XEN, DOMID_IO, etc. */ struct domain *__read_mostly dom_xen; struct domain *__read_mostly dom_io; -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING struct domain *__read_mostly dom_cow; #endif @@ -549,7 +549,7 @@ void __init setup_system_domains(void) if ( IS_ERR(dom_io) ) panic("Failed to create d[IO]: %ld\n", PTR_ERR(dom_io)); -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING /* * Initialise our COW domain. * This domain owns sharable pages. @@ -966,7 +966,7 @@ static void complete_domain_destroy(struct rcu_head *head) xfree(d->vm_event_paging); #endif xfree(d->vm_event_monitor); -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING xfree(d->vm_event_share); #endif diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index b6b45c21e1..97695a221a 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -3798,7 +3798,7 @@ void grant_table_init_vcpu(struct vcpu *v) v->maptrack_tail = MAPTRACK_TAIL; } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref, gfn_t *gfn, uint16_t *status) { diff --git a/xen/common/memory.c b/xen/common/memory.c index 30d210fc08..d9b35a608c 100644 --- a/xen/common/memory.c +++ b/xen/common/memory.c @@ -1676,7 +1676,7 @@ int check_get_page_from_gfn(struct domain *d, gfn_t gfn, bool readonly, return -EAGAIN; } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) ) { if ( page ) diff --git a/xen/common/vm_event.c b/xen/common/vm_event.c index e8726805e7..2a1c87e44b 100644 --- a/xen/common/vm_event.c +++ b/xen/common/vm_event.c @@ -531,7 +531,7 @@ static void monitor_notification(struct vcpu *v, unsigned int port) vm_event_resume(v->domain, v->domain->vm_event_monitor); } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING /* Registered with Xen-bound event channel for incoming notifications. */ static void mem_sharing_notification(struct vcpu *v, unsigned int port) { @@ -561,7 +561,7 @@ void vm_event_cleanup(struct domain *d) destroy_waitqueue_head(&d->vm_event_monitor->wq); (void)vm_event_disable(d, &d->vm_event_monitor); } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING if ( vm_event_check_ring(d->vm_event_share) ) { destroy_waitqueue_head(&d->vm_event_share->wq); @@ -703,7 +703,7 @@ int vm_event_domctl(struct domain *d, struct xen_domctl_vm_event_op *vec) } break; -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING case XEN_DOMCTL_VM_EVENT_OP_SHARING: { rc = -EINVAL; diff --git a/xen/include/asm-x86/mem_sharing.h b/xen/include/asm-x86/mem_sharing.h index afd0c17292..db22468744 100644 --- a/xen/include/asm-x86/mem_sharing.h +++ b/xen/include/asm-x86/mem_sharing.h @@ -24,6 +24,8 @@ #include #include +#ifdef CONFIG_MEM_SHARING + /* Auditing of memory sharing code? */ #ifndef NDEBUG #define MEM_SHARING_AUDIT 1 @@ -99,4 +101,30 @@ int mem_sharing_domctl(struct domain *d, */ int relinquish_shared_pages(struct domain *d); +#else + +static inline unsigned int mem_sharing_get_nr_saved_mfns(void) +{ + return 0; +} +static inline unsigned int mem_sharing_get_nr_shared_mfns(void) +{ + return 0; +} +static inline int mem_sharing_unshare_page(struct domain *d, + unsigned long gfn, + uint16_t flags) +{ + ASSERT_UNREACHABLE(); + return -EOPNOTSUPP; +} +static inline int mem_sharing_notify_enomem(struct domain *d, unsigned long gfn, + bool allow_sleep) +{ + ASSERT_UNREACHABLE(); + return -EOPNOTSUPP; +} + +#endif + #endif /* __MEM_SHARING_H__ */ diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 087ad97351..35c2527265 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -127,6 +127,8 @@ struct page_info /* For non-pinnable single-page shadows, a higher entry that points * at us. */ paddr_t up; + +#ifdef CONFIG_MEM_SHARING /* For shared/sharable pages, we use a doubly-linked list * of all the {pfn,domain} pairs that map this page. We also include * an opaque handle, which is effectively a version, so that clients @@ -134,6 +136,7 @@ struct page_info * This list is allocated and freed when a page is shared/unshared. */ struct page_sharing_info *sharing; +#endif }; /* Reference count and various PGC_xxx flags and fields. */ diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index 47c8412b3a..977e45aae7 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -275,7 +275,7 @@ struct npfec { /* Private domain structs for DOMID_XEN, DOMID_IO, etc. */ extern struct domain *dom_xen, *dom_io; -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING extern struct domain *dom_cow; #else # define dom_cow NULL diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 97a3ab55aa..b40c8fd138 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -459,7 +459,7 @@ struct domain /* Various vm_events */ /* Memory sharing support */ -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING struct vm_event_domain *vm_event_share; #endif /* Memory paging support */ diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h index 01d2814fed..ef52bb1764 100644 --- a/xen/include/xsm/dummy.h +++ b/xen/include/xsm/dummy.h @@ -597,7 +597,7 @@ static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static XSM_INLINE int xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d) { XSM_ASSERT_ACTION(XSM_DM_PRIV); diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index b6141f6ab1..e22d6160b5 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -150,7 +150,7 @@ struct xsm_operations { int (*mem_paging) (struct domain *d); #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING int (*mem_sharing) (struct domain *d); #endif @@ -597,7 +597,7 @@ static inline int xsm_mem_paging (xsm_default_t def, struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static inline int xsm_mem_sharing (xsm_default_t def, struct domain *d) { return xsm_ops->mem_sharing(d); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index c9a566f2b5..5705e52791 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -128,7 +128,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, mem_paging); #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING set_to_dummy_if_null(ops, mem_sharing); #endif diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index a7d690ac3c..791c1f66af 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1262,7 +1262,7 @@ static int flask_mem_paging(struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static int flask_mem_sharing(struct domain *d) { return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__MEM_SHARING); @@ -1829,7 +1829,7 @@ static struct xsm_operations flask_ops = { .mem_paging = flask_mem_paging, #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING .mem_sharing = flask_mem_sharing, #endif From patchwork Wed Jul 17 19:33:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 11048183 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 865076C5 for ; Wed, 17 Jul 2019 19:35:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7539028686 for ; Wed, 17 Jul 2019 19:35:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 68F7928789; Wed, 17 Jul 2019 19:35:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EFFA028686 for ; Wed, 17 Jul 2019 19:35:29 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgk-0000KX-B5; Wed, 17 Jul 2019 19:34:06 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hnpgj-0000Ju-2C for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 19:34:05 +0000 X-Inumbo-ID: d4362ae0-a8c9-11e9-8c7b-2bb8db163187 Received: from mail-io1-f68.google.com (unknown [209.85.166.68]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d4362ae0-a8c9-11e9-8c7b-2bb8db163187; Wed, 17 Jul 2019 19:34:01 +0000 (UTC) Received: by mail-io1-f68.google.com with SMTP id i10so47542946iol.13 for ; Wed, 17 Jul 2019 12:34:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DZeS6BHvOxQrDAME9IQTHZlH+LOilULLIYwYdA0+/Qg=; b=XiPw7HuCNpZwtHhs2saUqrwAoPzK0EhLqmI22OTnJTc08nINU6q25wX9FXGEyjNkcp +vriEDEnuwWZob6Kvh71hSofILwXFXOOi/Rw8yUUEhljDbEe0GqOs1yf41WQgbt+jNfe qT/vfI6QzN3rXB2HwVhZL272usgDxZlayA5EcHwJM1HARSbDNun4kt5qbeKhbFQZX5yP zVQNwNVtS0sqylBEDqYC8507hPwgbmwCTNA3oYOr6akgpiSZmKj0r8eOMtf3E39HtSSN q4buqWXQx+5Cg/UH2umcQkmikUBcvazRxIq2vSU+bn4ZVWyqk/7KqSob1673omAdWmbJ b2ew== X-Gm-Message-State: APjAAAW7gWb37BIU5bF7W6cV87JCt3i8DNn1PW9bbNDtz69LfvngBjW5 0sUZInmASt8OJh+yOuYdq50HEv5f X-Google-Smtp-Source: APXvYqztdtJ/vOpct7p7oU8uFaGuVm+k56N2dMhnDiDrrQuJ2Kb+UdTum51T4dLHh53zjL4Nndiydw== X-Received: by 2002:a5e:881a:: with SMTP id l26mr14801631ioj.185.1563392040268; Wed, 17 Jul 2019 12:34:00 -0700 (PDT) Received: from l1.lan (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id v13sm21743421ioq.13.2019.07.17.12.33.58 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 17 Jul 2019 12:33:59 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Wed, 17 Jul 2019 13:33:35 -0600 Message-Id: <20190717193335.11991-6-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190717193335.11991-1-tamas@tklengyel.com> References: <20190717193335.11991-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v6 5/5] x86/mem_sharing: style cleanup X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change Signed-off-by: Tamas K Lengyel --- xen/arch/x86/mm/mem_sharing.c | 265 ++++++++++++++++++---------------- 1 file changed, 138 insertions(+), 127 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index a5fe89e339..aaf8c2f2c8 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -42,7 +42,8 @@ static shr_handle_t next_handle = 1; -typedef struct pg_lock_data { +typedef struct pg_lock_data +{ int mm_unlock_level; unsigned short recurse_count; } pg_lock_data_t; @@ -88,8 +89,8 @@ static inline void page_sharing_dispose(struct page_info *page) { /* Unlikely given our thresholds, but we should be careful. */ if ( unlikely(RMAP_USES_HASHTAB(page)) ) - free_xenheap_pages(page->sharing->hash_table.bucket, - RMAP_HASHTAB_ORDER); + free_xenheap_pages(page->sharing->hash_table.bucket, + RMAP_HASHTAB_ORDER); spin_lock(&shr_audit_lock); list_del_rcu(&page->sharing->entry); @@ -105,8 +106,8 @@ static inline void page_sharing_dispose(struct page_info *page) { /* Unlikely given our thresholds, but we should be careful. */ if ( unlikely(RMAP_USES_HASHTAB(page)) ) - free_xenheap_pages(page->sharing->hash_table.bucket, - RMAP_HASHTAB_ORDER); + free_xenheap_pages(page->sharing->hash_table.bucket, + RMAP_HASHTAB_ORDER); xfree(page->sharing); } @@ -136,8 +137,8 @@ static inline bool _page_lock(struct page_info *page) cpu_relax(); nx = x + (1 | PGT_locked); if ( !(x & PGT_validated) || - !(x & PGT_count_mask) || - !(nx & PGT_count_mask) ) + !(x & PGT_count_mask) || + !(nx & PGT_count_mask) ) return false; } while ( cmpxchg(&page->u.inuse.type_info, x, nx) != x ); @@ -168,7 +169,7 @@ static inline bool mem_sharing_page_lock(struct page_info *pg) if ( rc ) { preempt_disable(); - page_sharing_mm_post_lock(&pld->mm_unlock_level, + page_sharing_mm_post_lock(&pld->mm_unlock_level, &pld->recurse_count); } return rc; @@ -178,7 +179,7 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) { pg_lock_data_t *pld = &(this_cpu(__pld)); - page_sharing_mm_unlock(pld->mm_unlock_level, + page_sharing_mm_unlock(pld->mm_unlock_level, &pld->recurse_count); preempt_enable(); _page_unlock(pg); @@ -186,19 +187,18 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) static inline shr_handle_t get_next_handle(void) { - /* Get the next handle get_page style */ + /* Get the next handle get_page style */ uint64_t x, y = next_handle; do { x = y; - } - while ( (y = cmpxchg(&next_handle, x, x + 1)) != x ); + } while ( (y = cmpxchg(&next_handle, x, x + 1)) != x ); return x + 1; } #define mem_sharing_enabled(d) \ (is_hvm_domain(d) && (d)->arch.hvm.mem_sharing_enabled) -static atomic_t nr_saved_mfns = ATOMIC_INIT(0); +static atomic_t nr_saved_mfns = ATOMIC_INIT(0); static atomic_t nr_shared_mfns = ATOMIC_INIT(0); /** Reverse map **/ @@ -210,7 +210,7 @@ static atomic_t nr_shared_mfns = ATOMIC_INIT(0); typedef struct gfn_info { unsigned long gfn; - domid_t domain; + domid_t domain; struct list_head list; } gfn_info_t; @@ -225,7 +225,7 @@ rmap_init(struct page_info *page) #define HASH(domain, gfn) \ (((gfn) + (domain)) % RMAP_HASHTAB_SIZE) -/* Conversions. Tuned by the thresholds. Should only happen twice +/* Conversions. Tuned by the thresholds. Should only happen twice * (once each) during the lifetime of a shared page */ static inline int rmap_list_to_hash_table(struct page_info *page) @@ -288,13 +288,13 @@ rmap_count(struct page_info *pg) } /* The page type count is always decreased after removing from the rmap. - * Use a convert flag to avoid mutating the rmap if in the middle of an + * Use a convert flag to avoid mutating the rmap if in the middle of an * iterator, or if the page will be soon destroyed anyways. */ static inline void rmap_del(gfn_info_t *gfn_info, struct page_info *page, int convert) { if ( RMAP_USES_HASHTAB(page) && convert && - (rmap_count(page) <= RMAP_LIGHT_SHARED_PAGE) ) + (rmap_count(page) <= RMAP_LIGHT_SHARED_PAGE) ) rmap_hash_table_to_list(page); /* Regardless of rmap type, same removal operation */ @@ -308,30 +308,30 @@ rmap_add(gfn_info_t *gfn_info, struct page_info *page) struct list_head *head; if ( !RMAP_USES_HASHTAB(page) && - (rmap_count(page) >= RMAP_HEAVY_SHARED_PAGE) ) + (rmap_count(page) >= RMAP_HEAVY_SHARED_PAGE) ) /* The conversion may fail with ENOMEM. We'll be less efficient, * but no reason to panic. */ (void)rmap_list_to_hash_table(page); head = (RMAP_USES_HASHTAB(page)) ? - page->sharing->hash_table.bucket + - HASH(gfn_info->domain, gfn_info->gfn) : - &page->sharing->gfns; + page->sharing->hash_table.bucket + + HASH(gfn_info->domain, gfn_info->gfn) : + &page->sharing->gfns; INIT_LIST_HEAD(&gfn_info->list); list_add(&gfn_info->list, head); } static inline gfn_info_t * -rmap_retrieve(uint16_t domain_id, unsigned long gfn, - struct page_info *page) +rmap_retrieve(uint16_t domain_id, unsigned long gfn, + struct page_info *page) { gfn_info_t *gfn_info; struct list_head *le, *head; head = (RMAP_USES_HASHTAB(page)) ? - page->sharing->hash_table.bucket + HASH(domain_id, gfn) : - &page->sharing->gfns; + page->sharing->hash_table.bucket + HASH(domain_id, gfn) : + &page->sharing->gfns; list_for_each(le, head) { @@ -358,7 +358,8 @@ static inline int rmap_has_entries(struct page_info *page) /* The iterator hides the details of how the rmap is implemented. This * involves splitting the list_for_each_safe macro into two steps. */ -struct rmap_iterator { +struct rmap_iterator +{ struct list_head *curr; struct list_head *next; unsigned int bucket; @@ -368,9 +369,9 @@ static inline void rmap_seed_iterator(struct page_info *page, struct rmap_iterator *ri) { ri->curr = (RMAP_USES_HASHTAB(page)) ? - page->sharing->hash_table.bucket : - &page->sharing->gfns; - ri->next = ri->curr->next; + page->sharing->hash_table.bucket : + &page->sharing->gfns; + ri->next = ri->curr->next; ri->bucket = 0; } @@ -378,8 +379,8 @@ static inline gfn_info_t * rmap_iterate(struct page_info *page, struct rmap_iterator *ri) { struct list_head *head = (RMAP_USES_HASHTAB(page)) ? - page->sharing->hash_table.bucket + ri->bucket : - &page->sharing->gfns; + page->sharing->hash_table.bucket + ri->bucket : + &page->sharing->gfns; retry: if ( ri->next == head) @@ -394,7 +395,8 @@ retry: ri->curr = head; ri->next = ri->curr->next; goto retry; - } else + } + else /* List exhausted */ return NULL; } @@ -406,13 +408,13 @@ retry: } static inline gfn_info_t *mem_sharing_gfn_alloc(struct page_info *page, - struct domain *d, - unsigned long gfn) + struct domain *d, + unsigned long gfn) { gfn_info_t *gfn_info = xmalloc(gfn_info_t); if ( gfn_info == NULL ) - return NULL; + return NULL; gfn_info->gfn = gfn; gfn_info->domain = d->domain_id; @@ -426,8 +428,8 @@ static inline gfn_info_t *mem_sharing_gfn_alloc(struct page_info *page, } static inline void mem_sharing_gfn_destroy(struct page_info *page, - struct domain *d, - gfn_info_t *gfn_info) + struct domain *d, + gfn_info_t *gfn_info) { /* Decrement the number of pages. */ atomic_dec(&d->shr_pages); @@ -437,15 +439,15 @@ static inline void mem_sharing_gfn_destroy(struct page_info *page, xfree(gfn_info); } -static struct page_info* mem_sharing_lookup(unsigned long mfn) +static struct page_info *mem_sharing_lookup(unsigned long mfn) { if ( mfn_valid(_mfn(mfn)) ) { - struct page_info* page = mfn_to_page(_mfn(mfn)); + struct page_info *page = mfn_to_page(_mfn(mfn)); if ( page_get_owner(page) == dom_cow ) { /* Count has to be at least two, because we're called - * with the mfn locked (1) and this is supposed to be + * with the mfn locked (1) and this is supposed to be * a shared page (1). */ unsigned long t = read_atomic(&page->u.inuse.type_info); ASSERT((t & PGT_type_mask) == PGT_shared_page); @@ -486,44 +488,44 @@ static int audit(void) /* If we can't lock it, it's definitely not a shared page */ if ( !mem_sharing_page_lock(pg) ) { - MEM_SHARING_DEBUG("mfn %lx in audit list, but cannot be locked (%lx)!\n", + MEM_SHARING_DEBUG("mfn %lx in audit list, but cannot be locked (%lx)!\n", mfn_x(mfn), pg->u.inuse.type_info); - errors++; - continue; + errors++; + continue; } - /* Check if the MFN has correct type, owner and handle. */ + /* Check if the MFN has correct type, owner and handle. */ if ( (pg->u.inuse.type_info & PGT_type_mask) != PGT_shared_page ) { - MEM_SHARING_DEBUG("mfn %lx in audit list, but not PGT_shared_page (%lx)!\n", + MEM_SHARING_DEBUG("mfn %lx in audit list, but not PGT_shared_page (%lx)!\n", mfn_x(mfn), pg->u.inuse.type_info & PGT_type_mask); - errors++; - continue; + errors++; + continue; } /* Check the page owner. */ if ( page_get_owner(pg) != dom_cow ) { - MEM_SHARING_DEBUG("mfn %lx shared, but wrong owner (%hu)!\n", - mfn_x(mfn), page_get_owner(pg)->domain_id); - errors++; + MEM_SHARING_DEBUG("mfn %lx shared, but wrong owner (%hu)!\n", + mfn_x(mfn), page_get_owner(pg)->domain_id); + errors++; } /* Check the m2p entry */ if ( !SHARED_M2P(get_gpfn_from_mfn(mfn_x(mfn))) ) { - MEM_SHARING_DEBUG("mfn %lx shared, but wrong m2p entry (%lx)!\n", - mfn_x(mfn), get_gpfn_from_mfn(mfn_x(mfn))); - errors++; + MEM_SHARING_DEBUG("mfn %lx shared, but wrong m2p entry (%lx)!\n", + mfn_x(mfn), get_gpfn_from_mfn(mfn_x(mfn))); + errors++; } /* Check we have a list */ if ( (!pg->sharing) || !rmap_has_entries(pg) ) { - MEM_SHARING_DEBUG("mfn %lx shared, but empty gfn list!\n", - mfn_x(mfn)); - errors++; - continue; + MEM_SHARING_DEBUG("mfn %lx shared, but empty gfn list!\n", + mfn_x(mfn)); + errors++; + continue; } /* We've found a page that is shared */ @@ -545,7 +547,7 @@ static int audit(void) errors++; continue; } - o_mfn = get_gfn_query_unlocked(d, g->gfn, &t); + o_mfn = get_gfn_query_unlocked(d, g->gfn, &t); if ( !mfn_eq(o_mfn, mfn) ) { MEM_SHARING_DEBUG("Incorrect P2M for d=%hu, PFN=%lx." @@ -568,7 +570,7 @@ static int audit(void) { MEM_SHARING_DEBUG("Mismatched counts for MFN=%lx." "nr_gfns in list %lu, in type_info %lx\n", - mfn_x(mfn), nr_gfns, + mfn_x(mfn), nr_gfns, (pg->u.inuse.type_info & PGT_count_mask)); errors++; } @@ -596,15 +598,16 @@ int mem_sharing_notify_enomem(struct domain *d, unsigned long gfn, { struct vcpu *v = current; int rc; - vm_event_request_t req = { + vm_event_request_t req = + { .reason = VM_EVENT_REASON_MEM_SHARING, .vcpu_id = v->vcpu_id, .u.mem_sharing.gfn = gfn, .u.mem_sharing.p2mt = p2m_ram_shared }; - if ( (rc = __vm_event_claim_slot(d, - d->vm_event_share, allow_sleep)) < 0 ) + if ( (rc = __vm_event_claim_slot(d, + d->vm_event_share, allow_sleep)) < 0 ) return rc; if ( v->domain == d ) @@ -629,9 +632,9 @@ unsigned int mem_sharing_get_nr_shared_mfns(void) } /* Functions that change a page's type and ownership */ -static int page_make_sharable(struct domain *d, - struct page_info *page, - int expected_refcnt) +static int page_make_sharable(struct domain *d, + struct page_info *page, + int expected_refcnt) { bool_t drop_dom_ref; @@ -684,7 +687,7 @@ static int page_make_private(struct domain *d, struct page_info *page) if ( !get_page(page, dom_cow) ) return -EINVAL; - + spin_lock(&d->page_alloc_lock); if ( d->is_dying ) @@ -729,7 +732,7 @@ static inline struct page_info *__grab_shared_page(mfn_t mfn) return NULL; pg = mfn_to_page(mfn); - /* If the page is not validated we can't lock it, and if it's + /* If the page is not validated we can't lock it, and if it's * not validated it's obviously not shared. */ if ( !mem_sharing_page_lock(pg) ) return NULL; @@ -754,12 +757,12 @@ static int debug_mfn(mfn_t mfn) return -EINVAL; } - MEM_SHARING_DEBUG( - "Debug page: MFN=%lx is ci=%lx, ti=%lx, owner_id=%d\n", - mfn_x(page_to_mfn(page)), - page->count_info, - page->u.inuse.type_info, - page_get_owner(page)->domain_id); + MEM_SHARING_DEBUG( + "Debug page: MFN=%lx is ci=%lx, ti=%lx, owner_id=%d\n", + mfn_x(page_to_mfn(page)), + page->count_info, + page->u.inuse.type_info, + page_get_owner(page)->domain_id); /* -1 because the page is locked and that's an additional type ref */ num_refs = ((int) (page->u.inuse.type_info & PGT_count_mask)) - 1; @@ -775,7 +778,7 @@ static int debug_gfn(struct domain *d, gfn_t gfn) mfn = get_gfn_query(d, gfn_x(gfn), &p2mt); - MEM_SHARING_DEBUG("Debug for dom%d, gfn=%" PRI_gfn "\n", + MEM_SHARING_DEBUG("Debug for dom%d, gfn=%" PRI_gfn "\n", d->domain_id, gfn_x(gfn)); num_refs = debug_mfn(mfn); put_gfn(d, gfn_x(gfn)); @@ -796,10 +799,10 @@ static int debug_gref(struct domain *d, grant_ref_t ref) d->domain_id, ref, rc); return rc; } - + MEM_SHARING_DEBUG( - "==> Grant [dom=%d,ref=%d], status=%x. ", - d->domain_id, ref, status); + "==> Grant [dom=%d,ref=%d], status=%x. ", + d->domain_id, ref, status); return debug_gfn(d, gfn); } @@ -824,12 +827,14 @@ static int nominate_page(struct domain *d, gfn_t gfn, goto out; /* Return the handle if the page is already shared */ - if ( p2m_is_shared(p2mt) ) { + if ( p2m_is_shared(p2mt) ) + { struct page_info *pg = __grab_shared_page(mfn); if ( !pg ) { gprintk(XENLOG_ERR, - "Shared p2m entry gfn %" PRI_gfn ", but could not grab mfn %" PRI_mfn " dom%d\n", + "Shared p2m entry gfn %" PRI_gfn ", but could not grab mfn %" PRI_mfn + " dom%d\n", gfn_x(gfn), mfn_x(mfn), d->domain_id); BUG(); } @@ -876,12 +881,12 @@ static int nominate_page(struct domain *d, gfn_t gfn, /* Try to convert the mfn to the sharable type */ page = mfn_to_page(mfn); - ret = page_make_sharable(d, page, expected_refcnt); - if ( ret ) + ret = page_make_sharable(d, page, expected_refcnt); + if ( ret ) goto out; - /* Now that the page is validated, we can lock it. There is no - * race because we're holding the p2m entry, so no one else + /* Now that the page is validated, we can lock it. There is no + * race because we're holding the p2m entry, so no one else * could be nominating this gfn */ ret = -ENOENT; if ( !mem_sharing_page_lock(page) ) @@ -889,8 +894,8 @@ static int nominate_page(struct domain *d, gfn_t gfn, /* Initialize the shared state */ ret = -ENOMEM; - if ( (page->sharing = - xmalloc(struct page_sharing_info)) == NULL ) + if ( (page->sharing = + xmalloc(struct page_sharing_info)) == NULL ) { /* Making a page private atomically unlocks it */ BUG_ON(page_make_private(d, page) != 0); @@ -900,7 +905,7 @@ static int nominate_page(struct domain *d, gfn_t gfn, rmap_init(page); /* Create the handle */ - page->sharing->handle = get_next_handle(); + page->sharing->handle = get_next_handle(); /* Create the local gfn info */ if ( mem_sharing_gfn_alloc(page, d, gfn_x(gfn)) == NULL ) @@ -946,7 +951,7 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, get_two_gfns(sd, sgfn, &smfn_type, NULL, &smfn, cd, cgfn, &cmfn_type, NULL, &cmfn, 0, &tg); - /* This tricky business is to avoid two callers deadlocking if + /* This tricky business is to avoid two callers deadlocking if * grabbing pages in opposite client/source order */ if ( mfn_eq(smfn, cmfn) ) { @@ -972,7 +977,9 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, mem_sharing_page_unlock(spage); goto err_out; } - } else { + } + else + { ret = XENMEM_SHARING_OP_C_HANDLE_INVALID; cpage = firstpg = __grab_shared_page(cmfn); if ( cpage == NULL ) @@ -1010,7 +1017,7 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, rmap_seed_iterator(cpage, &ri); while ( (gfn = rmap_iterate(cpage, &ri)) != NULL) { - /* Get the source page and type, this should never fail: + /* Get the source page and type, this should never fail: * we are under shr lock, and got a successful lookup */ BUG_ON(!get_page_and_type(spage, dom_cow, PGT_shared_page)); /* Move the gfn_info from client list to source list. @@ -1043,14 +1050,15 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, atomic_dec(&nr_shared_mfns); atomic_inc(&nr_saved_mfns); ret = 0; - + err_out: put_two_gfns(&tg); return ret; } -int mem_sharing_add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle_t sh, - struct domain *cd, unsigned long cgfn) +int mem_sharing_add_to_physmap(struct domain *sd, unsigned long sgfn, + shr_handle_t sh, + struct domain *cd, unsigned long cgfn) { struct page_info *spage; int ret = -EINVAL; @@ -1101,7 +1109,9 @@ int mem_sharing_add_to_physmap(struct domain *sd, unsigned long sgfn, shr_handle { mem_sharing_gfn_destroy(spage, cd, gfn_info); put_page_and_type(spage); - } else { + } + else + { /* There is a chance we're plugging a hole where a paged out page was */ if ( p2m_is_paging(cmfn_type) && (cmfn_type != p2m_ram_paging_out) ) { @@ -1136,10 +1146,10 @@ err_out: /* A note on the rationale for unshare error handling: * 1. Unshare can only fail with ENOMEM. Any other error conditions BUG_ON()'s * 2. We notify a potential dom0 helper through a vm_event ring. But we - * allow the notification to not go to sleep. If the event ring is full + * allow the notification to not go to sleep. If the event ring is full * of ENOMEM warnings, then it's on the ball. * 3. We cannot go to sleep until the unshare is resolved, because we might - * be buried deep into locks (e.g. something -> copy_to_user -> __hvm_copy) + * be buried deep into locks (e.g. something -> copy_to_user -> __hvm_copy) * 4. So, we make sure we: * 4.1. return an error * 4.2. do not corrupt shared memory @@ -1147,19 +1157,20 @@ err_out: * 4.4. let the guest deal with it if the error propagation will reach it */ int __mem_sharing_unshare_page(struct domain *d, - unsigned long gfn, - uint16_t flags) + unsigned long gfn, + uint16_t flags) { p2m_type_t p2mt; mfn_t mfn; struct page_info *page, *old_page; int last_gfn; gfn_info_t *gfn_info = NULL; - + mfn = get_gfn(d, gfn, &p2mt); - + /* Has someone already unshared it? */ - if ( !p2m_is_shared(p2mt) ) { + if ( !p2m_is_shared(p2mt) ) + { put_gfn(d, gfn); return 0; } @@ -1168,7 +1179,7 @@ int __mem_sharing_unshare_page(struct domain *d, if ( page == NULL ) { gdprintk(XENLOG_ERR, "Domain p2m is shared, but page is not: " - "%lx\n", gfn); + "%lx\n", gfn); BUG(); } @@ -1176,12 +1187,12 @@ int __mem_sharing_unshare_page(struct domain *d, if ( unlikely(gfn_info == NULL) ) { gdprintk(XENLOG_ERR, "Could not find gfn_info for shared gfn: " - "%lx\n", gfn); + "%lx\n", gfn); BUG(); } /* Do the accounting first. If anything fails below, we have bigger - * bigger fish to fry. First, remove the gfn from the list. */ + * bigger fish to fry. First, remove the gfn from the list. */ last_gfn = rmap_has_one_entry(page); if ( last_gfn ) { @@ -1195,7 +1206,7 @@ int __mem_sharing_unshare_page(struct domain *d, else atomic_dec(&nr_saved_mfns); - /* If the GFN is getting destroyed drop the references to MFN + /* If the GFN is getting destroyed drop the references to MFN * (possibly freeing the page), and exit early */ if ( flags & MEM_SHARING_DESTROY_GFN ) { @@ -1212,7 +1223,7 @@ int __mem_sharing_unshare_page(struct domain *d, return 0; } - + if ( last_gfn ) { /* Making a page private atomically unlocks it */ @@ -1222,7 +1233,7 @@ int __mem_sharing_unshare_page(struct domain *d, old_page = page; page = alloc_domheap_page(d, 0); - if ( !page ) + if ( !page ) { /* Undo dec of nr_saved_mfns, as the retry will decrease again. */ atomic_inc(&nr_saved_mfns); @@ -1240,11 +1251,11 @@ int __mem_sharing_unshare_page(struct domain *d, mem_sharing_page_unlock(old_page); put_page_and_type(old_page); -private_page_found: +private_page_found: if ( p2m_change_type_one(d, gfn, p2m_ram_shared, p2m_ram_rw) ) { - gdprintk(XENLOG_ERR, "Could not change p2m type d %hu gfn %lx.\n", - d->domain_id, gfn); + gdprintk(XENLOG_ERR, "Could not change p2m type d %hu gfn %lx.\n", + d->domain_id, gfn); BUG(); } @@ -1270,7 +1281,7 @@ int relinquish_shared_pages(struct domain *d) p2m_lock(p2m); for ( gfn = p2m->next_shared_gfn_to_relinquish; - gfn <= p2m->max_mapped_pfn; gfn++ ) + gfn <= p2m->max_mapped_pfn; gfn++ ) { p2m_access_t a; p2m_type_t t; @@ -1283,8 +1294,8 @@ int relinquish_shared_pages(struct domain *d) if ( mfn_valid(mfn) && (t == p2m_ram_shared) ) { /* Does not fail with ENOMEM given the DESTROY flag */ - BUG_ON(__mem_sharing_unshare_page(d, gfn, - MEM_SHARING_DESTROY_GFN)); + BUG_ON(__mem_sharing_unshare_page(d, gfn, + MEM_SHARING_DESTROY_GFN)); /* Clear out the p2m entry so no one else may try to * unshare. Must succeed: we just read the old entry and * we hold the p2m lock. */ @@ -1454,8 +1465,8 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) if ( XENMEM_SHARING_OP_FIELD_IS_GREF(mso.u.share.source_gfn) ) { - grant_ref_t gref = (grant_ref_t) - (XENMEM_SHARING_OP_FIELD_GET_GREF( + grant_ref_t gref = (grant_ref_t) + (XENMEM_SHARING_OP_FIELD_GET_GREF( mso.u.share.source_gfn)); rc = mem_sharing_gref_to_gfn(d->grant_table, gref, &sgfn, NULL); @@ -1470,8 +1481,8 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) if ( XENMEM_SHARING_OP_FIELD_IS_GREF(mso.u.share.client_gfn) ) { - grant_ref_t gref = (grant_ref_t) - (XENMEM_SHARING_OP_FIELD_GET_GREF( + grant_ref_t gref = (grant_ref_t) + (XENMEM_SHARING_OP_FIELD_GET_GREF( mso.u.share.client_gfn)); rc = mem_sharing_gref_to_gfn(cd->grant_table, gref, &cgfn, NULL); @@ -1534,7 +1545,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) sh = mso.u.share.source_handle; cgfn = mso.u.share.client_gfn; - rc = mem_sharing_add_to_physmap(d, sgfn, sh, cd, cgfn); + rc = mem_sharing_add_to_physmap(d, sgfn, sh, cd, cgfn); rcu_unlock_domain(cd); } @@ -1547,8 +1558,8 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) rc = -EINVAL; if ( mso.u.range._pad[0] || mso.u.range._pad[1] || - mso.u.range._pad[2] ) - goto out; + mso.u.range._pad[2] ) + goto out; /* * We use opaque for the hypercall continuation value. @@ -1557,8 +1568,8 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) * that it's at least in range. */ if ( mso.u.range.opaque && - (mso.u.range.opaque < mso.u.range.first_gfn || - mso.u.range.opaque > mso.u.range.last_gfn) ) + (mso.u.range.opaque < mso.u.range.first_gfn || + mso.u.range.opaque > mso.u.range.last_gfn) ) goto out; if ( !mem_sharing_enabled(d) ) @@ -1593,7 +1604,7 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) * the duration of this op. */ if ( !atomic_read(&d->pause_count) || - !atomic_read(&cd->pause_count) ) + !atomic_read(&cd->pause_count) ) { rcu_unlock_domain(cd); rc = -EINVAL; @@ -1604,9 +1615,9 @@ int mem_sharing_memop(XEN_GUEST_HANDLE_PARAM(xen_mem_sharing_op_t) arg) max_cgfn = domain_get_maximum_gpfn(cd); if ( max_sgfn < mso.u.range.first_gfn || - max_sgfn < mso.u.range.last_gfn || - max_cgfn < mso.u.range.first_gfn || - max_cgfn < mso.u.range.last_gfn ) + max_sgfn < mso.u.range.last_gfn || + max_cgfn < mso.u.range.first_gfn || + max_cgfn < mso.u.range.last_gfn ) { rcu_unlock_domain(cd); rc = -EINVAL; @@ -1657,9 +1668,9 @@ int mem_sharing_domctl(struct domain *d, struct xen_domctl_mem_sharing_op *mec) /* Only HAP is supported */ if ( !hap_enabled(d) ) - return -ENODEV; + return -ENODEV; - switch(mec->op) + switch (mec->op) { case XEN_DOMCTL_MEM_SHARING_CONTROL: {