From patchwork Tue Oct 15 03:49:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hui Zhu X-Patchwork-Id: 11189647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB58C1390 for ; Tue, 15 Oct 2019 03:49:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE6EA217F9 for ; Tue, 15 Oct 2019 03:49:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE6EA217F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E65E98E0005; Mon, 14 Oct 2019 23:49:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DEF1C8E0001; Mon, 14 Oct 2019 23:49:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDD178E0005; Mon, 14 Oct 2019 23:49:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id A74838E0001 for ; Mon, 14 Oct 2019 23:49:32 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 460F16D73 for ; Tue, 15 Oct 2019 03:49:32 +0000 (UTC) X-FDA: 76044639384.30.fear22_36092820fdc5f X-Spam-Summary: 2,0,0,82492f170f1bd376,d41d8cd98f00b204,teawaterz@linux.alibaba.com,:konrad.wilk@oracle.com:sjenning@redhat.com:ddstreet@ieee.org:linux-kernel@vger.kernel.org::teawater@gmail.com:teawaterz@linux.alibaba.com,RULES_HIT:41:355:379:541:960:973:988:989:1260:1261:1345:1437:1534:1540:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3870:3872:4250:4321:5007:6119:6261:7903:9413:10004:11026:11658:11914:12043:12048:12297:12438:12555:12895:13069:13311:13357:14096:14181:14384:14394:14687:14721:21080:21627:21666:30054,0,RBL:115.124.30.42:@linux.alibaba.com:.lbl8.mailshell.net-62.20.2.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: fear22_36092820fdc5f X-Filterd-Recvd-Size: 1958 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Oct 2019 03:49:30 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04395;MF=teawaterz@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Tf5lel1_1571111361; Received: from localhost(mailfrom:teawaterz@linux.alibaba.com fp:SMTPD_---0Tf5lel1_1571111361) by smtp.aliyun-inc.com(127.0.0.1); Tue, 15 Oct 2019 11:49:27 +0800 From: Hui Zhu To: konrad.wilk@oracle.com, sjenning@redhat.com, ddstreet@ieee.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Hui Zhu , Hui Zhu Subject: [PATCH 1/2] mm, frontswap: Fix frontswap_map issue with THP Date: Tue, 15 Oct 2019 11:49:08 +0800 Message-Id: <1571111349-5041-1-git-send-email-teawater@gmail.com> X-Mailer: git-send-email 2.7.4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.242123, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Shrink will try to use frontswap interface store the THP as a normal page in __frontswap_store: if (ret == 0) { __frontswap_set(sis, offset); inc_frontswap_succ_stores(); } else { It should set all bits with THP. This commit set all bits with THP. Signed-off-by: Hui Zhu --- mm/frontswap.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/mm/frontswap.c b/mm/frontswap.c index 60bb20e..f07ea63 100644 --- a/mm/frontswap.c +++ b/mm/frontswap.c @@ -274,8 +274,12 @@ int __frontswap_store(struct page *page) break; } if (ret == 0) { - __frontswap_set(sis, offset); - inc_frontswap_succ_stores(); + int i, nr = hpage_nr_pages(page); + + for (i = 0; i < nr; i++) { + __frontswap_set(sis, offset + i); + inc_frontswap_succ_stores(); + } } else { inc_frontswap_failed_stores(); } From patchwork Tue Oct 15 03:49:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hui Zhu X-Patchwork-Id: 11189649 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2DDD01390 for ; Tue, 15 Oct 2019 03:49:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC54F217F9 for ; Tue, 15 Oct 2019 03:49:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC54F217F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3916F8E0006; Mon, 14 Oct 2019 23:49:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 340C68E0001; Mon, 14 Oct 2019 23:49:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 230778E0006; Mon, 14 Oct 2019 23:49:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 009788E0001 for ; Mon, 14 Oct 2019 23:49:38 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 73BD4180ACF8F for ; Tue, 15 Oct 2019 03:49:38 +0000 (UTC) X-FDA: 76044639636.03.music90_36eb2c34b7415 X-Spam-Summary: 2,0,0,9ad617ccca6b9256,d41d8cd98f00b204,teawaterz@linux.alibaba.com,:konrad.wilk@oracle.com:sjenning@redhat.com:ddstreet@ieee.org:linux-kernel@vger.kernel.org::teawater@gmail.com:teawaterz@linux.alibaba.com,RULES_HIT:2:41:69:355:379:541:960:966:968:973:988:989:1260:1261:1345:1359:1437:1535:1605:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:2895:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3872:4049:4118:4250:4385:5007:6117:6119:6261:7903:7904:8957:9413:9592:10004:11026:11658:11914:12043:12048:12291:12296:12297:12438:12555:12679:12683:12895:14110:14394:14687:21080:21324:21627:21666:30034:30054:30070,0,RBL:115.124.30.132:@linux.alibaba.com:.lbl8.mailshell.net-62.20.2.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: music90_36eb2c34b7415 X-Filterd-Recvd-Size: 7702 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Oct 2019 03:49:37 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R341e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=teawaterz@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Tf5len0_1571111370; Received: from localhost(mailfrom:teawaterz@linux.alibaba.com fp:SMTPD_---0Tf5len0_1571111370) by smtp.aliyun-inc.com(127.0.0.1); Tue, 15 Oct 2019 11:49:33 +0800 From: Hui Zhu To: konrad.wilk@oracle.com, sjenning@redhat.com, ddstreet@ieee.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Hui Zhu , Hui Zhu Subject: [PATCH 2/2] mm, zswap: Support THP Date: Tue, 15 Oct 2019 11:49:09 +0800 Message-Id: <1571111349-5041-2-git-send-email-teawater@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1571111349-5041-1-git-send-email-teawater@gmail.com> References: <1571111349-5041-1-git-send-email-teawater@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This commit let zswap treats THP as continuous normal pages in zswap_frontswap_store. It will store them to a lot of "zswap_entry". These "zswap_entry" will be inserted to "zswap_tree" together. Signed-off-by: Hui Zhu --- mm/zswap.c | 170 +++++++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 109 insertions(+), 61 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 46a3223..36aa10d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -316,11 +316,7 @@ static void zswap_rb_erase(struct rb_root *root, struct zswap_entry *entry) } } -/* - * Carries out the common pattern of freeing and entry's zpool allocation, - * freeing the entry itself, and decrementing the number of stored pages. - */ -static void zswap_free_entry(struct zswap_entry *entry) +static void zswap_free_entry_1(struct zswap_entry *entry) { if (!entry->length) atomic_dec(&zswap_same_filled_pages); @@ -329,6 +325,15 @@ static void zswap_free_entry(struct zswap_entry *entry) zswap_pool_put(entry->pool); } zswap_entry_cache_free(entry); +} + +/* + * Carries out the common pattern of freeing and entry's zpool allocation, + * freeing the entry itself, and decrementing the number of stored pages. + */ +static void zswap_free_entry(struct zswap_entry *entry) +{ + zswap_free_entry_1(entry); atomic_dec(&zswap_stored_pages); zswap_update_total_size(); } @@ -980,15 +985,11 @@ static void zswap_fill_page(void *ptr, unsigned long value) memset_l(page, value, PAGE_SIZE / sizeof(unsigned long)); } -/********************************* -* frontswap hooks -**********************************/ -/* attempts to compress and store an single page */ -static int zswap_frontswap_store(unsigned type, pgoff_t offset, - struct page *page) +static int zswap_frontswap_store_1(unsigned type, pgoff_t offset, + struct page *page, + struct zswap_entry **entry_pointer) { - struct zswap_tree *tree = zswap_trees[type]; - struct zswap_entry *entry, *dupentry; + struct zswap_entry *entry; struct crypto_comp *tfm; int ret; unsigned int hlen, dlen = PAGE_SIZE; @@ -998,36 +999,6 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, struct zswap_header zhdr = { .swpentry = swp_entry(type, offset) }; gfp_t gfp; - /* THP isn't supported */ - if (PageTransHuge(page)) { - ret = -EINVAL; - goto reject; - } - - if (!zswap_enabled || !tree) { - ret = -ENODEV; - goto reject; - } - - /* reclaim space if needed */ - if (zswap_is_full()) { - zswap_pool_limit_hit++; - if (zswap_shrink()) { - zswap_reject_reclaim_fail++; - ret = -ENOMEM; - goto reject; - } - - /* A second zswap_is_full() check after - * zswap_shrink() to make sure it's now - * under the max_pool_percent - */ - if (zswap_is_full()) { - ret = -ENOMEM; - goto reject; - } - } - /* allocate entry */ entry = zswap_entry_cache_alloc(GFP_KERNEL); if (!entry) { @@ -1035,6 +1006,7 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, ret = -ENOMEM; goto reject; } + *entry_pointer = entry; if (zswap_same_filled_pages_enabled) { src = kmap_atomic(page); @@ -1044,7 +1016,7 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); - goto insert_entry; + goto out; } kunmap_atomic(src); } @@ -1093,31 +1065,105 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, entry->handle = handle; entry->length = dlen; -insert_entry: +out: + return 0; + +put_dstmem: + put_cpu_var(zswap_dstmem); + zswap_pool_put(entry->pool); +freepage: + zswap_entry_cache_free(entry); +reject: + return ret; +} + +/********************************* +* frontswap hooks +**********************************/ +/* attempts to compress and store an single page */ +static int zswap_frontswap_store(unsigned type, pgoff_t offset, + struct page *page) +{ + struct zswap_tree *tree = zswap_trees[type]; + struct zswap_entry **entries = NULL, *dupentry; + struct zswap_entry *single_entry[1]; + int ret; + int i, nr; + + if (!zswap_enabled || !tree) { + ret = -ENODEV; + goto reject; + } + + /* reclaim space if needed */ + if (zswap_is_full()) { + zswap_pool_limit_hit++; + if (zswap_shrink()) { + zswap_reject_reclaim_fail++; + ret = -ENOMEM; + goto reject; + } + + /* A second zswap_is_full() check after + * zswap_shrink() to make sure it's now + * under the max_pool_percent + */ + if (zswap_is_full()) { + ret = -ENOMEM; + goto reject; + } + } + + nr = hpage_nr_pages(page); + + if (unlikely(nr > 1)) { + entries = kvmalloc(sizeof(struct zswap_entry *) * nr, + GFP_KERNEL); + if (!entries) { + ret = -ENOMEM; + goto reject; + } + } else + entries = single_entry; + + for (i = 0; i < nr; i++) { + ret = zswap_frontswap_store_1(type, offset + i, page + i, + &entries[i]); + if (ret) + goto freepage; + } + /* map */ spin_lock(&tree->lock); - do { - ret = zswap_rb_insert(&tree->rbroot, entry, &dupentry); - if (ret == -EEXIST) { - zswap_duplicate_entry++; - /* remove from rbtree */ - zswap_rb_erase(&tree->rbroot, dupentry); - zswap_entry_put(tree, dupentry); - } - } while (ret == -EEXIST); + for (i = 0; i < nr; i++) { + do { + ret = zswap_rb_insert(&tree->rbroot, entries[i], + &dupentry); + if (ret == -EEXIST) { + zswap_duplicate_entry++; + /* remove from rbtree */ + zswap_rb_erase(&tree->rbroot, dupentry); + zswap_entry_put(tree, dupentry); + } + } while (ret == -EEXIST); + } spin_unlock(&tree->lock); /* update stats */ - atomic_inc(&zswap_stored_pages); + atomic_add(nr, &zswap_stored_pages); zswap_update_total_size(); - return 0; - -put_dstmem: - put_cpu_var(zswap_dstmem); - zswap_pool_put(entry->pool); + ret = 0; freepage: - zswap_entry_cache_free(entry); + if (unlikely(nr > 1)) { + if (ret) { + int j; + + for (j = 0; j < i; j++) + zswap_free_entry_1(entries[j]); + } + kvfree(entries); + } reject: return ret; } @@ -1136,6 +1182,8 @@ static int zswap_frontswap_load(unsigned type, pgoff_t offset, unsigned int dlen; int ret; + BUG_ON(PageTransHuge(page)); + /* find */ spin_lock(&tree->lock); entry = zswap_entry_find_get(&tree->rbroot, offset);