From patchwork Tue Jul 30 22:27:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13747947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E535C3DA49 for ; Tue, 30 Jul 2024 22:27:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E392D6B007B; Tue, 30 Jul 2024 18:27:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE9596B0082; Tue, 30 Jul 2024 18:27:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB0B36B0083; Tue, 30 Jul 2024 18:27:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A798E6B007B for ; Tue, 30 Jul 2024 18:27:11 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 008B4A1654 for ; Tue, 30 Jul 2024 22:27:10 +0000 (UTC) X-FDA: 82397855862.11.F57013F Received: from mail-yw1-f180.google.com (mail-yw1-f180.google.com [209.85.128.180]) by imf20.hostedemail.com (Postfix) with ESMTP id 4CB0B1C0038 for ; Tue, 30 Jul 2024 22:27:09 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IZzdCxbe; spf=pass (imf20.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722378374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=cfkeNSLPTC1ikCyuKHlEYPZ+QWefpm7U5tn/Vpm9NRk=; b=DiF1roNnxpUEqzc+9UkYITVGNjhaYEzcft9vsEe+VdxCR6h7fVGJ56oW/7uRBGOMXEuECY 5DtZ1PhzFV9zBaCzxLcUvurpggF7XdAPuY4xprpOPAQg3tqufE6iAzG9gAr1uJJWJGXlxT G8FN0XXVdRFbHtKADFvamExzDkNHQ5E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722378374; a=rsa-sha256; cv=none; b=4YcVVx0NxTYj5xoffLMZCUOAXInX0wypvOB6t5+plQjDd8bWfa4/umYnE+2k+u2Fp6+taD vTAErDON/MAj3ahPyLjGv7O7IxoYUSXw916eTXoRjf/gaEGFn0gRtjMLHFvNsl3P0DhJtj cUgh7rHXR2Z5giFI/8QoU43Q+EKwWUk= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IZzdCxbe; spf=pass (imf20.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yw1-f180.google.com with SMTP id 00721157ae682-65f880c56b1so36455167b3.3 for ; Tue, 30 Jul 2024 15:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722378428; x=1722983228; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=cfkeNSLPTC1ikCyuKHlEYPZ+QWefpm7U5tn/Vpm9NRk=; b=IZzdCxbeSOflRoNN6HEzsRcrb1yYkZA6d9Ez+s8JJQoCFaZc7OyIyrrBafQk+510+r YxjMBFFgxcbXjx45Kr8AwRjeIt+6ZkleoX8pIot8gZGN1MzdrSPVlAm/STkKukfF4Z6n eN/SqyV0TiXouDMn8THSvJ+HSOb7p0vHbnUCE+s+szr8Xv39EZ2BkwyvanG3V5otvbZv zqKOwfMQrwyV0jhXWJd3W3ZWCNGiH7p6kQDGQWCS5MolxxAbpnFTHq0g++6jUzkwdEYq ggainoIIWC0GjtJjDcfG7Dx2vumoXIN5fb3Ltfw/3598NCgrYKGGJx7AiM2MclN+nFEj Yo1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722378428; x=1722983228; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cfkeNSLPTC1ikCyuKHlEYPZ+QWefpm7U5tn/Vpm9NRk=; b=KsvMnSHAHZSy38oH/JqVpsGdyE4t9wz7yBL6dafZ+u9cpBIYNIJgg8yQ/76twIVB7d 3m+kYqYPK/WHLTNFl0qIx9euIgZUGvmN4LH+gARbws2tEqa6IB9Y0XyuHtzs3p2+Oxkh v9m4S5vjyEY6cuvBN16mRhk6DXT4MjylEM8vQ/vx+DXBwUwpXy3qvMIS2wOwaZ5s+Sb7 EJRjQ439BK0nZiH6Nd9ewWmRwH2HP23gHgvaYZwEqd+njNKn6vPQv1vEjUXqZBw5VbDq Pvpf5Rzdm6L77ueovsz1zCg4zfxf/fyL2abUCiHcO/xZk4n2Ji2q5wEVhalA62rBATJJ sScg== X-Forwarded-Encrypted: i=1; AJvYcCXh2AEB6e3xOO8TpJVmrqA+c8QqOcRNJl5mdtbTVcM2530c6UU3G85E5RyY6L32WZRBSqOSSChFTYFsGIzaqTDO/+k= X-Gm-Message-State: AOJu0YyLjYrhT9YZapDBez+/TPlxuERbLe5Cw1tyAsxwd5hRqyrUwLsU 67e6k0pojEn8ws0mgf2gG31jE2pGb+LjCN0yWe7Fn6e4xbis6zJU X-Google-Smtp-Source: AGHT+IFaSmxbn8C5jRNFT82wnON6P4Dzy9JZ0JAuF+9TVBsmRxvIL89DPOlLv3cl7kGGiAAQd/UNqA== X-Received: by 2002:a0d:edc6:0:b0:62c:c5e1:ff1a with SMTP id 00721157ae682-67a0a3253f2mr103397147b3.45.1722378428231; Tue, 30 Jul 2024 15:27:08 -0700 (PDT) Received: from localhost (fwdproxy-nha-006.fbsv.net. [2a03:2880:25ff:6::face:b00c]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6756795e78esm27322247b3.43.2024.07.30.15.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jul 2024 15:27:08 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, yosryahmed@google.com, shakeel.butt@linux.dev, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, flintglass@gmail.com, chengming.zhou@linux.dev Subject: [PATCH v2 0/2] improving dynamic zswap shrinker protection scheme Date: Tue, 30 Jul 2024 15:27:05 -0700 Message-ID: <20240730222707.2324536-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Stat-Signature: aakgmke5xb168k7jpogm56e6x7emhhtt X-Rspamd-Queue-Id: 4CB0B1C0038 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722378429-986813 X-HE-Meta: U2FsdGVkX1+aXfFJqoQxSSSznFF7D/3omudtSvrudu1gHRJxyCdSUt5WPQRpDbCBke/eYWLPrYtL1nfyEv5snFowY0NwVVnfNQNTLh0RGi9VDDURrH4GMrm307PFth9aMwdiSmwMvG+x9m6m4ITFtfZ/hzlUtPjKzva0Zd+7/FL5zm/ObEVnX73ja7Sx3DkUfis7zhQ77ZVUHMF5XtUUBEDhrItrGGD9dks7fRDX/+Fzpeyx8s1zjxS1h5glpsphYKcECQqw0ie93VeFGkdnwWy3dmF+bBd/DIv8CqGIO/pEAwhS8tHT9ujZoMlNeIn5y35KB0CR+Lp4v832DvcxYM6VQlKl754hkeoR/4XlZOMOOkDgQEdCvdcQkt4iiz6gjvPnZSKm/gdFqNlUz2cAI5KudcK0iGTsqv5zH7Zqg3ItFfnz7zVltWt2y9+jjlJqFiMv72AT0MAx/bI/myrb9CI6tNRFE/3Gwd0iZ9nU4cUtMSBSc7dMOKXIJTxhUrDj+3w9XcxhqKdnNzP85zAN3SNF/8rOCwxtnzc0ANCxABsjqYog3VefTyi6p803imE1Doa6Imy+xq+Tuj+6c2zOOWFg5ilBk7tILfSRiIvPQHgJFnabpzcCIIpuHyPUMVl9R4jd5/7VFjUzpHbKGB1rCvvoPYBBN8as6QyLg0430JE5SADt/a4yyg+UkWTEe3y3jO9aAks7MkDS/ptxTPNbmL6wszkKekWLgbcdUpWmR9LxTBJPc87xwdLlkAiLQzh2SNnHfhLleNZwwNfmYCWe3+Xmt7BNkxcSN4EN48JTGhad1+D2dO3EOKY900MTS1kJy9hbp+w1kTxy1qIcMFghbFoW7i4UiaX6TQZuZeCTyeuwkqTZlr4XRK2KgTWLoVjW2KMIhDp/jyjclplU7ognPvk5uAfweQ3QyVOURQJ42p/l3ZyOf47GmxQpdJuJbcTabrzjd28SbEDWjNA9dGs t6f/JKCV 9+FkjUXVkjbzhzs5KcpwE67GpN5UO83pIF9Mufhsf1n23mc9tReq4+0AjvPom1K4wN1JlR/DcBDYFnCrj9Hj3ogePrI8EbS4fQaLWlHXm4D/Je2/g//vwyLLCqftesDUj5oDheYUQRtkvH/fZVajm7nYlPCvICiN7UEkXaT0ed9RAkuHGc57F8i6UxJsBGaHWP9gCwh8CTKc268aLLMuHTq36g+4aVLKghms/wSG81snhKM4htrjSrR+n48/8ej++pVFNrVAj02ouUOpi03kNZQPviQVN8CeuxSevz4FikGRBwm5HAqAwG0q/5vy+4a1HQ9pZO9TaVvdS01jTNZOBhmXS97Thrjv/bUr6OSt/czp3o/yhHS0fPHmN7IGH8XHReimiGyuLjNCMJcQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.086644, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: v2: * Add more details in comments, patch changelog, documentation, etc. about the second chance scheme and its ability to modulate the writeback rate (patch 1) (suggested by Yosry Ahmed). * Move the referenced bit (patch 1) (suggested by Yosry Ahmed). When experimenting with the memory-pressure based (i.e "dynamic") zswap shrinker in production, we observed a sharp increase in the number of swapins, which led to performance regression. We were able to trace this regression to the following problems with the shrinker's warm pages protection scheme: 1. The protection decays way too rapidly, and the decaying is coupled with zswap stores, leading to anomalous patterns, in which a small batch of zswap stores effectively erase all the protection in place for the warmer pages in the zswap LRU. This observation has also been corroborated upstream by Takero Funaki (in [1]). 2. We inaccurately track the number of swapped in pages, missing the non-pivot pages that are part of the readahead window, while counting the pages that are found in the zswap pool. To alleviate these two issues, this patch series improve the dynamic zswap shrinker in the following manner: 1. Replace the protection size tracking scheme with a second chance algorithm. This new scheme removes the need for haphazard stats decaying, and automatically adjusts the pace of pages aging with memory pressure, and writeback rate with pool activities: slowing down when the pool is dominated with zswpouts, and speeding up when the pool is dominated with stale entries. 2. Fix the tracking of the number of swapins to take into account non-pivot pages in the readahead window. With these two changes in place, in a kernel-building benchmark without any cold data added, the number of swapins is reduced by 64.12%. This translate to a 10.32% reduction in build time. We also observe a 3% reduction in kernel CPU time. In another benchmark, with cold data added (to gauge the new algorithm's ability to offload cold data), the new second chance scheme outperforms the old protection scheme by around 0.7%, and actually written back around 21% more pages to backing swap device. So the new scheme is just as good, if not even better than the old scheme on this front as well. [1]: https://lore.kernel.org/linux-mm/CAPpodddcGsK=0Xczfuk8usgZ47xeyf4ZjiofdT+ujiyz6V2pFQ@mail.gmail.com/ Nhat Pham (2): zswap: implement a second chance algorithm for dynamic zswap shrinker zswap: increment swapin count for non-pivot swapped in pages include/linux/zswap.h | 16 +++--- mm/page_io.c | 11 ++++- mm/swap_state.c | 8 +-- mm/zswap.c | 110 ++++++++++++++++++++++++------------------ 4 files changed, 82 insertions(+), 63 deletions(-) base-commit: cca1345bd26a67fc61a92ff0c6d81766c259e522