From patchwork Thu Sep 13 21:41:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timofey Titovets X-Patchwork-Id: 10600131 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1168E112B for ; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B3D52B511 for ; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F1D112B515; Thu, 13 Sep 2018 21:41:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98EA22B511 for ; Thu, 13 Sep 2018 21:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728166AbeINCwe (ORCPT ); Thu, 13 Sep 2018 22:52:34 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:41495 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726914AbeINCwd (ORCPT ); Thu, 13 Sep 2018 22:52:33 -0400 Received: by mail-wr1-f68.google.com with SMTP id z96-v6so8234841wrb.8 for ; Thu, 13 Sep 2018 14:41:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3x/vmeOQFQqTYre8gpQTbELvaL++cjvhCa8l4tMQ/lo=; b=k5CtG2WbN/Oo3JLHcXIpLfJmsttOvwMxZhZwhOfFsguVxMlD3J3y+wYM970BeH8aWt NXKkQyWWynh4eBWzOux4L9yPsZjzm4r2fs9MRgwZ05dhmAk2k+usMtFcd5wwN2SzrJ89 XtMNin72C461DdkX71mYQilZM92RFikPngta0JXOcOv/O4M9Rk9hAXK2YHw1l3ouvtxE /b0VVTsC4JDlK0taUGtteH6OaRQt0OTP6fUkTaS6T091bqjjP1NTm/jaSSfWRO1ObqAj H09J0kLc+iEUtNnq2Yzds2MndexQoX5v1UI3PY+Ie1BTmw+1ecHV/AvbyG5f2KPAWS2k M/gg== X-Gm-Message-State: APzg51BfzDYBKFPBjkCmROpf2CGZ08hoVjWNAsH4VMCVyzBBprHWfg6M WoIiv0IZcCx8rSTcxOEe8Vw= X-Google-Smtp-Source: ANB0VdaJy75U23ICHnTnslRsfvVP9NP5GchkUrgvRiIYpkuOsX8deGl6TQK3J709FFpJ7on5z6ludA== X-Received: by 2002:a1c:99c2:: with SMTP id b185-v6mr7198217wme.15.1536874873118; Thu, 13 Sep 2018 14:41:13 -0700 (PDT) Received: from TitovetsT.synesis.local ([178.121.227.155]) by smtp.gmail.com with ESMTPSA id j66-v6sm7644411wrj.28.2018.09.13.14.41.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 14:41:12 -0700 (PDT) From: Timofey Titovets To: linux-mm@kvack.org Cc: rppt@linux.vnet.ibm.com, Timofey Titovets , Andrea Arcangeli , kvm@vger.kernel.org, leesioh Subject: [PATCH V8 1/2] xxHash: create arch dependent 32/64-bit xxhash() Date: Fri, 14 Sep 2018 00:41:01 +0300 Message-Id: <20180913214102.28269-2-timofey.titovets@synesis.ru> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20180913214102.28269-1-timofey.titovets@synesis.ru> References: <20180913214102.28269-1-timofey.titovets@synesis.ru> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Timofey Titovets xxh32() - fast on both 32/64-bit platforms xxh64() - fast only on 64-bit platform Create xxhash() which will pickup fastest version on compile time. As result depends on cpu word size, the main proporse of that - in memory hashing. Changes: v2: - Create that patch v3 -> v8: - Nothing, whole patchset version bump Signed-off-by: Timofey Titovets Reviewed-by: Pavel Tatashin CC: Andrea Arcangeli CC: linux-mm@kvack.org CC: kvm@vger.kernel.org CC: leesioh Reviewed-by: Mike Rapoport --- include/linux/xxhash.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/include/linux/xxhash.h b/include/linux/xxhash.h index 9e1f42cb57e9..52b073fea17f 100644 --- a/include/linux/xxhash.h +++ b/include/linux/xxhash.h @@ -107,6 +107,29 @@ uint32_t xxh32(const void *input, size_t length, uint32_t seed); */ uint64_t xxh64(const void *input, size_t length, uint64_t seed); +/** + * xxhash() - calculate wordsize hash of the input with a given seed + * @input: The data to hash. + * @length: The length of the data to hash. + * @seed: The seed can be used to alter the result predictably. + * + * If the hash does not need to be comparable between machines with + * different word sizes, this function will call whichever of xxh32() + * or xxh64() is faster. + * + * Return: wordsize hash of the data. + */ + +static inline unsigned long xxhash(const void *input, size_t length, + uint64_t seed) +{ +#if BITS_PER_LONG == 64 + return xxh64(input, length, seed); +#else + return xxh32(input, length, seed); +#endif +} + /*-**************************** * Streaming Hash Functions *****************************/ From patchwork Thu Sep 13 21:41:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timofey Titovets X-Patchwork-Id: 10600133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9BCA17D5 for ; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A51062B511 for ; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 98D3B2B515; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27A792B511 for ; Thu, 13 Sep 2018 21:41:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728174AbeINCwe (ORCPT ); Thu, 13 Sep 2018 22:52:34 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:34027 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726824AbeINCwe (ORCPT ); Thu, 13 Sep 2018 22:52:34 -0400 Received: by mail-wr1-f67.google.com with SMTP id g33-v6so8266139wrd.1 for ; Thu, 13 Sep 2018 14:41:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xsJk5RY8DkMZyyLYnBA/RCWoSGav79Bq9FXIAsC9xEw=; b=NE3RtbkmUpfq526aT1iH/+HwEA0F2/c2kzgzuIqDxWBUh2slNiQzWtBTuLrgemxDUK sey3E/EQg5dn9WDqfKFKRjk4HYqxfuBrKYrng8UXaKPOs5mxqM1POLZV+dL43pyj7xeJ KYCQ/FotJ9arHhug3HtP2vdW6T0AxSNXPFAWui1cTIjXfg6YlXqw0NedeIIe51WUyx77 /a8tTz9aTsfEvXVyCDDbqmdS9tVwHP+F68bv2OEQa4vZlyOX+3b0DTa29T2UVI/6bF4s GbbRQcII5/jqUz7WTqoAe6XaWpoyO7YoNNVP6u27llj/NHo1lDDr0W00Ctq5fi96koml iC5Q== X-Gm-Message-State: APzg51DalA9MU+WgHt5IzZzNlZj4V+z+fm58w5sY4YjrDkaKlZElUrhX +zTC8KN/V7DeSOEGB/Q+cFg= X-Google-Smtp-Source: ANB0VdYMm4n1F976+ukPKAboWLgLAhmyHQedrh4olshABw0PN0hO3yXOm8m4I6Lm3NtltM7U+SNy/w== X-Received: by 2002:a5d:4089:: with SMTP id o9-v6mr7346459wrp.133.1536874874092; Thu, 13 Sep 2018 14:41:14 -0700 (PDT) Received: from TitovetsT.synesis.local ([178.121.227.155]) by smtp.gmail.com with ESMTPSA id j66-v6sm7644411wrj.28.2018.09.13.14.41.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 14:41:13 -0700 (PDT) From: Timofey Titovets To: linux-mm@kvack.org Cc: rppt@linux.vnet.ibm.com, Timofey Titovets , leesioh , Andrea Arcangeli , kvm@vger.kernel.org Subject: [PATCH V8 2/2] ksm: replace jhash2 with xxhash Date: Fri, 14 Sep 2018 00:41:02 +0300 Message-Id: <20180913214102.28269-3-timofey.titovets@synesis.ru> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20180913214102.28269-1-timofey.titovets@synesis.ru> References: <20180913214102.28269-1-timofey.titovets@synesis.ru> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Timofey Titovets Replace jhash2 with xxhash. Perf numbers: Intel(R) Xeon(R) CPU E5-2420 v2 @ 2.20GHz ksm: crc32c hash() 12081 MB/s ksm: xxh64 hash() 8770 MB/s ksm: xxh32 hash() 4529 MB/s ksm: jhash2 hash() 1569 MB/s From Sioh Lee: crc32c_intel: 1084.10ns crc32c (no hardware acceleration): 7012.51ns xxhash32: 2227.75ns xxhash64: 1413.16ns jhash2: 5128.30ns As jhash2 always will be slower (for data size like PAGE_SIZE). Don't use it in ksm at all. Use only xxhash for now, because for using crc32c, cryptoapi must be initialized first - that require some tricky solution to work good in all situations. Thanks. Changes: v1 -> v2: - Move xxhash() to xxhash.h/c and separate patches v2 -> v3: - Move xxhash() xxhash.c -> xxhash.h - replace xxhash_t with 'unsigned long' - update kerneldoc above xxhash() v3 -> v4: - Merge xxhash/crc32 patches - Replace crc32 with crc32c (crc32 have same as jhash2 speed) - Add auto speed test and auto choice of fastest hash function v4 -> v5: - Pickup missed xxhash patch - Update code with compile time choicen xxhash - Add more macros to make code more readable - As now that only possible use xxhash or crc32c, on crc32c allocation error, skip speed test and fallback to xxhash - For workaround too early init problem (crc32c not avaliable), move zero_checksum init to first call of fastcall() - Don't alloc page for hash testing, use arch zero pages for that v5 -> v6: - Use libcrc32c instead of CRYPTO API, mainly for code/Kconfig deps Simplification - Add crc32c_available(): libcrc32c will BUG_ON on crc32c problems, so test crc32c avaliable by crc32c_available() - Simplify choice_fastest_hash() - Simplify fasthash() - struct rmap_item && stable_node have sizeof == 64 on x86_64, that makes them cache friendly. As we don't suffer from hash collisions, change hash type from unsigned long back to u32. - Fix kbuild robot warning, make all local functions static v6 -> v7: - Drop crc32c for now and use only xxhash in ksm. v7 -> v8: - Remove empty line changes Signed-off-by: Timofey Titovets Signed-off-by: leesioh Reviewed-by: Pavel Tatashin CC: Andrea Arcangeli CC: linux-mm@kvack.org CC: kvm@vger.kernel.org Reviewed-by: Mike Rapoport --- mm/Kconfig | 1 + mm/ksm.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index a550635ea5c3..b5f923081bce 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -297,6 +297,7 @@ config MMU_NOTIFIER config KSM bool "Enable KSM for page merging" depends on MMU + select XXHASH help Enable Kernel Samepage Merging: KSM periodically scans those areas of an application's address space that an app has advised may be diff --git a/mm/ksm.c b/mm/ksm.c index 5b0894b45ee5..1a088306ef81 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -1009,7 +1009,7 @@ static u32 calc_checksum(struct page *page) { u32 checksum; void *addr = kmap_atomic(page); - checksum = jhash2(addr, PAGE_SIZE / 4, 17); + checksum = xxhash(addr, PAGE_SIZE, 0); kunmap_atomic(addr); return checksum; }