From patchwork Fri Apr 17 02:03:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bernard Zhao X-Patchwork-Id: 11494013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB5036CA for ; Fri, 17 Apr 2020 02:03:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8984221D93 for ; Fri, 17 Apr 2020 02:03:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8984221D93 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=vivo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9D87D8E0003; Thu, 16 Apr 2020 22:03:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 961528E0001; Thu, 16 Apr 2020 22:03:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 828C98E0003; Thu, 16 Apr 2020 22:03:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 60B648E0001 for ; Thu, 16 Apr 2020 22:03:43 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1BA34180AD82F for ; Fri, 17 Apr 2020 02:03:43 +0000 (UTC) X-FDA: 76715700726.17.mist31_3268c6b99e34 X-Spam-Summary: 2,0,0,58eaff4a540aeb57,d41d8cd98f00b204,bernard@vivo.com,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1345:1437:1535:1544:1605:1711:1730:1747:1777:1792:1801:2198:2199:2380:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3874:4321:4605:5007:6119:6261:7875:7903:8603:9121:10004:10026:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12555:12679:12683:12895:12986:13548:14181:14394:14721:21080:21451:21611:21627:21740:21939:21990:30012:30054:30085:30090,0,RBL:59.111.176.13:@vivo.com:.lbl8.mailshell.net-62.14.2.100 66.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: mist31_3268c6b99e34 X-Filterd-Recvd-Size: 5914 Received: from mail-m17613.qiye.163.com (mail-m17613.qiye.163.com [59.111.176.13]) by imf43.hostedemail.com (Postfix) with SMTP for ; Fri, 17 Apr 2020 02:03:41 +0000 (UTC) Received: from ubuntu.localdomain (unknown [157.0.31.122]) by mail-m17613.qiye.163.com (Hmail) with ESMTPA id 3F349482635; Fri, 17 Apr 2020 10:03:38 +0800 (CST) From: Bernard Zhao To: Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: kernel@vivo.com, Bernard Zhao Subject: [PATCH] kmalloc_index optimization(code size & runtime stable) Date: Thu, 16 Apr 2020 19:03:30 -0700 Message-Id: <1587089010-110083-1-git-send-email-bernard@vivo.com> X-Mailer: git-send-email 2.7.4 X-HM-Spam-Status: e1kfGhgUHx5ZQUtXWQgYFAkeWUFZT1VNQ0JLS0tKSUJNQkJMSllXWShZQU hPN1dZLVlBSVdZCQ4XHghZQVk1NCk2OjckKS43PlkG X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6OjI6Ajo4ETg9PA8VTjoXGhMX IyoaFE9VSlVKTkNMS0NCS0pDTklLVTMWGhIXVRkeCRUaCR87DRINFFUYFBZFWVdZEgtZQVlKTkxV S1VISlVKSUlZV1kIAVlBTk1MQzcG X-HM-Tid: 0a7185e08e2993bakuws3f349482635 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kmalloc_index inline function code size optimization and runtime performance stability optimization. After optimization, the function kmalloc_index is more stable, the size will never affecte the function`s execution efficiency. And follow test data shows that the performance of new optimization exceeds the original algorithm when applying for more than 512 Bytes (include 512B).And new optimization runtime is more stable than before. Test platform:install vmware ubuntu 16.04, ram 2G, cpu 1, i5-8500 3.00GHz Compiler: gcc -O2 optimization, gcc version 5.4.0. Just test diff code part. Follow is detailed test data: size time/Per 100 million times old fun new fun with optimise 8 203777 241934 16 245611 409278 32 236384 408419 64 275499 447732 128 354909 416439 256 360472 406598 512 431072 409168 1024 463822 407401 2 * 1024 548519 407710 4 * 1024 623378 422326 8 * 1024 655932 407457 16 * 1024 744673 417574 32 * 1024 824889 415316 64 * 1024 854374 408577 128 * 1024 968079 433582 256 * 1024 985527 412080 512 * 1024 1196877 448199 1024 * 1024 1310315 448969 2 * 1024 * 1024 1367441 513117 4 * 1024 * 1024 1264623 415019 8 * 1024 * 1024 1255727 417197 16 * 1024 * 1024 1401431 411087 32 * 1024 * 1024 1440415 416616 64 * 1024 * 1024 1428122 417459 Signed-off-by: Bernard Zhao Reported-by: kbuild test robot Reported-by: kbuild test robot Reported-by: kbuild test robot --- include/linux/slab.h | 61 +++++++++++++++++++++++++++++++--------------------- 1 file changed, 37 insertions(+), 24 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 6d45488..6ccee7a 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -301,6 +301,22 @@ static inline void __check_heap_object(const void *ptr, unsigned long n, #define SLAB_OBJ_MIN_SIZE (KMALLOC_MIN_SIZE < 16 ? \ (KMALLOC_MIN_SIZE) : 16) +#ifndef CONFIG_SLOB +/* + * This used to show the relation between size`s last (most-significant) + * bit set & index of kmalloc_info[] + * If size%2 ==0, then fls - 1, else fls(round up) + * size 8(b 1000)-(b 1xxx)-16(b 10000)-(b 1xxxx)-32(b 100000)-(b 1xxxxx) + * | | | | | | + * index 3 4 4 5 5 6 + * 64(b 1000000)-(b 1xxxxxx)-128(b 10000000)-(b 1xxxxxxx)-256.... + * | | | | | + * 6 7 7 8 8... + */ +#define KMALLOC_SIZE_POW_2_SHIFT_BIT (2) +#define KMALLOC_SIZE_POW_2_INDEX_BIT (1) +#endif + /* * Whenever changing this, take care of that kmalloc_type() and * create_kmalloc_caches() still work as intended. @@ -358,30 +374,27 @@ static __always_inline unsigned int kmalloc_index(size_t size) return 1; if (KMALLOC_MIN_SIZE <= 64 && size > 128 && size <= 192) return 2; - if (size <= 8) return 3; - if (size <= 16) return 4; - if (size <= 32) return 5; - if (size <= 64) return 6; - if (size <= 128) return 7; - if (size <= 256) return 8; - if (size <= 512) return 9; - if (size <= 1024) return 10; - if (size <= 2 * 1024) return 11; - if (size <= 4 * 1024) return 12; - if (size <= 8 * 1024) return 13; - if (size <= 16 * 1024) return 14; - if (size <= 32 * 1024) return 15; - if (size <= 64 * 1024) return 16; - if (size <= 128 * 1024) return 17; - if (size <= 256 * 1024) return 18; - if (size <= 512 * 1024) return 19; - if (size <= 1024 * 1024) return 20; - if (size <= 2 * 1024 * 1024) return 21; - if (size <= 4 * 1024 * 1024) return 22; - if (size <= 8 * 1024 * 1024) return 23; - if (size <= 16 * 1024 * 1024) return 24; - if (size <= 32 * 1024 * 1024) return 25; - if (size <= 64 * 1024 * 1024) return 26; + + if (size <= 8) + return 3; + + /* size over KMALLOC_MAX_SIZE should trigger BUG */ + if (size <= KMALLOC_MAX_SIZE) { + /* + * kmalloc_info[index] + * size 8----16----32----64----128---256---512---1024---2048. + * | | | | | | | | | | | | | | | | | + * index 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 + */ + + high_bit = fls((int)size); + + if (size == (2 << (high_bit - KMALLOC_SIZE_POW_2_SHIFT_BIT))) + return (high_bit - KMALLOC_SIZE_POW_2_INDEX_BIT); + + return high_bit; + } + BUG(); /* Will never be reached. Needed because the compiler may complain */