From patchwork Mon Nov 12 15:41:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA1AF13BF for ; Mon, 12 Nov 2018 15:41:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C81EE2A07E for ; Mon, 12 Nov 2018 15:41:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC7BB2A0A2; Mon, 12 Nov 2018 15:41:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 468AD2A098 for ; Mon, 12 Nov 2018 15:41:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49ADD6B0292; Mon, 12 Nov 2018 10:41:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 44A136B0294; Mon, 12 Nov 2018 10:41:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33A8F6B0295; Mon, 12 Nov 2018 10:41:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id 03C756B0292 for ; Mon, 12 Nov 2018 10:41:37 -0500 (EST) Received: by mail-qk1-f199.google.com with SMTP id s19so24637431qke.20 for ; Mon, 12 Nov 2018 07:41:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=nMfZ9RiIK02lAYL+37T5dlgwkmxWF9Yw93NdTM8F2A8=; b=epJQYFGzu/wW/RRa8oHj7HMsb6jfVKYKRx0uADcKfbbd8didF7ij20aQmYbN/C+kjJ 9IyVPV83Tm2mevGUv9UqsQni2lWOw21lkJJN9ypzPF5ehgHjJVvAkCV9kU2PbmV9P3m0 8HGb8fwT6jpQ8edICMARGbgWevz1x5BEBQqvmeUqJLc58fWlUhMyWy0DqW3t3Dr4SVpT 0ytUwrRNfbIK1ZzDo7MfZmhUqwylyZDl5lP7dQfXKv6zsS2Dj5HHjwxscvn531yTEL6j KwIeiwCehVK+f+FKO+54e7gDalIMRlKKW/zQuWWkWRnL+yyAHrVQmDqM9F/5SXPfI42c JxVQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gKGWB8JP2uSPG6eaZWvh6ws+vcLDQIdcg+GFVHhf7rlHDJ+jCkw r8UHy2GoZPF0y0fBwZurOVpMStxV62tI0bpi2uYuymSECPnGstyrckq9tkH6LfA01UAJ76I6rsr ta8sOyrAe/SyGyZdPvEyMgl9gdpH6YheenRG4BStvSBSKrAW0y/Jwo1adyWfOSfCsEw== X-Received: by 2002:ac8:7950:: with SMTP id r16mr1410850qtt.12.1542037296782; Mon, 12 Nov 2018 07:41:36 -0800 (PST) X-Google-Smtp-Source: AJdET5fHNHoz+14SwPHj3WE70vX/vCKYLg2SMNfyu8gqbdN7uqh/7spxPGcRTAc5AOkUY31KjH6+ X-Received: by 2002:ac8:7950:: with SMTP id r16mr1410806qtt.12.1542037296174; Mon, 12 Nov 2018 07:41:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037296; cv=none; d=google.com; s=arc-20160816; b=tTu04XhKwUsKSTOmOooburUXaEKDANCFDoreTBK/EjSjnDFo9VbnMkUcBL5r+4bslV GoLDojzKXlFXY1VOi5MnWIWkU0fqZJ+2z+C8XEywVu0WUnodmWU8fAqUTdtFOPkVRZ7S uqlSIXt66tIgS2F227MzSLIomDxbWwxpbpmLHMR9+ILUyUORgZUomT77ZzLNoCax3To2 rgMhsAmO59eKULGQy5o/4xPjFtuyboiLF+0buCQopX1vzqv/2saCOrhv3UwDGSLiGuKt Px9BNWRkiLBGg49EcOcW2jTHh6j/pl5YK+BKSBUIDfJd81opFSc40v1QVlmFqcmFWjIG HwTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=nMfZ9RiIK02lAYL+37T5dlgwkmxWF9Yw93NdTM8F2A8=; b=L1JI5TJqqoPOt9ongFNe6MbpPMVh7q/PyVBPwIHHxxpe83mIAMFBf85cq4Gg+hJuFm Cx+F3vnYfEXh3YQflJX3gPUVFjkDhMT8CLl+aIi4kXrakLsbIP/5oGCPBVF4lon59Tli 8RmHkvm7U7WAHe9uzMntnBg/+iGhue34VWEAdswPJi1GiUrhzjnB6Ty1L3pLPMeK3ZMZ L6Agu83xvYjZZ4ued5d3Y1MDn/g633+mpRfY0tTufrpkZmlMQ30l20cHqNvIgY2Htv0y JfgICj2BbjgTj6FYIkkU1enlp3iq3tMPeCo60K3pJSkQhzvTawQjsK1Goc4dFElvmOBj 3amw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id f196si1428932qka.61.2018.11.12.07.41.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:41:36 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037294-0fb3b01fb3add4a0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id x0wO1xpBp7wEfKEO (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:41:34 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529337; Mon, 12 Nov 2018 10:41:34 -0500 From: Tony Battersby Subject: [PATCH v4 1/9] dmapool: fix boundary comparison To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , "iommu@lists.linux-foundation.org" , linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 1/9] dmapool: fix boundary comparison Cc: "linux-scsi@vger.kernel.org" Message-ID: Date: Mon, 12 Nov 2018 10:41:34 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037294 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 1451 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Fix the boundary comparison when constructing the list of free blocks for the case that 'size' is a power of two. Since 'boundary' is also a power of two, that would make 'boundary' a multiple of 'size', in which case a single block would never cross the boundary. This bug would cause some of the allocated memory to be wasted (but not leaked). Example: size = 512 boundary = 2048 allocation = 4096 Address range 0 - 511 512 - 1023 1024 - 1535 1536 - 2047 * 2048 - 2559 2560 - 3071 3072 - 3583 3584 - 4095 * Prior to this fix, the address ranges marked with "*" would not have been used even though they didn't cross the given boundary. Fixes: e34f44b3517f ("pool: Improve memory usage for devices which can't cross boundaries") Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- Even though I described this as a "fix", it does not seem important enough to Cc: stable from a strict reading of the stable kernel rules. IOW, it is not "bothering" anyone. --- linux/mm/dmapool.c.orig 2018-08-01 17:57:04.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-01 17:57:16.000000000 -0400 @@ -210,7 +210,7 @@ static void pool_initialise_page(struct do { unsigned int next = offset + pool->size; - if (unlikely((next + pool->size) >= next_boundary)) { + if (unlikely((next + pool->size) > next_boundary)) { next = next_boundary; next_boundary += pool->boundary; } From patchwork Mon Nov 12 15:42:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F103B14E2 for ; Mon, 12 Nov 2018 15:42:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFBC12A074 for ; Mon, 12 Nov 2018 15:42:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD8A22A0AE; Mon, 12 Nov 2018 15:42:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64E462A074 for ; Mon, 12 Nov 2018 15:42:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BFB66B0294; Mon, 12 Nov 2018 10:42:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 66D086B0296; Mon, 12 Nov 2018 10:42:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55EDE6B0297; Mon, 12 Nov 2018 10:42:16 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 228F66B0294 for ; Mon, 12 Nov 2018 10:42:16 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id f22so24028094qkm.11 for ; Mon, 12 Nov 2018 07:42:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=ZAOGdbB9f7ejNhWfDnLeWaCFbXBV/H1CoaxWk7lHKuc=; b=JzGwK1Vp665GOSxE9bGhYB7iZLPWKy65qwxeiYAgO9UojR3M0I538Ra++Du7AkoBuO jjnOUteU/hYnvh0vNchJM2L5MDNvhiFP4ZEKLll+B2pKYjsiSgxzDQ0p46A/wkkTvT+q XwGjwxLbzvFrUZ9qrujpmqU+C9yMJTQJ63xcZNySHJx2iE4lhtnHZ4w2vEc1QrfymseW lCyA43g0aYkSuF7SnBIMWimzRoNFLoa0gdmgpbp3qzqV/q03Ov57DM6CzE1Nhouv6U8o 7L+53BCgOsRB5e6WXIeB93ACsMcd7OaeXLGJC8hO0tBXIIUksfZ4hUIIZt/4TLGAXr1R +4+Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gKJHgi5eCN9CXIAy6IuIUjVbgLXc1ea2WmI6hvu95Ujq734rCdZ It1RPvj4xPIaEdbsLyAoGPNPzVd0yBd6NwJcAKFg8v/zsC5j7P//1tGOs8oJXggkntvEx7vfM+c hJflDv4h3GjsHuwvBM+6z+2LuNfigq/UiAGJAkbNOq9YyIenVqk2Mg+92UiRoka4zdw== X-Received: by 2002:a0c:91e8:: with SMTP id r37mr1405509qvr.141.1542037335803; Mon, 12 Nov 2018 07:42:15 -0800 (PST) X-Google-Smtp-Source: AJdET5eJfVbOjP7zaiLCw5VCiRw+TUQEVW7soZxR+b6PUcTc6E16QmhnWV7nPAH9b6S9qw/1rEqc X-Received: by 2002:a0c:91e8:: with SMTP id r37mr1405472qvr.141.1542037335024; Mon, 12 Nov 2018 07:42:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037334; cv=none; d=google.com; s=arc-20160816; b=rS+8Qy+xMaLLN7uzqI14OJSgoHlDtLbq9rfh8FlTLN6NrxSpGo84O/FNNoB4nkCxzO 7XBGWRLM7/S/Iiimu297lo8Ew5QpVAuRNU4v4rU3rY4gzcqUgt39NPvsQmwY1VzaqZKT pxx1MYnVv7ZOw51WoGoq7qA/I4ZjBn1+rIl5W4du13n8rfV9cayTjeNsRUrJoO1yOPpt kd3aQPHwabJdckn9jjj752wsY5mx10XKPCAOj2pCTTtQYZ/4o5J0oc3eyY/jSyKtgiw2 aLVrHrT6DZ5YhBpUIL2qoUl6gIJEZw855ys5LAHivs8opLYS8X+zxf0G1uovYxJVF5/j PZQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=ZAOGdbB9f7ejNhWfDnLeWaCFbXBV/H1CoaxWk7lHKuc=; b=bzRm37muU3L5DXYpd5s3OVBYeBtZYBje/ZvHuUj/dzd/gidmAzuJUHxAyBWlbl9DZX 2H7hGq74qKeAzZ0tqGl2gnDjOcV7k7cD0AW+p6l3M3WrpHdr/NfPUueRdAkcP9tL0NPd m/KV4wFu2W0OeBha1ZWu3ZrgnjYA21BfgZGdr+IeMSyawOxFAu0189E2N+bK38z8a9il nwdgf+pjChIXEcVzdAGjL2/MmQSsd1smItaYphjwqnd4R9iZlmGsXjCcX6JN5z3j6ch7 tOpj9J0Pu4WevkcTT3Vx/ABrut96jIcUEU7jA1o1mXUihZDYQ1UcrC7/dyne4pgyAMKq nEEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id y80si12964580qkg.186.2018.11.12.07.42.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:42:14 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037332-0fb3b01fb3add4f0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id LqfhyaLirQDmIW9f (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:42:13 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529340; Mon, 12 Nov 2018 10:42:12 -0500 From: Tony Battersby Subject: [PATCH v4 2/9] dmapool: remove checks for dev == NULL To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 2/9] dmapool: remove checks for dev == NULL Cc: "linux-scsi@vger.kernel.org" Message-ID: Date: Mon, 12 Nov 2018 10:42:12 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037333 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3468 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dmapool originally tried to support pools without a device because dma_alloc_coherent() supports allocations without a device. But nobody ended up using dma pools without a device, so the current checks in dmapool.c for pool->dev == NULL are both insufficient and causing bloat. Remove them. Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/mm/dmapool.c.orig 2018-08-03 16:12:23.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 16:13:44.000000000 -0400 @@ -277,7 +277,7 @@ void dma_pool_destroy(struct dma_pool *p mutex_lock(&pools_reg_lock); mutex_lock(&pools_lock); list_del(&pool->pools); - if (pool->dev && list_empty(&pool->dev->dma_pools)) + if (list_empty(&pool->dev->dma_pools)) empty = true; mutex_unlock(&pools_lock); if (empty) @@ -289,13 +289,9 @@ void dma_pool_destroy(struct dma_pool *p page = list_entry(pool->page_list.next, struct dma_page, page_list); if (is_page_busy(page)) { - if (pool->dev) - dev_err(pool->dev, - "dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); - else - pr_err("dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); + dev_err(pool->dev, + "dma_pool_destroy %s, %p busy\n", + pool->name, page->vaddr); /* leak the still-in-use consistent memory */ list_del(&page->page_list); kfree(page); @@ -357,13 +353,9 @@ void *dma_pool_alloc(struct dma_pool *po for (i = sizeof(page->offset); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; - if (pool->dev) - dev_err(pool->dev, - "dma_pool_alloc %s, %p (corrupted)\n", - pool->name, retval); - else - pr_err("dma_pool_alloc %s, %p (corrupted)\n", - pool->name, retval); + dev_err(pool->dev, + "dma_pool_alloc %s, %p (corrupted)\n", + pool->name, retval); /* * Dump the first 4 bytes even if they are not @@ -418,13 +410,9 @@ void dma_pool_free(struct dma_pool *pool page = pool_find_page(pool, dma); if (!page) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, - "dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); - else - pr_err("dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); + dev_err(pool->dev, + "dma_pool_free %s, %p/%lx (bad dma)\n", + pool->name, vaddr, (unsigned long)dma); return; } @@ -432,13 +420,9 @@ void dma_pool_free(struct dma_pool *pool #ifdef DMAPOOL_DEBUG if ((dma - page->dma) != offset) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, - "dma_pool_free %s, %p (bad vaddr)/%pad\n", - pool->name, vaddr, &dma); - else - pr_err("dma_pool_free %s, %p (bad vaddr)/%pad\n", - pool->name, vaddr, &dma); + dev_err(pool->dev, + "dma_pool_free %s, %p (bad vaddr)/%pad\n", + pool->name, vaddr, &dma); return; } { @@ -449,12 +433,9 @@ void dma_pool_free(struct dma_pool *pool continue; } spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, "dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); - else - pr_err("dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); + dev_err(pool->dev, + "dma_pool_free %s, dma %pad already free\n", + pool->name, &dma); return; } } From patchwork Mon Nov 12 15:42:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9556114E2 for ; Mon, 12 Nov 2018 15:42:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82E8F2A000 for ; Mon, 12 Nov 2018 15:42:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 731D92A07D; Mon, 12 Nov 2018 15:42:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 555F22A000 for ; Mon, 12 Nov 2018 15:42:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FA7E6B0296; Mon, 12 Nov 2018 10:42:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5D4406B0298; Mon, 12 Nov 2018 10:42:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 498BA6B0299; Mon, 12 Nov 2018 10:42:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 18E5F6B0296 for ; Mon, 12 Nov 2018 10:42:57 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id 92so24498462qkx.19 for ; Mon, 12 Nov 2018 07:42:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=QH1IIg0CaNqSGBNxYjpd2D1uslcpAwDjwG+aYibniNg=; b=k1+KRbloaSs5TcU0OPCBA2zzWzdpdsBe79mL0k15fA8Ry+zoEOoCg7GgCAZGpYHmWi VGanFkpCo+ykmFBjJDQBsPa82wKWafc2DIxVtdzlFMr1cA8IKXjVKj6HTLR0zSZez/Gg EdnObpanFkGpqO5BD9SJRZueIhs1QCylE50vXYs5GeCGg/Vv/+sgat4FLmC43/Sra7Qv LROenc4tk4gretKLJxZtL8sEbSxskxQ5oHaoxa9LoPPv6Rl9g9oObgF+RPfoWBIYH+jS 9iYEhYd7NE9GRFcJHa5FJzBhSJOIhc7F6YPFuC6sj+Oco8cBL/vZu+0iq2mbDc0S2QSu iSbw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gKAZOKtW8ACCtV/IvXkvVAkvwf6VK7nUCevKJqpyRh/0HlJ7LNp it4z+VvHS0G9yX44Lqyb9BteNolr5Qi2OjJgB16VQ++6srBTuY73wDOQeIajC6OZNETu6Z73qZq kruL/T7Eky2ggHossBMjz4O339OyZSfvURITbkgDBM/QOtR3cPOLS8OwDK+anPMCGWA== X-Received: by 2002:a37:781:: with SMTP id 123mr1336267qkh.231.1542037376770; Mon, 12 Nov 2018 07:42:56 -0800 (PST) X-Google-Smtp-Source: AJdET5dAXbrA/nQUjE6Hm/F/Xcg8jjPcXyLHFzHS5AAEGU0WIQ1PDUzC0yqwnWEMEzq4BFYQqjU8 X-Received: by 2002:a37:781:: with SMTP id 123mr1336205qkh.231.1542037375901; Mon, 12 Nov 2018 07:42:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037375; cv=none; d=google.com; s=arc-20160816; b=dE9CKZECNo2HTRyPgYHbkNZz4wOyMlmFdT6SX2+0OqrP8Y/TbPBCpSwy5DGLVwD8rj Y/a147jhmVuEGQu0/k9FmX/A8t6euFUVv8DTQlLvDHe/YoAE0MLTznj5zscAv1kb+OK4 bsy8jCz+4bS7F3oe54xkAQoCTA9Bz9Fdby6Zfcggr7MlyBxHAgs7vA5zU3PbzZp2O64J vCte/e5iF8fDokad3+icFyOOsd5bLxbMigssoVllFoKL177o4ld9OnTQn3dt25EePvDX hDuZ9A19GW+N956bUMGjNBe7DPj8QGBSsvOqw0stBIrEM5Byu0s/fa/BAyLTrjVknZ4Y d+vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=QH1IIg0CaNqSGBNxYjpd2D1uslcpAwDjwG+aYibniNg=; b=Qpxy+D/l2rxGCNYkd5bTLO3Bd/7+MzMihjXEj8kdCZCpAES1LQqRpy09xKoZ14dyaa ZJuyv+aYrrynopDGH+YI43iMdBiieCpcUJSurZ+BEgQ1N+QswFv6FQuzUYe7uQFeublB Cm+vMM6SDBsxDU9pdnXPvr1/xnJtfPp/jRKoZyuJmcu1QRYwiSdbWmYHabt8F6o8qGhg 7+soaBw1QHhCghOm8e2A41Pn9BMJD3+28h29OUulgVuRx7LVqqXMl7l8S63Q+EaT0vCw JpUUvf/7Hk8C4T3Ka0WgHmFWc2ahg08BoLh7wUA8QxTUWAe7eyl8Yi96OQe7Fy0jjw0l K5gg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id d9si3485494qto.61.2018.11.12.07.42.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:42:55 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037368-0fb3b01fb3add520001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id jmZTy47Ba0bTisqy (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:42:48 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529345; Mon, 12 Nov 2018 10:42:48 -0500 From: Tony Battersby Subject: [PATCH v4 3/9] dmapool: cleanup dma_pool_destroy To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 3/9] dmapool: cleanup dma_pool_destroy Cc: linux-scsi@vger.kernel.org Message-ID: <2ff327bb-59f7-5105-0bba-72329cb73154@cybernetics.com> Date: Mon, 12 Nov 2018 10:42:48 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037368 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2073 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Remove a small amount of code duplication between dma_pool_destroy() and pool_free_page() in preparation for adding more code without having to duplicate it. No functional changes. Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/mm/dmapool.c.orig 2018-08-02 09:59:15.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-02 10:01:26.000000000 -0400 @@ -249,13 +249,22 @@ static inline bool is_page_busy(struct d static void pool_free_page(struct dma_pool *pool, struct dma_page *page) { + void *vaddr = page->vaddr; dma_addr_t dma = page->dma; + list_del(&page->page_list); + + if (is_page_busy(page)) { + dev_err(pool->dev, + "dma_pool_destroy %s, %p busy\n", + pool->name, vaddr); + /* leak the still-in-use consistent memory */ + } else { #ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); + memset(vaddr, POOL_POISON_FREED, pool->allocation); #endif - dma_free_coherent(pool->dev, pool->allocation, page->vaddr, dma); - list_del(&page->page_list); + dma_free_coherent(pool->dev, pool->allocation, vaddr, dma); + } kfree(page); } @@ -269,6 +278,7 @@ static void pool_free_page(struct dma_po */ void dma_pool_destroy(struct dma_pool *pool) { + struct dma_page *page; bool empty = false; if (unlikely(!pool)) @@ -284,19 +294,10 @@ void dma_pool_destroy(struct dma_pool *p device_remove_file(pool->dev, &dev_attr_pools); mutex_unlock(&pools_reg_lock); - while (!list_empty(&pool->page_list)) { - struct dma_page *page; - page = list_entry(pool->page_list.next, - struct dma_page, page_list); - if (is_page_busy(page)) { - dev_err(pool->dev, - "dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); - /* leak the still-in-use consistent memory */ - list_del(&page->page_list); - kfree(page); - } else - pool_free_page(pool, page); + while ((page = list_first_entry_or_null(&pool->page_list, + struct dma_page, + page_list))) { + pool_free_page(pool, page); } kfree(pool); From patchwork Mon Nov 12 15:43:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678869 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B44D14E2 for ; Mon, 12 Nov 2018 15:43:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84E3D287F3 for ; Mon, 12 Nov 2018 15:43:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 783FE2A0AE; Mon, 12 Nov 2018 15:43:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA5112A07C for ; Mon, 12 Nov 2018 15:43:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2A676B0298; Mon, 12 Nov 2018 10:43:29 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD9456B029A; Mon, 12 Nov 2018 10:43:29 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B02A6B029B; Mon, 12 Nov 2018 10:43:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 6ADC66B0298 for ; Mon, 12 Nov 2018 10:43:29 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id y83so24255925qka.7 for ; Mon, 12 Nov 2018 07:43:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=ZKosqTz7mNSRhZGnFWHvk6gtxp5FA22N+5H113WFItA=; b=EiNNKGcc6rSBeCN2jv9U4FTM56Lvv36WjcpqA9K2cch+7fyGYFpRWfr/yE9a/xh4jG oY7nhSLL1D6rsVoZmQL8DTftm+64AWe836CMhqiDZptmqeoMHOzuqdBGTDN9x/Dp0Zx3 XjbQFGOXk3VcFe4IIicL5733SF88Vmfyzp0/vs5TMPA7zDUCyLu0YJ4O/4VCn/MeiDc5 8hY9cU59Kt/ZK7gCFauCiWX3AlT5fE8zDdcoP0sgLacJk7ymglgWBkBxne4o7v3Bhg+u ldfmwz1wJ8XNO26EoD97LZ8SGfJy5P3E2hAoARhC7JZWmsfnZAq4RHEy7afpVFhHKTXs Ltrw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gLPUV1GyAtjb1h8m0BeYY4inV14lAHIqy4xtPdBIH7vEglTuteS T8q6emY+5SLRt4tN4L2thfeHbEPCn03mw7pJZ3N8ZUailO7tB7Q9mbST6q6S6tuZTVuTWIL7QXv XhJDpHnuv3/yc1f8UrQZRweCouE7Ex1rkQVppTA1X0F4AaGbTKYSyBY4VVOCj/4zMZw== X-Received: by 2002:aed:3641:: with SMTP id e59mr1381241qtb.59.1542037409170; Mon, 12 Nov 2018 07:43:29 -0800 (PST) X-Google-Smtp-Source: AJdET5elth1o6dU2DjHK482MKdSJIYa3+N0YfwCCfJ/fgmiA1qFH9efQ6og4y6IC79IJAnFxEP+u X-Received: by 2002:aed:3641:: with SMTP id e59mr1381179qtb.59.1542037408198; Mon, 12 Nov 2018 07:43:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037408; cv=none; d=google.com; s=arc-20160816; b=JqYaLLjZblATqrV6qWecdWko/2ofysvKjvQJcFqN8gzTEzMcv/YWh9ivS1REn7/fEK 83k/cDPnG8MRB5nqIq3I91ng7jh2NtOkQmtcpGpQYpaxt7bLSknCIagMQXk7JB/CkkbV ijIn9tQVSUot4ussEO8F7ryqUNuJ+Ptyb5oBxlDVOXpSOVawUJkbl3JMASdHejv2t509 ekpDieEh04FqitCuz15vxDofe+8NKAHuGfYtjDpBRXh9J0LKtNG4rqgUJ9lIZf+tPF2e U+tqkzFdLIuo5d5wO8Refq6+K4uDQhJ4Gmc+WDIA78xHuhbKhNrjb4vxHD2o2HY6XvBF yvnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=ZKosqTz7mNSRhZGnFWHvk6gtxp5FA22N+5H113WFItA=; b=dBheCn81K5hg/IjkOSzMJSEilQ4MKzUdvEIgH1oO3qg9f4/aSQIPeuaykb29l6BIUC 5AhjS8bMcC888SyA5RyUTdhu0xoWbL8ynEwTsT3LpUHfbspVale+zrh+7ELLQ7WgHMPu RCDQf/pJR7Hq2ILVq4vfSfYhNH3e1PDrbG2nIvsjaYi94E+szIIlwjCQehKI1QzZhJpT Rqyg0wbO5Pq5H4ScEGTwbWX9qU9nKnKvdH6WEU9QbWsRO1QBqucCTOYnWSJRnCs/48P1 31CVZEtdlmtbrlu0vA3+Fc5XADrGvBPTdNUt7Mg8p+L07ia50nsMS+wsDLzNvCPcRtar jZxA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id k17si8091222qkh.130.2018.11.12.07.43.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:43:28 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037406-0fb3b01fb3add540001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id xFxYtrZlvdXMpMCa (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:43:26 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529348; Mon, 12 Nov 2018 10:43:26 -0500 From: Tony Battersby Subject: [PATCH v4 4/9] dmapool: improve scalability of dma_pool_alloc To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 4/9] dmapool: improve scalability of dma_pool_alloc Cc: "linux-scsi@vger.kernel.org" Message-ID: Date: Mon, 12 Nov 2018 10:43:25 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037406 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 6584 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dma_pool_alloc() scales poorly when allocating a large number of pages because it does a linear scan of all previously-allocated pages before allocating a new one. Improve its scalability by maintaining a separate list of pages that have free blocks ready to (re)allocate. In big O notation, this improves the algorithm from O(n^2) to O(n). Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/mm/dmapool.c.orig 2018-08-03 16:16:49.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 16:45:33.000000000 -0400 @@ -15,11 +15,16 @@ * Many older drivers still have their own code to do this. * * The current design of this allocator is fairly simple. The pool is - * represented by the 'struct dma_pool' which keeps a doubly-linked list of - * allocated pages. Each page in the page_list is split into blocks of at - * least 'size' bytes. Free blocks are tracked in an unsorted singly-linked - * list of free blocks within the page. Used blocks aren't tracked, but we - * keep a count of how many are currently allocated from each page. + * represented by the 'struct dma_pool'. Each allocated page is split into + * blocks of at least 'size' bytes. Free blocks are tracked in an unsorted + * singly-linked list of free blocks within the page. Used blocks aren't + * tracked, but we keep a count of how many are currently allocated from each + * page. + * + * The pool keeps two doubly-linked list of allocated pages. The 'available' + * list tracks pages that have one or more free blocks, and the 'full' list + * tracks pages that have no free blocks. Pages are moved from one list to + * the other as their blocks are allocated and freed. */ #include @@ -43,7 +48,10 @@ #endif struct dma_pool { /* the pool */ - struct list_head page_list; +#define POOL_FULL_IDX 0 +#define POOL_AVAIL_IDX 1 +#define POOL_MAX_IDX 2 + struct list_head page_list[POOL_MAX_IDX]; spinlock_t lock; size_t size; struct device *dev; @@ -54,7 +62,7 @@ struct dma_pool { /* the pool */ }; struct dma_page { /* cacheable header for 'allocation' bytes */ - struct list_head page_list; + struct list_head dma_list; void *vaddr; dma_addr_t dma; unsigned int in_use; @@ -70,7 +78,6 @@ show_pools(struct device *dev, struct de unsigned temp; unsigned size; char *next; - struct dma_page *page; struct dma_pool *pool; next = buf; @@ -84,11 +91,18 @@ show_pools(struct device *dev, struct de list_for_each_entry(pool, &dev->dma_pools, pools) { unsigned pages = 0; unsigned blocks = 0; + int list_idx; spin_lock_irq(&pool->lock); - list_for_each_entry(page, &pool->page_list, page_list) { - pages++; - blocks += page->in_use; + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + list_for_each_entry(page, + &pool->page_list[list_idx], + dma_list) { + pages++; + blocks += page->in_use; + } } spin_unlock_irq(&pool->lock); @@ -163,7 +177,8 @@ struct dma_pool *dma_pool_create(const c retval->dev = dev; - INIT_LIST_HEAD(&retval->page_list); + INIT_LIST_HEAD(&retval->page_list[POOL_FULL_IDX]); + INIT_LIST_HEAD(&retval->page_list[POOL_AVAIL_IDX]); spin_lock_init(&retval->lock); retval->size = size; retval->boundary = boundary; @@ -252,7 +267,7 @@ static void pool_free_page(struct dma_po void *vaddr = page->vaddr; dma_addr_t dma = page->dma; - list_del(&page->page_list); + list_del(&page->dma_list); if (is_page_busy(page)) { dev_err(pool->dev, @@ -278,8 +293,8 @@ static void pool_free_page(struct dma_po */ void dma_pool_destroy(struct dma_pool *pool) { - struct dma_page *page; bool empty = false; + int list_idx; if (unlikely(!pool)) return; @@ -294,10 +309,15 @@ void dma_pool_destroy(struct dma_pool *p device_remove_file(pool->dev, &dev_attr_pools); mutex_unlock(&pools_reg_lock); - while ((page = list_first_entry_or_null(&pool->page_list, - struct dma_page, - page_list))) { - pool_free_page(pool, page); + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + while ((page = list_first_entry_or_null( + &pool->page_list[list_idx], + struct dma_page, + dma_list))) { + pool_free_page(pool, page); + } } kfree(pool); @@ -325,10 +345,11 @@ void *dma_pool_alloc(struct dma_pool *po might_sleep_if(gfpflags_allow_blocking(mem_flags)); spin_lock_irqsave(&pool->lock, flags); - list_for_each_entry(page, &pool->page_list, page_list) { - if (page->offset < pool->allocation) - goto ready; - } + page = list_first_entry_or_null(&pool->page_list[POOL_AVAIL_IDX], + struct dma_page, + dma_list); + if (page) + goto ready; /* pool_alloc_page() might sleep, so temporarily drop &pool->lock */ spin_unlock_irqrestore(&pool->lock, flags); @@ -339,11 +360,15 @@ void *dma_pool_alloc(struct dma_pool *po spin_lock_irqsave(&pool->lock, flags); - list_add(&page->page_list, &pool->page_list); + list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: page->in_use++; offset = page->offset; page->offset = *(int *)(page->vaddr + offset); + if (page->offset >= pool->allocation) + /* Move page from the "available" list to the "full" list. */ + list_move_tail(&page->dma_list, + &pool->page_list[POOL_FULL_IDX]); retval = offset + page->vaddr; *handle = offset + page->dma; #ifdef DMAPOOL_DEBUG @@ -381,13 +406,19 @@ EXPORT_SYMBOL(dma_pool_alloc); static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) { - struct dma_page *page; + int list_idx; - list_for_each_entry(page, &pool->page_list, page_list) { - if (dma < page->dma) - continue; - if ((dma - page->dma) < pool->allocation) - return page; + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + list_for_each_entry(page, + &pool->page_list[list_idx], + dma_list) { + if (dma < page->dma) + continue; + if ((dma - page->dma) < pool->allocation) + return page; + } } return NULL; } @@ -444,6 +475,9 @@ void dma_pool_free(struct dma_pool *pool #endif page->in_use--; + if (page->offset >= pool->allocation) + /* Move page from the "full" list to the "available" list. */ + list_move(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); *(int *)vaddr = page->offset; page->offset = offset; /* From patchwork Mon Nov 12 15:44:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678871 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1B0414E2 for ; Mon, 12 Nov 2018 15:44:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF02A2929B for ; Mon, 12 Nov 2018 15:44:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D305929857; Mon, 12 Nov 2018 15:44:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54C602929B for ; Mon, 12 Nov 2018 15:44:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7326D6B029A; Mon, 12 Nov 2018 10:44:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6E0876B029C; Mon, 12 Nov 2018 10:44:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D0586B029D; Mon, 12 Nov 2018 10:44:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 350C86B029A for ; Mon, 12 Nov 2018 10:44:05 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id f22so24037888qkm.11 for ; Mon, 12 Nov 2018 07:44:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=gisvINFkS/iGnzT4KigfhtklRZpszQjoE1eqkyrTQiY=; b=MLstLXz3elgUqaE7jxw4aUPdvH5QRIdTXUYRCpfHNMYc6Rb/r5gdrYQR8ef/Q8WfH+ eXk6J3Dlifng+8vZ+2D9BfEd7GUi77sSRs+Ej5eXbBac+k4Qyrr2LE7yB1X4iXS93AoL 25PsXQSegFRanym8naF/P+2XEyJXa04QTpWtWiDlHmekUcSjHerivDpCeWx0EquDpULO wX/MdKPEZ1IlkrMr0MxB+ds8DwQgMiG4xfEGcM5RP6Ho4biTkRBOVp0YP1oPeuGjLxuF GvwgMYPe/EtDwKjshkPKUZv9tnDYjkBG73GLUZmwV4UvUyyPRS1VAnfnh34OHdVMHm58 0lsA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gKysEaV/DnpXbocOAhYCHHe4yuaqoUSo2K1l5rgjcKvS2gC7BoN ww1o78hddPV1POMuwJuSAgLGIX1gB2/k5nu00RYZNBVIoko83ORWkC5WN/eEswfVn9y6Lw5ALOs hDXH2dbpqnt1G1NZ0/tSqUZ6N5pFK9bSI6vYULO0PbH9Qggsm7v9D0paFaDesGEhttQ== X-Received: by 2002:a37:77c5:: with SMTP id s188mr1319846qkc.174.1542037444953; Mon, 12 Nov 2018 07:44:04 -0800 (PST) X-Google-Smtp-Source: AJdET5eN0OcNLOr4rr2SgZkullx6UsVwfExOYLRnna5im9tNdmiOr08sWKyMJy3rtGOkNM+0yd0M X-Received: by 2002:a37:77c5:: with SMTP id s188mr1319806qkc.174.1542037444259; Mon, 12 Nov 2018 07:44:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037444; cv=none; d=google.com; s=arc-20160816; b=T9nmvWLEAJUqjp7PUu3boL/C3a6Rlp28ygqbz+T7E21yrx78MyOb8bTeIVln5qH+M9 PTNU9laLqWFyqqblDn5nzB2dc4DkkIdbtKtOtVobIMjoqxxq5e/d718eesDzmpUB/Qbv jTl6FgRmleaTCMQkCAJZYvoGJ08r0bOVUkARK6kSUjecGRsIjUojHJ9orud/iBMXEYMq bMO7Y9XqqvYmWwTjTGE71+IEdGag320uJmXHpvUpApy0nRWhyye5sk0CQMjpjZFLdYw0 JJoR4kJYB/6Mp1fMqdh+8ffPl/sglsZUAM+grAyMAbcGYZ3MU12+AfEVsQ4UpaBHRLgS zBaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=gisvINFkS/iGnzT4KigfhtklRZpszQjoE1eqkyrTQiY=; b=VHHDfiP5gI4LFj74o7giU9J18JMBSGnRvqVlpSRD2bMJqmzhbI6/PwFdbUbT1AG9wb w0J0TgE/AaGrmTGggsD4NhdisvcZJqeP7auNgDVeBtQiAF+GRTAQfo3Ygi75nI70rLYY ieztshn+jDOZEiV6iSjHq0aSj/oT/UMS13sOch4mo5qLtDd+hGDDQThIITHmfQ1i2wZ3 Hz/erxWDrTpK3/FPNRh/mmmpTE9Ryl03m+Ea6b/iVOaT79z8zEuAg52glSlYHG5T8hCg YZ4ieQa1tvImr1IqipDyothvozhDosv7jnvq8yuMGzAUfyDD52tDOIvx1oRaHF7lC2/2 j+ig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id i21si2088384qtp.305.2018.11.12.07.44.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:44:04 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037442-0fb3b01fb3add560001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id ue3TjEock6lPxDAA (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:44:02 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529355; Mon, 12 Nov 2018 10:44:02 -0500 From: Tony Battersby Subject: [PATCH v4 5/9] dmapool: rename fields in dma_page To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 5/9] dmapool: rename fields in dma_page Cc: "linux-scsi@vger.kernel.org" Message-ID: <4ac76051-74fc-0a70-4d17-7618823d24c3@cybernetics.com> Date: Mon, 12 Nov 2018 10:44:02 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037442 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3346 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Rename fields in 'struct dma_page' in preparation for moving them into 'struct page'. No functional changes. in_use -> dma_in_use offset -> dma_free_off Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/mm/dmapool.c.orig 2018-08-03 17:46:13.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 17:46:24.000000000 -0400 @@ -65,8 +65,8 @@ struct dma_page { /* cacheable header f struct list_head dma_list; void *vaddr; dma_addr_t dma; - unsigned int in_use; - unsigned int offset; + unsigned int dma_in_use; + unsigned int dma_free_off; }; static DEFINE_MUTEX(pools_lock); @@ -101,7 +101,7 @@ show_pools(struct device *dev, struct de &pool->page_list[list_idx], dma_list) { pages++; - blocks += page->in_use; + blocks += page->dma_in_use; } } spin_unlock_irq(&pool->lock); @@ -248,8 +248,8 @@ static struct dma_page *pool_alloc_page( memset(page->vaddr, POOL_POISON_FREED, pool->allocation); #endif pool_initialise_page(pool, page); - page->in_use = 0; - page->offset = 0; + page->dma_in_use = 0; + page->dma_free_off = 0; } else { kfree(page); page = NULL; @@ -259,7 +259,7 @@ static struct dma_page *pool_alloc_page( static inline bool is_page_busy(struct dma_page *page) { - return page->in_use != 0; + return page->dma_in_use != 0; } static void pool_free_page(struct dma_pool *pool, struct dma_page *page) @@ -362,10 +362,10 @@ void *dma_pool_alloc(struct dma_pool *po list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: - page->in_use++; - offset = page->offset; - page->offset = *(int *)(page->vaddr + offset); - if (page->offset >= pool->allocation) + page->dma_in_use++; + offset = page->dma_free_off; + page->dma_free_off = *(int *)(page->vaddr + offset); + if (page->dma_free_off >= pool->allocation) /* Move page from the "available" list to the "full" list. */ list_move_tail(&page->dma_list, &pool->page_list[POOL_FULL_IDX]); @@ -375,8 +375,8 @@ void *dma_pool_alloc(struct dma_pool *po { int i; u8 *data = retval; - /* page->offset is stored in first 4 bytes */ - for (i = sizeof(page->offset); i < pool->size; i++) { + /* page->dma_free_off is stored in first 4 bytes */ + for (i = sizeof(page->dma_free_off); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; dev_err(pool->dev, @@ -458,7 +458,7 @@ void dma_pool_free(struct dma_pool *pool return; } { - unsigned int chain = page->offset; + unsigned int chain = page->dma_free_off; while (chain < pool->allocation) { if (chain != offset) { chain = *(int *)(page->vaddr + chain); @@ -474,12 +474,12 @@ void dma_pool_free(struct dma_pool *pool memset(vaddr, POOL_POISON_FREED, pool->size); #endif - page->in_use--; - if (page->offset >= pool->allocation) + page->dma_in_use--; + if (page->dma_free_off >= pool->allocation) /* Move page from the "full" list to the "available" list. */ list_move(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); - *(int *)vaddr = page->offset; - page->offset = offset; + *(int *)vaddr = page->dma_free_off; + page->dma_free_off = offset; /* * Resist a temptation to do * if (!is_page_busy(page)) pool_free_page(pool, page); From patchwork Mon Nov 12 15:44:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678873 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60B1A13BF for ; Mon, 12 Nov 2018 15:44:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C1AB2929B for ; Mon, 12 Nov 2018 15:44:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D56729875; Mon, 12 Nov 2018 15:44:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78B242929B for ; Mon, 12 Nov 2018 15:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 905B36B029F; Mon, 12 Nov 2018 10:44:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B6436B02A0; Mon, 12 Nov 2018 10:44:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6406B02A1; Mon, 12 Nov 2018 10:44:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 4D1436B029F for ; Mon, 12 Nov 2018 10:44:44 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id d196so24618728qkb.6 for ; Mon, 12 Nov 2018 07:44:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=1toGkssQ5UghREN1GkXPP4w4ERJv1TsbjB0WDnViXMM=; b=Q2S/zI9Zb3cAraTHHzPjIoMTEKCbl/ohPzMtjf6YOHdIzofsERUpLtFLMTbrZTxL+Z pcFp6vOEyCj43FjcTNefUct17M6w79mWhs8fbyVlHlhUTL3FZSecG6xRTOR1xGo3tZZv Z5HFKs1cFxEF9uGt3J5kDZ3YMOzlsctrsgi7UMs6yA++LQZy6lpezXSnoV6sASWWU7nf kGylZ3WPP9+6ltWB7+VLJSpE4HJLZCEv7W532gayYH0W6XYlKLkrZtUkns12PBAHrsQU 2YdGLPyjLUKSaDKLqbrbF9/gD8+Ya+57izluxf9Qb4Et0rGL7fgATG6tvQBC4Eige/8S fd6w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gInuLLXG6eC9lmevV0GMhhOHjuQG7ovfZIzPdONk1VVpPaSlajg roG3OdiPv9kBfCk2CEHxGPG985rMkP4s5kXOYCDwEqoOoWNE2X3pjJ1214GuFWGgfE8tZEWiM+c 1EgxVh3urvU7VH73N+ujZvc164e9xKvrDwqJKc9408DLdPV9w3lW7eZmojPLmogoe7A== X-Received: by 2002:ac8:203:: with SMTP id k3mr1450376qtg.64.1542037484015; Mon, 12 Nov 2018 07:44:44 -0800 (PST) X-Google-Smtp-Source: AJdET5dTmtwAy95gmbBiv7NsgfiGOndL3f8rGPtvfw7TuQuQG/PLxwNYr6wLwVUXtcEuHsCL4yEj X-Received: by 2002:ac8:203:: with SMTP id k3mr1450317qtg.64.1542037482911; Mon, 12 Nov 2018 07:44:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037482; cv=none; d=google.com; s=arc-20160816; b=rCdycy0E4qEtgtgzGHhHR/kPTu/1pJVDUmexDZGG0PynLmqLTFVa8gRhEqhSxMDoEs QSzsKHOv9DGmzLiJbbz0lWl8YUjglBYL1hMjshDv3HR8uyYAOj4Rq5U9YQwVUpAocjwZ XoOzTthjI/as47BJKONawjnu9oZDBNdW+u76TjZ7hZoBhPu56qQwyHzJgCMow2c5YrDE faWOD9/8N5URH3mu6fJKdXJUzE58b8dRiTorAL1LqRTfMH73T/3rqJtyChEXd8ibYvVO yZfWBGjEcIzF3r/ObaO9YLz/SE4Bk6+uav0+Bu67/2hMH2fPkFw/haI3Et3m5akOIDRz aCjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=1toGkssQ5UghREN1GkXPP4w4ERJv1TsbjB0WDnViXMM=; b=tlJt2c8IvWQ24a9WtOmTTOyfVr6w1YeQBGf2u9R+IMQJU1SsI+qfIYEhafITCDsOS1 i42oVm1EyUKtbiSYiMWov6N8RYpE3lG/jpjzEEkgSMf4QIrySvEKN+fyp5xG1yMlmlgN +f4Ccbw2RrLE9yEhBP9mWGrD2qBrK+PT1AyeXmCj75Dog9cxW/VrbyPjRN3WaFaG8jCh ai9Gyy9O5kvVxl1VzJ1VQGNlX9BdlkunHhCbE8n4Sq4wmit8b9PayXe30dcSlSg/af4T UR7XG/nQngHgP14bO1CeLiouxx6dSd30K2AuoJK/2/Y5+YwdJgMvZEeSVqstKeoapQte MvSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id j5si7014859qtd.148.2018.11.12.07.44.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:44:42 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037480-0fb3b01fb3add5b0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id EfGs2cqo6C8ilRIB (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:44:41 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529359; Mon, 12 Nov 2018 10:44:40 -0500 From: Tony Battersby Subject: [PATCH v4 6/9] dmapool: improve scalability of dma_pool_free To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 6/9] dmapool: improve scalability of dma_pool_free Cc: "linux-scsi@vger.kernel.org" Message-ID: <4c3c25ab-5793-c394-9fe4-221b81805536@cybernetics.com> Date: Mon, 12 Nov 2018 10:44:40 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037481 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 8779 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dma_pool_free() scales poorly when the pool contains many pages because pool_find_page() does a linear scan of all allocated pages. Improve its scalability by replacing the linear scan with virt_to_page() and storing dmapool private data directly in 'struct page', thereby eliminating 'struct dma_page'. In big O notation, this improves the algorithm from O(n^2) to O(n) while also reducing memory usage. Thanks to Matthew Wilcox for the suggestion to use struct page. Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/include/linux/mm_types.h.orig 2018-08-01 17:59:46.000000000 -0400 +++ linux/include/linux/mm_types.h 2018-08-01 17:59:56.000000000 -0400 @@ -156,6 +156,12 @@ struct page { unsigned long _zd_pad_1; /* uses mapping */ }; + struct { /* dma_pool pages */ + struct list_head dma_list; + dma_addr_t dma; + unsigned int dma_free_off; + }; + /** @rcu_head: You can use this to free a page by RCU. */ struct rcu_head rcu_head; }; @@ -177,6 +183,8 @@ struct page { unsigned int active; /* SLAB */ int units; /* SLOB */ + + unsigned int dma_in_use; /* dma_pool pages */ }; /* Usage count. *DO NOT USE DIRECTLY*. See page_ref.h */ --- linux/mm/dmapool.c.orig 2018-08-03 17:47:03.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 17:47:22.000000000 -0400 @@ -25,6 +25,10 @@ * list tracks pages that have one or more free blocks, and the 'full' list * tracks pages that have no free blocks. Pages are moved from one list to * the other as their blocks are allocated and freed. + * + * When allocating DMA pages, we use some available space in 'struct page' to + * store data private to dmapool; search 'dma_pool' in the definition of + * 'struct page' for details. */ #include @@ -61,14 +65,6 @@ struct dma_pool { /* the pool */ struct list_head pools; }; -struct dma_page { /* cacheable header for 'allocation' bytes */ - struct list_head dma_list; - void *vaddr; - dma_addr_t dma; - unsigned int dma_in_use; - unsigned int dma_free_off; -}; - static DEFINE_MUTEX(pools_lock); static DEFINE_MUTEX(pools_reg_lock); @@ -95,7 +91,7 @@ show_pools(struct device *dev, struct de spin_lock_irq(&pool->lock); for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; + struct page *page; list_for_each_entry(page, &pool->page_list[list_idx], @@ -218,7 +214,7 @@ struct dma_pool *dma_pool_create(const c } EXPORT_SYMBOL(dma_pool_create); -static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) +static void pool_initialize_free_block_list(struct dma_pool *pool, void *vaddr) { unsigned int offset = 0; unsigned int next_boundary = pool->boundary; @@ -229,47 +225,57 @@ static void pool_initialise_page(struct next = next_boundary; next_boundary += pool->boundary; } - *(int *)(page->vaddr + offset) = next; + *(int *)(vaddr + offset) = next; offset = next; } while (offset < pool->allocation); } -static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) +static struct page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) { - struct dma_page *page; + struct page *page; + dma_addr_t dma; + void *vaddr; - page = kmalloc(sizeof(*page), mem_flags); - if (!page) + vaddr = dma_alloc_coherent(pool->dev, pool->allocation, &dma, + mem_flags); + if (!vaddr) return NULL; - page->vaddr = dma_alloc_coherent(pool->dev, pool->allocation, - &page->dma, mem_flags); - if (page->vaddr) { + #ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); + memset(vaddr, POOL_POISON_FREED, pool->allocation); #endif - pool_initialise_page(pool, page); - page->dma_in_use = 0; - page->dma_free_off = 0; - } else { - kfree(page); - page = NULL; - } + pool_initialize_free_block_list(pool, vaddr); + + page = virt_to_page(vaddr); + page->dma = dma; + page->dma_free_off = 0; + page->dma_in_use = 0; + return page; } -static inline bool is_page_busy(struct dma_page *page) +static inline bool is_page_busy(struct page *page) { return page->dma_in_use != 0; } -static void pool_free_page(struct dma_pool *pool, struct dma_page *page) +static void pool_free_page(struct dma_pool *pool, struct page *page) { - void *vaddr = page->vaddr; + /* Save local copies of some page fields. */ + void *vaddr = page_to_virt(page); + bool busy = is_page_busy(page); dma_addr_t dma = page->dma; list_del(&page->dma_list); - if (is_page_busy(page)) { + /* Clear all the page fields we use. */ + page->dma_list.next = NULL; + page->dma_list.prev = NULL; + page->dma = 0; + page->dma_free_off = 0; + page_mapcount_reset(page); /* clear dma_in_use */ + + if (busy) { dev_err(pool->dev, "dma_pool_destroy %s, %p busy\n", pool->name, vaddr); @@ -280,7 +286,6 @@ static void pool_free_page(struct dma_po #endif dma_free_coherent(pool->dev, pool->allocation, vaddr, dma); } - kfree(page); } /** @@ -310,11 +315,11 @@ void dma_pool_destroy(struct dma_pool *p mutex_unlock(&pools_reg_lock); for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; + struct page *page; while ((page = list_first_entry_or_null( &pool->page_list[list_idx], - struct dma_page, + struct page, dma_list))) { pool_free_page(pool, page); } @@ -338,15 +343,16 @@ void *dma_pool_alloc(struct dma_pool *po dma_addr_t *handle) { unsigned long flags; - struct dma_page *page; + struct page *page; size_t offset; void *retval; + void *vaddr; might_sleep_if(gfpflags_allow_blocking(mem_flags)); spin_lock_irqsave(&pool->lock, flags); page = list_first_entry_or_null(&pool->page_list[POOL_AVAIL_IDX], - struct dma_page, + struct page, dma_list); if (page) goto ready; @@ -362,14 +368,15 @@ void *dma_pool_alloc(struct dma_pool *po list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: + vaddr = page_to_virt(page); page->dma_in_use++; offset = page->dma_free_off; - page->dma_free_off = *(int *)(page->vaddr + offset); + page->dma_free_off = *(int *)(vaddr + offset); if (page->dma_free_off >= pool->allocation) /* Move page from the "available" list to the "full" list. */ list_move_tail(&page->dma_list, &pool->page_list[POOL_FULL_IDX]); - retval = offset + page->vaddr; + retval = offset + vaddr; *handle = offset + page->dma; #ifdef DMAPOOL_DEBUG { @@ -404,25 +411,6 @@ void *dma_pool_alloc(struct dma_pool *po } EXPORT_SYMBOL(dma_pool_alloc); -static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) -{ - int list_idx; - - for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; - - list_for_each_entry(page, - &pool->page_list[list_idx], - dma_list) { - if (dma < page->dma) - continue; - if ((dma - page->dma) < pool->allocation) - return page; - } - } - return NULL; -} - /** * dma_pool_free - put block back into dma pool * @pool: the dma pool holding the block @@ -434,34 +422,35 @@ static struct dma_page *pool_find_page(s */ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) { - struct dma_page *page; + struct page *page; unsigned long flags; unsigned int offset; - spin_lock_irqsave(&pool->lock, flags); - page = pool_find_page(pool, dma); - if (!page) { - spin_unlock_irqrestore(&pool->lock, flags); + if (unlikely(!virt_addr_valid(vaddr))) { dev_err(pool->dev, - "dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); + "dma_pool_free %s, %p (bad vaddr)/%pad\n", + pool->name, vaddr, &dma); return; } - offset = vaddr - page->vaddr; -#ifdef DMAPOOL_DEBUG - if ((dma - page->dma) != offset) { - spin_unlock_irqrestore(&pool->lock, flags); + page = virt_to_page(vaddr); + offset = offset_in_page(vaddr); + + if (unlikely((dma - page->dma) != offset)) { dev_err(pool->dev, - "dma_pool_free %s, %p (bad vaddr)/%pad\n", + "dma_pool_free %s, %p (bad vaddr)/%pad (or bad dma)\n", pool->name, vaddr, &dma); return; } + + spin_lock_irqsave(&pool->lock, flags); +#ifdef DMAPOOL_DEBUG { + void *page_vaddr = vaddr - offset; unsigned int chain = page->dma_free_off; while (chain < pool->allocation) { if (chain != offset) { - chain = *(int *)(page->vaddr + chain); + chain = *(int *)(page_vaddr + chain); continue; } spin_unlock_irqrestore(&pool->lock, flags); From patchwork Mon Nov 12 15:45:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678875 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 883F5109C for ; Mon, 12 Nov 2018 15:45:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 763412929B for ; Mon, 12 Nov 2018 15:45:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A43229875; Mon, 12 Nov 2018 15:45:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCC822929B for ; Mon, 12 Nov 2018 15:45:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A2046B02A2; Mon, 12 Nov 2018 10:45:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 077A86B02A3; Mon, 12 Nov 2018 10:45:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA7606B02A4; Mon, 12 Nov 2018 10:45:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id B580D6B02A2 for ; Mon, 12 Nov 2018 10:45:29 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id w185so24919232qka.9 for ; Mon, 12 Nov 2018 07:45:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=OrPrrJdFao+JDP++x3m1WXFk+3jNvzZQEC7WFio8Rik=; b=kpNkqelM9R75QscBhhQWVpO9fl4ELau3PGjLC5KBiL9N8dEpJ9pTI1il7emJ5W1Vqb gNomEEFwnOsK49yirrhbSVu/ZnTthtAyQ92VJ0P3AckGBzCJfxH/Q+Q9bFE3iZgOOL4o 8SE32iaD5DbErYTueY6vcjdTHFnaA+T2qPCpPo6ZuWl0+ZvjVf9Dm8R0tDdWBqtiyNdW CokvB19m8wasyANVidmKB1GEnyhn7/x5i02+nC5Nyabg5hATATdJuKH5YcLk+oseRg61 UlmQfP6HCxqBMqYEz9v1AfT4Axa6hw3YonjVcFSzadJTqLu+x/E5VW6AbpsUOMEEt2mz 2hvg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gJf0ktNRQjH0lVAsfTOlTR3ypkaZ5mMxSDObABexQiyZGyzc3j/ MM5JXzuDw5e8/1dfSxkoN+PaQeFovG42NIeA4D5gBBqTaxTSMvEau+47JM9rY8glEZCNCKDgX3A GRPR0LMdNhFbQJi0IT4vq2YLCPSrH6mhrKpZZAwLYcYhNEP6WwdMPDgG7RrUpvL/fOA== X-Received: by 2002:a0c:e904:: with SMTP id a4mr1386177qvo.181.1542037529452; Mon, 12 Nov 2018 07:45:29 -0800 (PST) X-Google-Smtp-Source: AJdET5eVchrLWGH249359g7D6uIyR1Ze3be8LaZABUvZkXcaGJmmJc1RszBD8pSfT8nU53rmKgVQ X-Received: by 2002:a0c:e904:: with SMTP id a4mr1386132qvo.181.1542037528694; Mon, 12 Nov 2018 07:45:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037528; cv=none; d=google.com; s=arc-20160816; b=EF7QDeSef/6cqVm7DQXbHz3+wA/aEAKNDT9HaaGxPz5r5qgA0r0HLNJNua/YWNMpfB JycJNnKnA7nkRcg1nA7Fu5+TzR5qrMULDDupvD5M8+XQE/b1QRk6OmfNTn4u7T4UZa2j kDnt9ch76bd5kUfr8BYk560HkQR9F7EyoMfo8DEf2WfaiZAZB9rNEorRxuPUYQTZ8d/G 27j27iDVr+oRfBKcZOUy3wAosVlrAd+UIng6skbGLsRGAaE9sQjmQSQ1LdCKQ2Xg7tdR 2PQ8uvJgYD3lXZ4DbV0aAKLBPR3FCLtOrXAt8fLQr3lDkzT8FqcghadXGvBhfYFYEbgr 1NfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=OrPrrJdFao+JDP++x3m1WXFk+3jNvzZQEC7WFio8Rik=; b=mYcw7NkEa/gnSwOxAmxlmPxVxwwCsBdE0Le7G47lN4Vt6QOwTAW/l7N3Z9yRDUgxLw 5Ewa7bnmV9PMUMN56HvZAfVHH84DlfuM9EAEtg81u3iFwQqFV8/zPWCHYrcR9i6J9iub 8YCFDPXt0pNAiHLAUzWbK6EqxQJiFHK6hB4oVYnidCvB7OqLKdNYbpPiutLDZIBxushR GV3ICGvkw9HxdiigCORrfnjJsfUV36lWYB0QWgEj0S+ETJmAQNKl7Y57eu36FXlrMySC 9I2yaClCLHTe12Z556dqUFx+jv/UsAQygWKCAh7ZCjiLsOi2eTSh+fRff35dbFh3k/5O +dJg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id k5si3203197qkb.174.2018.11.12.07.45.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:45:28 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037521-0fb3b01fb3add620001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id p6tq8PCSe8JgaLG2 (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:45:21 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529361; Mon, 12 Nov 2018 10:45:21 -0500 From: Tony Battersby Subject: [PATCH v4 7/9] dmapool: cleanup integer types To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 7/9] dmapool: cleanup integer types Cc: linux-scsi@vger.kernel.org Message-ID: <39edbec6-9c58-e6f0-61ab-02cb94ab4146@cybernetics.com> Date: Mon, 12 Nov 2018 10:45:21 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037521 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3509 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To represent the size of a single allocation, dmapool currently uses 'unsigned int' in some places and 'size_t' in other places. Standardize on 'unsigned int' to reduce overhead, but use 'size_t' when counting all the blocks in the entire pool. Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- This puts an upper bound on 'size' of INT_MAX to avoid overflowing the following comparison in pool_initialize_free_block_list(): unsigned int offset = 0; unsigned int next = offset + pool->size; if (unlikely((next + pool->size) > ... The actual maximum allocation size is probably lower anyway, probably KMALLOC_MAX_SIZE, but that gets into the implementation details of other subsystems which don't export a predefined maximum, so I didn't want to hardcode it here. The purpose of the added bounds check is to avoid overflowing integers, not to check the actual (platform/device/config-specific?) maximum allocation size. 'boundary' is passed in as a size_t but gets stored as an unsigned int. 'boundary' values >= 'allocation' do not have any effect, so clipping 'boundary' to 'allocation' keeps it within the range of unsigned int without affecting anything else. A few lines above (not in the diff) you can see that if 'boundary' is passed in as 0 then it is set to 'allocation', so it is nothing new. For reference, here is the relevant code after being patched: if (!boundary) boundary = allocation; else if ((boundary < size) || (boundary & (boundary - 1))) return NULL; boundary = min(boundary, allocation); --- linux/mm/dmapool.c.orig 2018-08-06 17:48:19.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:48:54.000000000 -0400 @@ -57,10 +57,10 @@ struct dma_pool { /* the pool */ #define POOL_MAX_IDX 2 struct list_head page_list[POOL_MAX_IDX]; spinlock_t lock; - size_t size; + unsigned int size; struct device *dev; - size_t allocation; - size_t boundary; + unsigned int allocation; + unsigned int boundary; char name[32]; struct list_head pools; }; @@ -86,7 +86,7 @@ show_pools(struct device *dev, struct de mutex_lock(&pools_lock); list_for_each_entry(pool, &dev->dma_pools, pools) { unsigned pages = 0; - unsigned blocks = 0; + size_t blocks = 0; int list_idx; spin_lock_irq(&pool->lock); @@ -103,9 +103,10 @@ show_pools(struct device *dev, struct de spin_unlock_irq(&pool->lock); /* per-pool info, no real statistics yet */ - temp = scnprintf(next, size, "%-16s %4u %4zu %4zu %2u\n", + temp = scnprintf(next, size, "%-16s %4zu %4zu %4u %2u\n", pool->name, blocks, - pages * (pool->allocation / pool->size), + (size_t) pages * + (pool->allocation / pool->size), pool->size, pages); size -= temp; next += temp; @@ -150,7 +151,7 @@ struct dma_pool *dma_pool_create(const c else if (align & (align - 1)) return NULL; - if (size == 0) + if (size == 0 || size > INT_MAX) return NULL; else if (size < 4) size = 4; @@ -165,6 +166,8 @@ struct dma_pool *dma_pool_create(const c else if ((boundary < size) || (boundary & (boundary - 1))) return NULL; + boundary = min(boundary, allocation); + retval = kmalloc_node(sizeof(*retval), GFP_KERNEL, dev_to_node(dev)); if (!retval) return retval; @@ -344,7 +347,7 @@ void *dma_pool_alloc(struct dma_pool *po { unsigned long flags; struct page *page; - size_t offset; + unsigned int offset; void *retval; void *vaddr; From patchwork Mon Nov 12 15:45:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678879 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3773314E2 for ; Mon, 12 Nov 2018 15:46:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 288CE2929B for ; Mon, 12 Nov 2018 15:46:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C8EA29920; Mon, 12 Nov 2018 15:46:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB0E629385 for ; Mon, 12 Nov 2018 15:46:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFD746B02A5; Mon, 12 Nov 2018 10:46:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD1606B02A6; Mon, 12 Nov 2018 10:46:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C5186B02A7; Mon, 12 Nov 2018 10:46:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id 6EE4B6B02A5 for ; Mon, 12 Nov 2018 10:46:01 -0500 (EST) Received: by mail-qk1-f199.google.com with SMTP id f81-v6so24750722qkb.14 for ; Mon, 12 Nov 2018 07:46:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=iX8zRGNPu62ZIigyZz2yhUPQAoybGxa86J9C5XNhZEs=; b=VCZcyTbRD0QqrSZVS+9shC/lMVymib7ZOHMUik/gK3omJUtoUHCu4eOhx+uC3qrXe3 peYykIQGNsUgA/2P5lKeZCTBiDUaCwSEJyIlXsRV6daFJAs+1RZFHNvvkSxPjm9E5hfy oHdblmNbhcA2lw+RRAlbTWXbOj4AjCd/jX6F30gELc29nfapszPp9bBB0hfzngBYbZZx eG4RkctftSom8toamCFqHW33gNjOYTW5OpAtB+fbey9J+Gw0vjRvIiF/zqUyoUXCJT66 gb+Rw3RS3kUo0SL7ec9myQQHZT4bZdjVNK7YryPld25SgZJtOw+CPs753zeKEV3zpnG3 U4vw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gIdLzMLRV60OwVcdrOr1AOdrnusQJvLU+P2Vdt69/hdrkbW1gS0 inbC1G9WHWbYlDNmWltwaANAL9ERJS1COfB5/e+k0gAl/HifIc3mjvM2ngWgW/VLs/4wotNZCsU SFga+Ahx02dBllypNP4hg6ylOZvc8AngTLwd/L8qzcqZCChi5fZyfyt+82PHVH5W8Ag== X-Received: by 2002:a37:a053:: with SMTP id j80mr1363821qke.356.1542037561177; Mon, 12 Nov 2018 07:46:01 -0800 (PST) X-Google-Smtp-Source: AJdET5fxe3K9aaJPVw8L5uHW8mRKkWWc/L5lKbfg6M7H8ZLQvlyH9ItqmsM6vICdEGZz7SX537uf X-Received: by 2002:a37:a053:: with SMTP id j80mr1363791qke.356.1542037560451; Mon, 12 Nov 2018 07:46:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037560; cv=none; d=google.com; s=arc-20160816; b=ZVmCG1diXqPiXNTYTmnbNlYmjAaRhz8IAhqHW/fzH0JfUdEUI80uVPHUyjqaBa46xL cPA2q1vsenbdbCikKOJ0qQA8rpzjO0zCLbPtTK+tHkCve8pqktVMIs0zfKKCtzlWjuBx 1okQhY93Czj8UO/DKy2fecDPuEfygosZV8CzzPdSKN5GL+pMZgvy1nlpSUQDw6fXu3e0 vArT9bwK2f1X9tBCPlD3pWVa/Yz15aoXbT2uwRe/4iDna2F0VvFph1SJaZUQ8v4E9cZo 2Vk62HeNK0X/Pyf3udeLAcLLrZi1IPDXA/N0RaYo7jneRp01O4DFdfyphDsQt+Lwhe7G gupw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=iX8zRGNPu62ZIigyZz2yhUPQAoybGxa86J9C5XNhZEs=; b=A4ZsSlpsn6Kde8fkACL3o5coVxSIAk3GAM1+wirz805MF9Hn4+iWRGRMX/3eYoIlDw CkBRrqF62aU3MYYHYBDfus0Pu1k07gScK6zHhHKIEAn5eMWUuHiTxmcN29i5zVieu9ph BkzP+2OgZVJK0E+8P+9VdWYXFDLo83SVlO+x9e7fWC6QN/Ct8a2ZIVYc6sZOSMhls5oK joGQeeKfG6i57vDKnSwj3+/1WTswS9nAsmMbtk46Q88KPtqCCXCisAHuroJqfQLWQ05H y89sF8/QsHzYrI9f6k5yt24tGrexZnRxsrvT3u90zx80twk4FyHTFpRNBFH+DJAQDR9b 4IlQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id r41si497493qtj.56.2018.11.12.07.46.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:46:00 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037558-0fb3b01fb3add6a0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id SkzT8XAKr4RyHkVH (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:45:58 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529365; Mon, 12 Nov 2018 10:45:58 -0500 From: Tony Battersby Subject: [PATCH v4 8/9] dmapool: improve accuracy of debug statistics To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 8/9] dmapool: improve accuracy of debug statistics Cc: linux-scsi@vger.kernel.org Message-ID: Date: Mon, 12 Nov 2018 10:45:58 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037558 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 1632 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The "total number of blocks in pool" debug statistic currently does not take the boundary value into account, so it diverges from the "total number of blocks in use" statistic when a boundary is in effect. Add a calculation for the number of blocks per allocation that takes the boundary into account, and use it to replace the inaccurate calculation. Signed-off-by: Tony Battersby --- This depends on patch #1 "dmapool: fix boundary comparison" for the calculated blks_per_alloc value to be correct. The added blks_per_alloc value will also be used in the next patch. --- linux/mm/dmapool.c.orig 2018-08-06 17:48:54.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:52:53.000000000 -0400 @@ -61,6 +61,7 @@ struct dma_pool { /* the pool */ struct device *dev; unsigned int allocation; unsigned int boundary; + unsigned int blks_per_alloc; char name[32]; struct list_head pools; }; @@ -105,8 +106,7 @@ show_pools(struct device *dev, struct de /* per-pool info, no real statistics yet */ temp = scnprintf(next, size, "%-16s %4zu %4zu %4u %2u\n", pool->name, blocks, - (size_t) pages * - (pool->allocation / pool->size), + (size_t) pages * pool->blks_per_alloc, pool->size, pages); size -= temp; next += temp; @@ -182,6 +182,9 @@ struct dma_pool *dma_pool_create(const c retval->size = size; retval->boundary = boundary; retval->allocation = allocation; + retval->blks_per_alloc = + (allocation / boundary) * (boundary / size) + + (allocation % boundary) / size; INIT_LIST_HEAD(&retval->pools); From patchwork Mon Nov 12 15:46:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10678881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4665C14E2 for ; Mon, 12 Nov 2018 15:46:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3481229BED for ; Mon, 12 Nov 2018 15:46:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 283A629DD0; Mon, 12 Nov 2018 15:46:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAB4329BED for ; Mon, 12 Nov 2018 15:46:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC2C06B02A6; Mon, 12 Nov 2018 10:46:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B99426B02A8; Mon, 12 Nov 2018 10:46:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A620F6B02A9; Mon, 12 Nov 2018 10:46:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id 7A01F6B02A6 for ; Mon, 12 Nov 2018 10:46:40 -0500 (EST) Received: by mail-qk1-f199.google.com with SMTP id g22so24594514qke.15 for ; Mon, 12 Nov 2018 07:46:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:subject :to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=sDVSTewdUwY/1+HBQHfNCUhNWvzrGS9h8XMZm9tQu6A=; b=kybUqCS219CvlS3CGSrg5QbCt+avAhz1DPCHd1uILgA8Pw1V8ga+svyYtxQw1qHTou VVeb+OT2eC/Mir9Ij4iFCMtWl5BaIiW8MXmo3581vDJ1uOU/0aTXKaOVMBgaE8682ZdQ dlh5CuLI83upshXPOPiG3ppQwK+mrM/jk83wGvsgcPYiCBG4sGYOdMMkZHjrHIEG5gYM R0ip2+ahfT18k+0aQaiG8+m0X6E55eLq9Ycf1tSlwMC7dq1Vy++c3HuOi+9j0hF2OYzi VcF9SJbWJTdh5ExELIETZ4FJcn0trB8ZjtJTa948kAIqfp98h6sL8RyYRDLkqG4cM2Aj aPPw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-Gm-Message-State: AGRZ1gJcTbUtBqP2gtnhUYpNSAuiswEQznyvBWQ8WaXH77819wHCaRTW yRnbrQ94v0/+20vAsQa0RYrv8Q3l1GGUY8uJDoWGmGXOjVIg2LBaDz5kfzqamp2QPzROYaeTkBE v/YJKvdOxl/8p/aUHBHgrmnakD+/rSbVc06xua71cc9GAEm8/NmAHKgNck7zg+L35tw== X-Received: by 2002:aed:2f82:: with SMTP id m2mr1451185qtd.4.1542037600279; Mon, 12 Nov 2018 07:46:40 -0800 (PST) X-Google-Smtp-Source: AJdET5dNMvH1tn03gnImDOUacDdlA4MBDJjSneWoIaQH/+ei0WoCSzDBqpoc0kLaY5jYqp/YN7T8 X-Received: by 2002:aed:2f82:: with SMTP id m2mr1451143qtd.4.1542037599617; Mon, 12 Nov 2018 07:46:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542037599; cv=none; d=google.com; s=arc-20160816; b=N8Rw9mEg7xzsdb5qK1sUnhR1EyJa9C74eLZhDI5hicF1AVNkRFB+yFae8Gj/Ssc/NU Qrb8URiTp/aCTEujRwdPejmHu+NS98fljvpYCYlqxH2Ca6kegb6+1cOrhiThOBicO3fu RVNK7f7LWYDhR4maCFHkgSoultu8/RBwB9mntvAMhCBSq3ElDXKYFkRXW/0oaPn2mTsz XMlOIqarDzSc5oj6SNOZE95dBL46jf0xWKxiffGZ47NPiJmliP0gGXglUpa/95IdTHl0 P5anAv+2wqbf4Vw4eZphofThapwmrAMzxMstx6NDA42q5qNVV8EQSfth/L1gigzCTwfJ 1OGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=sDVSTewdUwY/1+HBQHfNCUhNWvzrGS9h8XMZm9tQu6A=; b=OwktlSYAdxuo74RCqZoYujyrpdWu+wIjHlp5ob6Dbq2ocdPf+iw4d7kMiG424ZovHS 47TmRqF2pAPiPiBbtmhsDM+L8FqmD5VcRF/hn6RtIgWQ8AFuj3CAHOufguxr3K+TFg1V Ln2JrEdyNVZDYDMKhquoe85Po18gDTb1eRUQw2UeDFg0c7454RRTwQ99Gxzjc5TMK3Gj OgqtB2O+AtmdzBDKjehoBrpaHFLVDEvFbIiDqP7R2gfry3dgE8Vhli0ipfWDGoqaFE9B 5PdxCd6sV8LEG6vBnwUEW4qo/Zj57uiURPANLf6KfVyCR7n57zmpTOy/tThp0DbqSAyR 7E3w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id k17si8097660qkh.130.2018.11.12.07.46.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Nov 2018 07:46:39 -0800 (PST) Received-SPF: pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==854ac0a7dab==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==854ac0a7dab==tonyb@cybernetics.com" X-ASG-Debug-ID: 1542037595-0fb3b01fb3add6f0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id aXECOlFwimmGkViH (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Mon, 12 Nov 2018 10:46:35 -0500 (EST) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8529370; Mon, 12 Nov 2018 10:46:35 -0500 From: Tony Battersby Subject: [PATCH v4 9/9] dmapool: debug: prevent endless loop in case of corruption To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , iommu@lists.linux-foundation.org, linux-mm@kvack.org X-ASG-Orig-Subj: [PATCH v4 9/9] dmapool: debug: prevent endless loop in case of corruption Cc: "linux-scsi@vger.kernel.org" Message-ID: <9e65ec2e-5e22-4f65-7b92-ca2af0c555f3@cybernetics.com> Date: Mon, 12 Nov 2018 10:46:35 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1542037595 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 1752 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Prevent a possible endless loop with DMAPOOL_DEBUG enabled if a buggy driver corrupts DMA pool memory. Signed-off-by: Tony Battersby Acked-by: Matthew Wilcox --- linux/mm/dmapool.c.orig 2018-08-06 17:52:53.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:53:31.000000000 -0400 @@ -454,17 +454,39 @@ void dma_pool_free(struct dma_pool *pool { void *page_vaddr = vaddr - offset; unsigned int chain = page->dma_free_off; + unsigned int free_blks = 0; + while (chain < pool->allocation) { - if (chain != offset) { - chain = *(int *)(page_vaddr + chain); - continue; + if (unlikely(chain == offset)) { + spin_unlock_irqrestore(&pool->lock, flags); + dev_err(pool->dev, + "dma_pool_free %s, dma %pad already free\n", + pool->name, &dma); + return; + } + + /* + * A buggy driver could corrupt the freelist by + * use-after-free, buffer overflow, etc. Besides + * checking for corruption, this also prevents an + * endless loop in case corruption causes a circular + * loop in the freelist. + */ + if (unlikely(++free_blks + page->dma_in_use > + pool->blks_per_alloc)) { + freelist_corrupt: + spin_unlock_irqrestore(&pool->lock, flags); + dev_err(pool->dev, + "dma_pool_free %s, freelist corrupted\n", + pool->name); + return; } - spin_unlock_irqrestore(&pool->lock, flags); - dev_err(pool->dev, - "dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); - return; + + chain = *(int *)(page_vaddr + chain); } + if (unlikely(free_blks + page->dma_in_use != + pool->blks_per_alloc)) + goto freelist_corrupt; } memset(vaddr, POOL_POISON_FREED, pool->size); #endif