From patchwork Tue Aug 7 16:45:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558905 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A22FC13B4 for ; Tue, 7 Aug 2018 16:45:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DD8D2A688 for ; Tue, 7 Aug 2018 16:45:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81D952A6B5; Tue, 7 Aug 2018 16:45:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 120E22A688 for ; Tue, 7 Aug 2018 16:45:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 39F606B0006; Tue, 7 Aug 2018 12:45:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34FC56B0007; Tue, 7 Aug 2018 12:45:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 265FB6B0008; Tue, 7 Aug 2018 12:45:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id EDE346B0006 for ; Tue, 7 Aug 2018 12:45:45 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id e3-v6so16980573qkj.17 for ; Tue, 07 Aug 2018 09:45:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=eAYLw5kccLQ6ciuy3I6VIAz+nbFJ/0P9qzDJ15YN/uE=; b=QHCDCcXAb2QfGR/YzlpCdIQ7KlhNLvf7qxbsixAoTjwR70qJlwEuU4juQbr67pp88r AFLEZ6ZRNwRMg/h/+i72BG6eTMjW07bH+5bV0MNWJNDOoK57FAyLIO+CSYWOX8gk8LEB AacN+KTjrskg6cPfJOi4cCABRDsFwjCLmmyD1qIIOFqaWtE0Jd8fe9AJM2B60l4TpKw/ uPxVW5HsxK7Oy3U3AjgO3ho2tdRnOq1Uy1D+6fJwTasDzQtAm6RfGwOnoVbdSq2IqRGm 4j6zSCeUxrKMcZwVgrhyLgvQa9Gk+r3yvnHznBvKfV8UWOE7MfOdSN1gc3YUdkvJysON GN/w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlHiznRLdVk+52bxAfi5NU8i/U+rvnDNmwA0CdmE+R0MXr7nmSm1 bsDDZjNuLVAmB966fhjttV/pZzrzA4ZEUZbtFU6NYQD7I+OajElpE9tKjDaEIBMvlj0IcrZcNBD R14l+XeWTt29em5bHSbiCuialgPlzFYTkHZm2deb1EVfBseEAV1VMOX8niz8IUKtfgg== X-Received: by 2002:ac8:3417:: with SMTP id u23-v6mr18691894qtb.237.1533660345672; Tue, 07 Aug 2018 09:45:45 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeLSTM/B1bjbFChuF5ZfXUX488Svikg/ojoL81Sdi0JUyPAuMVajplj8rsiGJXKc/cqZTdl X-Received: by 2002:ac8:3417:: with SMTP id u23-v6mr18691848qtb.237.1533660344898; Tue, 07 Aug 2018 09:45:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660344; cv=none; d=google.com; s=arc-20160816; b=FACy/JFAVr17w/OLycYocFt8cFWnU8yP9Hs4YLvqgCwET3GPvEkKlC6Vhc44ah1v9+ RVoEJloBI5eCDKO+Cis1WBeDLS8EoZryVy0dl7rIILwrc+aPCsuZleTeSEvQulvEMiNJ vR51nF3D2XdbV00aEIBWSnpbcfhvBMcmWFMGJVBx4rIYf+MmBMrxU2W+oGSqj0hGFyDz 1+lo1zig7Bzpmew6OoNqaoDaB6IXik7nX7W2Gn3HwpfiiyhR1PEW3/LDKrf6JQXiptO1 YwX/57GF1puKxDPwo1daS6EoDNVdg06abyaCZZVonjdZbp+qLGuCpgCQr0y+imUynaV8 bt9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=eAYLw5kccLQ6ciuy3I6VIAz+nbFJ/0P9qzDJ15YN/uE=; b=Ceto/lPvBJpjSeP+1VDt/qRuAZocK0p2YCEjXqG8/tgouhS3jCWla9IyS1KzFWa/ZN FRE53o2s3zqh0CZLJPof4EpxdEEVMedy1OKmVBFDwarFvWLT2LDwsvjX3Q6j2/Rnjfi7 4Jceo8i1fCegxJQkmiXmE6cHxEhFQZGXUGIvXYKobPlX+fm8Wpqch3wnhbuuosihlkji N735H5KpgJsnVsiKlXbeg/8L+K5UgWCqQOdxoPw98iTz/L++FcGyxWC9qGF1ySoKss/8 hNm7+omG3Mkeg0+94ZbU9zpA+gEkXpg33UZ8wu3b4FWQpLrf8EdYV8bbmZWvrNYZJc6m 7jYg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id r24-v6si312050qkk.207.2018.08.07.09.45.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:45:44 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660342-0fb3b01fb342b830001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id ckqYIo1C7Ii7OcuH (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:45:42 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324853; Tue, 07 Aug 2018 12:45:42 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 02/10] dmapool: remove checks for dev == NULL Message-ID: <99c5f149-90a7-dcc5-73a5-179ca009f914@cybernetics.com> X-ASG-Orig-Subj: [PATCH v3 02/10] dmapool: remove checks for dev == NULL Date: Tue, 7 Aug 2018 12:45:42 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660342 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3650 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dmapool originally tried to support pools without a device because dma_alloc_coherent() supports allocations without a device. But nobody ended up using dma pools without a device, so the current checks in dmapool.c for pool->dev == NULL are both insufficient and causing bloat. Remove them. Signed-off-by: Tony Battersby --- Changes since v2: *) This was "dmapool: cleanup error messages" in v2. *) Remove one more check for dev == NULL in dma_pool_destroy() that is unrelated to error messages. --- linux/mm/dmapool.c.orig 2018-08-03 16:12:23.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 16:13:44.000000000 -0400 @@ -277,7 +277,7 @@ void dma_pool_destroy(struct dma_pool *p mutex_lock(&pools_reg_lock); mutex_lock(&pools_lock); list_del(&pool->pools); - if (pool->dev && list_empty(&pool->dev->dma_pools)) + if (list_empty(&pool->dev->dma_pools)) empty = true; mutex_unlock(&pools_lock); if (empty) @@ -289,13 +289,9 @@ void dma_pool_destroy(struct dma_pool *p page = list_entry(pool->page_list.next, struct dma_page, page_list); if (is_page_busy(page)) { - if (pool->dev) - dev_err(pool->dev, - "dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); - else - pr_err("dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); + dev_err(pool->dev, + "dma_pool_destroy %s, %p busy\n", + pool->name, page->vaddr); /* leak the still-in-use consistent memory */ list_del(&page->page_list); kfree(page); @@ -357,13 +353,9 @@ void *dma_pool_alloc(struct dma_pool *po for (i = sizeof(page->offset); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; - if (pool->dev) - dev_err(pool->dev, - "dma_pool_alloc %s, %p (corrupted)\n", - pool->name, retval); - else - pr_err("dma_pool_alloc %s, %p (corrupted)\n", - pool->name, retval); + dev_err(pool->dev, + "dma_pool_alloc %s, %p (corrupted)\n", + pool->name, retval); /* * Dump the first 4 bytes even if they are not @@ -418,13 +410,9 @@ void dma_pool_free(struct dma_pool *pool page = pool_find_page(pool, dma); if (!page) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, - "dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); - else - pr_err("dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); + dev_err(pool->dev, + "dma_pool_free %s, %p/%lx (bad dma)\n", + pool->name, vaddr, (unsigned long)dma); return; } @@ -432,13 +420,9 @@ void dma_pool_free(struct dma_pool *pool #ifdef DMAPOOL_DEBUG if ((dma - page->dma) != offset) { spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, - "dma_pool_free %s, %p (bad vaddr)/%pad\n", - pool->name, vaddr, &dma); - else - pr_err("dma_pool_free %s, %p (bad vaddr)/%pad\n", - pool->name, vaddr, &dma); + dev_err(pool->dev, + "dma_pool_free %s, %p (bad vaddr)/%pad\n", + pool->name, vaddr, &dma); return; } { @@ -449,12 +433,9 @@ void dma_pool_free(struct dma_pool *pool continue; } spin_unlock_irqrestore(&pool->lock, flags); - if (pool->dev) - dev_err(pool->dev, "dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); - else - pr_err("dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); + dev_err(pool->dev, + "dma_pool_free %s, dma %pad already free\n", + pool->name, &dma); return; } } From patchwork Tue Aug 7 16:46:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 55BC314E5 for ; Tue, 7 Aug 2018 16:46:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45CD32A621 for ; Tue, 7 Aug 2018 16:46:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 383E22A6AF; Tue, 7 Aug 2018 16:46:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCD6C2A621 for ; Tue, 7 Aug 2018 16:46:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06B9E6B0008; Tue, 7 Aug 2018 12:46:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 043E16B000A; Tue, 7 Aug 2018 12:46:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9A336B000C; Tue, 7 Aug 2018 12:46:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f197.google.com (mail-qk0-f197.google.com [209.85.220.197]) by kanga.kvack.org (Postfix) with ESMTP id C0AFE6B0008 for ; Tue, 7 Aug 2018 12:46:21 -0400 (EDT) Received: by mail-qk0-f197.google.com with SMTP id 17-v6so17276387qkz.15 for ; Tue, 07 Aug 2018 09:46:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=/Y+T8Euumkx2WMsRvhpHY8+YGLSTwy7S6ZHoxPHvzZE=; b=KH2bm9v4SAkcuKYZ2fs/XFpQDZ6NeRkc8O+IlWlHE/hIQRB65zydIk0Urgfq79oklF L62pVMfc6kQVPTojerzvTALsNuK8BBUYi424V0GrCjIuVawp5ZSZ0a36HN2p6zl2IrLR obsE17gJsWfxs6ZzX8UTfEKDCru+B7YXfJsQdjs/zfSGd99+4xJ3OqaWIUZUYFVC4hCb uL5uLI6NV1yjZrye1tMPc/YYr5TgBlFe0AY6VupTFkh3yaMBBOnAfHubL8xJFD59jaWg dOYDzxSPU2pzlzMadTcxzgOrHVhpM6CWBN4QLRybAk/EmYpTuxuQbca1uyLkQb422Zyq +4UA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlHhpaP37IxYF/wEuqSchMa8vc2bmpDgsX2jYDJAXzhK/rUAqepn 5xSfSHfHjUeSfiUU33nge5+OQF9Nc1Ln/upUC51/OAK6ICwqsyIpWjS2uYWP0v1th3/f2UwsdKE /pjC5/x9z7XDlboPq7/l+ScpJPXPcbBTISZPxbPqFrORoawMP1pPHGirU0IoxiDQqhg== X-Received: by 2002:ac8:27f4:: with SMTP id x49-v6mr19851771qtx.422.1533660381558; Tue, 07 Aug 2018 09:46:21 -0700 (PDT) X-Google-Smtp-Source: AAOMgpcs+LAmz6hKcKiS1sbKceUbFsMnAWcLdtAJYKyr/0vRGlDRwfsXlP9uJhrrZRtchS0hB638 X-Received: by 2002:ac8:27f4:: with SMTP id x49-v6mr19851716qtx.422.1533660380618; Tue, 07 Aug 2018 09:46:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660380; cv=none; d=google.com; s=arc-20160816; b=utAbGMGAhuVSYxkgPyiyt21PXZ1E0y8PZCqU1YCo5BcHLYYxDfi0sHTV9R7qbMQL1M aUH3TkpYGsCwzRXwAfINQdBokV1417df3p381PtiA46YEVyHGUAYJDDkuHzpbuWKGqaU w7AKjDC16/5+75xmhHKbIFLC2HiowwTGy0GIF+ENifMLfnPXfkPoqulXpyjeHsYdqjPW qCKtzrm/wJGXc3dz2QbOboYiTB18P4pYVkIQUX9Qa/7cyjrIrpuy5Cvbv71E0NuxNqQf 7NGQ7ny3+WqQRR6gLuw/lUuTOsG+Uf+BYj5XG7Zvq34Q7AWGJ8XuzEI+GSEOvY3ma7Kt QqwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=/Y+T8Euumkx2WMsRvhpHY8+YGLSTwy7S6ZHoxPHvzZE=; b=gQVdj0XHZI3bwcYR/aEAQsZeWGp+PBsznXLK9vWNPxbNO+3v1E+o14QvwGPOzLTLZx RxjL1ve1dlJKj8rpKudApXdcpk0D1aPT6XMgmn0mMXVX02PRNejvPD1xc3bZkZu8M1nr GoNHTh8wdewRZeVLjnmc4eS2FfrJybHqeluwJrvXta0HhpKPJkEBWoLgH4z05R1Ha6LU veEgUlcN7GTENZdwoOqsHxqOnL0vCrYhbH3OjmA0sJYCrM9V1cnoO9CVRROzqyWwD8Nk bQ6X0urjcC2N8EGz1l2cFRQE2BsdBJTIlhZNFSKZDaBpRjN6KtCVLo8KERHseOO2IJJ6 umLw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id 9-v6si1848731qkv.364.2018.08.07.09.46.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:46:20 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660378-0fb3b01fb342b850001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id 1q7UScR8LZmYJ9oQ (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:46:18 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324855; Tue, 07 Aug 2018 12:46:18 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 03/10] dmapool: cleanup dma_pool_destroy Message-ID: <9fb39095-4dd8-877a-b857-649e76fedd59@cybernetics.com> X-ASG-Orig-Subj: [PATCH v3 03/10] dmapool: cleanup dma_pool_destroy Date: Tue, 7 Aug 2018 12:46:18 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660378 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2099 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Remove a small amount of code duplication between dma_pool_destroy() and pool_free_page() in preparation for adding more code without having to duplicate it. No functional changes. Signed-off-by: Tony Battersby --- No changes since v2. --- linux/mm/dmapool.c.orig 2018-08-02 09:59:15.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-02 10:01:26.000000000 -0400 @@ -249,13 +249,22 @@ static inline bool is_page_busy(struct d static void pool_free_page(struct dma_pool *pool, struct dma_page *page) { + void *vaddr = page->vaddr; dma_addr_t dma = page->dma; + list_del(&page->page_list); + + if (is_page_busy(page)) { + dev_err(pool->dev, + "dma_pool_destroy %s, %p busy\n", + pool->name, vaddr); + /* leak the still-in-use consistent memory */ + } else { #ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); + memset(vaddr, POOL_POISON_FREED, pool->allocation); #endif - dma_free_coherent(pool->dev, pool->allocation, page->vaddr, dma); - list_del(&page->page_list); + dma_free_coherent(pool->dev, pool->allocation, vaddr, dma); + } kfree(page); } @@ -269,6 +278,7 @@ static void pool_free_page(struct dma_po */ void dma_pool_destroy(struct dma_pool *pool) { + struct dma_page *page; bool empty = false; if (unlikely(!pool)) @@ -284,19 +294,10 @@ void dma_pool_destroy(struct dma_pool *p device_remove_file(pool->dev, &dev_attr_pools); mutex_unlock(&pools_reg_lock); - while (!list_empty(&pool->page_list)) { - struct dma_page *page; - page = list_entry(pool->page_list.next, - struct dma_page, page_list); - if (is_page_busy(page)) { - dev_err(pool->dev, - "dma_pool_destroy %s, %p busy\n", - pool->name, page->vaddr); - /* leak the still-in-use consistent memory */ - list_del(&page->page_list); - kfree(page); - } else - pool_free_page(pool, page); + while ((page = list_first_entry_or_null(&pool->page_list, + struct dma_page, + page_list))) { + pool_free_page(pool, page); } kfree(pool); From patchwork Tue Aug 7 16:46:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D249D13B4 for ; Tue, 7 Aug 2018 16:46:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C25662A621 for ; Tue, 7 Aug 2018 16:46:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B66C52A692; Tue, 7 Aug 2018 16:46:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27C6E2A621 for ; Tue, 7 Aug 2018 16:46:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AFCA6B000C; Tue, 7 Aug 2018 12:46:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 487EB6B000D; Tue, 7 Aug 2018 12:46:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 376646B000E; Tue, 7 Aug 2018 12:46:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f197.google.com (mail-qk0-f197.google.com [209.85.220.197]) by kanga.kvack.org (Postfix) with ESMTP id 0AA856B000C for ; Tue, 7 Aug 2018 12:46:57 -0400 (EDT) Received: by mail-qk0-f197.google.com with SMTP id y130-v6so17350490qka.1 for ; Tue, 07 Aug 2018 09:46:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=b6urxPbDKN+LpF7jLNeBWPle4OAE2OIgdKNDb3CcBxE=; b=NaStVfZE1BEpKVEV9pKZcAXSaZF14WQn/NoWQCa/TiMoAwcK+aDgKkbZDCrVuWXcKs TcP1s2S9srl/XgTW7+R0JqXL4u0lHrep8RKjz3nlazdn5wp7+nG+wUMGnqKLnLRDY/Ei A/relB4YAq/9piGqS1STkszGsaSLHqwTHEFgk9IsOBA6kKgKFNH4hy/sAiD1FcDrY3Bi ibjlPCAiY4efuk0SBEUgL7cNbF71yyWgwA+HM7MUo5zAAF7YTm2WsYot2HlGzh1Cib4X 7nfQPHdGfU44kvyOzJxWxWrxpqeAyUXqcnC7Xx5vVjI7rzYlNruyrTVfznhN42qIefdj SUQA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlGuQcK4kizg8JrJ+pVwCOcPNx6UHPomf/vQPYTuVswcvdpVB1gr l2EOnKxfazZ80no2jv56qTxUR2Mjv1QJxnDp3/G6voWusa4jXZjnkIr3eeEK6D75vZMh8Q3Hid0 DYykmYbBluk9ecPgnuVPkBjqBxK4uAef46IPn1RrUmz6NRlY7Q/Yoqie2pYa/mkQiGA== X-Received: by 2002:ac8:2b45:: with SMTP id 5-v6mr18427122qtv.395.1533660416804; Tue, 07 Aug 2018 09:46:56 -0700 (PDT) X-Google-Smtp-Source: AAOMgpf9qmIjr/HnVsRA2qev/RiN4p70OeztMi19orNjXyqzvzvA1SlvXcfBrGvMMhkCCnhKKTIT X-Received: by 2002:ac8:2b45:: with SMTP id 5-v6mr18427063qtv.395.1533660415817; Tue, 07 Aug 2018 09:46:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660415; cv=none; d=google.com; s=arc-20160816; b=YqppiTRMTrOrliIgmydBsVqW2WFvt3LG5L0jF7OJOTPD4+XD42gNejXz0uxTRYyDU3 3IEd0T4TGdpk6LTwIUiXvMi85bb5YL2+qq6oOtgt1ReVbJh+PTzTsQaF9kdaV+phzrvD Ikmgb+d4Gf/fxNApRoNbDXRj4Gum2IP5x3HAbT0w4W162cZINeJG+h7X/SeW+sq8tYVL RRbvKd2PcZS+1EFyKboYiIRgp5+eZAVRFg0/Kfal770DCAsFwJAjM8meioUCo0ms9Cvh kFC0AkqwK6/84bXPaWRiXqWDPn3x1Mncy1Um+OvplmBY1uTKJwVvrvOckiwycNJzgM3U h/xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=b6urxPbDKN+LpF7jLNeBWPle4OAE2OIgdKNDb3CcBxE=; b=f14zK31Vo0c+ZonQNsgAkvYiB1n9hXqi7UBO3S9wZf/WjFQbILBfCw6Q2+EuX2vjPz JKt7zfkyOLlASadBdG6dDy6pM4I3LkX/5owvq6ADfI8pJLKlSLoRLW26JlsXaAVnftUZ T4Hd5r1An/pgDu73BvnUeVYJrdI/q+nbhV8oVLGLXaCe2yG7gARC5cc5bG+6Z4WCvj8G Bu7iQMVvFl6KRQ8QwULOX3iXkBtOInkSLW3d/8D5EdxBRSw8P/y2E6ypApSJzVqtSFAy wVSNAqOuCiDr8zy2I7RZIbV0DHF/RvGpHCqo7qPhMkpGxGC4tEA056qmNm1NHBrqGK3h xb7Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id v42-v6si1869579qtv.203.2018.08.07.09.46.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:46:55 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660413-0fb3b01fb342b8a0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id tpA8NN95pS060sn0 (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:46:53 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324863; Tue, 07 Aug 2018 12:46:53 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 04/10] dmapool: improve scalability of dma_pool_alloc Message-ID: <1186a929-8fd8-5ead-606d-d3699527c795@cybernetics.com> X-ASG-Orig-Subj: [PATCH v3 04/10] dmapool: improve scalability of dma_pool_alloc Date: Tue, 7 Aug 2018 12:46:53 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660413 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 6785 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dma_pool_alloc() scales poorly when allocating a large number of pages because it does a linear scan of all previously-allocated pages before allocating a new one. Improve its scalability by maintaining a separate list of pages that have free blocks ready to (re)allocate. In big O notation, this improves the algorithm from O(n^2) to O(n). Signed-off-by: Tony Battersby --- Changes since v2: *) Use list_move()/list_move_tail() instead of list_del+list_add(). *) Renamed POOL_N_LISTS to POOL_MAX_IDX. *) Use defined names instead of 0/1 indexes for INIT_LIST_HEAD(). --- linux/mm/dmapool.c.orig 2018-08-03 16:16:49.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 16:45:33.000000000 -0400 @@ -15,11 +15,16 @@ * Many older drivers still have their own code to do this. * * The current design of this allocator is fairly simple. The pool is - * represented by the 'struct dma_pool' which keeps a doubly-linked list of - * allocated pages. Each page in the page_list is split into blocks of at - * least 'size' bytes. Free blocks are tracked in an unsorted singly-linked - * list of free blocks within the page. Used blocks aren't tracked, but we - * keep a count of how many are currently allocated from each page. + * represented by the 'struct dma_pool'. Each allocated page is split into + * blocks of at least 'size' bytes. Free blocks are tracked in an unsorted + * singly-linked list of free blocks within the page. Used blocks aren't + * tracked, but we keep a count of how many are currently allocated from each + * page. + * + * The pool keeps two doubly-linked list of allocated pages. The 'available' + * list tracks pages that have one or more free blocks, and the 'full' list + * tracks pages that have no free blocks. Pages are moved from one list to + * the other as their blocks are allocated and freed. */ #include @@ -43,7 +48,10 @@ #endif struct dma_pool { /* the pool */ - struct list_head page_list; +#define POOL_FULL_IDX 0 +#define POOL_AVAIL_IDX 1 +#define POOL_MAX_IDX 2 + struct list_head page_list[POOL_MAX_IDX]; spinlock_t lock; size_t size; struct device *dev; @@ -54,7 +62,7 @@ struct dma_pool { /* the pool */ }; struct dma_page { /* cacheable header for 'allocation' bytes */ - struct list_head page_list; + struct list_head dma_list; void *vaddr; dma_addr_t dma; unsigned int in_use; @@ -70,7 +78,6 @@ show_pools(struct device *dev, struct de unsigned temp; unsigned size; char *next; - struct dma_page *page; struct dma_pool *pool; next = buf; @@ -84,11 +91,18 @@ show_pools(struct device *dev, struct de list_for_each_entry(pool, &dev->dma_pools, pools) { unsigned pages = 0; unsigned blocks = 0; + int list_idx; spin_lock_irq(&pool->lock); - list_for_each_entry(page, &pool->page_list, page_list) { - pages++; - blocks += page->in_use; + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + list_for_each_entry(page, + &pool->page_list[list_idx], + dma_list) { + pages++; + blocks += page->in_use; + } } spin_unlock_irq(&pool->lock); @@ -163,7 +177,8 @@ struct dma_pool *dma_pool_create(const c retval->dev = dev; - INIT_LIST_HEAD(&retval->page_list); + INIT_LIST_HEAD(&retval->page_list[POOL_FULL_IDX]); + INIT_LIST_HEAD(&retval->page_list[POOL_AVAIL_IDX]); spin_lock_init(&retval->lock); retval->size = size; retval->boundary = boundary; @@ -252,7 +267,7 @@ static void pool_free_page(struct dma_po void *vaddr = page->vaddr; dma_addr_t dma = page->dma; - list_del(&page->page_list); + list_del(&page->dma_list); if (is_page_busy(page)) { dev_err(pool->dev, @@ -278,8 +293,8 @@ static void pool_free_page(struct dma_po */ void dma_pool_destroy(struct dma_pool *pool) { - struct dma_page *page; bool empty = false; + int list_idx; if (unlikely(!pool)) return; @@ -294,10 +309,15 @@ void dma_pool_destroy(struct dma_pool *p device_remove_file(pool->dev, &dev_attr_pools); mutex_unlock(&pools_reg_lock); - while ((page = list_first_entry_or_null(&pool->page_list, - struct dma_page, - page_list))) { - pool_free_page(pool, page); + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + while ((page = list_first_entry_or_null( + &pool->page_list[list_idx], + struct dma_page, + dma_list))) { + pool_free_page(pool, page); + } } kfree(pool); @@ -325,10 +345,11 @@ void *dma_pool_alloc(struct dma_pool *po might_sleep_if(gfpflags_allow_blocking(mem_flags)); spin_lock_irqsave(&pool->lock, flags); - list_for_each_entry(page, &pool->page_list, page_list) { - if (page->offset < pool->allocation) - goto ready; - } + page = list_first_entry_or_null(&pool->page_list[POOL_AVAIL_IDX], + struct dma_page, + dma_list); + if (page) + goto ready; /* pool_alloc_page() might sleep, so temporarily drop &pool->lock */ spin_unlock_irqrestore(&pool->lock, flags); @@ -339,11 +360,15 @@ void *dma_pool_alloc(struct dma_pool *po spin_lock_irqsave(&pool->lock, flags); - list_add(&page->page_list, &pool->page_list); + list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: page->in_use++; offset = page->offset; page->offset = *(int *)(page->vaddr + offset); + if (page->offset >= pool->allocation) + /* Move page from the "available" list to the "full" list. */ + list_move_tail(&page->dma_list, + &pool->page_list[POOL_FULL_IDX]); retval = offset + page->vaddr; *handle = offset + page->dma; #ifdef DMAPOOL_DEBUG @@ -381,13 +406,19 @@ EXPORT_SYMBOL(dma_pool_alloc); static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) { - struct dma_page *page; + int list_idx; - list_for_each_entry(page, &pool->page_list, page_list) { - if (dma < page->dma) - continue; - if ((dma - page->dma) < pool->allocation) - return page; + for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { + struct dma_page *page; + + list_for_each_entry(page, + &pool->page_list[list_idx], + dma_list) { + if (dma < page->dma) + continue; + if ((dma - page->dma) < pool->allocation) + return page; + } } return NULL; } @@ -444,6 +475,9 @@ void dma_pool_free(struct dma_pool *pool #endif page->in_use--; + if (page->offset >= pool->allocation) + /* Move page from the "full" list to the "available" list. */ + list_move(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); *(int *)vaddr = page->offset; page->offset = offset; /* From patchwork Tue Aug 7 16:47:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C91814E5 for ; Tue, 7 Aug 2018 16:47:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C3372A621 for ; Tue, 7 Aug 2018 16:47:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 002572A692; Tue, 7 Aug 2018 16:47:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93ACB2A621 for ; Tue, 7 Aug 2018 16:47:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF1636B000E; Tue, 7 Aug 2018 12:47:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AA15C6B0010; Tue, 7 Aug 2018 12:47:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 991D96B0266; Tue, 7 Aug 2018 12:47:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f199.google.com (mail-qt0-f199.google.com [209.85.216.199]) by kanga.kvack.org (Postfix) with ESMTP id 714836B000E for ; Tue, 7 Aug 2018 12:47:31 -0400 (EDT) Received: by mail-qt0-f199.google.com with SMTP id e14-v6so13745466qtp.17 for ; Tue, 07 Aug 2018 09:47:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=sKpmnAcRyha5G2We6zNr26W0mYYtlt+pU+AiMoJ/JwM=; b=hL1iv3iBCOiTQ+gbsdWAIh4jQDz9PluFhbb3uOKH0Kkr2L8c2/qY+XhupaKoQlrJmL nIhI6zrQyTCLHEJBbi0stxgz8KqGJn2nQ8rRLQX1hZoch/TeMzc/PrqV1QkxMv88+nDb 7KT++ocbGotjvXjmgd7wTXTP7HQiTDZdnJQUS9popdfCF1uOcy+AlH8fMzmKR9/7qEsi 4mP0KnSrwgwCxQ2zhuNt/THoLaSCP7EUyNaYy1tmnU7TBapYzsOj9SaJmiagItjhHonP mUZKRf1ARf85LuvavwU5hKFIvmJC1UZAZyBahU4K8/StCPplEpCt77Vp+CgxKR9wRSP4 CCaA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlGmdJGFfRFORG5v7H+G8txFxeI6bTppDo0aBY08+fgRr7SSq7Dl E/tan5nNJ+iJmh4HS2tM3f/WDCWo1oc6wS2qeKZpgenLYylXk4peNzuD8BYUhkMQ4sTUmxV4ecB hn2dFTeuSqLR91mkHMNGedEt+nRRKrzOvEd8avk8oh+PN7fbY2MDRT5LSJHY2kRM8eA== X-Received: by 2002:a37:cf43:: with SMTP id e64-v6mr17281253qkj.68.1533660451238; Tue, 07 Aug 2018 09:47:31 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeT3wTaNaSMW1zlSQkWMgXzLVsJDq23IG2afZKNN8sWEomof0ZcTGuvgKkvPy5dshTz6z0o X-Received: by 2002:a37:cf43:: with SMTP id e64-v6mr17281193qkj.68.1533660450235; Tue, 07 Aug 2018 09:47:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660450; cv=none; d=google.com; s=arc-20160816; b=XgwKSrR4DwmudcGYpoIFMKpGtI9PuOVoid4WrRvjI8bh8i2yJOz5+89jbyzkrMObxp gqeC4aVWXbtN4lNU5Sbc3Ga+F9qOMmPTsNQLpeB0x6jBc6wy0bX+Jh58zDOD687fMGWF /Ux9ely/wNPNnrTRLsPhs5S6UHYjhMGGCumHXmo2K+ArqBbWX0zC4kSacs0riAc2WUIK YNaxAEOHnzPEKTN7SRdyiG2xieY7X7Q0r3/KFwB12zxNX1N9L317HeHZyjzhfLPV/5Fy NsPh+YCAK1ZtSF7x0EhoRDpo4TXz2ptPxtT5iciYe4M4nPpcjGI/SI8PCFlnDlmrJ9RE S08w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=sKpmnAcRyha5G2We6zNr26W0mYYtlt+pU+AiMoJ/JwM=; b=U0QwRuD2833oJgCeDCooQl1Aw5LByWLdnxdMGbjvMXCMbiTEQZqdKAFH6B5GLrhnPA O1yMj6sh7XpW5cPMPBMajccYZ1wtrx84ru4YFNv9JGjDNR90PYi5BmdvCJvey1Cs+OkA GqvMPeQ/Qnmtma+C/vatSUgp8DLiIJWCrxFGSjJMTzZjIdqkOfvup2fexCj0R6uTag2D OS4VsM1/WHt39+F3zywKbXchbZXE2Ide2xdlg93qdVzslYQ+BAjFQ3GC134GMUgFcmze v6P9/2UG7N2HeIc7LbgKHH4RBvkdzxX3tjSPqQgaj9l0dGj4y2Zs1v6OBNgdxW0eTUbJ fXig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id f20-v6si1334366qtp.48.2018.08.07.09.47.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:47:30 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660448-0fb3b01fb342b8c0001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id aoLRbHEitRbZoBdk (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:47:28 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324865; Tue, 07 Aug 2018 12:47:28 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 05/10] dmapool: rename fields in dma_page Message-ID: X-ASG-Orig-Subj: [PATCH v3 05/10] dmapool: rename fields in dma_page Date: Tue, 7 Aug 2018 12:47:28 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660448 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3410 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Rename fields in 'struct dma_page' in preparation for moving them into 'struct page'. No functional changes. in_use -> dma_in_use offset -> dma_free_off Signed-off-by: Tony Battersby --- Changes since v2: Use dma_free_off instead of dma_free_o. --- linux/mm/dmapool.c.orig 2018-08-03 17:46:13.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 17:46:24.000000000 -0400 @@ -65,8 +65,8 @@ struct dma_page { /* cacheable header f struct list_head dma_list; void *vaddr; dma_addr_t dma; - unsigned int in_use; - unsigned int offset; + unsigned int dma_in_use; + unsigned int dma_free_off; }; static DEFINE_MUTEX(pools_lock); @@ -101,7 +101,7 @@ show_pools(struct device *dev, struct de &pool->page_list[list_idx], dma_list) { pages++; - blocks += page->in_use; + blocks += page->dma_in_use; } } spin_unlock_irq(&pool->lock); @@ -248,8 +248,8 @@ static struct dma_page *pool_alloc_page( memset(page->vaddr, POOL_POISON_FREED, pool->allocation); #endif pool_initialise_page(pool, page); - page->in_use = 0; - page->offset = 0; + page->dma_in_use = 0; + page->dma_free_off = 0; } else { kfree(page); page = NULL; @@ -259,7 +259,7 @@ static struct dma_page *pool_alloc_page( static inline bool is_page_busy(struct dma_page *page) { - return page->in_use != 0; + return page->dma_in_use != 0; } static void pool_free_page(struct dma_pool *pool, struct dma_page *page) @@ -362,10 +362,10 @@ void *dma_pool_alloc(struct dma_pool *po list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: - page->in_use++; - offset = page->offset; - page->offset = *(int *)(page->vaddr + offset); - if (page->offset >= pool->allocation) + page->dma_in_use++; + offset = page->dma_free_off; + page->dma_free_off = *(int *)(page->vaddr + offset); + if (page->dma_free_off >= pool->allocation) /* Move page from the "available" list to the "full" list. */ list_move_tail(&page->dma_list, &pool->page_list[POOL_FULL_IDX]); @@ -375,8 +375,8 @@ void *dma_pool_alloc(struct dma_pool *po { int i; u8 *data = retval; - /* page->offset is stored in first 4 bytes */ - for (i = sizeof(page->offset); i < pool->size; i++) { + /* page->dma_free_off is stored in first 4 bytes */ + for (i = sizeof(page->dma_free_off); i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; dev_err(pool->dev, @@ -458,7 +458,7 @@ void dma_pool_free(struct dma_pool *pool return; } { - unsigned int chain = page->offset; + unsigned int chain = page->dma_free_off; while (chain < pool->allocation) { if (chain != offset) { chain = *(int *)(page->vaddr + chain); @@ -474,12 +474,12 @@ void dma_pool_free(struct dma_pool *pool memset(vaddr, POOL_POISON_FREED, pool->size); #endif - page->in_use--; - if (page->offset >= pool->allocation) + page->dma_in_use--; + if (page->dma_free_off >= pool->allocation) /* Move page from the "full" list to the "available" list. */ list_move(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); - *(int *)vaddr = page->offset; - page->offset = offset; + *(int *)vaddr = page->dma_free_off; + page->dma_free_off = offset; /* * Resist a temptation to do * if (!is_page_busy(page)) pool_free_page(pool, page); From patchwork Tue Aug 7 16:48:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BD0013B4 for ; Tue, 7 Aug 2018 16:48:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B5952A440 for ; Tue, 7 Aug 2018 16:48:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2DE902A446; Tue, 7 Aug 2018 16:48:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D9FC2A440 for ; Tue, 7 Aug 2018 16:48:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DF646B0266; Tue, 7 Aug 2018 12:48:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9B9076B0269; Tue, 7 Aug 2018 12:48:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A8E16B026A; Tue, 7 Aug 2018 12:48:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 627F66B0266 for ; Tue, 7 Aug 2018 12:48:07 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id l13-v6so13808082qth.8 for ; Tue, 07 Aug 2018 09:48:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=QjaROLoVqnHy6va8AXxZ6kuoQJh9ZARaqURGNLQ4hgY=; b=SsIqgihchy9mCOy/KsS/XkLcvIzKdi9luUGI8hvDTkhAXOhzYk6MfkwurfyaSN0t2N iT24R4bQ0GT5TT9sOAOR0Da9eLXzHHkb49wijZwImsKJONe9rvXR2LPVDmvEP/XkuiB6 D3WENoCTGrtp1y3raDiovISMkZt1tQfJEgIOfh2+2/y87nz5/m4HM9fTS5wem0GCdCGT ZxgacmeJkY8rNAwP264WzdZSINjIvH97hwovI/DHL7cC8ZtCFPVMErXKZOceSEwzALGH W01DW2jbzgUMkNEv4lPyXsRWZdeu8MTqJp4UHccZXeibcMUEfItxm2KLbkxkqAzbKrs/ l1Ew== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlEuS2DNf1YL00k4+aYo00ujd45TYkFtfRTKvJQKlwSFhLIk4ENq cLAXr2UYiF0bZTkspzSRhW0KmDno4QD0NLONzMzg954VwXDYynxutD0iy8DtffyHo1q9JrvMmOb lPn+/mq/Zi5zubyaS+dkDgBzb6CnqexRoJLTjvIEQu3eGd3BHeOAdrh3MetYCuqv8cw== X-Received: by 2002:a0c:fac4:: with SMTP id p4-v6mr17679648qvo.179.1533660487085; Tue, 07 Aug 2018 09:48:07 -0700 (PDT) X-Google-Smtp-Source: AAOMgpccBTbo5GeNPugCTq3/A2GVYEygudr3cgqYzH60M1OeMLC5gp3wFQi4DOssp3p/SorpP3RT X-Received: by 2002:a0c:fac4:: with SMTP id p4-v6mr17679597qvo.179.1533660486013; Tue, 07 Aug 2018 09:48:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660485; cv=none; d=google.com; s=arc-20160816; b=PmuqFYeZjgyzSFEuN5sT8PklHlfvPXXiLypOhAei6VgcWj9hWJP8aSJeZe9Z3Q1yxx t9W5t6KkTIoeG4Ys+yf90aeq5qBfhP/xb7WwzTT0YdDNzp3SUHvm//UGdyw1/8iYZx5g S1lW5amgYPZUrPoODFTY+H60JjpImB1kdTqGRwFgG6u/l3WWxDGLPXZ+dn7xFpRUBVzl kYyrl1ScX9Vaw996p4q+XOZcCib5MTNYusb3hkDh8klXWwFF4J5ZM8i8G8MZVrdewF0N ezdUueZAbMHrPdlKnT3vUAVyFE/CB0cIaAztQ74gL3utTgEBYugpfXKhsV1Gxa6qU617 ZdYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=QjaROLoVqnHy6va8AXxZ6kuoQJh9ZARaqURGNLQ4hgY=; b=d6ApDaybwjk/nFfx3RkwxZU3BlH7jky+eeiDr2d/6YBlRM7NjkgV4OM8dRadO36fw0 MMe3q1K3LNJ6+Qp60XRib7dQfr5sgVppO/h5QuQSV7PzaEdsLYQOweZ1/rbYs0k6nUl7 yQzsrRhK4/IMGNHXk99hdhn/J5g1LnlFGTOWWBfNPA68HdIVsDMQdN+d/51gt3XnXE+Y 01RnYMxKNehjfcay1I611tuiQVCTxBoTWh48NLgvjYMQUnfBEMgdqsg/8Oh0GbCzF+uF x3vL8UvD/bhQykfI3YNjJZcsI14a8+sBGphwkb4JnRk8bm+uooqcR8JnMJVDLsJmKhQZ t+PQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id f39-v6si1032031qtk.237.2018.08.07.09.48.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:48:05 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660483-0fb3b01fb342b910001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id folF0MDlsSuJcxid (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:48:03 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324868; Tue, 07 Aug 2018 12:48:03 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 06/10] dmapool: improve scalability of dma_pool_free Message-ID: X-ASG-Orig-Subj: [PATCH v3 06/10] dmapool: improve scalability of dma_pool_free Date: Tue, 7 Aug 2018 12:48:03 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660483 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 8854 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP dma_pool_free() scales poorly when the pool contains many pages because pool_find_page() does a linear scan of all allocated pages. Improve its scalability by replacing the linear scan with virt_to_page() and storing dmapool private data directly in 'struct page', thereby eliminating 'struct dma_page'. In big O notation, this improves the algorithm from O(n^2) to O(n) while also reducing memory usage. Thanks to Matthew Wilcox for the suggestion to use struct page. Signed-off-by: Tony Battersby --- Changes since v2: Just a re-diff after the changes in prior patches. --- linux/include/linux/mm_types.h.orig 2018-08-01 17:59:46.000000000 -0400 +++ linux/include/linux/mm_types.h 2018-08-01 17:59:56.000000000 -0400 @@ -153,6 +153,12 @@ struct page { unsigned long _zd_pad_1; /* uses mapping */ }; + struct { /* dma_pool pages */ + struct list_head dma_list; + dma_addr_t dma; + unsigned int dma_free_off; + }; + /** @rcu_head: You can use this to free a page by RCU. */ struct rcu_head rcu_head; }; @@ -174,6 +180,8 @@ struct page { unsigned int active; /* SLAB */ int units; /* SLOB */ + + unsigned int dma_in_use; /* dma_pool pages */ }; /* Usage count. *DO NOT USE DIRECTLY*. See page_ref.h */ --- linux/mm/dmapool.c.orig 2018-08-03 17:47:03.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-03 17:47:22.000000000 -0400 @@ -25,6 +25,10 @@ * list tracks pages that have one or more free blocks, and the 'full' list * tracks pages that have no free blocks. Pages are moved from one list to * the other as their blocks are allocated and freed. + * + * When allocating DMA pages, we use some available space in 'struct page' to + * store data private to dmapool; search 'dma_pool' in the definition of + * 'struct page' for details. */ #include @@ -61,14 +65,6 @@ struct dma_pool { /* the pool */ struct list_head pools; }; -struct dma_page { /* cacheable header for 'allocation' bytes */ - struct list_head dma_list; - void *vaddr; - dma_addr_t dma; - unsigned int dma_in_use; - unsigned int dma_free_off; -}; - static DEFINE_MUTEX(pools_lock); static DEFINE_MUTEX(pools_reg_lock); @@ -95,7 +91,7 @@ show_pools(struct device *dev, struct de spin_lock_irq(&pool->lock); for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; + struct page *page; list_for_each_entry(page, &pool->page_list[list_idx], @@ -218,7 +214,7 @@ struct dma_pool *dma_pool_create(const c } EXPORT_SYMBOL(dma_pool_create); -static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) +static void pool_initialize_free_block_list(struct dma_pool *pool, void *vaddr) { unsigned int offset = 0; unsigned int next_boundary = pool->boundary; @@ -229,47 +225,57 @@ static void pool_initialise_page(struct next = next_boundary; next_boundary += pool->boundary; } - *(int *)(page->vaddr + offset) = next; + *(int *)(vaddr + offset) = next; offset = next; } while (offset < pool->allocation); } -static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) +static struct page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) { - struct dma_page *page; + struct page *page; + dma_addr_t dma; + void *vaddr; - page = kmalloc(sizeof(*page), mem_flags); - if (!page) + vaddr = dma_alloc_coherent(pool->dev, pool->allocation, &dma, + mem_flags); + if (!vaddr) return NULL; - page->vaddr = dma_alloc_coherent(pool->dev, pool->allocation, - &page->dma, mem_flags); - if (page->vaddr) { + #ifdef DMAPOOL_DEBUG - memset(page->vaddr, POOL_POISON_FREED, pool->allocation); + memset(vaddr, POOL_POISON_FREED, pool->allocation); #endif - pool_initialise_page(pool, page); - page->dma_in_use = 0; - page->dma_free_off = 0; - } else { - kfree(page); - page = NULL; - } + pool_initialize_free_block_list(pool, vaddr); + + page = virt_to_page(vaddr); + page->dma = dma; + page->dma_free_off = 0; + page->dma_in_use = 0; + return page; } -static inline bool is_page_busy(struct dma_page *page) +static inline bool is_page_busy(struct page *page) { return page->dma_in_use != 0; } -static void pool_free_page(struct dma_pool *pool, struct dma_page *page) +static void pool_free_page(struct dma_pool *pool, struct page *page) { - void *vaddr = page->vaddr; + /* Save local copies of some page fields. */ + void *vaddr = page_to_virt(page); + bool busy = is_page_busy(page); dma_addr_t dma = page->dma; list_del(&page->dma_list); - if (is_page_busy(page)) { + /* Clear all the page fields we use. */ + page->dma_list.next = NULL; + page->dma_list.prev = NULL; + page->dma = 0; + page->dma_free_off = 0; + page_mapcount_reset(page); /* clear dma_in_use */ + + if (busy) { dev_err(pool->dev, "dma_pool_destroy %s, %p busy\n", pool->name, vaddr); @@ -280,7 +286,6 @@ static void pool_free_page(struct dma_po #endif dma_free_coherent(pool->dev, pool->allocation, vaddr, dma); } - kfree(page); } /** @@ -310,11 +315,11 @@ void dma_pool_destroy(struct dma_pool *p mutex_unlock(&pools_reg_lock); for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; + struct page *page; while ((page = list_first_entry_or_null( &pool->page_list[list_idx], - struct dma_page, + struct page, dma_list))) { pool_free_page(pool, page); } @@ -338,15 +343,16 @@ void *dma_pool_alloc(struct dma_pool *po dma_addr_t *handle) { unsigned long flags; - struct dma_page *page; + struct page *page; size_t offset; void *retval; + void *vaddr; might_sleep_if(gfpflags_allow_blocking(mem_flags)); spin_lock_irqsave(&pool->lock, flags); page = list_first_entry_or_null(&pool->page_list[POOL_AVAIL_IDX], - struct dma_page, + struct page, dma_list); if (page) goto ready; @@ -362,14 +368,15 @@ void *dma_pool_alloc(struct dma_pool *po list_add(&page->dma_list, &pool->page_list[POOL_AVAIL_IDX]); ready: + vaddr = page_to_virt(page); page->dma_in_use++; offset = page->dma_free_off; - page->dma_free_off = *(int *)(page->vaddr + offset); + page->dma_free_off = *(int *)(vaddr + offset); if (page->dma_free_off >= pool->allocation) /* Move page from the "available" list to the "full" list. */ list_move_tail(&page->dma_list, &pool->page_list[POOL_FULL_IDX]); - retval = offset + page->vaddr; + retval = offset + vaddr; *handle = offset + page->dma; #ifdef DMAPOOL_DEBUG { @@ -404,25 +411,6 @@ void *dma_pool_alloc(struct dma_pool *po } EXPORT_SYMBOL(dma_pool_alloc); -static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) -{ - int list_idx; - - for (list_idx = 0; list_idx < POOL_MAX_IDX; list_idx++) { - struct dma_page *page; - - list_for_each_entry(page, - &pool->page_list[list_idx], - dma_list) { - if (dma < page->dma) - continue; - if ((dma - page->dma) < pool->allocation) - return page; - } - } - return NULL; -} - /** * dma_pool_free - put block back into dma pool * @pool: the dma pool holding the block @@ -434,34 +422,35 @@ static struct dma_page *pool_find_page(s */ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) { - struct dma_page *page; + struct page *page; unsigned long flags; unsigned int offset; - spin_lock_irqsave(&pool->lock, flags); - page = pool_find_page(pool, dma); - if (!page) { - spin_unlock_irqrestore(&pool->lock, flags); + if (unlikely(!virt_addr_valid(vaddr))) { dev_err(pool->dev, - "dma_pool_free %s, %p/%lx (bad dma)\n", - pool->name, vaddr, (unsigned long)dma); + "dma_pool_free %s, %p (bad vaddr)/%pad\n", + pool->name, vaddr, &dma); return; } - offset = vaddr - page->vaddr; -#ifdef DMAPOOL_DEBUG - if ((dma - page->dma) != offset) { - spin_unlock_irqrestore(&pool->lock, flags); + page = virt_to_page(vaddr); + offset = offset_in_page(vaddr); + + if (unlikely((dma - page->dma) != offset)) { dev_err(pool->dev, - "dma_pool_free %s, %p (bad vaddr)/%pad\n", + "dma_pool_free %s, %p (bad vaddr)/%pad (or bad dma)\n", pool->name, vaddr, &dma); return; } + + spin_lock_irqsave(&pool->lock, flags); +#ifdef DMAPOOL_DEBUG { + void *page_vaddr = vaddr - offset; unsigned int chain = page->dma_free_off; while (chain < pool->allocation) { if (chain != offset) { - chain = *(int *)(page->vaddr + chain); + chain = *(int *)(page_vaddr + chain); continue; } spin_unlock_irqrestore(&pool->lock, flags); From patchwork Tue Aug 7 16:48:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2355E13B4 for ; Tue, 7 Aug 2018 16:48:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D6DC2A70A for ; Tue, 7 Aug 2018 16:48:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0173B2A70D; Tue, 7 Aug 2018 16:48:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C5752A70A for ; Tue, 7 Aug 2018 16:48:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A304E6B026A; Tue, 7 Aug 2018 12:48:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A04526B026C; Tue, 7 Aug 2018 12:48:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91C1A6B026D; Tue, 7 Aug 2018 12:48:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f197.google.com (mail-qk0-f197.google.com [209.85.220.197]) by kanga.kvack.org (Postfix) with ESMTP id 698D76B026A for ; Tue, 7 Aug 2018 12:48:41 -0400 (EDT) Received: by mail-qk0-f197.google.com with SMTP id l15-v6so17061394qki.18 for ; Tue, 07 Aug 2018 09:48:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=lslOrQ3VU/qhT82QQArSXyHZa5d7ZlKdm5nn85023Do=; b=QENa8M9EEWNFtC0opi9HfxKCMsLmjb0ge+c+k8uzO/x7cjmuwUkGh/8uF2+JfokPuK eJDipWJBbUrecmG+fyrDOvYv8qY+MfzcPmX9sYGq4TbCeyPD8GCifZdLKqpriRbD2eM/ iU4LwbUze8bnw+Pex7YyMLfggHqmq4oCG6ndkycAytShlLTN6QVn95YKh1udc3ZpxmZT t5wyfpUjfdRyDnNwXFua8tIaNzYCHeIiNaEgU3pCRbCE6ZfxQJm9v7X0GjSLWReoSUFr 1cP6OTkh4hxFXDF1krzmjh8j1UMt77qx5p86YnSQBp5dZjN4qxf3Z7PW8nrG9z3rHroY UYgQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlE2dk5fx3mdMRukMmHzbX3es2PA2vmjvpZ2kBBhcD01kuVGuaJK GhJZBe0LVLUGw8J5b5OzDZ7pzrlAKpSMYfwuHV3xNqvaysJxpM3MpHTNDnFBcrbgq4jo6c/ZOfO hsIGqbyFJuNUCSEB3IPUBvfcZZfp56lEUho14RobR4iUrj2Bpi79AMMosmS0k+fdIIA== X-Received: by 2002:aed:355b:: with SMTP id b27-v6mr18812374qte.399.1533660521200; Tue, 07 Aug 2018 09:48:41 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdifmsLVRQ2OyEONAExKl01PdIlnWbyjYRpliYb+yPjyLc4vETUGtQWPHRUcnCErHcIahYE X-Received: by 2002:aed:355b:: with SMTP id b27-v6mr18812326qte.399.1533660520540; Tue, 07 Aug 2018 09:48:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660520; cv=none; d=google.com; s=arc-20160816; b=lpqFmOATHZKAvwnhshrHWs//xMTxLKOFPOhXIZk6r2H6Ibn0acgxt2hX036MzS47xy 2+9jDGU6OPq1wCeoU1HZdeF1O2ftSRvQ9U5Mrlg1QFK93pEkn/sDLCSaM5lFCsTXnqBQ DyVWLEX8QrOboWdFdh83QDiB9s+Gg0DqhhJswFtKXOS3uYzbQRudvMiE7OnzHtn0xx47 5bMd6HkjtoLpd7Q979nE9c1drCfJw5PH7fMKEihrG4Ofx7E2ZDr/TjCrcyLYJGtIKwa5 UqKX1aTwvtGG9bLI0TswdoHl6MmfMk4GH83FjGP/Kt8EKTQSYdWF9a6FqtUZG9pli8qa 2Tag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=lslOrQ3VU/qhT82QQArSXyHZa5d7ZlKdm5nn85023Do=; b=jDOqMKgmHtn1kkWp7fUtEOBI7qfk3JNqQLFCVvsWt0QzJFVYywDQfBd61OwkkKAWW2 kDzPDZrOVXLy5cRYVBDEDluLnc+mW66NX/baQehsS50vNwXMqB20NpEhaRWtxAvAIIoj T0FDUx1jFCQnpKAt7pQIhIW+lq1+3SamdcQk3tj7UM53vRKwH+cmC2L5stql5TNAFOTS R1pac6+3uQtPFDGOdRPf5p8KZaRpRmgNUUi7DMcQcxvqW0w8ehnIb+4Y/2OqNWQFAyvf kf4hDHOY3D5QP6BZawOKSRIVgXLAsdu4tI8h9H6IuDO8DGAQH15ggIhYLit9aXhw/Ngu +4uA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id d20-v6si276777qvf.180.2018.08.07.09.48.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:48:40 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660518-0fb3b01fb342b940001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id e9CaRxsAFf7Ms5D1 (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:48:38 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324871; Tue, 07 Aug 2018 12:48:38 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 07/10] dmapool: cleanup integer types Message-ID: <5d0aec14-73e0-280d-62fb-2b0fe6c01418@cybernetics.com> X-ASG-Orig-Subj: [PATCH v3 07/10] dmapool: cleanup integer types Date: Tue, 7 Aug 2018 12:48:38 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660518 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3586 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To represent the size of a single allocation, dmapool currently uses 'unsigned int' in some places and 'size_t' in other places. Standardize on 'unsigned int' to reduce overhead, but use 'size_t' when counting all the blocks in the entire pool. Signed-off-by: Tony Battersby --- This was split off from "dmapool: reduce footprint in struct page" in v2. This puts an upper bound on 'size' of INT_MAX to avoid overflowing the following comparison in pool_initialize_free_block_list(): unsigned int offset = 0; unsigned int next = offset + pool->size; if (unlikely((next + pool->size) > ... The actual maximum allocation size is probably lower anyway, probably KMALLOC_MAX_SIZE, but that gets into the implementation details of other subsystems which don't export a predefined maximum, so I didn't want to hardcode it here. The purpose of the added bounds check is to avoid overflowing integers, not to check the actual (platform/device/config-specific?) maximum allocation size. 'boundary' is passed in as a size_t but gets stored as an unsigned int. 'boundary' values >= 'allocation' do not have any effect, so clipping 'boundary' to 'allocation' keeps it within the range of unsigned int without affecting anything else. A few lines above (not in the diff) you can see that if 'boundary' is passed in as 0 then it is set to 'allocation', so it is nothing new. For reference, here is the relevant code after being patched: if (!boundary) boundary = allocation; else if ((boundary < size) || (boundary & (boundary - 1))) return NULL; boundary = min(boundary, allocation); --- linux/mm/dmapool.c.orig 2018-08-06 17:48:19.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:48:54.000000000 -0400 @@ -57,10 +57,10 @@ struct dma_pool { /* the pool */ #define POOL_MAX_IDX 2 struct list_head page_list[POOL_MAX_IDX]; spinlock_t lock; - size_t size; + unsigned int size; struct device *dev; - size_t allocation; - size_t boundary; + unsigned int allocation; + unsigned int boundary; char name[32]; struct list_head pools; }; @@ -86,7 +86,7 @@ show_pools(struct device *dev, struct de mutex_lock(&pools_lock); list_for_each_entry(pool, &dev->dma_pools, pools) { unsigned pages = 0; - unsigned blocks = 0; + size_t blocks = 0; int list_idx; spin_lock_irq(&pool->lock); @@ -103,9 +103,10 @@ show_pools(struct device *dev, struct de spin_unlock_irq(&pool->lock); /* per-pool info, no real statistics yet */ - temp = scnprintf(next, size, "%-16s %4u %4zu %4zu %2u\n", + temp = scnprintf(next, size, "%-16s %4zu %4zu %4u %2u\n", pool->name, blocks, - pages * (pool->allocation / pool->size), + (size_t) pages * + (pool->allocation / pool->size), pool->size, pages); size -= temp; next += temp; @@ -150,7 +151,7 @@ struct dma_pool *dma_pool_create(const c else if (align & (align - 1)) return NULL; - if (size == 0) + if (size == 0 || size > INT_MAX) return NULL; else if (size < 4) size = 4; @@ -165,6 +166,8 @@ struct dma_pool *dma_pool_create(const c else if ((boundary < size) || (boundary & (boundary - 1))) return NULL; + boundary = min(boundary, allocation); + retval = kmalloc_node(sizeof(*retval), GFP_KERNEL, dev_to_node(dev)); if (!retval) return retval; @@ -344,7 +347,7 @@ void *dma_pool_alloc(struct dma_pool *po { unsigned long flags; struct page *page; - size_t offset; + unsigned int offset; void *retval; void *vaddr; From patchwork Tue Aug 7 16:49:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558929 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A262D14E5 for ; Tue, 7 Aug 2018 16:49:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 91C442A70A for ; Tue, 7 Aug 2018 16:49:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 85BE12A713; Tue, 7 Aug 2018 16:49:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C45F2A70A for ; Tue, 7 Aug 2018 16:49:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 571826B026D; Tue, 7 Aug 2018 12:49:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5229D6B026E; Tue, 7 Aug 2018 12:49:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4381C6B026F; Tue, 7 Aug 2018 12:49:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id 1CFD86B026D for ; Tue, 7 Aug 2018 12:49:16 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id w14-v6so17386566qkw.2 for ; Tue, 07 Aug 2018 09:49:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=InR8UYivZt3Do63AQUtrjdguD20rxdIFWErYGtEp+YI=; b=emZSZV8u++/YdQBlwBNNwX1JrwziYlyYeaIorNJWCiPWzmZFnhclXL18QoNKKpUsX/ wSFQbbHr/yjLtYi98VMMos1cCPqMQKya+fgnp8uOY4rgSEL5nHIvm0CSYZSBctlOZdXe 7JDtnS53nXCoJ3j8Xy7k++q2aTDDM7lO4aE61Lw9pQJChsHiOITB/G92wNovZrz4G4/r 0tfaunDJLEGEWxiNJeIEWVMY3y/p+021rgfyE+9+7XBkeUhQeDHdPdCK9QmP9wlMCcLJ NiWzisoxtgsLJrFCfkqzvBFfuXgiiJgGPBuMY40Q645C+XbB74GiVCCBnVqeVZYLjJd4 3xHA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlEoxcXyIFd8NakRF7bVIrSgAIQ6u1kZFPuJGPXFikQI9cLg/tMB yC438z7fuMV4jYG0MntvF+l1/KuAsW7sOrDdgaIlG29L37svLOikYf8EG49vGhYoaK9GliRXrZB mT62ftzmBxWyzVR47C4YOlXpFZKaRwOYhlzByvCY9tnMzRmCpJEM43oNQD+g7sRzRUg== X-Received: by 2002:a37:4150:: with SMTP id o77-v6mr17948223qka.78.1533660555893; Tue, 07 Aug 2018 09:49:15 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeMx7PMz6pRgr35lOyXpP3Vaz78y9eSKATuVKgdz8qNMPIzzAEUJLPNi79ve5TZ1rB1QSks X-Received: by 2002:a37:4150:: with SMTP id o77-v6mr17948189qka.78.1533660555283; Tue, 07 Aug 2018 09:49:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660555; cv=none; d=google.com; s=arc-20160816; b=dCzdJiJV23mWDMrFgZjlDdpYhW2ssFL50nV6MlNiIRcbtVmQePITB0f7KeJwI5/BFq GS1WTOVQ5Dp53HMGqi/hqU4DZQtkz7GG+1KNhuXLSOVJZcczp2dRI+p3OMK7hOJcPwZ6 9yIkB6usPWwB3Ggk/P0RbqG9mPb+xWXeZq+MybKF3VWNHrHMU6hCfJNybBwev1hEdh7d CpuVA7Zcbhu1nYJB6j83uoMhMdmxV81PUCx83UPw7jLEdD05vmWPAhBoitSBHZdHufaM E85Wlxtd9nH+/Vq8MhbjGAkPjLkNn0S41oxgRJkQyD2hKmnIoNdyTANk7tFvsXCWEzvl yY4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=InR8UYivZt3Do63AQUtrjdguD20rxdIFWErYGtEp+YI=; b=GSrioRG1ejmiPrIx6X4xegAin2IuGp2LSjGu144+t1MUpC1h7IVFcbk8/056+me0vC DxVWk/avcxejjhyvDpltuj29iPYm7cfMN5ItKF2mlZfq1TnDbZwbek0EukE5zHjVPkfB qnEU3S8kNGA3dE+w5rogIK8phULT0PI1nM44z/YtF5q9G71aqh+agF3vI4PFsJGRFlmL y5+I5db7WqMff/WodjVcjIUJpasYMTSRz07JQyovF4xIudmqnetUpr4EAmTS0aJ29k6A PPrlzRLunVTZJxM8tfa3FHxR+IUB4GiLwNcugepoQWs6/I9UqVOIQ+i+zCqqaTpEhuGM V/2A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id y74-v6si1741573qkg.315.2018.08.07.09.49.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:49:15 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660553-0fb3b01fb342b970001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id cMcEferQEOTLuIkw (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:49:13 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324867; Tue, 07 Aug 2018 12:49:13 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 08/10] dmapool: improve accuracy of debug statistics Message-ID: X-ASG-Orig-Subj: [PATCH v3 08/10] dmapool: improve accuracy of debug statistics Date: Tue, 7 Aug 2018 12:49:13 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660553 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 1709 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The "total number of blocks in pool" debug statistic currently does not take the boundary value into account, so it diverges from the "total number of blocks in use" statistic when a boundary is in effect. Add a calculation for the number of blocks per allocation that takes the boundary into account, and use it to replace the inaccurate calculation. Signed-off-by: Tony Battersby --- This was split off from "dmapool: reduce footprint in struct page" in v2. This depends on patch #1 "dmapool: fix boundary comparison" for the calculated blks_per_alloc value to be correct. The added blks_per_alloc value will also be used in the next patch. --- linux/mm/dmapool.c.orig 2018-08-06 17:48:54.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:52:53.000000000 -0400 @@ -61,6 +61,7 @@ struct dma_pool { /* the pool */ struct device *dev; unsigned int allocation; unsigned int boundary; + unsigned int blks_per_alloc; char name[32]; struct list_head pools; }; @@ -105,8 +106,7 @@ show_pools(struct device *dev, struct de /* per-pool info, no real statistics yet */ temp = scnprintf(next, size, "%-16s %4zu %4zu %4u %2u\n", pool->name, blocks, - (size_t) pages * - (pool->allocation / pool->size), + (size_t) pages * pool->blks_per_alloc, pool->size, pages); size -= temp; next += temp; @@ -182,6 +182,9 @@ struct dma_pool *dma_pool_create(const c retval->size = size; retval->boundary = boundary; retval->allocation = allocation; + retval->blks_per_alloc = + (allocation / boundary) * (boundary / size) + + (allocation % boundary) / size; INIT_LIST_HEAD(&retval->pools); From patchwork Tue Aug 7 16:50:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D15013B4 for ; Tue, 7 Aug 2018 16:50:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E7382A710 for ; Tue, 7 Aug 2018 16:50:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5CAB32A739; Tue, 7 Aug 2018 16:50:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 083B22A710 for ; Tue, 7 Aug 2018 16:50:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE7616B026F; Tue, 7 Aug 2018 12:50:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E70676B0270; Tue, 7 Aug 2018 12:50:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D60996B0271; Tue, 7 Aug 2018 12:50:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id A73D66B026F for ; Tue, 7 Aug 2018 12:50:18 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id d194-v6so17099998qkb.12 for ; Tue, 07 Aug 2018 09:50:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=qsnW8tRX4OIj7zd2gtaXNu7odEk3O8vE/fsQylMHbBU=; b=oyIp2fN3MCpD2siXiW1ou2oVCkl7c3WZAJUyQR0BK1rW4LmN3SiPKb3GfcjJyna1Ig b19H3vGep/md7d9itghYd52USamWohgTUnDv+t9QqpL3oQETyx7GJ/5PewNdAkodFi8N IGuQcImjumqwhTcLUbhBvQKXtAi673s0HhEBqwoceyG5kbVHUMf7L9F67yFqiVZuU9Jl vqUXG45GP6Wrq8WbhEsVQp3HvHVFFaiHacsPlP1FVq0cq1EKwtwj5BZIG6dPcMdxIaq9 KgFLGB5x/0Eqshe+5XlnqNQSHVADz7Z8UwMIHjP+5GQUAvKIYsuEn4XoqMZFt4Ro1Bxk lEAw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlHEmYvJry22/AC2XnKf483Oyp5yXHgg9eiUtewc91/mEljP80tw WKrYl1JhB1kt7WegpJgNkvObKrtHaKMI951zA6eDUGg1e9VDPymNbhTvVgvvB3Z1E8LMQaEvn/l mYxdC2eBWkxaXvwcuTvLQ5bFRfykmArAHV/Fbw+ZollvhWhlSxew3gMbdUidC8i93Jw== X-Received: by 2002:a37:b91:: with SMTP id 139-v6mr18084598qkl.429.1533660618475; Tue, 07 Aug 2018 09:50:18 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeXwKe9AnQ2jm/QnF9P6As1+QTODBzRulNDkVONK8iatElRNwK7CotKfAFFVM9wPF2Cnd7/ X-Received: by 2002:a37:b91:: with SMTP id 139-v6mr18084564qkl.429.1533660617848; Tue, 07 Aug 2018 09:50:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660617; cv=none; d=google.com; s=arc-20160816; b=DNFDOd8bAYAuMh4YZKXknRQxoIT8/RfU9adwKbPBYpSP/Kbapp0Zh9owhid/tIAHnk LEmuaoT8jVwN9Mya+crZJMGZbXMIU27BObUBDTsnzIIajZpvb+8EXfjpjtnzKwsVstQp frLUhl0uQU8Cz8TmIKxkyI0yg8YSE8xFTmlOU5bzfG0iQxzCfwX7SypS8dV1yx+vwBCr S0i86kfbg61vz/3LoEwmb3QJXQZSq+nSu8aYgoot2K7PRPvJ60BqWBVWV4rxZupo7B3B hY7PLAgipndOz7VESf5/M++qQDR2zjYSyumDIXPTteR3/IGBRNGT1QyAzreffUvsp8xK q1QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=qsnW8tRX4OIj7zd2gtaXNu7odEk3O8vE/fsQylMHbBU=; b=NkqVN/WGMcJTU2zZS4nK8GQyRILkADCvQkiMxf+CFOO9eUZBlza/cSn8vQQXL1uktE z9N92H20YId9na8DzAawM9X1qZr+w+2A3NnQmbr0VwzJI5qBmoKCw96F0SIvxMFHYuIj QavYSEqtvcKQgfpt7tkQi6lJkCgdEpg1+hGlXFaSqoI4zlcHmAXHgBnZv0s95VlcwDH5 lMfcMPUK+yWJ5arBVr+O9xi9WuIwHmn7E1Unb6rGwhFB2I29P9BN6Uhf8fJWXaRRY7+O MZulhwgPlIpatRduC36LpnnkvyGUvZLkn/ID0ZXmFGzg4S4YwYPrETxcSMdNl/4aD2c0 0deg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id f129-v6si1707198qkb.232.2018.08.07.09.50.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:50:17 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660615-0fb3b01fb342ba20001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id Kd0YbZwU8wOeYaST (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:50:15 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324877; Tue, 07 Aug 2018 12:50:15 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 09/10] dmapool: debug: prevent endless loop in case of corruption Message-ID: <63e9878d-4819-572d-7288-b1b2329f056b@cybernetics.com> X-ASG-Orig-Subj: [PATCH v3 09/10] dmapool: debug: prevent endless loop in case of corruption Date: Tue, 7 Aug 2018 12:50:15 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660615 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 1920 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Prevent a possible endless loop with DMAPOOL_DEBUG enabled if a buggy driver corrupts DMA pool memory. Signed-off-by: Tony Battersby --- Changes since v2: This is closer to the improved version from "dmapool: reduce footprint in struct page" in v2 thanks to a previous patch adding blks_per_alloc. --- linux/mm/dmapool.c.orig 2018-08-06 17:52:53.000000000 -0400 +++ linux/mm/dmapool.c 2018-08-06 17:53:31.000000000 -0400 @@ -454,17 +454,39 @@ void dma_pool_free(struct dma_pool *pool { void *page_vaddr = vaddr - offset; unsigned int chain = page->dma_free_off; + unsigned int free_blks = 0; + while (chain < pool->allocation) { - if (chain != offset) { - chain = *(int *)(page_vaddr + chain); - continue; + if (unlikely(chain == offset)) { + spin_unlock_irqrestore(&pool->lock, flags); + dev_err(pool->dev, + "dma_pool_free %s, dma %pad already free\n", + pool->name, &dma); + return; + } + + /* + * A buggy driver could corrupt the freelist by + * use-after-free, buffer overflow, etc. Besides + * checking for corruption, this also prevents an + * endless loop in case corruption causes a circular + * loop in the freelist. + */ + if (unlikely(++free_blks + page->dma_in_use > + pool->blks_per_alloc)) { + freelist_corrupt: + spin_unlock_irqrestore(&pool->lock, flags); + dev_err(pool->dev, + "dma_pool_free %s, freelist corrupted\n", + pool->name); + return; } - spin_unlock_irqrestore(&pool->lock, flags); - dev_err(pool->dev, - "dma_pool_free %s, dma %pad already free\n", - pool->name, &dma); - return; + + chain = *(int *)(page_vaddr + chain); } + if (unlikely(free_blks + page->dma_in_use != + pool->blks_per_alloc)) + goto freelist_corrupt; } memset(vaddr, POOL_POISON_FREED, pool->size); #endif From patchwork Tue Aug 7 16:51:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Battersby X-Patchwork-Id: 10558937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4028513B4 for ; Tue, 7 Aug 2018 16:52:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2FB4B2A70A for ; Tue, 7 Aug 2018 16:52:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 242072A714; Tue, 7 Aug 2018 16:52:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F8892A70A for ; Tue, 7 Aug 2018 16:52:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B0C36B0271; Tue, 7 Aug 2018 12:52:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 82DED6B0272; Tue, 7 Aug 2018 12:52:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71D016B0273; Tue, 7 Aug 2018 12:52:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 46CBE6B0271 for ; Tue, 7 Aug 2018 12:52:28 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id x9-v6so13792216qto.18 for ; Tue, 07 Aug 2018 09:52:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:to:from :subject:message-id:date:user-agent:mime-version :content-transfer-encoding:content-language; bh=+x6kYjlA0T+mCYv39uZCAj8gyVtFZtsm0EMCmnnXRfg=; b=KMN1UXGcJdspTo9w/6wRAYmZFrZa5FmP2Qv1t5yKX+vNI+DL7QqR+0HjMf44dRZwes +gkx/VzSih6EF8/PHa8s2v222BkIyRq9oH7PBcs9oBdh4k6EwlBzp/WYvpR3KFCEzszT qSApC1f9CmOmYIsI3+vKfNUc2XVxZ9HEVrJw4xTUwcCC9+Mb6i90yF/XWLVm9d3alp3x 4dmOfzblNgPwPWnf/P4im4ug5d2PxRRb5eM0QPyKs2JJO6godsI/aUJ9iql2hbO9v/5q dlQoHV/ySgxmfkDnNBH0SRQktwJ1DD9WMXnmIN9/e+jVLnMXZUseiMYWMbPuZ+gaBIOP 8G0w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-Gm-Message-State: AOUpUlFuZfTtYF2fC199raGplpOvI55WoQoAAz7B0zTl+S3W3MHlcHj5 aqtFjSZJZG3OJuTrW1261qYuceZxxNI/zmV0UDwcdV5eD7GGx3702r2h1qMDXgjaYwOp4RhC/3c sHu24IMA5iYmE9EaVDRfekaIZ9OODChd2o/SYbY4RKXHu4CUTBezywqOA9YPVpkEb1A== X-Received: by 2002:a0c:f708:: with SMTP id w8-v6mr17089851qvn.91.1533660748046; Tue, 07 Aug 2018 09:52:28 -0700 (PDT) X-Google-Smtp-Source: AAOMgpf5F0lDAgL+StuOqsOr1s4cSGCXQSLDRUcsBAjt7buN7gwqJ2YaOONldPHHWYik5Mx0G8X5 X-Received: by 2002:a0c:f708:: with SMTP id w8-v6mr17089786qvn.91.1533660747023; Tue, 07 Aug 2018 09:52:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533660746; cv=none; d=google.com; s=arc-20160816; b=x3BNTS2c+sNivy/OyMH36SBZN4n9ulha833YseSJSLkeBK5TyVgwuoZMBfP/c7DfLj SVnxEDnwsaWdeCNzSdkHqoFkoKdlnB3KcmRm8HcenIS1QO+5RIjJh6DRFui5nOaTaMFM E1/k+wu8xRDpRcZ43Gj1pjS8r3kI0n6oM8TsTm3gPvgH/E43iw50P7kapUHM2rT5AJPb 77yYObGwj79EnD3F6BZY41KCXRtK1j1YezXrsLF/QtcSAGKddNQEkN46slVct9Hstc2x 0oxH+XCUbWyl63pOr2EyDKwOVAkuIBZ8t7PQyGjRoQ8T3thfBLMijq8BfcmgVtYGkFJX WjyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:mime-version:user-agent :date:message-id:subject:from:to:arc-authentication-results; bh=+x6kYjlA0T+mCYv39uZCAj8gyVtFZtsm0EMCmnnXRfg=; b=pyXK1nj+XIkU9yS+5vu0cXvRNxjalVZ9199HNRlw+ti+YY+9HFi979hNH9dTFpkalZ HRYdMVWmHapVpUJt7EmfT04gIQ40lHpcRPgnE/yruDr8ksdBipuLcyfxj9hDjdH80MaR vannsygBqvvdgZMBkRh8VrzSXHuhm2gHLNNIGlfnSvJlHsZO9GQXQknFp5d7VaVBKBEI txfaiDaTdu1mHB+cwT/TVW4asiEa1DV2WJh6qWNs2k5KrjVvYL1mJXOe9navZ16MvdD0 5Hf+/6lqkevFtgyGJ7v9Md40/mfvZUnioAzKWLP4t8hCepbdz4fbnH17CZdkvbCekdMM us4g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" Received: from mail.cybernetics.com (mail.cybernetics.com. [173.71.130.66]) by mx.google.com with ESMTPS id b49-v6si362870qta.299.2018.08.07.09.52.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:52:26 -0700 (PDT) Received-SPF: pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) client-ip=173.71.130.66; Authentication-Results: mx.google.com; spf=pass (google.com: domain of btv1==757ef7a2255==tonyb@cybernetics.com designates 173.71.130.66 as permitted sender) smtp.mailfrom="btv1==757ef7a2255==tonyb@cybernetics.com" X-ASG-Debug-ID: 1533660683-0fb3b01fb342ba60001-v9ZeMO Received: from cybernetics.com ([10.157.1.126]) by mail.cybernetics.com with ESMTP id DJIyl8MvvXFmy7ja (version=SSLv3 cipher=DES-CBC3-SHA bits=112 verify=NO); Tue, 07 Aug 2018 12:51:23 -0400 (EDT) X-Barracuda-Envelope-From: tonyb@cybernetics.com X-ASG-Whitelist: Client Received: from [10.157.2.224] (account tonyb HELO [192.168.200.1]) by cybernetics.com (CommuniGate Pro SMTP 5.1.14) with ESMTPSA id 8324890; Tue, 07 Aug 2018 12:51:23 -0400 To: Matthew Wilcox , Christoph Hellwig , Marek Szyprowski , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-scsi@vger.kernel.org" , "MPT-FusionLinux.pdl@broadcom.com" From: Tony Battersby Subject: [PATCH v3 10/10] [SCSI] mpt3sas: replace chain_dma_pool Message-ID: X-ASG-Orig-Subj: [PATCH v3 10/10] [SCSI] mpt3sas: replace chain_dma_pool Date: Tue, 7 Aug 2018 12:51:23 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Language: en-US X-Barracuda-Connect: UNKNOWN[10.157.1.126] X-Barracuda-Start-Time: 1533660683 X-Barracuda-Encrypted: DES-CBC3-SHA X-Barracuda-URL: https://10.157.1.122:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 7782 X-Virus-Scanned: by bsmtpd at cybernetics.com X-Barracuda-BRTS-Status: 1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Replace chain_dma_pool with direct calls to dma_alloc_coherent() and dma_free_coherent(). Since the chain lookup can involve hundreds of thousands of allocations, it is worthwile to avoid the overhead of the dma_pool API. Signed-off-by: Tony Battersby --- No changes since v1. The original code called _base_release_memory_pools() before "goto out" if dma_pool_alloc() failed, but this was unnecessary because mpt3sas_base_attach() will call _base_release_memory_pools() after "goto out_free_resources". It may have been that way because the out-of-tree vendor driver (from https://www.broadcom.com/support/download-search) has a slightly-more-complicated error handler there that adjusts max_request_credit, calls _base_release_memory_pools() and then does "goto retry_allocation" under some circumstances, but that is missing from the in-tree driver. diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 569392d..2cb567a 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -4224,6 +4224,134 @@ void mpt3sas_base_clear_st(struct MPT3SAS_ADAPTER *ioc, } /** + * _base_release_chain_lookup - release chain_lookup memory pools + * @ioc: per adapter object + * + * Free memory allocated from _base_allocate_chain_lookup. + */ +static void +_base_release_chain_lookup(struct MPT3SAS_ADAPTER *ioc) +{ + unsigned int chains_avail = 0; + struct chain_tracker *ct; + int i, j; + + if (!ioc->chain_lookup) + return; + + /* + * NOTE + * + * To make this code easier to understand and maintain, the for loops + * and the management of the chains_avail value are designed to be + * similar to the _base_allocate_chain_lookup() function. That way, + * the code for freeing the memory is similar to the code for + * allocating the memory. + */ + for (i = 0; i < ioc->scsiio_depth; i++) { + if (!ioc->chain_lookup[i].chains_per_smid) + break; + + for (j = ioc->chains_per_prp_buffer; + j < ioc->chains_needed_per_io; j++) { + /* + * If chains_avail is 0, then the chain represents a + * real allocation, so free it. + * + * If chains_avail is nonzero, then the chain was + * initialized at an offset from a previous allocation, + * so don't free it. + */ + if (chains_avail == 0) { + ct = &ioc->chain_lookup[i].chains_per_smid[j]; + if (ct->chain_buffer) + dma_free_coherent( + &ioc->pdev->dev, + ioc->chain_allocation_sz, + ct->chain_buffer, + ct->chain_buffer_dma); + chains_avail = ioc->chains_per_allocation; + } + chains_avail--; + } + kfree(ioc->chain_lookup[i].chains_per_smid); + } + + kfree(ioc->chain_lookup); + ioc->chain_lookup = NULL; +} + +/** + * _base_allocate_chain_lookup - allocate chain_lookup memory pools + * @ioc: per adapter object + * @total_sz: external value that tracks total amount of memory allocated + * + * Return: 0 success, anything else error + */ +static int +_base_allocate_chain_lookup(struct MPT3SAS_ADAPTER *ioc, u32 *total_sz) +{ + unsigned int aligned_chain_segment_sz; + const unsigned int align = 16; + unsigned int chains_avail = 0; + struct chain_tracker *ct; + dma_addr_t dma_addr = 0; + void *vaddr = NULL; + int i, j; + + /* Round up the allocation size for alignment. */ + aligned_chain_segment_sz = ioc->chain_segment_sz; + if (aligned_chain_segment_sz % align != 0) + aligned_chain_segment_sz = + ALIGN(aligned_chain_segment_sz, align); + + /* Allocate a page of chain buffers at a time. */ + ioc->chain_allocation_sz = + max_t(unsigned int, aligned_chain_segment_sz, PAGE_SIZE); + + /* Calculate how many chain buffers we can get from one allocation. */ + ioc->chains_per_allocation = + ioc->chain_allocation_sz / aligned_chain_segment_sz; + + for (i = 0; i < ioc->scsiio_depth; i++) { + for (j = ioc->chains_per_prp_buffer; + j < ioc->chains_needed_per_io; j++) { + /* + * Check if there are any chain buffers left in the + * previously-allocated block. + */ + if (chains_avail == 0) { + /* Allocate a new block of chain buffers. */ + vaddr = dma_alloc_coherent( + &ioc->pdev->dev, + ioc->chain_allocation_sz, + &dma_addr, + GFP_KERNEL); + if (!vaddr) { + pr_err(MPT3SAS_FMT + "chain_lookup: dma_alloc_coherent failed\n", + ioc->name); + return -1; + } + chains_avail = ioc->chains_per_allocation; + } + + ct = &ioc->chain_lookup[i].chains_per_smid[j]; + ct->chain_buffer = vaddr; + ct->chain_buffer_dma = dma_addr; + + /* Go to the next chain buffer in the block. */ + vaddr += aligned_chain_segment_sz; + dma_addr += aligned_chain_segment_sz; + *total_sz += ioc->chain_segment_sz; + chains_avail--; + } + } + + return 0; +} + +/** * _base_release_memory_pools - release memory * @ioc: per adapter object * @@ -4235,8 +4363,6 @@ void mpt3sas_base_clear_st(struct MPT3SAS_ADAPTER *ioc, _base_release_memory_pools(struct MPT3SAS_ADAPTER *ioc) { int i = 0; - int j = 0; - struct chain_tracker *ct; struct reply_post_struct *rps; dexitprintk(ioc, pr_info(MPT3SAS_FMT "%s\n", ioc->name, @@ -4326,22 +4452,7 @@ void mpt3sas_base_clear_st(struct MPT3SAS_ADAPTER *ioc, kfree(ioc->hpr_lookup); kfree(ioc->internal_lookup); - if (ioc->chain_lookup) { - for (i = 0; i < ioc->scsiio_depth; i++) { - for (j = ioc->chains_per_prp_buffer; - j < ioc->chains_needed_per_io; j++) { - ct = &ioc->chain_lookup[i].chains_per_smid[j]; - if (ct && ct->chain_buffer) - dma_pool_free(ioc->chain_dma_pool, - ct->chain_buffer, - ct->chain_buffer_dma); - } - kfree(ioc->chain_lookup[i].chains_per_smid); - } - dma_pool_destroy(ioc->chain_dma_pool); - kfree(ioc->chain_lookup); - ioc->chain_lookup = NULL; - } + _base_release_chain_lookup(ioc); } /** @@ -4784,29 +4895,8 @@ void mpt3sas_base_clear_st(struct MPT3SAS_ADAPTER *ioc, total_sz += sz * ioc->scsiio_depth; } - ioc->chain_dma_pool = dma_pool_create("chain pool", &ioc->pdev->dev, - ioc->chain_segment_sz, 16, 0); - if (!ioc->chain_dma_pool) { - pr_err(MPT3SAS_FMT "chain_dma_pool: dma_pool_create failed\n", - ioc->name); + if (_base_allocate_chain_lookup(ioc, &total_sz)) goto out; - } - for (i = 0; i < ioc->scsiio_depth; i++) { - for (j = ioc->chains_per_prp_buffer; - j < ioc->chains_needed_per_io; j++) { - ct = &ioc->chain_lookup[i].chains_per_smid[j]; - ct->chain_buffer = dma_pool_alloc( - ioc->chain_dma_pool, GFP_KERNEL, - &ct->chain_buffer_dma); - if (!ct->chain_buffer) { - pr_err(MPT3SAS_FMT "chain_lookup: " - " pci_pool_alloc failed\n", ioc->name); - _base_release_memory_pools(ioc); - goto out; - } - } - total_sz += ioc->chain_segment_sz; - } dinitprintk(ioc, pr_info(MPT3SAS_FMT "chain pool depth(%d), frame_size(%d), pool_size(%d kB)\n", diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index f02974c..7ee81d5 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -1298,7 +1298,6 @@ struct MPT3SAS_ADAPTER { /* chain */ struct chain_lookup *chain_lookup; struct list_head free_chain_list; - struct dma_pool *chain_dma_pool; ulong chain_pages; u16 max_sges_in_main_message; u16 max_sges_in_chain_message; @@ -1306,6 +1305,8 @@ struct MPT3SAS_ADAPTER { u32 chain_depth; u16 chain_segment_sz; u16 chains_per_prp_buffer; + u32 chain_allocation_sz; + u32 chains_per_allocation; /* hi-priority queue */ u16 hi_priority_smid;