From patchwork Fri Mar 29 15:27:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877449 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 373281390 for ; Fri, 29 Mar 2019 15:40:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CB1024B5B for ; Fri, 29 Mar 2019 15:40:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0F74F288DA; Fri, 29 Mar 2019 15:40:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90B51288E1 for ; Fri, 29 Mar 2019 15:40:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D9E86B0269; Fri, 29 Mar 2019 11:40:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 960466B026A; Fri, 29 Mar 2019 11:40:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 828CB6B026B; Fri, 29 Mar 2019 11:40:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 3D9BE6B0269 for ; Fri, 29 Mar 2019 11:40:11 -0400 (EDT) Received: by mail-pl1-f197.google.com with SMTP id e5so1931145plb.9 for ; Fri, 29 Mar 2019 08:40:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=OrXAwKmXL7Fav/fo5UPjMYEgsfO33kHnrLyMGw9gRbo=; b=pNEqDLa+8nM+LFMGS4iCAifJdU9dTa8rmA7nZOH0Y3lG4hgHnwRz/6kZgiYjYtvjih T8uGN9jwsIrFjBdGgwaKXUll+QPIH1TDwK/NBcw8ret/L7yQPdOoEe0AqdQYHgJQeNno 582eJbl3ulOFSDzWQoP26sWq5I/YFFDxBFDIR3vnmo3eklXCgj8GWpd4fq4jsgOGRV+p k1n32Kk29cb8j0mcTb+2iL0jmvHsWv0pkREAAoPwt2PDw+kFamnGRQcCqRVlT1WYT9N5 RRMjQywHcWS5VzFrVDPR+w1AKmVtgvWaIFS76yXInx/xlsPAZcDwUaYrTh7/vLcS/Puv eRnA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAVQ4FG8SzVdmXUBScPZsMY8Dvcjtg+IeHY+JfYy4tjFQUXLhwwh gFQAZV9LxknVVYeUpLuXLsVQ0XBYKpk8xYkCCKmZ/0uIFPnej2L/9SywsrNhBCGrhGqTmxkQirf /4XIsbyvNup2OyPbXrl8sXKsOMw5VajquYNDLxOp2mOpq5Z65aokjPllArFLEFJwR4A== X-Received: by 2002:a17:902:8c8b:: with SMTP id t11mr48539238plo.148.1553874010904; Fri, 29 Mar 2019 08:40:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqwtPzbEVnKfARpT/d/pOSuWxEClwR1sFULldMFLDU0oX0Wgy0EOuiJGTSLZdlh/vj65b0Nz X-Received: by 2002:a17:902:8c8b:: with SMTP id t11mr48539176plo.148.1553874010189; Fri, 29 Mar 2019 08:40:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874010; cv=none; d=google.com; s=arc-20160816; b=XhecvNuy5QxumKjRBNdCv0UUOek7Wt5dS7W4uVcXyPCvgKzIzV9wXjyPdGFJ/FCKJs jBDTV6HrTmHXgeXexEPRPJndKxM4fNW/HzP3QQrRAMUWikcKAaIeh2CZZHK5zrj4rfvr D0W7pet6lCaf7kS0x7181a4mpdpRM5OFNExHjb8mZH4Cla41dok1TGyAwB3CMls/WEwN xrcq/w8ypVxOJAk8amZZEd4GgcDu8p/tkALF1u3gc1B/ee7fCWjz3lfqNLlrS1YztARl J7vF9cyHmZPCuSwBRBQsSTOHnk4sRkh/KugMsqHMe7T1SZ6WeNTK3P/phvUUWs2Vkxr5 of8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=OrXAwKmXL7Fav/fo5UPjMYEgsfO33kHnrLyMGw9gRbo=; b=CWGLXxoIc5leCslxe3tS4d8TozmIs3o3pX55Zyy6F9V9ton7hFqabgf2nlnjeR5p+g FChW9QGy8zkS35j82H2kuzdUYYtE7Ewabza5Me6llBe0ytvvbe+PtsdV3FB50NwQ9Vmg OOPJH7CUfr+pohzf4Ui7mAAmA4zDxigAGhtHT1ix30j+BzfD67BpHRXJNfq/UZyBwzZT OFjb5TBmPm/Qye9TfFOBpbYYgX9/0FCeGe081/bzjPPIpNlS+/b4G33nAp7LqWsqHit0 BYHITTnJu8htpIn8W2JHu5WHOCU1Sssrlw0gcZPW5qVDcJi87HGKYGM1FRSrB2TfTBEs 9wQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga14.intel.com (mga14.intel.com. [192.55.52.115]) by mx.google.com with ESMTPS id f2si2160945pgv.441.2019.03.29.08.40.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:10 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) client-ip=192.55.52.115; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="126977111" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga007.jf.intel.com with ESMTP; 29 Mar 2019 08:40:08 -0700 Subject: [PATCH 1/6] drivers/base/devres: Introduce devm_release_action() From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Ira Weiny , Bjorn Helgaas , Christoph Hellwig , Greg Kroah-Hartman , "Rafael J. Wysocki" , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:29 -0700 Message-ID: <155387324904.2443841.8406657131473708859.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The devm_add_action() facility allows a resource allocation routine to add custom devm semantics. One such user is devm_memremap_pages(). There is now a need to manually trigger devm_memremap_pages_release(). Introduce devm_release_action() so the release action can be triggered via a new devm_memunmap_pages() api in a follow-on change. Cc: Logan Gunthorpe Cc: Ira Weiny Cc: Bjorn Helgaas Cc: Christoph Hellwig Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Signed-off-by: Dan Williams --- drivers/base/devres.c | 24 +++++++++++++++++++++++- include/linux/device.h | 1 + 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index e038e2b3b7ea..0bbb328bd17f 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -755,10 +755,32 @@ void devm_remove_action(struct device *dev, void (*action)(void *), void *data) WARN_ON(devres_destroy(dev, devm_action_release, devm_action_match, &devres)); - } EXPORT_SYMBOL_GPL(devm_remove_action); +/** + * devm_release_action() - release previously added custom action + * @dev: Device that owns the action + * @action: Function implementing the action + * @data: Pointer to data passed to @action implementation + * + * Releases and removes instance of @action previously added by + * devm_add_action(). Both action and data should match one of the + * existing entries. + */ +void devm_release_action(struct device *dev, void (*action)(void *), void *data) +{ + struct action_devres devres = { + .data = data, + .action = action, + }; + + WARN_ON(devres_release(dev, devm_action_release, devm_action_match, + &devres)); + +} +EXPORT_SYMBOL_GPL(devm_release_action); + /* * Managed kmalloc/kfree */ diff --git a/include/linux/device.h b/include/linux/device.h index b425a7ee04ce..02a3e45de9af 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -715,6 +715,7 @@ void __iomem *devm_of_iomap(struct device *dev, /* allows to add/remove a custom action to devres stack */ int devm_add_action(struct device *dev, void (*action)(void *), void *data); void devm_remove_action(struct device *dev, void (*action)(void *), void *data); +void devm_release_action(struct device *dev, void (*action)(void *), void *data); static inline int devm_add_action_or_reset(struct device *dev, void (*action)(void *), void *data) From patchwork Fri Mar 29 15:27:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877455 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F19811390 for ; Fri, 29 Mar 2019 15:40:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D8DAC24B5B for ; Fri, 29 Mar 2019 15:40:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC6A5288AE; Fri, 29 Mar 2019 15:40:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B4AE24B5B for ; Fri, 29 Mar 2019 15:40:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DAF96B026B; Fri, 29 Mar 2019 11:40:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 960246B026C; Fri, 29 Mar 2019 11:40:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D8FC6B026D; Fri, 29 Mar 2019 11:40:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 444AC6B026B for ; Fri, 29 Mar 2019 11:40:15 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id b11so1723946pfo.15 for ; Fri, 29 Mar 2019 08:40:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=bfbRtSXnvBNgnAga55D234OHrkN1o27J7rPg1iia1mI=; b=MYyCGEZKZEuo0s5YQK9kCeR6QnQeaPuVoURNwwotWBh3Z5kV+YeCZ5pAVfOh7vhcU+ UJxp68iYQxntjUs/Ix+0weeUmq59U41uY3HfhL3yqoTCQKcD7iJ4X/6Hva5mHM0MHpal 5gG9jkhwGod48whgRtDg/zBiw8R1yw1d2TZ2o8qyKMBfLFCso94wlziusnSQvnA72gxW VBQ6nM5Y9skN3bBJZxg7kvbQykIr3XqlLSBoMZbCQNpdVDd4xxkkuF+ymcI7J/Hm7L47 WjPfQTYcRi2b8QPi86Hx0+JT6frBedRSD9iT/EWQ2c8YZ3ISo4h2nBxWhmL6MZzkNcME Asog== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAX70ij+0BH4pVqnrdVC+1fEGaw3AlU3WTzuTV5nCLQs7KJpNkv7 MZYg/CDl6uA2LeA0qQp8JkYi39WDdOyUdXJ6fu7Ce7h5/8UJKD4/SFTtp7+RDL2mtoHLVcdzfe+ URuKO8MeJV2Ps53mkHYyyIei7A9rXRwNiOgUFY0MIv+ErdrLvzpHTEV+jFZo2nREZcQ== X-Received: by 2002:a62:ee0a:: with SMTP id e10mr29454917pfi.6.1553874014958; Fri, 29 Mar 2019 08:40:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqx7OsovC2/2aORHtoP97Krg4GVRnslY8RBz7yaxhXou1zzhUXZ/C85EEL1Lz3+Jbv7+TwT5 X-Received: by 2002:a62:ee0a:: with SMTP id e10mr29454874pfi.6.1553874014306; Fri, 29 Mar 2019 08:40:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874014; cv=none; d=google.com; s=arc-20160816; b=V77H03eLi5PJvKVoaMHCG9K31E+lrKqpP08z/RBkt1tDP75Cx1LLE85/7XGihkK/sP K0JoWoRk/ijHsas/yKHZ1p9VdBUdKQ+aBtnExl8IUTwBFFdvk+qbtqUMIQSogNZY38sW aV8HnTnuD/XA4l5ggYOphJ8Xl4PMPcoEtQ0nKTxi4lXm6wzxMHxGphLo40xALjQrcVZP CraKvufaGJjKBoiDOQQ3ikTrQNpeWyT3Z9tpHGd3R+9ki5bq5dNCzVGCluG0f4gGaSqu FjvwcYFNC3H+njnV3sdiDmHdZbrfQOxvUZj7mGwfuCc6yayH3C4lUGASGXYt5AG9XJGp 6NvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=bfbRtSXnvBNgnAga55D234OHrkN1o27J7rPg1iia1mI=; b=TcfWmaC0MFCQ2in3FVpow0sDJVUwn8DBOLw17tjOa4S5VdyatC0Zbor3+Gi+BtWmZW 9gN4xMxkUXqFIF4xCHVF+CpuDUgM7ZFnc87B8DpAjib81plIRYcpifPDBrcQtZe6xBUK S7+aOyUOKT+6vq1r0CQDAi84DJHqefipWpct8cVOfRC2S6OBQj0BPofl/1HsERx6RB48 IekHuIjCCQz1NDffgpc6dfwskAEgVC0+AFunOYzMkf9rHZaZ2yJPmzOTydh5W5rZ/gCR lAb6LeCIVKn2Qi9Kyb6rlSOtEJO3DGIFt85YmItYxxhLCU3U8QyxTEPnz+4uOZeZrLIf hHLA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga11.intel.com (mga11.intel.com. [192.55.52.93]) by mx.google.com with ESMTPS id k9si2139027pgb.532.2019.03.29.08.40.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:14 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) client-ip=192.55.52.93; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="287036931" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga004.jf.intel.com with ESMTP; 29 Mar 2019 08:40:13 -0700 Subject: [PATCH 2/6] mm/devm_memremap_pages: Introduce devm_memunmap_pages From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Ira Weiny , Bjorn Helgaas , Christoph Hellwig , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:34 -0700 Message-ID: <155387325416.2443841.6074930471073650165.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use the new devm_relase_action() facility to allow devm_memremap_pages_release() to be manually triggered. Cc: Logan Gunthorpe Cc: Ira Weiny Cc: Bjorn Helgaas Cc: Christoph Hellwig Signed-off-by: Dan Williams --- include/linux/memremap.h | 6 ++++++ kernel/memremap.c | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index f0628660d541..7601ee314c4a 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -100,6 +100,7 @@ struct dev_pagemap { #ifdef CONFIG_ZONE_DEVICE void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap); +void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap); struct dev_pagemap *get_dev_pagemap(unsigned long pfn, struct dev_pagemap *pgmap); @@ -118,6 +119,11 @@ static inline void *devm_memremap_pages(struct device *dev, return ERR_PTR(-ENXIO); } +static inline void devm_memunmap_pages(struct device *dev, + struct dev_pagemap *pgmap) +{ +} + static inline struct dev_pagemap *get_dev_pagemap(unsigned long pfn, struct dev_pagemap *pgmap) { diff --git a/kernel/memremap.c b/kernel/memremap.c index a856cb5ff192..65afbacab44e 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -266,6 +266,12 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) } EXPORT_SYMBOL_GPL(devm_memremap_pages); +void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap) +{ + devm_release_action(dev, devm_memremap_pages_release, pgmap); +} +EXPORT_SYMBOL_GPL(devm_memunmap_pages); + unsigned long vmem_altmap_offset(struct vmem_altmap *altmap) { /* number of pfns from base where pfn_to_page() is valid */ From patchwork Fri Mar 29 15:27:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1244C1390 for ; Fri, 29 Mar 2019 15:40:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEF7224B5B for ; Fri, 29 Mar 2019 15:40:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E385C288AE; Fri, 29 Mar 2019 15:40:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8390A24B5B for ; Fri, 29 Mar 2019 15:40:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 853A46B026C; Fri, 29 Mar 2019 11:40:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7DA016B026D; Fri, 29 Mar 2019 11:40:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A2236B026E; Fri, 29 Mar 2019 11:40:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 311F26B026C for ; Fri, 29 Mar 2019 11:40:20 -0400 (EDT) Received: by mail-pg1-f197.google.com with SMTP id 132so1863728pgc.18 for ; Fri, 29 Mar 2019 08:40:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=BGW5opGcfChg9t4SQtyg6XKafZpTItiwM+WiiS4PyGc=; b=erA+x0liE8JNlMJEQ1e70Tdydd4nt8j6RPFSy2wCQgdrll7EuHGk0LyLCoaQG2Ke8n I508I3zaxwfZd5ltiMZB4EA92DtfDhPFvRfXdYDPhItZbyRa6T+9Q0U8gSQFNeQYVUby lnrnrn3xwq5IfBkSs8QySJk5xljnH/wmE5gysjg8THtuSwZRcJArBX1AK6GRLd5w5bvn OeoEM6IbQpNCyz0re9nAcuibZiLhqk1bSF+RC6G5G4UpmiJEhN85UYjwNki/IvhAnkBP 7DrV0wF8WhvjnvyheJXsn7ged9AphWsRSZg2NF4DX+xTJiI/U3aUIh7eqGADE07lRjHX sdMg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAWN4KN+NnK4kGegJ45YtX46LQPov81gu1NPxQJoxN1a5CgDOtes OFNW21qQdq+R2K5aKhTF9oQhRGakO+NWWFXguuCa7eHnKwfuRNXETnP9yuNIUcRMAp7MXGXQ7td UhoVtlcMgR0D9maqAYRFWIiuW8XndiyGaokYoFSLylmyG0gd51CpnGhHkjVCUCozQGA== X-Received: by 2002:a62:29c5:: with SMTP id p188mr46566593pfp.203.1553874019861; Fri, 29 Mar 2019 08:40:19 -0700 (PDT) X-Google-Smtp-Source: APXvYqzZ7Vj+GhSe7doVWE7quJHMKPcpDcyieQJMChsITRkNhK7WFdBwJAMpW0H7AxVVEUYPXD8h X-Received: by 2002:a62:29c5:: with SMTP id p188mr46566530pfp.203.1553874019097; Fri, 29 Mar 2019 08:40:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874019; cv=none; d=google.com; s=arc-20160816; b=Z52BYTYAxCgQ49CbWIMRM8aT8TSP1sZ5OB561uyUMoxVb4JCm0yHIl6WXTDGYukBoh Qm7+znt1mjN0jSBXSvm6Z9KZGlwXohp63+HEKeST0O8TpoCa/UvnrB9zyXML5WFaZT1e wSTEoLGoBw6b7T4yD62l0aE4ebrBvU2zMk+tYOiswKXMCzKL2Iwp9enw5YgJBb+09h9y w9TfVu1K3/z0PJ8dRL1jv7oXju18OHadmKpIYbE5AnhP+RCpIGuzA1AcUKtpId9UShuz bNI5hPzMdCmYsPqNDtZtNRXrjywnyOtx0Q8HAcyIxvKacF2pG9vA3c2n7SNygRmI1edm 5ang== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=BGW5opGcfChg9t4SQtyg6XKafZpTItiwM+WiiS4PyGc=; b=VyB6nLKWRa/8KXDlxu0sTCu5jQaSsJ9oPAzL3mQmdI+C5jZlaQnjZD4W75iW75Vy7P VWjeFrQOZyygOxoNJd1UeqZgHsj4EpH2TLnonWEzM6FbLYQIQf1gBzc45bz3fWo66fAn EOU19AVhi/6JcBw7rljto4Xq8XEgzOuAfdyjb2C4knLBER4WQ4XLI32zcJKOOgaE95ls Sb6sAARW3WsmtTiz2fAtYDk/wIhMWeMzqA8Qe024swYORHO1+FMaoN5dAWPczgDdMifT Scc6ExYZi11+19/4IS4hapnlSOb6yGbUbYGXGKC4TywfRREA6lM4g/w+lADHL+0qV67q U7QA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga05.intel.com (mga05.intel.com. [192.55.52.43]) by mx.google.com with ESMTPS id g3si913456plq.400.2019.03.29.08.40.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:19 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) client-ip=192.55.52.43; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="138515813" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by fmsmga007.fm.intel.com with ESMTP; 29 Mar 2019 08:40:18 -0700 Subject: [PATCH 3/6] pci/p2pdma: Fix the gen_pool_add_virt() failure path From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Ira Weiny , Bjorn Helgaas , Christoph Hellwig , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:39 -0700 Message-ID: <155387325926.2443841.6674640070856872301.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The pci_p2pdma_add_resource() implementation immediately frees the pgmap if gen_pool_add_virt() fails. However, that means that when @dev triggers a devres release devm_memremap_pages_release() will crash trying to access the freed @pgmap. Use the new devm_memunmap_pages() to manually free the mapping in the error path. Fixes: 52916982af48 ("PCI/P2PDMA: Support peer-to-peer memory") Cc: Logan Gunthorpe Cc: Ira Weiny Cc: Bjorn Helgaas Cc: Christoph Hellwig Signed-off-by: Dan Williams Acked-by: Bjorn Helgaas --- drivers/pci/p2pdma.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index c52298d76e64..595a534bd749 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -208,13 +208,15 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, pci_bus_address(pdev, bar) + offset, resource_size(&pgmap->res), dev_to_node(&pdev->dev)); if (error) - goto pgmap_free; + goto pages_free; pci_info(pdev, "added peer-to-peer DMA memory %pR\n", &pgmap->res); return 0; +pages_free: + devm_memunmap_pages(&pdev->dev, pgmap); pgmap_free: devm_kfree(&pdev->dev, pgmap); return error; From patchwork Fri Mar 29 15:27:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877467 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E47711708 for ; Fri, 29 Mar 2019 15:40:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE91424B5B for ; Fri, 29 Mar 2019 15:40:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C228028779; Fri, 29 Mar 2019 15:40:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2E8C24B5B for ; Fri, 29 Mar 2019 15:40:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6EFE6B026D; Fri, 29 Mar 2019 11:40:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BF7E06B026E; Fri, 29 Mar 2019 11:40:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9B046B026F; Fri, 29 Mar 2019 11:40:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 633206B026D for ; Fri, 29 Mar 2019 11:40:26 -0400 (EDT) Received: by mail-pl1-f200.google.com with SMTP id e5so1931591plb.9 for ; Fri, 29 Mar 2019 08:40:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=KMuvdVhDgsmANRguFYy5WyGjH25xZmgLKgcz3Eba90g=; b=nqypfZxfz2nWT2wpg0Xzdm79JystlVVt1WO/9U0CZxGLi+CHQWhtwvbcNi1cYEiifh 8exoQvfh1rx6c/jvOMTg2dox+DIDihdQhUpteMzV6NpEmNhpkd9YzjTJHfLMCtq7HnQM Q+nWl86smmXGWbkJ+fG3wxzuUJp457HSRX5oU9yTCichjmhEs3Y2p85vuJB1XoMyUgzu PRfPvdxngGELaLcX2FOsIdZ+FSBJjmXA5hjIHHDovsK0JHqlRMA1716OkalHkkqL+WP3 KMeabAd/TJ8SQgtY1QvSIQj8BUK5drfMxjwKXWnAMF3R2r/dWRPyh+dmzhC+emZvf9EB SMhg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAX8Ix+uA63Hhc/+BXgUhOhbzhn9v6nRtvhtH4iDowkO5Ips4WaS FC2lkxY8XE4fypexoaTcSjCGUpv3WguFrQ7SrHnNoC/gl8fPqJgBrVIxrbsowtPuH6c8H49HqM/ 1o2MWobcSFEBnmEHDUKnRxcIEi/iJfYg/BSCfacgmKldjBuHjtf/GeKl9dRC6zbUXiQ== X-Received: by 2002:a17:902:d211:: with SMTP id t17mr14280416ply.214.1553874025986; Fri, 29 Mar 2019 08:40:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqxNarkD4mLYkw+SxG12PlmL35tvyDx4EULX9EAvCqGxuT2P4b5lSAqaaH4l9VPzAi+zOcsl X-Received: by 2002:a17:902:d211:: with SMTP id t17mr14280347ply.214.1553874025105; Fri, 29 Mar 2019 08:40:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874025; cv=none; d=google.com; s=arc-20160816; b=jimYu/2vYbaRU9g/ZDTUbpB7eUBmWzLc2PNe44h6WjsPt+eVd7tN80u7wFNZiIGSta 4v5+MJlL7hVKafBJacLs+4vmsJhkRbKJUPKgMWjGIDTbOfOJHf0q5ozc6IiskkGY3n/k p944Hn7uiwC1jnP+NlhgAKLrqzYXvPWnPSZIvDbcNTlR4cR06w1kZW6ty8dgK1sOkvEp xSO5spMgGBDI4YgStfEVv927cCnosUAfIsInH5jVrvZK0eKDrpExn1/27Gh9QmqLLizy Tba5yBXTSqVGMloP8VIxvy0cBputHJ1HNwTiKrbNf2js39n/y0PGt3npX2rvdNeHsNx7 DwIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=KMuvdVhDgsmANRguFYy5WyGjH25xZmgLKgcz3Eba90g=; b=FlvvizbEpciwKmprkrsHOiY+QKlauhr90z1a0VKk+Eo3KiP6FbmFY6qH0zZMAgMmQi 3OrgQxzBDOBjFPshtuCIS73kn5FFwJCqnfIm0f8B5vnz4CvRwiP0OkcA9GP9K99JUcCr TQz7yE13gR6ZQ1Hn5nevqQe98Ywydyt6glai9enDTPB9GWBM76DSl6t5KXvsZRN5KX5f rc01PXI5Qyd9W5NJCIZ9jlgEl4CLpNVgEr1XCrCAQ1Z8SM0hywspZdf6McvOv3zukaxx oIlCdUuhcTerUpTGMv78+d/V73Oey5YajjLrteMJCdBvJKzfs2vdt1MIdVrnrw2w+vWB AZTQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id f185si2162626pgc.182.2019.03.29.08.40.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:25 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="156424704" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by fmsmga002.fm.intel.com with ESMTP; 29 Mar 2019 08:40:23 -0700 Subject: [PATCH 4/6] lib/genalloc: Introduce chunk owners From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Ira Weiny , Bjorn Helgaas , =?utf-8?b?SsOpcsO0bWU=?= Glisse , Christoph Hellwig , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:44 -0700 Message-ID: <155387326480.2443841.4506646109682116121.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The p2pdma facility enables a provider to publish a pool of dma addresses for a consumer to allocate. A genpool is used internally by p2pdma to collect dma resources, 'chunks', to be handed out to consumers. Whenever a consumer allocates a resource it needs to pin the 'struct dev_pagemap' instance that backs the chunk selected by pci_alloc_p2pmem(). Currently that reference is taken globally on the entire provider device. That sets up a lifetime mismatch whereby the p2pdma core needs to maintain hacks to make sure the percpu_ref is not released twice. This lifetime mismatch also stands in the way of a fix to devm_memremap_pages() whereby devm_memremap_pages_release() must wait for the percpu_ref ->release() callback to complete before it can proceed to teardown pages. So, towards fixing this situation, introduce the ability to store a 'chunk owner' at gen_pool_add() time, and a facility to retrieve the owner at gen_pool_{alloc,free}() time. For p2pdma this will be used to store and recall individual dev_pagemap reference counter instances per-chunk. Cc: Logan Gunthorpe Cc: Ira Weiny Cc: Bjorn Helgaas Cc: "Jérôme Glisse" Cc: Christoph Hellwig Signed-off-by: Dan Williams --- include/linux/genalloc.h | 55 +++++++++++++++++++++++++++++++++++++++++----- lib/genalloc.c | 51 +++++++++++++++++++++---------------------- 2 files changed, 74 insertions(+), 32 deletions(-) diff --git a/include/linux/genalloc.h b/include/linux/genalloc.h index dd0a452373e7..a337313e064f 100644 --- a/include/linux/genalloc.h +++ b/include/linux/genalloc.h @@ -75,6 +75,7 @@ struct gen_pool_chunk { struct list_head next_chunk; /* next chunk in pool */ atomic_long_t avail; phys_addr_t phys_addr; /* physical starting address of memory chunk */ + void *owner; /* private data to retrieve at alloc time */ unsigned long start_addr; /* start address of memory chunk */ unsigned long end_addr; /* end address of memory chunk (inclusive) */ unsigned long bits[0]; /* bitmap for allocating memory chunk */ @@ -96,8 +97,15 @@ struct genpool_data_fixed { extern struct gen_pool *gen_pool_create(int, int); extern phys_addr_t gen_pool_virt_to_phys(struct gen_pool *pool, unsigned long); -extern int gen_pool_add_virt(struct gen_pool *, unsigned long, phys_addr_t, - size_t, int); +extern int gen_pool_add_owner(struct gen_pool *, unsigned long, phys_addr_t, + size_t, int, void *); + +static inline int gen_pool_add_virt(struct gen_pool *pool, unsigned long addr, + phys_addr_t phys, size_t size, int nid) +{ + return gen_pool_add_owner(pool, addr, phys, size, nid, NULL); +} + /** * gen_pool_add - add a new chunk of special memory to the pool * @pool: pool to add new memory chunk to @@ -116,12 +124,47 @@ static inline int gen_pool_add(struct gen_pool *pool, unsigned long addr, return gen_pool_add_virt(pool, addr, -1, size, nid); } extern void gen_pool_destroy(struct gen_pool *); -extern unsigned long gen_pool_alloc(struct gen_pool *, size_t); -extern unsigned long gen_pool_alloc_algo(struct gen_pool *, size_t, - genpool_algo_t algo, void *data); +unsigned long gen_pool_alloc_algo_owner(struct gen_pool *pool, size_t size, + genpool_algo_t algo, void *data, void **owner); + +static inline unsigned long gen_pool_alloc_owner(struct gen_pool *pool, + size_t size, void **owner) +{ + return gen_pool_alloc_algo_owner(pool, size, pool->algo, pool->data, + owner); +} + +static inline unsigned long gen_pool_alloc_algo(struct gen_pool *pool, + size_t size, genpool_algo_t algo, void *data) +{ + return gen_pool_alloc_algo_owner(pool, size, algo, data, NULL); +} + +/** + * gen_pool_alloc - allocate special memory from the pool + * @pool: pool to allocate from + * @size: number of bytes to allocate from the pool + * + * Allocate the requested number of bytes from the specified pool. + * Uses the pool allocation function (with first-fit algorithm by default). + * Can not be used in NMI handler on architectures without + * NMI-safe cmpxchg implementation. + */ +static inline unsigned long gen_pool_alloc(struct gen_pool *pool, size_t size) +{ + return gen_pool_alloc_algo(pool, size, pool->algo, pool->data); +} + extern void *gen_pool_dma_alloc(struct gen_pool *pool, size_t size, dma_addr_t *dma); -extern void gen_pool_free(struct gen_pool *, unsigned long, size_t); +extern void gen_pool_free_owner(struct gen_pool *pool, unsigned long addr, + size_t size, void **owner); +static inline void gen_pool_free(struct gen_pool *pool, unsigned long addr, + size_t size) +{ + gen_pool_free_owner(pool, addr, size, NULL); +} + extern void gen_pool_for_each_chunk(struct gen_pool *, void (*)(struct gen_pool *, struct gen_pool_chunk *, void *), void *); extern size_t gen_pool_avail(struct gen_pool *); diff --git a/lib/genalloc.c b/lib/genalloc.c index 7e85d1e37a6e..770c769d7cb7 100644 --- a/lib/genalloc.c +++ b/lib/genalloc.c @@ -168,20 +168,21 @@ struct gen_pool *gen_pool_create(int min_alloc_order, int nid) EXPORT_SYMBOL(gen_pool_create); /** - * gen_pool_add_virt - add a new chunk of special memory to the pool + * gen_pool_add_owner- add a new chunk of special memory to the pool * @pool: pool to add new memory chunk to * @virt: virtual starting address of memory chunk to add to pool * @phys: physical starting address of memory chunk to add to pool * @size: size in bytes of the memory chunk to add to pool * @nid: node id of the node the chunk structure and bitmap should be * allocated on, or -1 + * @owner: private data the publisher would like to recall at alloc time * * Add a new chunk of special memory to the specified pool. * * Returns 0 on success or a -ve errno on failure. */ -int gen_pool_add_virt(struct gen_pool *pool, unsigned long virt, phys_addr_t phys, - size_t size, int nid) +int gen_pool_add_owner(struct gen_pool *pool, unsigned long virt, phys_addr_t phys, + size_t size, int nid, void *owner) { struct gen_pool_chunk *chunk; int nbits = size >> pool->min_alloc_order; @@ -195,6 +196,7 @@ int gen_pool_add_virt(struct gen_pool *pool, unsigned long virt, phys_addr_t phy chunk->phys_addr = phys; chunk->start_addr = virt; chunk->end_addr = virt + size - 1; + chunk->owner = owner; atomic_long_set(&chunk->avail, size); spin_lock(&pool->lock); @@ -203,7 +205,7 @@ int gen_pool_add_virt(struct gen_pool *pool, unsigned long virt, phys_addr_t phy return 0; } -EXPORT_SYMBOL(gen_pool_add_virt); +EXPORT_SYMBOL(gen_pool_add_owner); /** * gen_pool_virt_to_phys - return the physical address of memory @@ -260,35 +262,20 @@ void gen_pool_destroy(struct gen_pool *pool) EXPORT_SYMBOL(gen_pool_destroy); /** - * gen_pool_alloc - allocate special memory from the pool - * @pool: pool to allocate from - * @size: number of bytes to allocate from the pool - * - * Allocate the requested number of bytes from the specified pool. - * Uses the pool allocation function (with first-fit algorithm by default). - * Can not be used in NMI handler on architectures without - * NMI-safe cmpxchg implementation. - */ -unsigned long gen_pool_alloc(struct gen_pool *pool, size_t size) -{ - return gen_pool_alloc_algo(pool, size, pool->algo, pool->data); -} -EXPORT_SYMBOL(gen_pool_alloc); - -/** - * gen_pool_alloc_algo - allocate special memory from the pool + * gen_pool_alloc_algo_owner - allocate special memory from the pool * @pool: pool to allocate from * @size: number of bytes to allocate from the pool * @algo: algorithm passed from caller * @data: data passed to algorithm + * @owner: optionally retrieve the chunk owner * * Allocate the requested number of bytes from the specified pool. * Uses the pool allocation function (with first-fit algorithm by default). * Can not be used in NMI handler on architectures without * NMI-safe cmpxchg implementation. */ -unsigned long gen_pool_alloc_algo(struct gen_pool *pool, size_t size, - genpool_algo_t algo, void *data) +unsigned long gen_pool_alloc_algo_owner(struct gen_pool *pool, size_t size, + genpool_algo_t algo, void *data, void **owner) { struct gen_pool_chunk *chunk; unsigned long addr = 0; @@ -299,6 +286,9 @@ unsigned long gen_pool_alloc_algo(struct gen_pool *pool, size_t size, BUG_ON(in_nmi()); #endif + if (owner) + *owner = NULL; + if (size == 0) return 0; @@ -326,12 +316,14 @@ unsigned long gen_pool_alloc_algo(struct gen_pool *pool, size_t size, addr = chunk->start_addr + ((unsigned long)start_bit << order); size = nbits << order; atomic_long_sub(size, &chunk->avail); + if (owner) + *owner = chunk->owner; break; } rcu_read_unlock(); return addr; } -EXPORT_SYMBOL(gen_pool_alloc_algo); +EXPORT_SYMBOL(gen_pool_alloc_algo_owner); /** * gen_pool_dma_alloc - allocate special memory from the pool for DMA usage @@ -367,12 +359,14 @@ EXPORT_SYMBOL(gen_pool_dma_alloc); * @pool: pool to free to * @addr: starting address of memory to free back to pool * @size: size in bytes of memory to free + * @owner: private data stashed at gen_pool_add() time * * Free previously allocated special memory back to the specified * pool. Can not be used in NMI handler on architectures without * NMI-safe cmpxchg implementation. */ -void gen_pool_free(struct gen_pool *pool, unsigned long addr, size_t size) +void gen_pool_free_owner(struct gen_pool *pool, unsigned long addr, size_t size, + void **owner) { struct gen_pool_chunk *chunk; int order = pool->min_alloc_order; @@ -382,6 +376,9 @@ void gen_pool_free(struct gen_pool *pool, unsigned long addr, size_t size) BUG_ON(in_nmi()); #endif + if (owner) + *owner = NULL; + nbits = (size + (1UL << order) - 1) >> order; rcu_read_lock(); list_for_each_entry_rcu(chunk, &pool->chunks, next_chunk) { @@ -392,6 +389,8 @@ void gen_pool_free(struct gen_pool *pool, unsigned long addr, size_t size) BUG_ON(remain); size = nbits << order; atomic_long_add(size, &chunk->avail); + if (owner) + *owner = chunk->owner; rcu_read_unlock(); return; } @@ -399,7 +398,7 @@ void gen_pool_free(struct gen_pool *pool, unsigned long addr, size_t size) rcu_read_unlock(); BUG(); } -EXPORT_SYMBOL(gen_pool_free); +EXPORT_SYMBOL(gen_pool_free_owner); /** * gen_pool_for_each_chunk - call func for every chunk of generic memory pool From patchwork Fri Mar 29 15:27:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877473 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D17E71390 for ; Fri, 29 Mar 2019 15:40:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B895124B5B for ; Fri, 29 Mar 2019 15:40:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC9682852C; Fri, 29 Mar 2019 15:40:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09C5624B5B for ; Fri, 29 Mar 2019 15:40:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCF6D6B026E; Fri, 29 Mar 2019 11:40:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D56BF6B026F; Fri, 29 Mar 2019 11:40:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF9DA6B0270; Fri, 29 Mar 2019 11:40:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 8305C6B026E for ; Fri, 29 Mar 2019 11:40:32 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id g1so1742927pfo.2 for ; Fri, 29 Mar 2019 08:40:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=9jYpiNvxrjAb8tEF/6LpJuv4nBpUNEzwmCJO9xz0jaY=; b=WlAEKrBJ3EMayedUxEMhaIDrif1X6lKx/pACvtJ+pDTKLUsmmPku6J5XsjVfQnpVwk OPFYqvHC69LmLWwAXFfTyv0WMdOwMo6B11icL7THbV4EOKcfxkPhcCPh/gh12PJxKgru wAx8V+Y3rKO6TnKgK6esM04OGqWO9/JTgDDInb4PvxQ3OK6fqTYeksKIaFFCl6dgQlMe fYkQostolbxiRklMW+FTSeddx2IphzAEcEKc+wdLJm1AEmoigJqZ/G3DytYa4z+CmCcL vMVjQykZnfrF/fJyJ/J8ZrZ/J3LMKyZdXQR1KQMCWTdbNlh0yWFROl7grmsDFsJPc0hT TXpA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAXOXVrCVRiLOTwKwkppnj7rcOcvxr3k6iep+3PH4sApIMno6mjN jwjhRLebh+UunI2Pw83+ixlHYiRUIh9TCQD/I4GwY9Holfqh5fvJOH1dwwv92ZRMvTDFhYJy/3J Di2DMAq/FicIcD6dGH9DOFU7pT/rDP8zzZgZI3dZeE0uoXFsLRLM88Z+6s6fAbqZJYA== X-Received: by 2002:a62:1d0d:: with SMTP id d13mr13126074pfd.96.1553874032110; Fri, 29 Mar 2019 08:40:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqy8Xdo0X9XDrJErtGB4zMH2wLpZKZeJavyohejLILZKNdcIfH2JbriK7Sdd8DplTjfepqAS X-Received: by 2002:a62:1d0d:: with SMTP id d13mr13125992pfd.96.1553874031179; Fri, 29 Mar 2019 08:40:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874031; cv=none; d=google.com; s=arc-20160816; b=ML6WPUCk2bg1tyZN6HbVAqiwcxt+WJoohXyJHVwW47SBOJsAwmSjWP/UPpWR2xWsHN 5UYwEczsxudBwFawsZalRQrz4I2ls3bCtY3AYQvopcn8HQOwnvzwLTTln4V4PuXQS42+ R2ysaYVVxO3SygKWJ1Pimm2f4dABqSK2bus0zW2b4NJYY/avHqc3y9fco2AjcTBBqul3 /xwyTA6LDV02Ex6Jw8yB3orkjq+yJvtExkukOgv6a6crdKlL1vKIQD9S9MlZNvprAvg5 +CoCG59jlaLTNSHMxUY96i84yOKEvEIlF0Kv7G4JV8q3y+xGDNsBvGvX3N3DKu3e8QPa 6gdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=9jYpiNvxrjAb8tEF/6LpJuv4nBpUNEzwmCJO9xz0jaY=; b=jbqN5ns+tOVwp1eYCSqbIGO4EkYfufblN35G1zcrkQxwnZ9ddHVZhNMBnpbwAenGpP 4uv5nLzt4j9AnEH5ECvFtuJQhSuhuBqAfSikFa8fZgJ74mQGm4lKcqU5tpEZfVLA7hiM +DNbwtFebN1NV1mhUF3mZ4FzMIZg6aBYLfGzBb6t82DmVrBzdHtGOZJGTavMq9UvLHuE XYeVvVSjRzf9ME0rr6QsIg8U/XUtM+7acc0erFlJ87AzjsAcMfRVprfmshSxdkOABsy4 JpHt4VuFU6o9qPYu6IUj6f4Ne2oNYnAdf5JG35ORth5huMXBRByvsEQ0Rj0PKY2ojAH5 QZ8A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTPS id b7si2240094pla.195.2019.03.29.08.40.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:31 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) client-ip=134.134.136.24; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="144965899" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by FMSMGA003.fm.intel.com with ESMTP; 29 Mar 2019 08:40:29 -0700 Subject: [PATCH 5/6] pci/p2pdma: Track pgmap references per resource, not globally From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Bjorn Helgaas , Christoph Hellwig , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:50 -0700 Message-ID: <155387327020.2443841.6446837127378298192.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In preparation for fixing a race between devm_memremap_pages_release() and the final put of a page from the device-page-map, allocate a percpu-ref per p2pdma resource mapping. Cc: Logan Gunthorpe Cc: Bjorn Helgaas Cc: Christoph Hellwig Signed-off-by: Dan Williams --- drivers/pci/p2pdma.c | 114 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 73 insertions(+), 41 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 595a534bd749..1b96c1688715 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -20,12 +20,16 @@ #include struct pci_p2pdma { - struct percpu_ref devmap_ref; - struct completion devmap_ref_done; struct gen_pool *pool; bool p2pmem_published; }; +struct p2pdma_pagemap { + struct dev_pagemap pgmap; + struct percpu_ref ref; + struct completion ref_done; +}; + static ssize_t size_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -74,28 +78,31 @@ static const struct attribute_group p2pmem_group = { .name = "p2pmem", }; +static struct p2pdma_pagemap *to_p2p_pgmap(struct percpu_ref *ref) +{ + return container_of(ref, struct p2pdma_pagemap, ref); +} + static void pci_p2pdma_percpu_release(struct percpu_ref *ref) { - struct pci_p2pdma *p2p = - container_of(ref, struct pci_p2pdma, devmap_ref); + struct p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(ref); - complete_all(&p2p->devmap_ref_done); + complete(&p2p_pgmap->ref_done); } static void pci_p2pdma_percpu_kill(struct percpu_ref *ref) { - /* - * pci_p2pdma_add_resource() may be called multiple times - * by a driver and may register the percpu_kill devm action multiple - * times. We only want the first action to actually kill the - * percpu_ref. - */ - if (percpu_ref_is_dying(ref)) - return; - percpu_ref_kill(ref); } +static void pci_p2pdma_percpu_cleanup(void *ref) +{ + struct p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(ref); + + wait_for_completion(&p2p_pgmap->ref_done); + percpu_ref_exit(&p2p_pgmap->ref); +} + static void pci_p2pdma_release(void *data) { struct pci_dev *pdev = data; @@ -103,12 +110,12 @@ static void pci_p2pdma_release(void *data) if (!pdev->p2pdma) return; - wait_for_completion(&pdev->p2pdma->devmap_ref_done); - percpu_ref_exit(&pdev->p2pdma->devmap_ref); + /* Flush and disable pci_alloc_p2p_mem() */ + pdev->p2pdma = NULL; + synchronize_rcu(); gen_pool_destroy(pdev->p2pdma->pool); sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group); - pdev->p2pdma = NULL; } static int pci_p2pdma_setup(struct pci_dev *pdev) @@ -124,12 +131,6 @@ static int pci_p2pdma_setup(struct pci_dev *pdev) if (!p2p->pool) goto out; - init_completion(&p2p->devmap_ref_done); - error = percpu_ref_init(&p2p->devmap_ref, - pci_p2pdma_percpu_release, 0, GFP_KERNEL); - if (error) - goto out_pool_destroy; - error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev); if (error) goto out_pool_destroy; @@ -163,6 +164,7 @@ static int pci_p2pdma_setup(struct pci_dev *pdev) int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, u64 offset) { + struct p2pdma_pagemap *p2p_pgmap; struct dev_pagemap *pgmap; void *addr; int error; @@ -185,14 +187,32 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, return error; } - pgmap = devm_kzalloc(&pdev->dev, sizeof(*pgmap), GFP_KERNEL); - if (!pgmap) + p2p_pgmap = devm_kzalloc(&pdev->dev, sizeof(*p2p_pgmap), GFP_KERNEL); + if (!p2p_pgmap) return -ENOMEM; + init_completion(&p2p_pgmap->ref_done); + error = percpu_ref_init(&p2p_pgmap->ref, + pci_p2pdma_percpu_release, 0, GFP_KERNEL); + if (error) + goto pgmap_free; + + /* + * FIXME: the percpu_ref_exit needs to be coordinated internal + * to devm_memremap_pages_release(). Duplicate the same ordering + * as other devm_memremap_pages() users for now. + */ + error = devm_add_action(&pdev->dev, pci_p2pdma_percpu_cleanup, + &p2p_pgmap->ref); + if (error) + goto ref_cleanup; + + pgmap = &p2p_pgmap->pgmap; + pgmap->res.start = pci_resource_start(pdev, bar) + offset; pgmap->res.end = pgmap->res.start + size - 1; pgmap->res.flags = pci_resource_flags(pdev, bar); - pgmap->ref = &pdev->p2pdma->devmap_ref; + pgmap->ref = &p2p_pgmap->ref; pgmap->type = MEMORY_DEVICE_PCI_P2PDMA; pgmap->pci_p2pdma_bus_offset = pci_bus_address(pdev, bar) - pci_resource_start(pdev, bar); @@ -201,12 +221,13 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, addr = devm_memremap_pages(&pdev->dev, pgmap); if (IS_ERR(addr)) { error = PTR_ERR(addr); - goto pgmap_free; + goto ref_exit; } - error = gen_pool_add_virt(pdev->p2pdma->pool, (unsigned long)addr, + error = gen_pool_add_owner(pdev->p2pdma->pool, (unsigned long)addr, pci_bus_address(pdev, bar) + offset, - resource_size(&pgmap->res), dev_to_node(&pdev->dev)); + resource_size(&pgmap->res), dev_to_node(&pdev->dev), + &p2p_pgmap->ref); if (error) goto pages_free; @@ -217,8 +238,10 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, pages_free: devm_memunmap_pages(&pdev->dev, pgmap); +ref_cleanup: + percpu_ref_exit(&p2p_pgmap->ref); pgmap_free: - devm_kfree(&pdev->dev, pgmap); + devm_kfree(&pdev->dev, p2p_pgmap); return error; } EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); @@ -555,19 +578,25 @@ EXPORT_SYMBOL_GPL(pci_p2pmem_find_many); */ void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) { - void *ret; + void *ret = NULL; + struct percpu_ref *ref; + rcu_read_lock(); if (unlikely(!pdev->p2pdma)) - return NULL; - - if (unlikely(!percpu_ref_tryget_live(&pdev->p2pdma->devmap_ref))) - return NULL; - - ret = (void *)gen_pool_alloc(pdev->p2pdma->pool, size); + goto out; - if (unlikely(!ret)) - percpu_ref_put(&pdev->p2pdma->devmap_ref); + ret = (void *)gen_pool_alloc_owner(pdev->p2pdma->pool, size, + (void **) &ref); + if (!ret) + goto out; + if (unlikely(!percpu_ref_tryget_live(ref))) { + gen_pool_free(pdev->p2pdma->pool, (unsigned long) ret, size); + ret = NULL; + goto out; + } +out: + rcu_read_unlock(); return ret; } EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); @@ -580,8 +609,11 @@ EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); */ void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size) { - gen_pool_free(pdev->p2pdma->pool, (uintptr_t)addr, size); - percpu_ref_put(&pdev->p2pdma->devmap_ref); + struct percpu_ref *ref; + + gen_pool_free_owner(pdev->p2pdma->pool, (uintptr_t)addr, size, + (void **) &ref); + percpu_ref_put(ref); } EXPORT_SYMBOL_GPL(pci_free_p2pmem); From patchwork Fri Mar 29 15:27:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10877479 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58FC31874 for ; Fri, 29 Mar 2019 15:40:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 484D024B5B for ; Fri, 29 Mar 2019 15:40:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C2C32852C; Fri, 29 Mar 2019 15:40:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43A1A24B5B for ; Fri, 29 Mar 2019 15:40:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D82D06B026F; Fri, 29 Mar 2019 11:40:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D30576B0270; Fri, 29 Mar 2019 11:40:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD0D96B0271; Fri, 29 Mar 2019 11:40:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 7ECF26B026F for ; Fri, 29 Mar 2019 11:40:37 -0400 (EDT) Received: by mail-pl1-f198.google.com with SMTP id s19so917600plp.6 for ; Fri, 29 Mar 2019 08:40:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=pvk/PF/EsK5FtRdAxRvaWVFNY0pLMM2xx7iS8ajtFZ8=; b=GjIIp/d+Ws/2/IokoNR0Bji16jkOlQAT0hVHaXyA2kb7dXvCXSR9xjCBrRPWkIH2dm Hv2LwzPMVBo67YFIpt92Dvh0K5V/k/fjMSM0fgN9n08J9i62LQakdoklFtnWT+wC2684 rBaP+E03R36NXHpX9W8udZlVihNaJ0qCwa3l1wfaHjzx/bfTfBIYO+Czks0axeJzUnD/ CerrtFCITwCW+4M6WgWoLveW+RFRuG1yyDzC/NYqzRiEeF7nyxvFZM/MzIYMbz3/s+pV VAsOtUYhZ5WY0nh0Z7a2fYbCWX2ae3ZLlEjJt9NpmEOQrPTDsSBHqjrj/eSnAEeG2aoq 2gbg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APjAAAUwybbrrPr6zpZIsSI8xFf0wakVv/tZ70AbNhkYbi22Qcwald90 WU+HdUt344zotzFYnwwOMb7N+Afu+19snWJXL9WYvJiLMuL9hUNKBR9B+pKaU4PDpATavLjL2O9 DdLtVIRtQl/Ipg+rZ7fNTdr/5QnYim6SxOhfKgT0aaugHDeskG/NjjKzF1IB77qt3Aw== X-Received: by 2002:a63:1659:: with SMTP id 25mr2987924pgw.275.1553874037108; Fri, 29 Mar 2019 08:40:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqxMm2kdfa9fhyAC8Fwco240MHmG0jnFm79/WFYFIO8rv3fJkySXfBoZpuNgPiARqIDO5m7k X-Received: by 2002:a63:1659:: with SMTP id 25mr2987849pgw.275.1553874036073; Fri, 29 Mar 2019 08:40:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553874036; cv=none; d=google.com; s=arc-20160816; b=vsZmOxGZQazgQCVta5BV0QznLaHGiDRhtcP1y4gv80hrmo5eJSOz0sI9OLraUIOrMF nurb4DRPy53oQrPtRUTjNo5UUP7oPdh19DreDU/MR8dqbtuz1CPN0Prc/U+Nt2ncr16k 8sNnhwiPxRKYe8Bwqs3rg74S7B4svJwiSIbKdxuf2MXBjcQPzqSSaEZ2zuJc3P3Ws6qQ 8mMNxERCpUUfLnfZPx2393x3aKGmbUmY+j0ynNTMUCCbdApYW6jMLmlVHPJyeTr9iJ9e 7BzNJ15lFftrxvvWbIGnRjvqxIMKiDqRuDJ2/oW1gtJdv90b8RZfViwFayAck9+rjdgE SgrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=pvk/PF/EsK5FtRdAxRvaWVFNY0pLMM2xx7iS8ajtFZ8=; b=sfBBtcWtvfH1YlsD/CfiEeSsuOMYhEBWO/w5pJkVF7j3m6j/i2zxVBNrVoObiyJHpw K8790aVQ2/LPThmJ+dMr2egQ9gNoqOK6xWF4KgMYW5PZgIaNPdXyYNinJThZ0K00KPXd r4IzXMxLC/MPTTJmORix22T/Z0e1SoGtQDan0c36SpP37pRxeuwXYobuU3H5P53+H5A8 xdf34ekKu02X3mGSzmaPY1OtMP1HfKM3d8Xzrd0aJhLTtlBMtZzHjBLwy1WI3wZ0rkcd Ew331vHsfhpRSjy2FrDHAmHITXYGyCAWw6QHWzh/tTVap2xdWFRcCP47LBX+sX4fhYZ1 Ll/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga18.intel.com (mga18.intel.com. [134.134.136.126]) by mx.google.com with ESMTPS id b92si2231665plb.348.2019.03.29.08.40.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Mar 2019 08:40:36 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) client-ip=134.134.136.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 08:40:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="333233861" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by fmsmga005.fm.intel.com with ESMTP; 29 Mar 2019 08:40:34 -0700 Subject: [PATCH 6/6] mm/devm_memremap_pages: Fix final page put race From: Dan Williams To: akpm@linux-foundation.org Cc: Logan Gunthorpe , Ira Weiny , Bjorn Helgaas , =?utf-8?b?SsOpcsO0bWU=?= Glisse , Christoph Hellwig , linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Date: Fri, 29 Mar 2019 08:27:55 -0700 Message-ID: <155387327591.2443841.1024616899609926902.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <155387324370.2443841.574715745262628837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Logan noticed that devm_memremap_pages_release() kills the percpu_ref drops all the page references that were acquired at init and then immediately proceeds to unplug, arch_remove_memory(), the backing pages for the pagemap. If for some reason device shutdown actually collides with a busy / elevated-ref-count page then arch_remove_memory() should be deferred until after that reference is dropped. As it stands the "wait for last page ref drop" happens *after* devm_memremap_pages_release() returns, which is obviously too late and can lead to crashes. Fix this situation by assigning the responsibility to wait for the percpu_ref to go idle to devm_memremap_pages() with a new ->cleanup() callback. Implement the new cleanup callback for all devm_memremap_pages() users: pmem, devdax, hmm, and p2pdma. Reported-by: Logan Gunthorpe Fixes: 41e94a851304 ("add devm_memremap_pages") Cc: Ira Weiny Cc: Bjorn Helgaas Cc: "Jérôme Glisse" Cc: Christoph Hellwig Signed-off-by: Dan Williams Reviewed-by: Ira Weiny --- drivers/dax/device.c | 13 +++---------- drivers/nvdimm/pmem.c | 17 +++++++++++++---- drivers/pci/p2pdma.c | 17 +++-------------- include/linux/memremap.h | 2 ++ kernel/memremap.c | 17 ++++++++++++----- mm/hmm.c | 14 +++----------- tools/testing/nvdimm/test/iomap.c | 2 ++ 7 files changed, 38 insertions(+), 44 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index e428468ab661..e3aa78dd1bb0 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -27,9 +27,8 @@ static void dev_dax_percpu_release(struct percpu_ref *ref) complete(&dev_dax->cmp); } -static void dev_dax_percpu_exit(void *data) +static void dev_dax_percpu_exit(struct percpu_ref *ref) { - struct percpu_ref *ref = data; struct dev_dax *dev_dax = ref_to_dev_dax(ref); dev_dbg(&dev_dax->dev, "%s\n", __func__); @@ -468,18 +467,12 @@ int dev_dax_probe(struct device *dev) if (rc) return rc; - rc = devm_add_action_or_reset(dev, dev_dax_percpu_exit, &dev_dax->ref); - if (rc) - return rc; - dev_dax->pgmap.ref = &dev_dax->ref; dev_dax->pgmap.kill = dev_dax_percpu_kill; + dev_dax->pgmap.cleanup = dev_dax_percpu_exit; addr = devm_memremap_pages(dev, &dev_dax->pgmap); - if (IS_ERR(addr)) { - devm_remove_action(dev, dev_dax_percpu_exit, &dev_dax->ref); - percpu_ref_exit(&dev_dax->ref); + if (IS_ERR(addr)) return PTR_ERR(addr); - } inode = dax_inode(dax_dev); cdev = inode->i_cdev; diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index bc2f700feef8..507b9eda42aa 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -304,11 +304,19 @@ static const struct attribute_group *pmem_attribute_groups[] = { NULL, }; -static void pmem_release_queue(void *q) +static void __pmem_release_queue(struct percpu_ref *ref) { + struct request_queue *q; + + q = container_of(ref, typeof(*q), q_usage_counter); blk_cleanup_queue(q); } +static void pmem_release_queue(void *ref) +{ + __pmem_release_queue(ref); +} + static void pmem_freeze_queue(struct percpu_ref *ref) { struct request_queue *q; @@ -400,12 +408,10 @@ static int pmem_attach_disk(struct device *dev, if (!q) return -ENOMEM; - if (devm_add_action_or_reset(dev, pmem_release_queue, q)) - return -ENOMEM; - pmem->pfn_flags = PFN_DEV; pmem->pgmap.ref = &q->q_usage_counter; pmem->pgmap.kill = pmem_freeze_queue; + pmem->pgmap.cleanup = __pmem_release_queue; if (is_nd_pfn(dev)) { if (setup_pagemap_fsdax(dev, &pmem->pgmap)) return -ENOMEM; @@ -426,6 +432,9 @@ static int pmem_attach_disk(struct device *dev, pmem->pfn_flags |= PFN_MAP; memcpy(&bb_res, &pmem->pgmap.res, sizeof(bb_res)); } else { + if (devm_add_action_or_reset(dev, pmem_release_queue, + &q->q_usage_counter)) + return -ENOMEM; addr = devm_memremap(dev, pmem->phys_addr, pmem->size, ARCH_MEMREMAP_PMEM); memcpy(&bb_res, &nsio->res, sizeof(bb_res)); diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 1b96c1688715..7ff5b8067670 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -95,7 +95,7 @@ static void pci_p2pdma_percpu_kill(struct percpu_ref *ref) percpu_ref_kill(ref); } -static void pci_p2pdma_percpu_cleanup(void *ref) +static void pci_p2pdma_percpu_cleanup(struct percpu_ref *ref) { struct p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(ref); @@ -197,16 +197,6 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, if (error) goto pgmap_free; - /* - * FIXME: the percpu_ref_exit needs to be coordinated internal - * to devm_memremap_pages_release(). Duplicate the same ordering - * as other devm_memremap_pages() users for now. - */ - error = devm_add_action(&pdev->dev, pci_p2pdma_percpu_cleanup, - &p2p_pgmap->ref); - if (error) - goto ref_cleanup; - pgmap = &p2p_pgmap->pgmap; pgmap->res.start = pci_resource_start(pdev, bar) + offset; @@ -217,11 +207,12 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, pgmap->pci_p2pdma_bus_offset = pci_bus_address(pdev, bar) - pci_resource_start(pdev, bar); pgmap->kill = pci_p2pdma_percpu_kill; + pgmap->cleanup = pci_p2pdma_percpu_cleanup; addr = devm_memremap_pages(&pdev->dev, pgmap); if (IS_ERR(addr)) { error = PTR_ERR(addr); - goto ref_exit; + goto pgmap_free; } error = gen_pool_add_owner(pdev->p2pdma->pool, (unsigned long)addr, @@ -238,8 +229,6 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, pages_free: devm_memunmap_pages(&pdev->dev, pgmap); -ref_cleanup: - percpu_ref_exit(&p2p_pgmap->ref); pgmap_free: devm_kfree(&pdev->dev, p2p_pgmap); return error; diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 7601ee314c4a..1732dea030b2 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -81,6 +81,7 @@ typedef void (*dev_page_free_t)(struct page *page, void *data); * @res: physical address range covered by @ref * @ref: reference count that pins the devm_memremap_pages() mapping * @kill: callback to transition @ref to the dead state + * @cleanup: callback to wait for @ref to be idle and reap it * @dev: host device of the mapping for debug * @data: private data pointer for page_free() * @type: memory type: see MEMORY_* in memory_hotplug.h @@ -92,6 +93,7 @@ struct dev_pagemap { struct resource res; struct percpu_ref *ref; void (*kill)(struct percpu_ref *ref); + void (*cleanup)(struct percpu_ref *ref); struct device *dev; void *data; enum memory_type type; diff --git a/kernel/memremap.c b/kernel/memremap.c index 65afbacab44e..05d1af5a2f15 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -96,6 +96,7 @@ static void devm_memremap_pages_release(void *data) pgmap->kill(pgmap->ref); for_each_device_pfn(pfn, pgmap) put_page(pfn_to_page(pfn)); + pgmap->cleanup(pgmap->ref); /* pages are dead and unused, undo the arch mapping */ align_start = res->start & ~(SECTION_SIZE - 1); @@ -134,8 +135,8 @@ static void devm_memremap_pages_release(void *data) * 2/ The altmap field may optionally be initialized, in which case altmap_valid * must be set to true * - * 3/ pgmap->ref must be 'live' on entry and will be killed at - * devm_memremap_pages_release() time, or if this routine fails. + * 3/ pgmap->ref must be 'live' on entry and will be killed and reaped + * at devm_memremap_pages_release() time, or if this routine fails. * * 4/ res is expected to be a host memory range that could feasibly be * treated as a "System RAM" range, i.e. not a device mmio range, but @@ -151,8 +152,10 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) pgprot_t pgprot = PAGE_KERNEL; int error, nid, is_ram; - if (!pgmap->ref || !pgmap->kill) + if (!pgmap->ref || !pgmap->kill || !pgmap->cleanup) { + WARN(1, "Missing reference count teardown definition\n"); return ERR_PTR(-EINVAL); + } align_start = res->start & ~(SECTION_SIZE - 1); align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE) @@ -163,14 +166,16 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) if (conflict_pgmap) { dev_WARN(dev, "Conflicting mapping in same section\n"); put_dev_pagemap(conflict_pgmap); - return ERR_PTR(-ENOMEM); + error = -ENOMEM; + goto err_array; } conflict_pgmap = get_dev_pagemap(PHYS_PFN(align_end), NULL); if (conflict_pgmap) { dev_WARN(dev, "Conflicting mapping in same section\n"); put_dev_pagemap(conflict_pgmap); - return ERR_PTR(-ENOMEM); + error = -ENOMEM; + goto err_array; } is_ram = region_intersects(align_start, align_size, @@ -262,6 +267,8 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) pgmap_array_delete(res); err_array: pgmap->kill(pgmap->ref); + pgmap->cleanup(pgmap->ref); + return ERR_PTR(error); } EXPORT_SYMBOL_GPL(devm_memremap_pages); diff --git a/mm/hmm.c b/mm/hmm.c index fe1cd87e49ac..225ade644058 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -975,9 +975,8 @@ static void hmm_devmem_ref_release(struct percpu_ref *ref) complete(&devmem->completion); } -static void hmm_devmem_ref_exit(void *data) +static void hmm_devmem_ref_exit(struct percpu_ref *ref) { - struct percpu_ref *ref = data; struct hmm_devmem *devmem; devmem = container_of(ref, struct hmm_devmem, ref); @@ -1054,10 +1053,6 @@ struct hmm_devmem *hmm_devmem_add(const struct hmm_devmem_ops *ops, if (ret) return ERR_PTR(ret); - ret = devm_add_action_or_reset(device, hmm_devmem_ref_exit, &devmem->ref); - if (ret) - return ERR_PTR(ret); - size = ALIGN(size, PA_SECTION_SIZE); addr = min((unsigned long)iomem_resource.end, (1UL << MAX_PHYSMEM_BITS) - 1); @@ -1096,6 +1091,7 @@ struct hmm_devmem *hmm_devmem_add(const struct hmm_devmem_ops *ops, devmem->pagemap.ref = &devmem->ref; devmem->pagemap.data = devmem; devmem->pagemap.kill = hmm_devmem_ref_kill; + devmem->pagemap.cleanup = hmm_devmem_ref_exit; result = devm_memremap_pages(devmem->device, &devmem->pagemap); if (IS_ERR(result)) @@ -1133,11 +1129,6 @@ struct hmm_devmem *hmm_devmem_add_resource(const struct hmm_devmem_ops *ops, if (ret) return ERR_PTR(ret); - ret = devm_add_action_or_reset(device, hmm_devmem_ref_exit, - &devmem->ref); - if (ret) - return ERR_PTR(ret); - devmem->pfn_first = devmem->resource->start >> PAGE_SHIFT; devmem->pfn_last = devmem->pfn_first + (resource_size(devmem->resource) >> PAGE_SHIFT); @@ -1150,6 +1141,7 @@ struct hmm_devmem *hmm_devmem_add_resource(const struct hmm_devmem_ops *ops, devmem->pagemap.ref = &devmem->ref; devmem->pagemap.data = devmem; devmem->pagemap.kill = hmm_devmem_ref_kill; + devmem->pagemap.cleanup = hmm_devmem_ref_exit; result = devm_memremap_pages(devmem->device, &devmem->pagemap); if (IS_ERR(result)) diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c index c6635fee27d8..219dd0a1cb08 100644 --- a/tools/testing/nvdimm/test/iomap.c +++ b/tools/testing/nvdimm/test/iomap.c @@ -108,7 +108,9 @@ static void nfit_test_kill(void *_pgmap) { struct dev_pagemap *pgmap = _pgmap; + WARN_ON(!pgmap || !pgmap->ref || !pgmap->kill || !pgmap->cleanup); pgmap->kill(pgmap->ref); + pgmap->cleanup(pgmap->ref); } void *__wrap_devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)