From patchwork Wed Jul 10 01:16:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11037673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D50A814F6 for ; Wed, 10 Jul 2019 01:16:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB19028864 for ; Wed, 10 Jul 2019 01:16:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE7A8288B4; Wed, 10 Jul 2019 01:16:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A585128864 for ; Wed, 10 Jul 2019 01:16:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 92B778E0062; Tue, 9 Jul 2019 21:16:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8DC918E0032; Tue, 9 Jul 2019 21:16:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 756F98E0062; Tue, 9 Jul 2019 21:16:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 555228E0032 for ; Tue, 9 Jul 2019 21:16:52 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id x1so780700qts.9 for ; Tue, 09 Jul 2019 18:16:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=tyS5k/4QJCxPGcV91OLPh8wYVhK9QVlUZJPQN7ARXzo=; b=oZxNd5hLuFDZWDx8yrMUd71x6emEC9h5lpXTw1TeSNM72bAiybzarrCWnoh/cOJhyp A601/u/qAnWeCuQ5/lIrJqCjH6/jE53oRTjnfM5uPiG7O5ARDf63vQ4Ffes1lAVdG8nS Hao4/w/5L8/hWQls+497mRV2XW/ovtAJDJ9DeNi/A6ExQXtK0AO+v0qY7FYSj7FQcSWP blGtzOpgCHoYDA9tDEaahQusPMaohqVV/k3f+Z900cFPlB63jXAZRBJiHTKdNwbLs7kW AqWR4hhf8NpIP3r/mX0QfIBUEkPjEs2M96Cmmg1i1ahZzeQyTC3yG373kmRM3fAHLUbd K+gw== X-Gm-Message-State: APjAAAWZok18Ab0tVZZ1B+VZimtKYBJbomcRVSAv9gfFDZsyOI/zgyNB vwiN0dXFkbqcr1z49hyr0rjcKES1uHzqyQ9q4W9iq64RHlSaidflCugp4yEE0kp1tmImbDjeost 0BBwmCw7uEhbbUaYTaHTtbGLpi2evxi4w3FF9Okfe2ItcVpByeTY57czMczUYERTMbQ== X-Received: by 2002:ac8:2e14:: with SMTP id r20mr21857082qta.241.1562721412071; Tue, 09 Jul 2019 18:16:52 -0700 (PDT) X-Received: by 2002:ac8:2e14:: with SMTP id r20mr21857053qta.241.1562721411353; Tue, 09 Jul 2019 18:16:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562721411; cv=none; d=google.com; s=arc-20160816; b=vRFSQOP8QpGZwkInUzm2bBaI78eVdOq3LPL9d5If/IPyFYzOdZOHZwc6rnl7X60ffU naqumuw8XFLgXv33RfMFHUEG8Qkb5jmG+8a29xDJAFKSdJ3a01lNXSW82wBx4lBhVJjm NGDq0wmtokK45yvDS6GaGluzcpErFOttpWqE3KO+S+bNiruD5tzlICENILaKU8Q85WQz NQ52z+zqO90t2C6P/ovHST8j+0iNsyKhMr9x8/DBDOW89JdGX/PDHrpjpTSTr4Go+xIZ V/RiUDcWKmI32vIFU2Y1v9Rn6e/SxWt3l7sBi8UZ8eidBySTGA5prZeaP6z26g7AAXTg H48g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=tyS5k/4QJCxPGcV91OLPh8wYVhK9QVlUZJPQN7ARXzo=; b=ALN4xPamQo3EMiumZ4sa/9/5piW+ZA0fTDgpsRVG0gjBnJVONGkG+3FdPyQIuhZrc8 htTAFlM5JI7jGBMOVhB32yadxyRTu5i2KcmIHoR0xbd6UrVZ5IECUL/1yS9uI4QuZs1q q5V8diKyEWvW3cov5CgoLXvMXtA5TGCtJuWG1WVlK9vkXNyIajm8xDDvpolyIr3XmuEG 4FvFEshIz5mUunaJNYHU6jpyfGA/mJzkuHYSV2bBXWatdOESCBLuaKzr1567AnTo4mdU RltEYNNSPLdSemY5lSjKYba4RpZFieIhpyYwIn2pDsrSL4EFHLXwewnsq8dwsRwLQuUR 3nFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=oIRaXzcB; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id d10sor259746qkl.81.2019.07.09.18.16.51 for (Google Transport Security); Tue, 09 Jul 2019 18:16:51 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=oIRaXzcB; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tyS5k/4QJCxPGcV91OLPh8wYVhK9QVlUZJPQN7ARXzo=; b=oIRaXzcBKN21PrMZxpFfBUUbAE7wF/UfbZoNG2Bo81kTMfH1jkYTY98GTIENUsItos V/WX4vpB1ciXEijN9rOhXuZxWeYIPcAwTN7nY6AcOWXL/QBUJIZ3udoI5EgSSrYP/v6Z n5zMxjMJN0LyV1nPiAgnF9KWNaXCjsqQ1RA2g69lzwit3bTSsd3CTb+ZpP8Yt1JHTAk+ 7mMvv1dHPucOHLPxqVU9EeLk6XkScChSXpj10FLFNDhUBRW4PeMiFSAEMN4izsN74QLv hSem2wQeYJr4YYe7fdf7Rk+3Py/i9ot4l52CQSRsBrhnyLj311qazIkAWg8uhCACaTgT L9Qg== X-Google-Smtp-Source: APXvYqzSbKGKdJu9YdO8wNTJ5v/G3QEIGSQCvwnfWWheykUuMY8FrLDyWCW6hyyPO85S9nAUTrUtAw== X-Received: by 2002:a37:bac2:: with SMTP id k185mr20774793qkf.211.1562721411119; Tue, 09 Jul 2019 18:16:51 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id u7sm260057qta.82.2019.07.09.18.16.49 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 09 Jul 2019 18:16:50 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v7 1/3] device-dax: fix memory and resource leak if hotplug fails Date: Tue, 9 Jul 2019 21:16:45 -0400 Message-Id: <20190710011647.10944-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190710011647.10944-1-pasha.tatashin@soleen.com> References: <20190710011647.10944-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When add_memory() function fails, the resource and the memory should be freed. Fixes: c221c0b0308f ("device-dax: "Hotplug" persistent memory for use like normal RAM") Signed-off-by: Pavel Tatashin Reviewed-by: Dave Hansen --- drivers/dax/kmem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index a02318c6d28a..4c0131857133 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -66,8 +66,11 @@ int dev_dax_kmem_probe(struct device *dev) new_res->name = dev_name(dev); rc = add_memory(numa_node, new_res->start, resource_size(new_res)); - if (rc) + if (rc) { + release_resource(new_res); + kfree(new_res); return rc; + } return 0; } From patchwork Wed Jul 10 01:16:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11037679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC32B1805 for ; Wed, 10 Jul 2019 01:16:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BCFBC28864 for ; Wed, 10 Jul 2019 01:16:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B0B972889E; Wed, 10 Jul 2019 01:16:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29499288A9 for ; Wed, 10 Jul 2019 01:16:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 906008E0063; Tue, 9 Jul 2019 21:16:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8DD3A8E0032; Tue, 9 Jul 2019 21:16:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7574B8E0063; Tue, 9 Jul 2019 21:16:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 553F88E0032 for ; Tue, 9 Jul 2019 21:16:54 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id r58so790368qtb.5 for ; Tue, 09 Jul 2019 18:16:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=oUgTnAZaywklydAAAPvB3a6I5/R1cYvy+iIkRwSNZnk=; b=F2OmTC+Buy/CBkWanuVllpngI1fvUzNI6Mp8Nbw1e3iShLW/8PmThXWhzEkDpHII/P nfmD/qpwGiGAt/Xqi7ACClbJazO2TpAw1Cd99iCDwiM2/IUPeWNiUnpUee7KrUICOK2E 0Yu6SZ7G4LZZj6FJHvRZJj2L2K9s6H0ZzntEVDzG9/PfcoHkhwwQu8V2oS03mpNHAhTP QSv6YVXbr+X8YmRQ4JPUh35tf4CZZdD2MVSTRN2QtqCEUJBfTh86f4Lx1cqbwtEBUIEV 7z+hysKzy67A9YqEQeJMaFqVaaPBWGSnlkgad8fljPCw8skvJkB4rSMk5ATkYsvXc2vk zIhg== X-Gm-Message-State: APjAAAXYyGKhqsaFbIBi+q6mM2jWuC8KhAk+3auDWEYQhQCbx+Y4gJ+d 64IOpbxl5AagVeabW5Gdz5QJDN0beUmnrUxVYE1wES2FVF8MiWg1rexT9oTXXtcqevSftof2Ueu J4e6dID240lgssS9ugHrZqyXBfuifX59ze6oOTaHAU/tjuhPlkRswyPveDvSEpIJbhQ== X-Received: by 2002:a0c:e001:: with SMTP id j1mr22236035qvk.110.1562721414079; Tue, 09 Jul 2019 18:16:54 -0700 (PDT) X-Received: by 2002:a0c:e001:: with SMTP id j1mr22235981qvk.110.1562721412964; Tue, 09 Jul 2019 18:16:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562721412; cv=none; d=google.com; s=arc-20160816; b=TR/oUmwmpb5hjopLpv7llAc7bdSePoRNcYR2xl29qw+kk5Bjx3kWSN770WtilS3JLJ ld3ScER9puBufQOErajUWEqk1BW1ZX3b60sqPXBeNhQt6e9UWj5ZZ9gJMHdO7Bgb4CRP ILclwCdo5gPiprBrFvXBtVHElP/1sdaT5hG8DKvhnwp0GXx3ApQVcrNlu71XSPqlLhBm 2U6AKGI3kOTf22SUWJJ0d2b4wRt9eTHsUmfQnBdr5DRglT5ptYHI1fiH98ud62bqskC8 rmyedVBpGY2TRRpSJ8n8npvJAXAc81v0LpZqQnY8XBooEA94Thoxo1rvN34qQc4EcmhQ t65A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=oUgTnAZaywklydAAAPvB3a6I5/R1cYvy+iIkRwSNZnk=; b=K4Ml9PVby2bZ8DzbMA5gxn20+DeRkMXREIEBnLXLGMAFcAPnGgrVbANCR+2daLgZ5E 7k/OYCdEIobt3MQcMXNfke1qT6jz62EP2f3Xepi4ug9iocWHkIfZnOdH7EvMKGusXBdk RTiAWuK+oLBiGf7eV9wYQ2kAp5Fg7/uQfeV7WK2skUIERaOX71v5h3gVgLBwoqM08dal eYehGJD8OZJBXmDTOJaqgPazLFZYaOFlUpcSeC2Z36bfPDMtBoe0vyHkEyZP1HcyPEAV MOcGbtvtbfwebBkdLkvVIqO75S0bDtgh3aa11V8hlBxw6uopOxl3j89xViKftJkmjCgZ sa/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=LntS6yWp; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id p184sor256278qkc.123.2019.07.09.18.16.52 for (Google Transport Security); Tue, 09 Jul 2019 18:16:52 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=LntS6yWp; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=oUgTnAZaywklydAAAPvB3a6I5/R1cYvy+iIkRwSNZnk=; b=LntS6yWpaejTt9SVF+fVC7Ci+89fr1DyTiF2xUKIH6h5NkvSH1k381FHUeqUUuEcc0 yTUCL82/lOdUkkiALPPXLeGUGi+WKdtti/fdXv7kou9x3YPyBEwjVcIl7USaCoXIZFUA fh2Y3StdFNp2P+hUfdI7X47L0fnYYCJ6Q27PFiFAX3EL6BnWqkIs1WJrholKT3m0OGvh QoQh/R6z6Axr1Gsk8qhhISOsyPX5mxP1AaJiB9yX8iBQS+y1DZqdo7yTBfxDyRPYS/HN y+M1N36aWpeympdukvQFHHSKXlVWGprnTISAipceqCsdtGqIPYS/cmxeFKXBruxe+2u5 JQ8A== X-Google-Smtp-Source: APXvYqyayfanR0LhFsg7XP0bJdPSuQa2GB1LgirxNwXiGmy32bzFbJzapk8lj3SFV7YobHsyO98cKQ== X-Received: by 2002:a37:4c4e:: with SMTP id z75mr21389763qka.230.1562721412692; Tue, 09 Jul 2019 18:16:52 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id u7sm260057qta.82.2019.07.09.18.16.51 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 09 Jul 2019 18:16:52 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v7 2/3] mm/hotplug: make remove_memory() interface useable Date: Tue, 9 Jul 2019 21:16:46 -0400 Message-Id: <20190710011647.10944-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190710011647.10944-1-pasha.tatashin@soleen.com> References: <20190710011647.10944-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP As of right now remove_memory() interface is inherently broken. It tries to remove memory but panics if some memory is not offline. The problem is that it is impossible to ensure that all memory blocks are offline as this function also takes lock_device_hotplug that is required to change memory state via sysfs. So, between calling this function and offlining all memory blocks there is always a window when lock_device_hotplug is released, and therefore, there is always a chance for a panic during this window. Make this interface to return an error if memory removal fails. This way it is safe to call this function without panicking machine, and also makes it symmetric to add_memory() which already returns an error. Signed-off-by: Pavel Tatashin Reviewed-by: David Hildenbrand Acked-by: Michal Hocko --- include/linux/memory_hotplug.h | 8 +++-- mm/memory_hotplug.c | 64 +++++++++++++++++++++++----------- 2 files changed, 49 insertions(+), 23 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index ae892eef8b82..988fde33cd7f 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -324,7 +324,7 @@ static inline void pgdat_resize_init(struct pglist_data *pgdat) {} extern bool is_mem_section_removable(unsigned long pfn, unsigned long nr_pages); extern void try_offline_node(int nid); extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages); -extern void remove_memory(int nid, u64 start, u64 size); +extern int remove_memory(int nid, u64 start, u64 size); extern void __remove_memory(int nid, u64 start, u64 size); #else @@ -341,7 +341,11 @@ static inline int offline_pages(unsigned long start_pfn, unsigned long nr_pages) return -EINVAL; } -static inline void remove_memory(int nid, u64 start, u64 size) {} +static inline int remove_memory(int nid, u64 start, u64 size) +{ + return -EBUSY; +} + static inline void __remove_memory(int nid, u64 start, u64 size) {} #endif /* CONFIG_MEMORY_HOTREMOVE */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index e096c987d261..77d1f69cdead 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1736,9 +1736,10 @@ static int check_memblock_offlined_cb(struct memory_block *mem, void *arg) endpa = PFN_PHYS(section_nr_to_pfn(mem->end_section_nr + 1))-1; pr_warn("removing memory fails, because memory [%pa-%pa] is onlined\n", &beginpa, &endpa); - } - return ret; + return -EBUSY; + } + return 0; } static int check_cpu_on_node(pg_data_t *pgdat) @@ -1821,19 +1822,9 @@ static void __release_memory_resource(resource_size_t start, } } -/** - * remove_memory - * @nid: the node ID - * @start: physical address of the region to remove - * @size: size of the region to remove - * - * NOTE: The caller must call lock_device_hotplug() to serialize hotplug - * and online/offline operations before this call, as required by - * try_offline_node(). - */ -void __ref __remove_memory(int nid, u64 start, u64 size) +static int __ref try_remove_memory(int nid, u64 start, u64 size) { - int ret; + int rc = 0; BUG_ON(check_hotplug_memory_range(start, size)); @@ -1841,13 +1832,13 @@ void __ref __remove_memory(int nid, u64 start, u64 size) /* * All memory blocks must be offlined before removing memory. Check - * whether all memory blocks in question are offline and trigger a BUG() + * whether all memory blocks in question are offline and return error * if this is not the case. */ - ret = walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1), NULL, - check_memblock_offlined_cb); - if (ret) - BUG(); + rc = walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1), NULL, + check_memblock_offlined_cb); + if (rc) + goto done; /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); @@ -1859,14 +1850,45 @@ void __ref __remove_memory(int nid, u64 start, u64 size) try_offline_node(nid); +done: mem_hotplug_done(); + return rc; } -void remove_memory(int nid, u64 start, u64 size) +/** + * remove_memory + * @nid: the node ID + * @start: physical address of the region to remove + * @size: size of the region to remove + * + * NOTE: The caller must call lock_device_hotplug() to serialize hotplug + * and online/offline operations before this call, as required by + * try_offline_node(). + */ +void __remove_memory(int nid, u64 start, u64 size) +{ + + /* + * trigger BUG() is some memory is not offlined prior to calling this + * function + */ + if (try_remove_memory(nid, start, size)) + BUG(); +} + +/* + * Remove memory if every memory block is offline, otherwise return -EBUSY is + * some memory is not offline + */ +int remove_memory(int nid, u64 start, u64 size) { + int rc; + lock_device_hotplug(); - __remove_memory(nid, start, size); + rc = try_remove_memory(nid, start, size); unlock_device_hotplug(); + + return rc; } EXPORT_SYMBOL_GPL(remove_memory); #endif /* CONFIG_MEMORY_HOTREMOVE */ From patchwork Wed Jul 10 01:16:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11037681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EBA414F6 for ; Wed, 10 Jul 2019 01:17:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42B1328864 for ; Wed, 10 Jul 2019 01:17:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3679D288A9; Wed, 10 Jul 2019 01:17:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF71128864 for ; Wed, 10 Jul 2019 01:16:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FC668E0064; Tue, 9 Jul 2019 21:16:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3AF9E8E0032; Tue, 9 Jul 2019 21:16:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18A2A8E0064; Tue, 9 Jul 2019 21:16:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id E443C8E0032 for ; Tue, 9 Jul 2019 21:16:55 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id q26so794666qtr.3 for ; Tue, 09 Jul 2019 18:16:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=CVTJCjL61cT1sLEtn2w9JvdFUT6cQxdBQbzitEZ8G3w=; b=TuME682oXRJ/7v7UXnSDNb0yeTKfMj1sitggQMjvXpNHdEk4Cegrh3caqUQq/DQHr3 wJmwIPkPsurG68sGyJNP/2ENxSNuQ75NXhYp2yusRf9sayWWqhrRZLE9YYmaZefZsB0p 6iXSY2dZ+EYYvRt1UKMjuy2WZoHNrxv8TzbYCBOc0vjyQSr1Ar0YKzWlsh9ElRQKB2Gy IaRgNlqzazzQ2hAIs7kQlWkQ4FbPxXQwd9riC/KjGEQGd/1a1AmlCVblApgbLh5PChJF 04ylkqrLdgiEU/Ae5N1PW6S7R4IPE8wyNR3mdSa3alpt2eI7wRpbSGEkTcCH3tC1t0i1 mZgw== X-Gm-Message-State: APjAAAUdR8SZmRz73QVSA3gXdjNICXh2CtMdFJXqzgVZGkdBnH4/Pfc0 tjk5vhVN2sKu/1EUaf50NyAQvjQBZAyZmLUTRGyNUD8Qkddag2vyiw8yzO8Bx/GE4TEaKo+mCNP KRvxaWMJdfrGPCTqAwi0vZ2VEhB4i29xI2MgTUjuVmS+e2+YNARNTtGxGkyInm7QhwQ== X-Received: by 2002:a0c:b4ae:: with SMTP id c46mr22510515qve.230.1562721415695; Tue, 09 Jul 2019 18:16:55 -0700 (PDT) X-Received: by 2002:a0c:b4ae:: with SMTP id c46mr22510454qve.230.1562721414585; Tue, 09 Jul 2019 18:16:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562721414; cv=none; d=google.com; s=arc-20160816; b=bk7t6ofeZzBHBJA8qeRz9xXmbl3xGvqFKj0tCoq0DPei6wscCgY9QCWpq+hWARm5LS sPO6BTMUjfKskgotumDttUEMEfUcnP5AKHw4GNAlJKB78QQ0ZNyXMSk+wrEGh3A7Twx8 3l1NJJ7H8cYNpwjqrXldKjMOsNhZDjole2J4H9t7DEIRNvEhAGO4QiQIMBR75F9JL431 tyvEVx32amJf5hlzNY7bPeyCmyBH4jySpIkD7nmXmrm53DljC3lhaf3WzWITRFpEihJj y0CMIwjJ7LPnV5Q0aYkvPZLB/XdD1GsFCHSCfJNpzAwbL+INtnL1KW0j30VfjRuYCCHy 4/1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=CVTJCjL61cT1sLEtn2w9JvdFUT6cQxdBQbzitEZ8G3w=; b=SXEP2txCGBooORCecJ+gwhRrGiqtnKPaXLuNZcJ4t78oYbJO64wIbL+7R4Z5LTD5tB MGqmnMJJxc1GS/XCk/wXLjYDmMcsGVfvvnHpfO5wbNHhlzTVDMHyk1ptNS/th/0JfZ5x kcWJ93NUv/uvVwuvf3i2EigVimEFh4HH7dnXIfgHKv2/DFaJnQviRFIdT6FD43oLivUz ZUdqijzzWMwJJy75jDeo/LGa2NLJIKmlHw9WbwT4BqdpKsfqrnOvI7IDpYXVpTbKX20y GJPlcdGo25kfASqaVCceQMkCHxWA6Sqrlebl4FTqXuFBlsHp3Lk47Ao4nfEjZtwR9q+Z Fm4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=CgnZ01od; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c27sor242686qkk.192.2019.07.09.18.16.54 for (Google Transport Security); Tue, 09 Jul 2019 18:16:54 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=CgnZ01od; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=CVTJCjL61cT1sLEtn2w9JvdFUT6cQxdBQbzitEZ8G3w=; b=CgnZ01od2X5o9ZxnU3YxLA5iP6DGIirNV+yqbmaGGd37pbDw+kMM0vJ0slJQWHOgGz 89n/NsZaNntd/HsoETEEEWONOoRpUvU+swsq3e0sogqmdn7o3c+6pgPWb0uMVxE0WqLg +7f3548VjFl28RcqVuZy3NVuR0FeaZeD2MTcqLaGipojj6pr7WHfHlZaKZDd6uR230L0 JT4sctjsx1qzKG+wB2YuOCU2TWArwX6x/FgTtjBrPZrz5jK0b+CWmWzZQlS4c5VoVZtr JemrX7sEW8IBBYGZikmfiz6eQN+ryIQdDAWkxCVDBjzH3tTwnd+byUmpjy4XiunxSWhU BXQQ== X-Google-Smtp-Source: APXvYqys9rzeaiHpJCQSMGWFJQogIT5Nl/kLTQmCtVzEY7YWfkQtFYUrb2NcrUN7cHm0A3qL/HJzCg== X-Received: by 2002:a05:620a:1661:: with SMTP id d1mr21322876qko.192.1562721414251; Tue, 09 Jul 2019 18:16:54 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id u7sm260057qta.82.2019.07.09.18.16.52 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 09 Jul 2019 18:16:53 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v7 3/3] device-dax: "Hotremove" persistent memory that is used like normal RAM Date: Tue, 9 Jul 2019 21:16:47 -0400 Message-Id: <20190710011647.10944-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190710011647.10944-1-pasha.tatashin@soleen.com> References: <20190710011647.10944-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It is now allowed to use persistent memory like a regular RAM, but currently there is no way to remove this memory until machine is rebooted. This work expands the functionality to also allows hotremoving previously hotplugged persistent memory, and recover the device for use for other purposes. To hotremove persistent memory, the management software must first offline all memory blocks of dax region, and than unbind it from device-dax/kmem driver. So, operations should look like this: echo offline > /sys/devices/system/memory/memoryN/state ... echo dax0.0 > /sys/bus/dax/drivers/kmem/unbind Note: if unbind is done without offlining memory beforehand, it won't be possible to do dax0.0 hotremove, and dax's memory is going to be part of System RAM until reboot. Signed-off-by: Pavel Tatashin Reviewed-by: David Hildenbrand Reviewed-by: Dan Williams --- drivers/dax/dax-private.h | 2 ++ drivers/dax/kmem.c | 41 +++++++++++++++++++++++++++++++++++---- 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/drivers/dax/dax-private.h b/drivers/dax/dax-private.h index b4177aafbbd1..9ee659ed5566 100644 --- a/drivers/dax/dax-private.h +++ b/drivers/dax/dax-private.h @@ -45,6 +45,7 @@ struct dax_region { * @pgmap - pgmap for memmap setup / lifetime (driver owned) * @ref: pgmap reference count (driver owned) * @cmp: @ref final put completion (driver owned) + * @dax_mem_res: physical address range of hotadded DAX memory */ struct dev_dax { struct dax_region *region; @@ -54,6 +55,7 @@ struct dev_dax { struct dev_pagemap pgmap; struct percpu_ref ref; struct completion cmp; + struct resource *dax_kmem_res; }; static inline struct dev_dax *to_dev_dax(struct device *dev) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 4c0131857133..c52a8e5f2345 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -71,21 +71,54 @@ int dev_dax_kmem_probe(struct device *dev) kfree(new_res); return rc; } + dev_dax->dax_kmem_res = new_res; return 0; } +#ifdef CONFIG_MEMORY_HOTREMOVE +static int dev_dax_kmem_remove(struct device *dev) +{ + struct dev_dax *dev_dax = to_dev_dax(dev); + struct resource *res = dev_dax->dax_kmem_res; + resource_size_t kmem_start = res->start; + resource_size_t kmem_size = resource_size(res); + int rc; + + /* + * We have one shot for removing memory, if some memory blocks were not + * offline prior to calling this function remove_memory() will fail, and + * there is no way to hotremove this memory until reboot because device + * unbind will succeed even if we return failure. + */ + rc = remove_memory(dev_dax->target_node, kmem_start, kmem_size); + if (rc) { + dev_err(dev, + "region %pR cannot be hotremoved, error (%d)\n", + res, rc); + return rc; + } + + /* Release and free dax resources */ + release_resource(res); + kfree(res); + dev_dax->dax_kmem_res = NULL; + + return 0; +} +#else static int dev_dax_kmem_remove(struct device *dev) { /* - * Purposely leak the request_mem_region() for the device-dax - * range and return '0' to ->remove() attempts. The removal of - * the device from the driver always succeeds, but the region - * is permanently pinned as reserved by the unreleased + * Without hotremove purposely leak the request_mem_region() for the + * device-dax range and return '0' to ->remove() attempts. The removal + * of the device from the driver always succeeds, but the region is + * permanently pinned as reserved by the unreleased * request_mem_region(). */ return 0; } +#endif /* CONFIG_MEMORY_HOTREMOVE */ static struct dax_device_driver device_dax_kmem_driver = { .drv = {