From patchwork Thu Dec 6 00:26:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Yang X-Patchwork-Id: 10715155 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B69261731 for ; Thu, 6 Dec 2018 00:26:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A50562DAA3 for ; Thu, 6 Dec 2018 00:26:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 98DBC2DAC6; Thu, 6 Dec 2018 00:26:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C53F2DAA3 for ; Thu, 6 Dec 2018 00:26:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 560636B771B; Wed, 5 Dec 2018 19:26:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E81D6B771C; Wed, 5 Dec 2018 19:26:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3891D6B771D; Wed, 5 Dec 2018 19:26:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id E7DFE6B771B for ; Wed, 5 Dec 2018 19:26:39 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id b8so18143119pfe.10 for ; Wed, 05 Dec 2018 16:26:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=URCyZCPL+TUEFjSpR/fenuVZXwVf0zMM4zm8B9xW2pA=; b=PP1NqslaxkxAms2oeBipZgJTVJpgrZkhCNMuc/3gSU0G7okeMZRuPB26GlpvV81N9j gt+y08j6VOPNYRE5vQ6whpMMvTp5JaDGJPC+7yKuGT1MYd1w3SaNYkU7R5IMciP0z8fr Z4oIJ1/aADlodQ54NNySDZYm2cOLPTbYbemA5S6WbjlwmdGGxk7UJuS9+t2PLuPUpjpH SLvc5DbigUZuVoqeBI1HTan7aYte41kQsynLcOVJLZbewARv5Yyqy13gFLbgJSwaRK9q ZduNLkhTEZ4NyZ1JRLF/IoHF90gAeCZsUCR+dsjetd2HXNoYbeTGGsztQ46zEw0o0hHz NKeg== X-Gm-Message-State: AA+aEWbD+OluhuHxN5wJusMP1sTbtCYHjZRyED4BA8IKDHHFectimNr0 d6cjk6fPZoQPmlePv1PMFYTew1Ii/2uSXXJ83tDdQCNWATwRsEsCgzIZfoaeL5ptSj+X1HNanQh gbdX9OmzXBpChbceMzgU00daO+7vgUps2AjfZusNZkBxpEkahLhSdagXVvtPlFzWXo2YzvCL6lX somXcNuXZGHdtdOMYvae9B9ZGcybIDjccFbq9i5uc7ky4J5+msWTrYwEunVupryLAGk2R+Oe0P/ /qDf7XcnBOW9hE2hzIgy6RDzz3PJ8zCLwIe0955Oou791Sx9jVF45t+tyvjM6D/0gug8q/RIJjc 8adbJzdLffP4d126uOsXdbI61Y/ap9HoW2Jm5u9nfEh5kcflK3h2zjHMBNFhf/VKm3y62ZvB9WB 5 X-Received: by 2002:a17:902:4681:: with SMTP id p1mr26939398pld.184.1544055999567; Wed, 05 Dec 2018 16:26:39 -0800 (PST) X-Received: by 2002:a17:902:4681:: with SMTP id p1mr26939373pld.184.1544055998791; Wed, 05 Dec 2018 16:26:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544055998; cv=none; d=google.com; s=arc-20160816; b=lqHnUhimUta+O7cBSr9a/M/XTQFyNawTIbVteo06ghFzlV8ZYCvAxfHaAgO2Cs6Y2U TwLm3Xmb36dDPfENHBHkcGi/4PBChAeJZULx6/TiUBOvUw2jLBrZHLWL9zjuUk+oL1+k gNdqk1NFiBuTnIVyRhPaQVJQhxzSJ3yIeyzP7iZx0xQyLSR00zDKFsh1pXGJg2WsDgDZ uwGRy/7prx65tGjfW88GrUQTjeTyPxBZTPCVj+pTG+4d0WBAxFH8UujjQId/lAk/fSLg XTd821r1TF1V6P1WUGCzzAnMB3UkrwdKTunoy4sKnqd8Bx/OiktdwPmGvT7acDpqhcfS 8eyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=URCyZCPL+TUEFjSpR/fenuVZXwVf0zMM4zm8B9xW2pA=; b=tBN9bW0ccP34IemXt2hpeJcURjTWgCsKcCt09ZSWPRKn4ICeIBdmrZ5rqe8bmyLu6A XYpHcFS5H4jfp1APyXpUXkAivKqTGoSBJrfrmP4dlpEat3NIlg42B1uyVmxMGOSNZSpx 2Gf99dlw9+29Lqbg2KLgSWgu/RwHrZhIKxwebkIGHr+puh8JRSl7X//h8xJckzwCnp6g 6sV90APygLYsbQzuZN49+T86eEk9lPLMjSDX8n/0RI+Z7BInFZ7IV75CNudVVz0Ut5GX Xx7F1cR+IoUqnvz7NuxTucyuWpDnKCUwzZvxPaJ8UChNhxpdQPRrSXECW7l1EyjGRFE5 Mdqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=X2SljuWn; spf=pass (google.com: domain of richard.weiyang@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 37sor30189546ple.23.2018.12.05.16.26.38 for (Google Transport Security); Wed, 05 Dec 2018 16:26:38 -0800 (PST) Received-SPF: pass (google.com: domain of richard.weiyang@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=X2SljuWn; spf=pass (google.com: domain of richard.weiyang@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=URCyZCPL+TUEFjSpR/fenuVZXwVf0zMM4zm8B9xW2pA=; b=X2SljuWn+9bHWDMbyEyaxzOXEa1CTVhuZwutL1yEfJLDYl9NKJm2Kgw+1uosRkYMNl 3QJwbPA2cJv6vDvAGJCcIsk7y1bqjyT0dDsQMOOrENr9O0mPGPACPkmlmbPA+70I/fse FzdEa6l+FYByRZ4nUDwk3mlQN2asxSvYj7SL7/DmyqBjY9LcKiMAY4MtMCGoH6MSmFyI cxBe70WguRLY4Y594pSW/MGy6XTO6e/sGVZwUWbWm8MeGYKgFWgLGF3luO45x7oJoEFI rL3gU4OPcd0w/A+gP55Qkj+ccP94wq1M6QKcAWVzE0d7oRY94CtfmCT9o0QRNkzxVsbD onhg== X-Google-Smtp-Source: AFSGD/W6/H8znQ6RfXPvhVzpcTkpVDdl7lQ7B+GA01lNNYw1QkCIeTJkfpeKW+9gzt6QzH2a3lYoGw== X-Received: by 2002:a17:902:70c6:: with SMTP id l6mr26947492plt.30.1544055998474; Wed, 05 Dec 2018 16:26:38 -0800 (PST) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id e24sm30646534pfi.153.2018.12.05.16.26.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Dec 2018 16:26:37 -0800 (PST) From: Wei Yang To: rppt@linux.ibm.com, david@redhat.com, mhocko@suse.com, osalvador@suse.de Cc: akpm@linux-foundation.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, Wei Yang Subject: [PATCH v2 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks Date: Thu, 6 Dec 2018 08:26:22 +0800 Message-Id: <20181206002622.30675-2-richard.weiyang@gmail.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20181206002622.30675-1-richard.weiyang@gmail.com> References: <20181205023426.24029-1-richard.weiyang@gmail.com> <20181206002622.30675-1-richard.weiyang@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently locking for memory hotplug is a little complicated. Generally speaking, we leverage the two global lock: * device_hotplug_lock * mem_hotplug_lock to serialise the process. While for the long term, we are willing to have more fine-grained lock to provide higher scalability. This patch divides Locking Internal section based on these two global locks to help readers to understand it. Also it adds some new finding to enrich it. [David: words arrangement] Signed-off-by: Wei Yang Reviewed-by: David Hildenbrand --- v2: adjustment based on David and Mike comment --- Documentation/core-api/memory-hotplug.rst | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst index de7467e48067..51d477ad4b80 100644 --- a/Documentation/core-api/memory-hotplug.rst +++ b/Documentation/core-api/memory-hotplug.rst @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification queue. Locking Internals ================= +In addition to fine grained locks like pgdat_resize_lock, there are three locks +involved + +- device_hotplug_lock +- mem_hotplug_lock +- device_lock + +Currently, they are twisted together for all kinds of reasons. The following +part is divided into device_hotplug_lock and mem_hotplug_lock parts +respectively to describe those tricky situations. + +device_hotplug_lock +--------------------- + When adding/removing memory that uses memory block devices (i.e. ordinary RAM), the device_hotplug_lock should be held to: @@ -111,13 +125,20 @@ As the device is visible to user space before taking the device_lock(), this can result in a lock inversion. onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) +device_offline() - to make sure it is properly synchronized to actions via +sysfs. Even if mem_hotplug_lock is used to protect the process, because of the +lock inversion described above, holding device_hotplug_lock is still advised +(to e.g. protect online_type) + +mem_hotplug_lock +--------------------- When adding/removing/onlining/offlining memory or adding/removing heterogeneous/device memory, we should always hold the mem_hotplug_lock in write mode to serialise memory hotplug (e.g. access to global/zone -variables). +variables). Currently, we take advantage of this to serialise sparsemem's +mem_section handling in sparse_add_one_section() and +sparse_remove_one_section(). In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read mode allows for a quite efficient get_online_mems/put_online_mems