From patchwork Thu May 23 15:34:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958069 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0D4B15A6 for ; Thu, 23 May 2019 15:34:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D60482869D for ; Thu, 23 May 2019 15:34:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA36C2869B; Thu, 23 May 2019 15:34:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9338628673 for ; Thu, 23 May 2019 15:34:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730899AbfEWPel (ORCPT ); Thu, 23 May 2019 11:34:41 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:45413 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730782AbfEWPek (ORCPT ); Thu, 23 May 2019 11:34:40 -0400 Received: by mail-qk1-f195.google.com with SMTP id j1so4038342qkk.12 for ; Thu, 23 May 2019 08:34:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gR5B+R57dzhfm++AjZVD0k9PFQ5ymVIe8oDIOybm+/Y=; b=K8z8HnLIYzj0CvIlrQpPXO2d+uOz8Pq+97N2YabABx1Vrr+qMaFLmh2VnVLx7a2oGq wOLg9W+DBwbkikfS0ff5K3MxYHAk3TSgoQYOvWZ/gmtirGdLKUC9DG9R8Ipu8mV0wg1O AVWPOOkR7fDNM3E0R1Ft5vDpgr37TJcvqhe3MhUEFNRKFUO4zvfCWS/qD4Zi6BZH+zi+ 2nHw0N8KVoBb1X1p7s0lsRt99Yq5hmwdLBLjHEaLE1oOXEBXSTOo4PBEa5CReklY8/7U mJVOX/VX4CF+TvyigLMWsvggRdPp9oUrqqQAiJtfOXeUBS65Sy7nS52+0lUgbQD2cN4+ 7aZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gR5B+R57dzhfm++AjZVD0k9PFQ5ymVIe8oDIOybm+/Y=; b=g/H2oOkZRt4A+/C7mA4gr564mpztCUY0cFPnUX8Sh/qL5C94AMaLAKQi3jN1SA0trO ARvxFpyOVB7AjbeXbbHxOMvJHDdhc/tazrzGyW/XcdYyp9SGQxkv9knTAbFbO5dwuTxE ox9aZkAqJFjpVQnVpDxBhh2KfR9bm2ZBpYvY5R3gJUzqTPesBHFEtIzB0T29UtuOlHlw SIyETnrnApV7kR4gy/3206mOVRw0O/R/1fVFmmBenPneH1PahaouPkJcXai85gOg5tqm zLfTLYyV0uk2v5S/KNLWI0m7KSiffF21y0vG3E95oGFX94YovBr6zuHGcrX/6EmPMAq5 sJmA== X-Gm-Message-State: APjAAAVecPG/csG21m59Ow1ZaoHr5uVLoXXkMU0griVWZLrkunXfPvox zT4TFEVWTtsL+A0zPh9hTePmRbI1uUE= X-Google-Smtp-Source: APXvYqx/CGFpFj+ej4XSzm0t6zCGOlTT7rmKjSiwJVztU6ABpwKrdUJowycnkB+WIld4Nb1SIXkSCw== X-Received: by 2002:a37:c445:: with SMTP id h5mr48174135qkm.105.1558625679185; Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id v126sm13212129qkh.86.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:38 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjp-0004zA-Sv; Thu, 23 May 2019 12:34:37 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 01/11] mm/hmm: Fix use after free with struct hmm in the mmu notifiers Date: Thu, 23 May 2019 12:34:26 -0300 Message-Id: <20190523153436.19102-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe mmu_notifier_unregister_no_release() is not a fence and the mmu_notifier system will continue to reference hmm->mn until the srcu grace period expires. Resulting in use after free races like this: CPU0 CPU1 __mmu_notifier_invalidate_range_start() srcu_read_lock hlist_for_each () // mn == hmm->mn hmm_mirror_unregister() hmm_put() hmm_free() mmu_notifier_unregister_no_release() hlist_del_init_rcu(hmm-mn->list) mn->ops->invalidate_range_start(mn, range); mm_get_hmm() mm->hmm = NULL; kfree(hmm) mutex_lock(&hmm->lock); Use SRCU to kfree the hmm memory so that the notifiers can rely on hmm existing. Get the now-safe hmm struct through container_of and directly check kref_get_unless_zero to lock it against free. Signed-off-by: Jason Gunthorpe Reviewed-by: Ira Weiny --- include/linux/hmm.h | 1 + mm/hmm.c | 25 +++++++++++++++++++------ 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 51ec27a8466816..8b91c90d3b88cb 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -102,6 +102,7 @@ struct hmm { struct mmu_notifier mmu_notifier; struct rw_semaphore mirrors_sem; wait_queue_head_t wq; + struct rcu_head rcu; long notifiers; bool dead; }; diff --git a/mm/hmm.c b/mm/hmm.c index 816c2356f2449f..824e7e160d8167 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -113,6 +113,11 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) return NULL; } +static void hmm_fee_rcu(struct rcu_head *rcu) +{ + kfree(container_of(rcu, struct hmm, rcu)); +} + static void hmm_free(struct kref *kref) { struct hmm *hmm = container_of(kref, struct hmm, kref); @@ -125,7 +130,7 @@ static void hmm_free(struct kref *kref) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); - kfree(hmm); + mmu_notifier_call_srcu(&hmm->rcu, hmm_fee_rcu); } static inline void hmm_put(struct hmm *hmm) @@ -153,10 +158,14 @@ void hmm_mm_destroy(struct mm_struct *mm) static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { - struct hmm *hmm = mm_get_hmm(mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; struct hmm_range *range; + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return; + /* Report this HMM as dying. */ hmm->dead = true; @@ -194,13 +203,15 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) static int hmm_invalidate_range_start(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { - struct hmm *hmm = mm_get_hmm(nrange->mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; struct hmm_update update; struct hmm_range *range; int ret = 0; - VM_BUG_ON(!hmm); + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return 0; update.start = nrange->start; update.end = nrange->end; @@ -248,9 +259,11 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, static void hmm_invalidate_range_end(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { - struct hmm *hmm = mm_get_hmm(nrange->mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); - VM_BUG_ON(!hmm); + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return; mutex_lock(&hmm->lock); hmm->notifiers--; From patchwork Thu May 23 15:34:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958073 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DDEA1395 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 298EE28688 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1DED1286C0; Thu, 23 May 2019 15:34:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0721C2851E for ; Thu, 23 May 2019 15:34:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730782AbfEWPel (ORCPT ); Thu, 23 May 2019 11:34:41 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:34488 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730752AbfEWPel (ORCPT ); Thu, 23 May 2019 11:34:41 -0400 Received: by mail-qk1-f194.google.com with SMTP id t64so3869067qkh.1 for ; Thu, 23 May 2019 08:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZePMKZ7jqBhopBdwVcCn1CrsRsr5VRg48k9A7BJXYb4=; b=jv6BZA8C1NV02Y1wmdXHZ90xqpN8v3U2iUJqYKuvZM/jGdctWUbv+nypqASupEpUuN vIJUJzSEQT+oiGmCOeMcLnf/zJZZJYkp9fMkQWdbVn6lbTE2PWKG+0XU8rQzC0Hq7Y+X r1gaLm6hZu3mcuSUXfzmFexbK/YKvGzQEV5NqwCdYopvQ50bAWFzwmX3JWwZuuR6KItI LqdMy9kYprQxYkWCbGlKs8lxcUTCSD7OTwxmzLrQKpMbz0TWlcMw3K9W6VInHK08n7Qd 2fajaOkWWQEZaJr0X7BQVoog4kSz1uBsRTCfvef4Gw8N0ebsa2yCNtTsdP4xqFS6YoQD xjuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZePMKZ7jqBhopBdwVcCn1CrsRsr5VRg48k9A7BJXYb4=; b=QfJuAOVWtuUDpgz5haLXG9T3BTp8/NaMlB3P+P6ZIKvjRgJYxArdSdkcPesh/n76sb El915RBW63AKw89OIol7QAEKDff7Yj3rgykABLxAG10cDeyzxpUMbt3y+W2QC6Ppfxkc osgLVAt9T4I49pcOqW8704goZo4lzSHSlI5Xs83Ihg/z+XTM4joNz5uIsuFeAiRBOmIK B5VMbn3y0MdYoXeKu9Ki8W0rJxisXLl3om+df+GYlGPcVRZwW+FziGVpAFK2OPtoGR96 K/WtosfwT0UXEqiuD1J5UYPkd70Fnkel+daAPIKtg0F8zZm9+1fKd98/0cQ9I9jXuKPf bhPA== X-Gm-Message-State: APjAAAXSFDdEooE4xf7gQxzADb3oWN9bk3VnK2KcfRhEPisKT624t8zC 41GyCUOHHlBsC9RPzZK8Nk5lgY9Gt7c= X-Google-Smtp-Source: APXvYqzE9Wf6pPLXAKTOOnViH6zjZXhhQeJF2uy5/0EwAhBMsAqbYZTx1jGzOFC4t22sUwexJeiqqQ== X-Received: by 2002:ae9:e208:: with SMTP id c8mr74383498qkc.154.1558625680068; Thu, 23 May 2019 08:34:40 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id x10sm2553300qkl.84.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:38 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjp-0004zH-Uv; Thu, 23 May 2019 12:34:37 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 02/11] mm/hmm: Use hmm_mirror not mm as an argument for hmm_register_range Date: Thu, 23 May 2019 12:34:27 -0300 Message-Id: <20190523153436.19102-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Ralph observes that hmm_register_range() can only be called by a driver while a mirror is registered. Make this clear in the API by passing in the mirror structure as a parameter. This also simplifies understanding the lifetime model for struct hmm, as the hmm pointer must be valid as part of a registered mirror so all we need in hmm_register_range() is a simple kref_get. Suggested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe --- include/linux/hmm.h | 7 ++++--- mm/hmm.c | 14 +++++--------- 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 8b91c90d3b88cb..87d29e085a69f7 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -503,7 +503,7 @@ static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror) * Please see Documentation/vm/hmm.rst for how to use the range API. */ int hmm_range_register(struct hmm_range *range, - struct mm_struct *mm, + struct hmm_mirror *mirror, unsigned long start, unsigned long end, unsigned page_shift); @@ -539,7 +539,8 @@ static inline bool hmm_vma_range_done(struct hmm_range *range) } /* This is a temporary helper to avoid merge conflict between trees. */ -static inline int hmm_vma_fault(struct hmm_range *range, bool block) +static inline int hmm_vma_fault(struct hmm_mirror *mirror, + struct hmm_range *range, bool block) { long ret; @@ -552,7 +553,7 @@ static inline int hmm_vma_fault(struct hmm_range *range, bool block) range->default_flags = 0; range->pfn_flags_mask = -1UL; - ret = hmm_range_register(range, range->vma->vm_mm, + ret = hmm_range_register(range, mirror, range->start, range->end, PAGE_SHIFT); if (ret) diff --git a/mm/hmm.c b/mm/hmm.c index 824e7e160d8167..fa1b04fcfc2549 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -927,7 +927,7 @@ static void hmm_pfns_clear(struct hmm_range *range, * Track updates to the CPU page table see include/linux/hmm.h */ int hmm_range_register(struct hmm_range *range, - struct mm_struct *mm, + struct hmm_mirror *mirror, unsigned long start, unsigned long end, unsigned page_shift) @@ -935,7 +935,6 @@ int hmm_range_register(struct hmm_range *range, unsigned long mask = ((1UL << page_shift) - 1UL); range->valid = false; - range->hmm = NULL; if ((start & mask) || (end & mask)) return -EINVAL; @@ -946,15 +945,12 @@ int hmm_range_register(struct hmm_range *range, range->start = start; range->end = end; - range->hmm = hmm_get_or_create(mm); - if (!range->hmm) - return -EFAULT; - /* Check if hmm_mm_destroy() was call. */ - if (range->hmm->mm == NULL || range->hmm->dead) { - hmm_put(range->hmm); + if (mirror->hmm->mm == NULL || mirror->hmm->dead) return -EFAULT; - } + + range->hmm = mirror->hmm; + kref_get(&range->hmm->kref); /* Initialize range to track CPU page table update */ mutex_lock(&range->hmm->lock); From patchwork Thu May 23 15:34:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958077 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E398017E0 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF343285D1 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD7FA286A0; Thu, 23 May 2019 15:34:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 85A4128682 for ; Thu, 23 May 2019 15:34:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731045AbfEWPem (ORCPT ); Thu, 23 May 2019 11:34:42 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:46711 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731070AbfEWPel (ORCPT ); Thu, 23 May 2019 11:34:41 -0400 Received: by mail-qt1-f194.google.com with SMTP id z19so4370497qtz.13 for ; Thu, 23 May 2019 08:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o0fnCIS1UU2Yl5HFhaP9USXWVXi3ioUfilGqBeg9WXo=; b=ZAWb6C2Zy9X1DesFJzdUY1NrKsPiS5Jvp1PCkg+zTa3+HZUKgu72kwwZR9Zz7cuUsN hqHmPHEl1SZqi6Ksxs8IWDmWa8NYcaq27MZg4o43q9cS1rMdNIwjOHXVvrNrUhmerPLH 2C4RM0d83niUVSA60rDZxLiCaCg0XIRLCrxlMxIAkpDCL2qGgOJrwg93mDBo0owbS9wG GigRKCJfgXFzftD8yNj5WhTzw0oUfA/5axgHUIGjENJUBHkLPDeOC2pI2f/yYxuyr07Y 3SKahFyaohSEsYHGIuLk/ql8MKS182Eez2RiWCFyOULJUSZK8ElKWKLASN/yeR0Yf0fy jaIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o0fnCIS1UU2Yl5HFhaP9USXWVXi3ioUfilGqBeg9WXo=; b=NuTquW/wbr3Q8cVy8/CyIMJZmaUnaghnqJBkgR0VYjeo1ux8bDUOtMQ4R0l+VcIhzk AulH8vwBf/y0betVBoJliGUOuzQpozeRz8oSgumA6UAdu2Fqm4+7lvIfq+USEQkSnI/C On3QfiyBDNryeG/hJjMYsQEVrZA0u6ZGaBL8JfbWtcDX5ml3gzW7X04hhRXYloOd6AVd /xAChal3FOhUctqhmxlNH4yVMbXYsuxh5xEVxZrwvAEJ583mixIsrk78Jef9j2SUyo+l Upt4pm1OW/eFyexsvAjLL0vNOAHedyMv55WsYwJA1AOeyhA5gT3ec4gh5RFdKPa4yrt5 WUvw== X-Gm-Message-State: APjAAAU8XHkXEcU4BQlXdVM26dlkxTHMwm+Mn4uLcBxiNGnaL8Pa5/oT rr9hX/ntO1UYU8EgNhKL8j7VFtKLoQc= X-Google-Smtp-Source: APXvYqyWStElkAeFxeRHIt8qw7jwn+e9hRhuM1POWa1lozD72Ww5AlAsfY/GROw/QPhwUqv3LHvK6A== X-Received: by 2002:a0c:8b6f:: with SMTP id d47mr30646505qvc.32.1558625680542; Thu, 23 May 2019 08:34:40 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id r47sm18148670qtc.14.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjp-0004zN-WB; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 03/11] mm/hmm: Hold a mmgrab from hmm to mm Date: Thu, 23 May 2019 12:34:28 -0300 Message-Id: <20190523153436.19102-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe So long a a struct hmm pointer exists, so should the struct mm it is linked too. Hold the mmgrab() as soon as a hmm is created, and mmdrop() it once the hmm refcount goes to zero. Since mmdrop() (ie a 0 kref on struct mm) is now impossible with a !NULL mm->hmm delete the hmm_hmm_destroy(). Signed-off-by: Jason Gunthorpe --- include/linux/hmm.h | 3 --- kernel/fork.c | 1 - mm/hmm.c | 21 +++------------------ 3 files changed, 3 insertions(+), 22 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 87d29e085a69f7..2a7346384ead13 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -584,14 +584,11 @@ static inline int hmm_vma_fault(struct hmm_mirror *mirror, } /* Below are for HMM internal use only! Not to be used by device driver! */ -void hmm_mm_destroy(struct mm_struct *mm); - static inline void hmm_mm_init(struct mm_struct *mm) { mm->hmm = NULL; } #else /* IS_ENABLED(CONFIG_HMM_MIRROR) */ -static inline void hmm_mm_destroy(struct mm_struct *mm) {} static inline void hmm_mm_init(struct mm_struct *mm) {} #endif /* IS_ENABLED(CONFIG_HMM_MIRROR) */ diff --git a/kernel/fork.c b/kernel/fork.c index b4cba953040a0f..51b114ec6c395c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -672,7 +672,6 @@ void __mmdrop(struct mm_struct *mm) WARN_ON_ONCE(mm == current->active_mm); mm_free_pgd(mm); destroy_context(mm); - hmm_mm_destroy(mm); mmu_notifier_mm_destroy(mm); check_mm(mm); put_user_ns(mm->user_ns); diff --git a/mm/hmm.c b/mm/hmm.c index fa1b04fcfc2549..e27058e92508b9 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -82,6 +83,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) hmm->notifiers = 0; hmm->dead = false; hmm->mm = mm; + mmgrab(hmm->mm); spin_lock(&mm->page_table_lock); if (!mm->hmm) @@ -130,6 +132,7 @@ static void hmm_free(struct kref *kref) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); + mmdrop(hmm->mm); mmu_notifier_call_srcu(&hmm->rcu, hmm_fee_rcu); } @@ -138,24 +141,6 @@ static inline void hmm_put(struct hmm *hmm) kref_put(&hmm->kref, hmm_free); } -void hmm_mm_destroy(struct mm_struct *mm) -{ - struct hmm *hmm; - - spin_lock(&mm->page_table_lock); - hmm = mm_get_hmm(mm); - mm->hmm = NULL; - if (hmm) { - hmm->mm = NULL; - hmm->dead = true; - spin_unlock(&mm->page_table_lock); - hmm_put(hmm); - return; - } - - spin_unlock(&mm->page_table_lock); -} - static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); From patchwork Thu May 23 15:34:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958075 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC95715A6 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4C2F286A1 for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1B7E285D1; Thu, 23 May 2019 15:34:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38BBA2851E for ; Thu, 23 May 2019 15:34:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731082AbfEWPem (ORCPT ); Thu, 23 May 2019 11:34:42 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:36909 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730752AbfEWPem (ORCPT ); Thu, 23 May 2019 11:34:42 -0400 Received: by mail-qt1-f196.google.com with SMTP id o7so7238417qtp.4 for ; Thu, 23 May 2019 08:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1UYOs1b1m4kqMa2Hcebr2aPMxfP9lZp6i0AyhspLKIc=; b=LV2mKDWl6z86EAVgXFrAKp9hIarl1v5CuBTZz0h3rmucixSlFycPZyRubUjCdCgxLA AtK7gRpiNk4P4k7v2McacqOVxWDzHDj1SIjoKD7Ak5PnFKzLI5rh/G1hioQExJlxX/qo 1aRAcCTHLzYgrrxhfgpVo6ETGueE0agyf8qN0BMT7WPG19ZYlmnMrra6KUdEXX3cEkGG BDEIIdpCyEyvvS0QDao0w2v4CajWdmWOKQIXYRhlqugAFNyeY4ReYthg9OXsCqyFabSi X0vD7hp+cgUvC8GCSciZF5t9Od7M9Jbnz3qcwMPWB1g/F6XF9aS65QgPOkraL59krHZQ OSog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1UYOs1b1m4kqMa2Hcebr2aPMxfP9lZp6i0AyhspLKIc=; b=EbvAJRXEq8C3RkBnBpU+oLuKIe//FOY7fXP6tDNcrxg26c/bEelsbiE1YzF1NGEePW Jg1KJz0m9mTVB8cAgEuLR0ggZ5BCRBDXvc3wB2vb522nJRtwA6lk16uDs+aVnS3evJca hVqtzewubiQm+wW2I0dxKaoksmsi7mV6eHIbd9tyRPN8zr2rYvGEZLl1tSz5o3Fovxk2 GXIXKRNDAl3CBeZdl9A6ZLeAPScimq60ZZocZLgrnzOsEYInnO2Am2NmA4jtAs0D6XJg dftuWLtvV/63hes4RAbUmx021tNLLQj7ebz2prhe2ChHi9hwjqo0NEH04dSeZ1e8KQFG pEew== X-Gm-Message-State: APjAAAVDP1CxmeYAJxIlSqfP+hFg5UBkZ3Qa4z9ykFSJ1F89AVa+QG6L mBhGtRDA8YhEC9KfGZ1EPTt79eoZdKQ= X-Google-Smtp-Source: APXvYqx4TikclVwMPUpuo7NFKA6Xd+Jq88ZLLLLwtvyEZvxo6rVnUXsWNS36lZk+68Gag43lw4B0JA== X-Received: by 2002:a0c:ad85:: with SMTP id w5mr8251178qvc.242.1558625680953; Thu, 23 May 2019 08:34:40 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id k53sm13877244qtb.65.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zT-1F; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 04/11] mm/hmm: Simplify hmm_get_or_create and make it reliable Date: Thu, 23 May 2019 12:34:29 -0300 Message-Id: <20190523153436.19102-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe As coded this function can false-fail in various racy situations. Make it reliable by running only under the write side of the mmap_sem and avoiding the false-failing compare/exchange pattern. Also make the locking very easy to understand by only ever reading or writing mm->hmm while holding the write side of the mmap_sem. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 75 ++++++++++++++++++++------------------------------------ 1 file changed, 27 insertions(+), 48 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index e27058e92508b9..ec54be54d81135 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -40,16 +40,6 @@ #if IS_ENABLED(CONFIG_HMM_MIRROR) static const struct mmu_notifier_ops hmm_mmu_notifier_ops; -static inline struct hmm *mm_get_hmm(struct mm_struct *mm) -{ - struct hmm *hmm = READ_ONCE(mm->hmm); - - if (hmm && kref_get_unless_zero(&hmm->kref)) - return hmm; - - return NULL; -} - /** * hmm_get_or_create - register HMM against an mm (HMM internal) * @@ -64,11 +54,20 @@ static inline struct hmm *mm_get_hmm(struct mm_struct *mm) */ static struct hmm *hmm_get_or_create(struct mm_struct *mm) { - struct hmm *hmm = mm_get_hmm(mm); - bool cleanup = false; + struct hmm *hmm; - if (hmm) - return hmm; + lockdep_assert_held_exclusive(mm->mmap_sem); + + if (mm->hmm) { + if (kref_get_unless_zero(&mm->hmm->kref)) + return mm->hmm; + /* + * The hmm is being freed by some other CPU and is pending a + * RCU grace period, but this CPU can NULL now it since we + * have the mmap_sem. + */ + mm->hmm = NULL; + } hmm = kmalloc(sizeof(*hmm), GFP_KERNEL); if (!hmm) @@ -85,54 +84,34 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) hmm->mm = mm; mmgrab(hmm->mm); - spin_lock(&mm->page_table_lock); - if (!mm->hmm) - mm->hmm = hmm; - else - cleanup = true; - spin_unlock(&mm->page_table_lock); - - if (cleanup) - goto error; - - /* - * We should only get here if hold the mmap_sem in write mode ie on - * registration of first mirror through hmm_mirror_register() - */ hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; - if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) - goto error_mm; + if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) { + kfree(hmm); + return NULL; + } + mm->hmm = hmm; return hmm; - -error_mm: - spin_lock(&mm->page_table_lock); - if (mm->hmm == hmm) - mm->hmm = NULL; - spin_unlock(&mm->page_table_lock); -error: - kfree(hmm); - return NULL; } static void hmm_fee_rcu(struct rcu_head *rcu) { + struct hmm *hmm = container_of(rcu, struct hmm, rcu); + + down_write(&hmm->mm->mmap_sem); + if (hmm->mm->hmm == hmm) + hmm->mm->hmm = NULL; + up_write(&hmm->mm->mmap_sem); + mmdrop(hmm->mm); + kfree(container_of(rcu, struct hmm, rcu)); } static void hmm_free(struct kref *kref) { struct hmm *hmm = container_of(kref, struct hmm, kref); - struct mm_struct *mm = hmm->mm; - - mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); - spin_lock(&mm->page_table_lock); - if (mm->hmm == hmm) - mm->hmm = NULL; - spin_unlock(&mm->page_table_lock); - - mmdrop(hmm->mm); + mmu_notifier_unregister_no_release(&hmm->mmu_notifier, hmm->mm); mmu_notifier_call_srcu(&hmm->rcu, hmm_fee_rcu); } From patchwork Thu May 23 15:34:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A065E15A6 for ; Thu, 23 May 2019 15:34:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C379285D1 for ; Thu, 23 May 2019 15:34:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80E1C28686; Thu, 23 May 2019 15:34:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2258C285D1 for ; Thu, 23 May 2019 15:34:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730752AbfEWPen (ORCPT ); Thu, 23 May 2019 11:34:43 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:39127 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731070AbfEWPem (ORCPT ); Thu, 23 May 2019 11:34:42 -0400 Received: by mail-qk1-f193.google.com with SMTP id i125so1942378qkd.6 for ; Thu, 23 May 2019 08:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8d3qZxuduenTYxmVtLo/HTiLB+fwW/lCyyNPj/qu33U=; b=hQux6/iJ6hqg+JBrYX1VSUyG5Ws/3mnhZg2BHMv3L6kyfjzf8PCRUOnHPNnDo5Ei52 EzmG7qHzV3jffzCQXKj0JUoEfaIP43zLaY5Wr6VzQ8nmDtD5NmA7j9TfXODD1YOs1F9H PZH2Za61bIL+znufcpgTJCwAGDEtZBaDT+uGjIxs4vHGmpuGCYXgRA67X3+xj2YEGoS7 qxppgbz5Ak0ZlKqhXrbw3bOcAA4n662B0aWl2e6V6tWw4A7HXJaDEV3RGWl1ZbpgHojB dZ3Mex3pGWIaqI5HTw4cpQ7GxfHh8wiS7DQizBjbqw30fLSk5qlGsvcbGE4opOCxHQ2o yokw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8d3qZxuduenTYxmVtLo/HTiLB+fwW/lCyyNPj/qu33U=; b=FSfFjRWtHAXXjSTk52l0TFZ8WG/gTu4vx0G6QCUHdpvlAzBNkAceNwqH8M5Diztt1W E2BlC6THelBh3yQ+7xdKY3YvP9cCXVk11QK8wpgLLqyNx7GBF6V5/o2UGjQFEKHkpCcr G33oX7VU+l82BjKVogC93Xv5lkHI7SpjfXpYP7umxDgA2Jp0eK2pxEXGgOjYGl2z8mAq s/yqOpLdBbVv3psiA+enIN4zcdSaKYPkgApxFbMLJNWPRnp1JXdRmj+cjlxjiMSsCmMR 6AqBC0aO6rtxR4RuEbYV1PCdUdvMXDGotw0fabDPWgGHmYhmNL2HLSYKA24l5H5CkXBD 7dtA== X-Gm-Message-State: APjAAAVSYyclcszt6yknjAVSwp5mbrjuFXMaiYVsCkQFYh3EK4X9z+jt Epi/BX/vVDkEcv7pQ6CE71FoBG3rkG0= X-Google-Smtp-Source: APXvYqxEddBfU0Rf4mgWBf/qIenpeQFY2LV64soWSgPl9ghoFmgdVdxNL+eh6sZGq9t3KI14a4o+gg== X-Received: by 2002:ae9:f70d:: with SMTP id s13mr75933217qkg.213.1558625681585; Thu, 23 May 2019 08:34:41 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id v69sm745374qkb.60.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zZ-2a; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 05/11] mm/hmm: Improve locking around hmm->dead Date: Thu, 23 May 2019 12:34:30 -0300 Message-Id: <20190523153436.19102-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This value is being read without any locking, so it is just an unreliable hint, however in many cases we need to have certainty that code is not racing with mmput()/hmm_release(). For the two functions doing find_vma(), document that the caller is expected to hold mmap_sem and thus also have a mmget(). For hmm_range_register acquire a mmget internally as it must not race with hmm_release() when it sets valid. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index ec54be54d81135..d97ec293336ea5 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -909,8 +909,10 @@ int hmm_range_register(struct hmm_range *range, range->start = start; range->end = end; - /* Check if hmm_mm_destroy() was call. */ - if (mirror->hmm->mm == NULL || mirror->hmm->dead) + /* + * We cannot set range->value to true if hmm_release has already run. + */ + if (!mmget_not_zero(mirror->hmm->mm)) return -EFAULT; range->hmm = mirror->hmm; @@ -928,6 +930,7 @@ int hmm_range_register(struct hmm_range *range, if (!range->hmm->notifiers) range->valid = true; mutex_unlock(&range->hmm->lock); + mmput(mirror->hmm->mm); return 0; } @@ -979,9 +982,13 @@ long hmm_range_snapshot(struct hmm_range *range) struct vm_area_struct *vma; struct mm_walk mm_walk; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) - return -EFAULT; + /* + * Caller must hold the mmap_sem, and that requires the caller to have + * a mmget. + */ + lockdep_assert_held(hmm->mm->mmap_sem); + if (WARN_ON(!atomic_read(&hmm->mm->mm_users))) + return -EINVAL; do { /* If range is no longer valid force retry. */ @@ -1077,9 +1084,13 @@ long hmm_range_fault(struct hmm_range *range, bool block) struct mm_walk mm_walk; int ret; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) - return -EFAULT; + /* + * Caller must hold the mmap_sem, and that requires the caller to have + * a mmget. + */ + lockdep_assert_held(hmm->mm->mmap_sem); + if (WARN_ON(!atomic_read(&hmm->mm->mm_users))) + return -EINVAL; do { /* If range is no longer valid force retry. */ From patchwork Thu May 23 15:34:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958083 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0BC9017E0 for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E952B2851E for ; Thu, 23 May 2019 15:34:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5986286AA; Thu, 23 May 2019 15:34:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61A652869B for ; Thu, 23 May 2019 15:34:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731070AbfEWPen (ORCPT ); Thu, 23 May 2019 11:34:43 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:43822 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731048AbfEWPen (ORCPT ); Thu, 23 May 2019 11:34:43 -0400 Received: by mail-qt1-f195.google.com with SMTP id g17so1163400qtq.10 for ; Thu, 23 May 2019 08:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DCPTNx8CjmJ5EZR4Q7fEVnlEGDk6XaLfhfnTJM/xZQM=; b=UscylUyC/LbE+hmwHcoTPty0rCYV8C9l6BH2lAPjaiQ8KCpQHYGVFqbwt9qHDAXr8/ 0k3qTMBhSJf3S42PRZfbrX9UEI/6LGFwoDjvFpmqQlcDDdNyD6dTDJrchuLLNc489jB7 Y/rZmj+WB81MnhoL6YnHiAQeUu1bwtK84oY4YfOfklhLOiyLAZUJPzOkPy0cRE3mZC87 G/yVQEsPneUO8B683OG8mQMLHQ79F1URNwp0dMjSHeuCd7ZWZGchFIcBB3L628nY5y71 wweFi2dQmZAt9lY8+eQpEay8IdpvBecKhqsuVKkK81+YslCO9og21vt7HuyZ/YQvqfFx 2JDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DCPTNx8CjmJ5EZR4Q7fEVnlEGDk6XaLfhfnTJM/xZQM=; b=Mbd6sv1/2gi3YKYf+SBuLw6zvpvlezVVfnnHHhFh27ffumfGFHJMeubOQAMOaci63/ bwCD1LCnkn032U10jeTKfamgoInjguKurGjOFHAJ6N3AP3XMbU5Oqh+xxwXGKMkq8YQF zQOskxUyMiQz+NEbG9eY/E379KmazZAEYp0KCvAbDNhr0uHQuI9LBkymM5VsMSzNidKx ukt3wiiUEjbDW3IOJQCh6Bs31OJ/7wj4KyvPest03qwtz5TUgoUE/+d5tYdGT3HwTl1m Ngn1dr4quNQe9FP8C4Ogx1+36wYrTor6ArR9qRQMJoFoWjDctdwNSxMgeodTtFA5j+VU sbUg== X-Gm-Message-State: APjAAAUvNMEMrzT/0as3v5Uv23EMQJ2qAa5Cm0lTdrlxKkyc76feCV5b aMuOkKoDh7NP5J3d03KdKlZg6b10LPg= X-Google-Smtp-Source: APXvYqxlKr/De7Ph6kX9Z/iOmfGCPd6u0UkBZ6roajbjLbPCO1l9qWnchzPbSB+6YXzx7C5cfiG9rg== X-Received: by 2002:ac8:28ac:: with SMTP id i41mr57963862qti.388.1558625682235; Thu, 23 May 2019 08:34:42 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id a51sm17403701qta.85.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zf-3t; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 06/11] mm/hmm: Remove duplicate condition test before wait_event_timeout Date: Thu, 23 May 2019 12:34:31 -0300 Message-Id: <20190523153436.19102-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The wait_event_timeout macro already tests the condition as its first action, so there is no reason to open code another version of this, all that does is skip the might_sleep() debugging in common cases, which is not helpful. Further, based on prior patches, we can no simplify the required condition test: - If range is valid memory then so is range->hmm - If hmm_release() has run then range->valid is set to false at the same time as dead, so no reason to check both. - A valid hmm has a valid hmm->mm. Also, add the READ_ONCE for range->valid as there is no lock held here. Signed-off-by: Jason Gunthorpe --- include/linux/hmm.h | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 2a7346384ead13..7f3b751fcab1ce 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -216,17 +216,9 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range) static inline bool hmm_range_wait_until_valid(struct hmm_range *range, unsigned long timeout) { - /* Check if mm is dead ? */ - if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) { - range->valid = false; - return false; - } - if (range->valid) - return true; - wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead, + wait_event_timeout(range->hmm->wq, range->valid, msecs_to_jiffies(timeout)); - /* Return current valid status just in case we get lucky */ - return range->valid; + return READ_ONCE(range->valid); } /* From patchwork Thu May 23 15:34:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EB1215A6 for ; Thu, 23 May 2019 15:34:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A2F11FF66 for ; Thu, 23 May 2019 15:34:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E9E928688; Thu, 23 May 2019 15:34:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16CA21FF66 for ; Thu, 23 May 2019 15:34:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731075AbfEWPeq (ORCPT ); Thu, 23 May 2019 11:34:46 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:36789 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731104AbfEWPep (ORCPT ); Thu, 23 May 2019 11:34:45 -0400 Received: by mail-qt1-f193.google.com with SMTP id a17so7250663qth.3 for ; Thu, 23 May 2019 08:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ncjASLGlHsj22sNocZs8CG3omWgokVRQnVtcSgoF4Do=; b=afbPPYk9gBj4SFUfupZZE4bCnA6gDZMhgy4+OgszgUI8N1f+l4L4QbGabEscrkzZio 04dKwEZTY1ZZpMvvyuXHs/Zur4eepQiD1W+lZazrUaj4nn7/t4j1M+InHHRzuagZXcr8 KKv520d1DP6I7X4/cfCmplGSWFB7qHZa/I9bvrzKq1ogoyjdw+q2JJfFWS8aPCq+8IpA qzget2M+jxJ0rPEHJZXgXf8cijcYlCmoGK6a41MkyHSp5f24D9rssOEcYPsQesCbV+n0 ERLiPP7EiRneYQ5/ivup+VNYTXqW/BHKDFa3+vKYdV9zuvfZzb8dXYO7o2N+FXeqLQBm RkYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ncjASLGlHsj22sNocZs8CG3omWgokVRQnVtcSgoF4Do=; b=Zy5luT8jvUUPBbYIlwoqKoECCIyQFmeD+C9BqdMF5B8gjdiZsKiYA9Zcvd969PiBfF drMHk/phPeMWsjpqPma/3aLi/KCx3X6kFZg9kNM6PXBgTE5jgz0WPK9EUrZXea15msC6 jFk/yZW9W0fQdiRCYZui5AGe1iWPZVyZtlnXMWi0G64Q4cptBa7z/82ujIsP3PQTS6C+ C2Z6d1kv+qyPtYgc7LzRixrJd9GNIOnW0yBNyGNJqUcPKUq3Jtr8WDTvC6TLUXKs6yi9 FpjmpEV8WTCuoAcZm11NFdl/AhUsxVZVn/x6+962H+R1lGzyW2IEnWTNcfnF8GqZjqFJ xxhg== X-Gm-Message-State: APjAAAVHhrsX8izTPd1vJiMfTimO557UXpSZfKQ/hJx+O07HvA6l1EhS EGbXLrSyRix6/P/88/E9bKTv7uibFn0= X-Google-Smtp-Source: APXvYqxAz8IO0yDJzMp5j01V0l+mHdFZgyUtfceu0FECv0lekCVZ9tSBPQKcxg2vezV1TSB8NGjW9g== X-Received: by 2002:ac8:18b8:: with SMTP id s53mr76217225qtj.232.1558625684721; Thu, 23 May 2019 08:34:44 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id g65sm2686228qkb.1.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zl-5F; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 07/11] mm/hmm: Delete hmm_mirror_mm_is_alive() Date: Thu, 23 May 2019 12:34:32 -0300 Message-Id: <20190523153436.19102-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Now that it is clarified that callers to hmm_range_dma_map() must hold the mmap_sem and thus the mmget, there is no purpose for this function. It was the last user of dead, so delete it as well. Signed-off-by: Jason Gunthorpe --- include/linux/hmm.h | 27 --------------------------- mm/hmm.c | 4 ---- 2 files changed, 31 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 7f3b751fcab1ce..6671643703a7ab 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -91,7 +91,6 @@ * @mirrors_sem: read/write semaphore protecting the mirrors list * @wq: wait queue for user waiting on a range invalidation * @notifiers: count of active mmu notifiers - * @dead: is the mm dead ? */ struct hmm { struct mm_struct *mm; @@ -104,7 +103,6 @@ struct hmm { wait_queue_head_t wq; struct rcu_head rcu; long notifiers; - bool dead; }; /* @@ -466,31 +464,6 @@ struct hmm_mirror { int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm); void hmm_mirror_unregister(struct hmm_mirror *mirror); -/* - * hmm_mirror_mm_is_alive() - test if mm is still alive - * @mirror: the HMM mm mirror for which we want to lock the mmap_sem - * Returns: false if the mm is dead, true otherwise - * - * This is an optimization it will not accurately always return -EINVAL if the - * mm is dead ie there can be false negative (process is being kill but HMM is - * not yet inform of that). It is only intented to be use to optimize out case - * where driver is about to do something time consuming and it would be better - * to skip it if the mm is dead. - */ -static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror) -{ - struct mm_struct *mm; - - if (!mirror || !mirror->hmm) - return false; - mm = READ_ONCE(mirror->hmm->mm); - if (mirror->hmm->dead || !mm) - return false; - - return true; -} - - /* * Please see Documentation/vm/hmm.rst for how to use the range API. */ diff --git a/mm/hmm.c b/mm/hmm.c index d97ec293336ea5..2695925c0c5927 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -80,7 +80,6 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) mutex_init(&hmm->lock); kref_init(&hmm->kref); hmm->notifiers = 0; - hmm->dead = false; hmm->mm = mm; mmgrab(hmm->mm); @@ -130,9 +129,6 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) if (!kref_get_unless_zero(&hmm->kref)) return; - /* Report this HMM as dying. */ - hmm->dead = true; - /* Wake-up everyone waiting on any range. */ mutex_lock(&hmm->lock); list_for_each_entry(range, &hmm->ranges, list) { From patchwork Thu May 23 15:34:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D7CA76 for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69264286AE for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D9A4286A0; Thu, 23 May 2019 15:34:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 146FE2851E for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731093AbfEWPeo (ORCPT ); Thu, 23 May 2019 11:34:44 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:45998 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731092AbfEWPeo (ORCPT ); Thu, 23 May 2019 11:34:44 -0400 Received: by mail-qt1-f196.google.com with SMTP id t1so7214117qtc.12 for ; Thu, 23 May 2019 08:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f4j069c7ZUzoxaaL4dTtRuO7XDhSw6R18jQGsXmwSSU=; b=Q1iR9o21bGF01UbFLx0luDqo+LhUazJlW9vM4h0+z3TY9W6sEyxEw1R8DRsXKzCFjm fcxDLu6fBdQ7+NFUWl/isFjecvb1MU2/EYhsJDaOAM7XH95HyJYuhAmdg/ngedBPOT2f A/UL3DDA9lTnFoM9t+t3e2u8hSjSbbYi6FkF8FVBMPrnALhwH2m8B9e+n4RvWRsmYIpR VWSYlsfP2MkG7ZigmeiUQ0n1/bZgKIlqHLtsxqRXpvC7SLAUBoMfzAuE7zfXaTtKTa8A VGeDoQ8pTFRVKqryBcVNxJtEZ8Cn0TIIINgbBg2xyE45bsPeuNCxS8mtks9fFUZS1sdG oPlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f4j069c7ZUzoxaaL4dTtRuO7XDhSw6R18jQGsXmwSSU=; b=PELF6RDPt3EzTD7yJT5gDuTH/KPXg0RTBAlGEm3dUenW5lf0IXllt1WB20OlYDZLUJ a1PV2CawhWLUvxEtSKM13bNUs1KPPvCMGqSjv27Cvy/pukWdLfT3PpDrPrAvMqoQDc59 HAC+Jl3nk0eyK3U86o5FEdIXLRVF0VKgH5chM8QCMCnMGMOSfAYru/qjBNLS0Ngk1zM2 mpSV7Iu5JYTEpkhKsUDv9UlfYIkGnG5hFfxdEdNo1Z+U54wfscCbTYEmkQRrulfJQeIQ x9jq3DqciJQXOnWR1p58CPRkh6YSqRqMudhiu/WBhtHzkdnbt97e9Y6DmtS+1OI52Lsb gHzg== X-Gm-Message-State: APjAAAWO0lMuX/WItHFMgSlivMEQiZh9lomcpg5Qot+dIWHmvusRWgNn B5s2/BTphQctLeOTj2QK/SUN8Ja7aDE= X-Google-Smtp-Source: APXvYqzldSamHhkoEu6IIunxCJ747qr3gst7TXBS5fLpoamgpQBiEE2VbTR+wtaPARW5aq21amozzQ== X-Received: by 2002:aed:3a0a:: with SMTP id n10mr82588942qte.145.1558625683002; Thu, 23 May 2019 08:34:43 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id p8sm15951242qta.24.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zr-6W; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 08/11] mm/hmm: Use lockdep instead of comments Date: Thu, 23 May 2019 12:34:33 -0300 Message-Id: <20190523153436.19102-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe So we can check locking at runtime. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 2695925c0c5927..46872306f922bb 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -256,11 +256,11 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops = { * * To start mirroring a process address space, the device driver must register * an HMM mirror struct. - * - * THE mm->mmap_sem MUST BE HELD IN WRITE MODE ! */ int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm) { + lockdep_assert_held_exclusive(mm->mmap_sem); + /* Sanity check */ if (!mm || !mirror || !mirror->ops) return -EINVAL; From patchwork Thu May 23 15:34:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958087 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CF6715A6 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 08BF528478 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE15F286A1; Thu, 23 May 2019 15:34:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B8E128682 for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731092AbfEWPeo (ORCPT ); Thu, 23 May 2019 11:34:44 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:35229 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731048AbfEWPeo (ORCPT ); Thu, 23 May 2019 11:34:44 -0400 Received: by mail-qk1-f195.google.com with SMTP id c15so4077391qkl.2 for ; Thu, 23 May 2019 08:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xpfktVbL8Us0qmdWRL9VRFwkpcEhMFdO2zVbp5WooYk=; b=AQy9qY5unh9pvUOhg9td8t1dNvtTOZFdIPbHYbv6sGctT7lxBlvtaDEwnhn4A+19Ea Rg7FPBiK+QeEfNuYYey9/yTJuLOz1p93njDBTGg1o1kwmziRbYbc7/r1RV9voanVidls 30IOKaMwGJFM53rpfO+9jyUNcc5sGg2amlLg17lxpwhDbVQBeZP7J9K2e+vswi8oBpRV I+kktxprDGfdJdLVSa1IlFeSv+iaXJXGJ2txMn51i04T0ADr1iIC1/SdYT+sTveLOLmP 1UTrdvaCWyzmYz7MafIRNnc5+Gu9nQKBqUn+NLtIyiTvORUJoBnsXlq+HNOLcEnckf0g Vxiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xpfktVbL8Us0qmdWRL9VRFwkpcEhMFdO2zVbp5WooYk=; b=G8XrGSU9mfxI07uuTJsr1bY1AvJfaiuRQEFZQzr0+tqMaye2DGCAUDb5K/Sqk4cY05 Qm1cHUcK9b2aEN4NqolMly+etNJFJktEkxUkXyXnRCq1+pXSWS/F+oQ+Gq0EKwgl7xJl 9Z9VnZlqxFJQx9k5vwMfV46sFCsMIUtt1DbbqE2MYuH0N7J2R3MMJdTtowDL2Iodb8QC F2bJelQGoA4LXpZx5eqXdhidxQsdIG/u/0vPGyVwNefuFFQr6pfOAcOMSXMuL34fWtZ4 oy3FFPOvHVupvSMu5BFl1KzKW+zLGyKN1pD+A+OR6DmkEwP6TFNrzCexVNJZL1cP4Q0S M8eA== X-Gm-Message-State: APjAAAW8puKs/isyQl+0uWRnVd4rnRpKKia0uoHCFPoZar/NbZmHSBJN 5ad9qjZ0iuU5M3AjF30EtKFx4I075xM= X-Google-Smtp-Source: APXvYqx9y4W6qkw9q6CFNTSxrDtAaedtA4a51WwLa3lqT4HhvTbeWF2PV7OrPjgqqyxZwLXJZH3cDQ== X-Received: by 2002:a37:50d4:: with SMTP id e203mr18097553qkb.83.1558625683357; Thu, 23 May 2019 08:34:43 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id k30sm11757172qte.49.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-0004zx-7m; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 09/11] mm/hmm: Remove racy protection against double-unregistration Date: Thu, 23 May 2019 12:34:34 -0300 Message-Id: <20190523153436.19102-10-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe No other register/unregister kernel API attempts to provide this kind of protection as it is inherently racy, so just drop it. Callers should provide their own protection, it appears nouveau already does, but just in case drop a debugging POISON. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 46872306f922bb..6c3b7398672c29 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -286,18 +286,13 @@ EXPORT_SYMBOL(hmm_mirror_register); */ void hmm_mirror_unregister(struct hmm_mirror *mirror) { - struct hmm *hmm = READ_ONCE(mirror->hmm); - - if (hmm == NULL) - return; + struct hmm *hmm = mirror->hmm; down_write(&hmm->mirrors_sem); list_del_init(&mirror->list); - /* To protect us against double unregister ... */ - mirror->hmm = NULL; up_write(&hmm->mirrors_sem); - hmm_put(hmm); + memset(&mirror->hmm, POISON_INUSE, sizeof(mirror->hmm)); } EXPORT_SYMBOL(hmm_mirror_unregister); From patchwork Thu May 23 15:34:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958091 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DAEF915A6 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6F8E1FF66 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BAF0D28688; Thu, 23 May 2019 15:34:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A95E1FF66 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731089AbfEWPep (ORCPT ); Thu, 23 May 2019 11:34:45 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:33373 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731075AbfEWPep (ORCPT ); Thu, 23 May 2019 11:34:45 -0400 Received: by mail-qt1-f196.google.com with SMTP id z5so884866qtb.0 for ; Thu, 23 May 2019 08:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fyMd/20M9wRDEt2CUwmbtJJl1RcuuD2R+QgKbl3Q/tc=; b=JIjtmXhIk+ouBJQ777LOHATYHiCxeddr9NRCIpfSEdrQ1ap9ixS0CW54V6HURwC9xA RKpjT/mKs8jM8bOrU+JpE9OrzRhV1PSZWwxlmuIpZMYfSawM5XO76xU4PAAGkcWLSm5s 7ndty4jjdbxARiXmFK166PeUPWVyFicpugoKluqPMkVT0YjQEoLXox9cyoZwmEcFagdx vDMEFLfHF1VypHEJSnZkoDjj4S4j7WOn++WkjrFxWKF7S2FzamGOtEw9nQxisne15cmS QxE2dETQ9lLxC5EcMBOuIOrkEj8esYg8qdZuZD+Z0aF0lIU814C/ZVRZzLowQaUo/lk1 YE8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fyMd/20M9wRDEt2CUwmbtJJl1RcuuD2R+QgKbl3Q/tc=; b=RhKCj8DufQHdzP/9CBeh8jI88O85sCnYqnyDtYFRIheAlpU7s4ZInEmC2Ers/rJUjh uHJvwv1KWXfn8RyuxTDJ0I2jXufVIdMQeV1+3VyzaJIrw+klGXqN1XZfXtQcwFaccSEf 2HWoeDN9fwopk+xcS6QK8o4PSiHDbjLlVbEWe1fLk6fAswl3CIwxWC3rhMIMVfjw7/bx 0ATkjOLGVhi+o3/4lnpRfmXexIi7rh+r0y7N3WGcbrLQXAb9b6O1pCUH0aF8619XAvqQ cEzDZdm2AVyd4+Du0lbh+lcxkUVAwfNy64cj6f9rEuvjB3dZ4DKRFVvZiPmRC3dz+Mib ruZA== X-Gm-Message-State: APjAAAVMfuT4IZ3xhikA0fCewGInoUnVIvEBBwLdUVEh+JDR6C/MRUV9 CGT81gt01ZhLPGqU1abxnrxNRGPiaRM= X-Google-Smtp-Source: APXvYqx/4tQV+dyjpchgQ23NYg/m02UvYl5N+Rh/idEA6TgEqyIlzTWmi6NKvXZCk10ahm2Boj71vA== X-Received: by 2002:ac8:7c9:: with SMTP id m9mr15162695qth.127.1558625684072; Thu, 23 May 2019 08:34:44 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id t17sm17461892qte.66.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-000503-9i; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 10/11] mm/hmm: Poison hmm_range during unregister Date: Thu, 23 May 2019 12:34:35 -0300 Message-Id: <20190523153436.19102-11-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Trying to misuse a range outside its lifetime is a kernel bug. Use WARN_ON and poison bytes to detect this condition. Signed-off-by: Jason Gunthorpe Acked-by: Souptick Joarder Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: John Hubbard --- mm/hmm.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 6c3b7398672c29..02752d3ef2ed92 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -936,8 +936,7 @@ EXPORT_SYMBOL(hmm_range_register); */ void hmm_range_unregister(struct hmm_range *range) { - /* Sanity check this really should not happen. */ - if (range->hmm == NULL || range->end <= range->start) + if (WARN_ON(range->end <= range->start)) return; mutex_lock(&range->hmm->lock); @@ -945,9 +944,13 @@ void hmm_range_unregister(struct hmm_range *range) mutex_unlock(&range->hmm->lock); /* Drop reference taken by hmm_range_register() */ - range->valid = false; hmm_put(range->hmm); - range->hmm = NULL; + + /* The range is now invalid, leave it poisoned. */ + range->valid = false; + range->start = ULONG_MAX; + range->end = 0; + memset(&range->hmm, POISON_INUSE, sizeof(range->hmm)); } EXPORT_SYMBOL(hmm_range_unregister); From patchwork Thu May 23 15:34:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10958089 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5AE011395 for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46878283EE for ; Thu, 23 May 2019 15:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3B0DE28478; Thu, 23 May 2019 15:34:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3574286A2 for ; Thu, 23 May 2019 15:34:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731048AbfEWPep (ORCPT ); Thu, 23 May 2019 11:34:45 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:44773 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731089AbfEWPeo (ORCPT ); Thu, 23 May 2019 11:34:44 -0400 Received: by mail-qt1-f196.google.com with SMTP id f24so7200660qtk.11 for ; Thu, 23 May 2019 08:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sVSVMj7qtVOG+pR++xGbUKEq7rSWx5vQVniXU7xAdVY=; b=DZpq16KCBGWseTw+Emu3YXxJ4qAUUBYOgC5S+OOEj/qjUePnCbs18YftNrqI5k72oF PF78BOqbqq6gsqFCloyWSHUujhEA6iLhbwmuZW+tDHMvYXyKsgXqWsFKUjift9trNwPn rqocr9xDbYrprtIAWRiT7tiljZiU9eyK5kUSizlvzHsiN4QQ+GhHFIt7Cl39+kytFtcP YEmfFYaselG4ky3Uaic1er6GJoWtkTtMUKSq5/K7iPHbfWN6p8Dto2g7QzWwkTANGhkB ROp6hnDt0PpHQofzdaPfaC8krT25q9BBGFxfk+c0x0zYuBd4O1EiwzV88vfNmYBJAYQz bG8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sVSVMj7qtVOG+pR++xGbUKEq7rSWx5vQVniXU7xAdVY=; b=IzDSbY+AO3mr21xH5iZ/FY2rT6bMOLse+i9rN7YvKrSlMEXBoUCLh8cseIgParyT4p sAepVgCGJNmVc9cQ/XN2g7vP6d6fv7sGheniroGMN0olwhX6rukeo8FT1x2vzO5O1goK CkkWTQqcZJKxOWxzgHHtt8bVl68KrSs/FrRIGd/Vc02pRjMuag1KhQwUSrnAlk0NfVa/ +xlzRRZwhQewg9xvfNmoaC7fEzFoGNsLejPz46RC0BbpSGCnxL7ME5kwLiihdEdsBDAC 4l+xsbjVUrY1zw0Ul2L8saJuF/F9noP+bzgsc0FUvBc9MzOtcOXZFa4tS2xXLpWSjVhT 2DAA== X-Gm-Message-State: APjAAAV7qatsBwjMTGptqZQGEE4p7aJUKIB2ULeOEWzOYYL8vp+QGBlF PE9/G5E69XJj5Y8u59ylpka8lJYDuts= X-Google-Smtp-Source: APXvYqwlZgmvtDLezrwrAV3Bsq4rJxczbRVrNsIiCVDFfpxHEZgg1uH2yntWWMscbHuucZ+UF0vtYg== X-Received: by 2002:ac8:18b8:: with SMTP id s53mr76217130qtj.232.1558625683755; Thu, 23 May 2019 08:34:43 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-49-251.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.49.251]) by smtp.gmail.com with ESMTPSA id h17sm12879104qkk.13.2019.05.23.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 May 2019 08:34:39 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hTpjq-000509-Bh; Thu, 23 May 2019 12:34:38 -0300 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard Cc: Jason Gunthorpe Subject: [RFC PATCH 11/11] mm/hmm: Do not use list*_rcu() for hmm->ranges Date: Thu, 23 May 2019 12:34:36 -0300 Message-Id: <20190523153436.19102-12-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523153436.19102-1-jgg@ziepe.ca> References: <20190523153436.19102-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This list is always read and written while holding hmm->lock so there is no need for the confusing _rcu annotations. Signed-off-by: Jason Gunthorpe Acked-by: Souptick Joarder --- mm/hmm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 02752d3ef2ed92..b4aafa90a109a5 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -912,7 +912,7 @@ int hmm_range_register(struct hmm_range *range, /* Initialize range to track CPU page table update */ mutex_lock(&range->hmm->lock); - list_add_rcu(&range->list, &range->hmm->ranges); + list_add(&range->list, &range->hmm->ranges); /* * If there are any concurrent notifiers we have to wait for them for @@ -940,7 +940,7 @@ void hmm_range_unregister(struct hmm_range *range) return; mutex_lock(&range->hmm->lock); - list_del_rcu(&range->list); + list_del(&range->list); mutex_unlock(&range->hmm->lock); /* Drop reference taken by hmm_range_register() */