From patchwork Fri Oct 16 23:09:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842441 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 175281580 for ; Fri, 16 Oct 2020 23:09:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BD67920874 for ; Fri, 16 Oct 2020 23:09:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GTdBD4iC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD67920874 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 81D0C6B0062; Fri, 16 Oct 2020 19:09:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7C9F36B0068; Fri, 16 Oct 2020 19:09:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A4006B006E; Fri, 16 Oct 2020 19:09:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 199E16B0062 for ; Fri, 16 Oct 2020 19:09:39 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AD89E181AEF1F for ; Fri, 16 Oct 2020 23:09:38 +0000 (UTC) X-FDA: 77379332436.15.tooth29_4106c4927220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 87F401814B0C1 for ; Fri, 16 Oct 2020 23:09:38 +0000 (UTC) X-Spam-Summary: 1,0,0,872f6bb3eabebb5a,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:2:41:69:355:379:541:800:960:966:973:982:988:989:1260:1311:1314:1345:1359:1437:1513:1515:1521:1535:1605:1606:1730:1747:1777:1792:2196:2198:2199:2200:2393:2553:2559:2562:2691:2693:2731:2894:2898:2901:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:4120:4250:4321:4385:4605:5007:6117:6120:6261:6653:7875:7901:7903:9592:9969:10004:11026:11232:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13161:13180:13229:13255:14096:14877:21080:21222:21324:21444:21451:21627:21939:21972:21987:30003:30045:30054:30070:30090,0,RBL:209.85.221.68:@google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04y8145b1d1of5xnnq3bg69hcowekyc4tzyr3u3mxmg5jc3imkd7ac1qcq3z9e1.cb11iithbnj9zsyh8mnm1eaem4mymm7bncid9dbss1hp8iqrgm913arehpn5yu7.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSB L:0,DNSB X-HE-Tag: tooth29_4106c4927220 X-Filterd-Recvd-Size: 9016 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:38 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id t9so4833439wrq.11 for ; Fri, 16 Oct 2020 16:09:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YKfumXOLgixAy84wx3Nv6Fkz9iX4qYebEti8jMCXhGY=; b=GTdBD4iCkIejmT0ii834rbvngwiM3DhYbyE0f6KICAmNujfqnubYf4FptFnJ7Ysp0D 0AbzjfrI4HXAM3FXcrrS7+gPcEpe3IBbotJjQILnBOSYFyRI7DnM/agiJPj9Sw+eK7PY q6wRtLdbq3KMM7iQzfrPslOjTuZ1JmpQ5JjGB/mU22P75fBHPmN7P+j+Lm4X1ybiyaIo iS3LK1vFKhl8x3xlYmY//o8eP72UCJ+bxVhgDZBWxb0Hq+qnU0OCajLdC4S5qV/8uPPD /vSGXzm3SdhEKPUQDvMeKGQ0DSf3xMwxv/7lMh522scj3xOCYR3IoxJ+sHeMtrj078aP IPrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YKfumXOLgixAy84wx3Nv6Fkz9iX4qYebEti8jMCXhGY=; b=Rw4ZdJN8KJmP81DtmiomdoZK39hv0syaUao1F19sQ0eI/86lddzXizsyWm1lNLo6W8 vrJSBFVIYn/nhdgja0VT9Y9uPahlJn5bxFMFWCZ+4wMRiTUE27llUfB9tyR+UYfBYHai Slf93nP1UDD00BdjAxeXW4po9DjeZJmaOTeFRRarnB28wKxoEIjhn4Z2gJIJzBejsfvR VOqgkuNjKwhBXZAuMa4xuptRxIzEzzrKfQKOcB5PrMkOg6vhB85zUCq7T0CMXiJ5vQD3 EnAqV4AG9M6Ft3F+1WzqTH8pacDID9Q5o1w00zB+1KGsVLh19IbXbV5Fy3tMLMRIAYJy qQ9Q== X-Gm-Message-State: AOAM532vcSWpRVX8l/0YXdmr3dFcU/4gd66dugx47IeopTT9+boiA9BP PuZYWymEVGXGY9Nlgm/EXkWQ5Q== X-Google-Smtp-Source: ABdhPJzPIxZWpPKHv3yxMqwsMLhL6oUdBGXHlkzC5ZPb9QzCCBu761qmb+/Ol2XykISQILgn3L+6XQ== X-Received: by 2002:a5d:4b49:: with SMTP id w9mr7014571wrs.41.1602889776793; Fri, 16 Oct 2020 16:09:36 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id s11sm5476456wrm.56.2020.10.16.16.09.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:36 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 1/6] ptrace: Keep mm around after exit_mm() for __ptrace_may_access() Date: Sat, 17 Oct 2020 01:09:10 +0200 Message-Id: <20201016230915.1972840-2-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __ptrace_may_access() checks can happen on target tasks that are in the middle of do_exit(), past exit_mm(). At that point, the ->mm pointer has been NULLed out, and the mm_struct has been mmput(). Unfortunately, the mm_struct contains the dumpability and the user_ns in which the task last went through execve(), and we need those for __ptrace_may_access(). Currently, that problem is handled by failing open: If the ->mm is gone, we assume that the task was dumpable. In some edge cases, this could potentially expose access to things like /proc/$pid/fd/$fd of originally non-dumpable processes. (exit_files() comes after exit_mm(), so the file descriptor table is still there when we've gone through exit_mm().) One way to fix this would be to move mm->user_ns and the dumpability state over into the task_struct. However, that gets quite ugly if we want to preserve existing semantics because e.g. PR_SET_DUMPABLE and commit_creds() would then have to scan through all tasks sharing the mm_struct and keep them in sync manually - that'd be a bit error-prone and overcomplicated. (Moving these things into the signal_struct is not an option because that is kept across executions, and pre-execve co-threads will share the signal_struct that is also used by the task that has gone through execve().) I believe that this patch may be the least bad option to fix this - keep the mm_struct (but not process memory) around with an mmgrab() reference from exit_mm() until the task goes away completely. Note that this moves free_task() down in order to make mmdrop_async() available without a forward declaration. Cc: stable@vger.kernel.org Fixes: bfedb589252c ("mm: Add a user_ns owner to mm_struct and fix ptrace permission checks") Signed-off-by: Jann Horn --- include/linux/sched.h | 8 +++++++ kernel/exit.c | 2 ++ kernel/fork.c | 54 ++++++++++++++++++++++--------------------- kernel/ptrace.c | 10 ++++++++ 4 files changed, 48 insertions(+), 26 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index afe01e232935..55bec6ff5626 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -747,6 +747,14 @@ struct task_struct { struct mm_struct *mm; struct mm_struct *active_mm; + /* + * When we exit and ->mm (the reference pinning ->mm's address space) + * goes away, we stash a reference to the mm_struct itself (counted via + * exit_mm->mm_count) in this member. + * This allows us to continue using the mm_struct for security checks + * and such even after the task has started exiting. + */ + struct mm_struct *exit_mm; /* Per-thread vma caching: */ struct vmacache vmacache; diff --git a/kernel/exit.c b/kernel/exit.c index 733e80f334e7..97253ef33486 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -476,6 +476,8 @@ static void exit_mm(void) /* more a memory barrier than a real lock */ task_lock(current); current->mm = NULL; + mmgrab(mm); /* for current->exit_mm */ + current->exit_mm = mm; mmap_read_unlock(mm); enter_lazy_tlb(mm, current); task_unlock(current); diff --git a/kernel/fork.c b/kernel/fork.c index da8d360fb032..4942428a217c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -438,32 +438,6 @@ void put_task_stack(struct task_struct *tsk) } #endif -void free_task(struct task_struct *tsk) -{ - scs_release(tsk); - -#ifndef CONFIG_THREAD_INFO_IN_TASK - /* - * The task is finally done with both the stack and thread_info, - * so free both. - */ - release_task_stack(tsk); -#else - /* - * If the task had a separate stack allocation, it should be gone - * by now. - */ - WARN_ON_ONCE(refcount_read(&tsk->stack_refcount) != 0); -#endif - rt_mutex_debug_task_free(tsk); - ftrace_graph_exit_task(tsk); - arch_release_task_struct(tsk); - if (tsk->flags & PF_KTHREAD) - free_kthread_struct(tsk); - free_task_struct(tsk); -} -EXPORT_SYMBOL(free_task); - #ifdef CONFIG_MMU static __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) @@ -722,6 +696,34 @@ static inline void put_signal_struct(struct signal_struct *sig) free_signal_struct(sig); } +void free_task(struct task_struct *tsk) +{ + scs_release(tsk); + +#ifndef CONFIG_THREAD_INFO_IN_TASK + /* + * The task is finally done with both the stack and thread_info, + * so free both. + */ + release_task_stack(tsk); +#else + /* + * If the task had a separate stack allocation, it should be gone + * by now. + */ + WARN_ON_ONCE(refcount_read(&tsk->stack_refcount) != 0); +#endif + rt_mutex_debug_task_free(tsk); + ftrace_graph_exit_task(tsk); + arch_release_task_struct(tsk); + if (tsk->flags & PF_KTHREAD) + free_kthread_struct(tsk); + if (tsk->exit_mm) + mmdrop_async(tsk->exit_mm); + free_task_struct(tsk); +} +EXPORT_SYMBOL(free_task); + void __put_task_struct(struct task_struct *tsk) { WARN_ON(!tsk->exit_state); diff --git a/kernel/ptrace.c b/kernel/ptrace.c index 43d6179508d6..0aedc6cf5bdc 100644 --- a/kernel/ptrace.c +++ b/kernel/ptrace.c @@ -342,7 +342,17 @@ static int __ptrace_may_access(struct task_struct *task, unsigned int mode) * Pairs with a write barrier in commit_creds(). */ smp_rmb(); + /* + * Look up the target task's mm_struct. If it fails because the task is + * exiting and has gone through exit_mm(), we can instead use ->exit_mm + * as long as we only use members that are preserved by an mmgrab() + * reference. + * The only case in which both ->mm and ->exit_mm can be NULL should be + * kernel threads. + */ mm = task->mm; + if (!mm) + mm = task->exit_mm; if (mm && ((get_dumpable(mm) != SUID_DUMP_USER) && !ptrace_has_cap(cred, mm->user_ns, mode))) From patchwork Fri Oct 16 23:09:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842443 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A6BF15E6 for ; Fri, 16 Oct 2020 23:09:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2B09122201 for ; Fri, 16 Oct 2020 23:09:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eL+60koM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B09122201 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F1E336B0068; Fri, 16 Oct 2020 19:09:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EF5006B006E; Fri, 16 Oct 2020 19:09:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE44A900002; Fri, 16 Oct 2020 19:09:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id AB5E56B0068 for ; Fri, 16 Oct 2020 19:09:40 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4D6358249980 for ; Fri, 16 Oct 2020 23:09:40 +0000 (UTC) X-FDA: 77379332520.01.twist99_610e3d127220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 1925C100480A0 for ; Fri, 16 Oct 2020 23:09:40 +0000 (UTC) X-Spam-Summary: 1,0,0,93bb8a1d20ee9c6e,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1513:1515:1521:1535:1542:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3152:3353:3865:3866:3867:3868:3870:3871:3874:4321:4385:5007:6120:6261:6653:7875:9592:9969:10004:11658:11914:12043:12048:12296:12297:12517:12519:12555:12895:12986:13138:13231:13618:14096:14181:14721:21080:21444:21451:21627:30054,0,RBL:209.85.221.68:@google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04yguuuis41yozownfyzhws86hb8sopaeqpnhdh1bx96wittjnw9fhesinukm5i.qsnga6b5uwnzyx9b9i7rny1iskts7ywy7s35x4bn4mbtz4zi9g1xfidqxzwb1at.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: twist99_610e3d127220 X-Filterd-Recvd-Size: 5169 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:39 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id t9so4833537wrq.11 for ; Fri, 16 Oct 2020 16:09:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4LL5NRC0cRVJNqBZIY3Vnesfk3vz5g8to7dyxws6//g=; b=eL+60koMwsyvcjNqBBva4am3HrD7R0Mk/MAMKX69iwE+8LKoVLd4bFn6bE0eHTD1a8 pybTDkHu2I9373x1/HctrF7ICCgGyBGldp0utHtZt+nXoOsQEo0B7ZloQyzhDGnUi4QI ESstpz23vYvfc2rFDn4MEgP6Hs/2gW6xF5W2MdSUjV6bCcE4jebP7DXYCIF3J8wXtaKC vXtM1KDeouCuOfGrPBB6gFrI8L2gf5FYi+HFxNI4B8rJfX+rewKEHzdqvozoItNZgA1O RtR4xP9dXBCq1SDJmnNNeZblxAotKmTItqDdHcoBnlkFHGGlbmlydFNtu3UYZ8gyHQJa IJHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4LL5NRC0cRVJNqBZIY3Vnesfk3vz5g8to7dyxws6//g=; b=WYg5pT/NUBis1juQJlRoIdCiVDTMkxYO9JWnELsgkvOvqyg9hSVG0NzFSUhU0WlpM3 8VVwAuORK2DAPcFWz5OuW5dGQoZgIQN3JBDEHCuGRnXjHhy+x4tZ1JMnJFEtTQw2rOWG kpE8L36gGfY1Aso6thpsxpbs4sXJ8ltTyQK9vjx2pppJRRVef+d1Cdok9CoUbG/b0ozs cXsx4rdh2Ndp4wpVr8tQq/IAYE6cwWEwU1jSycSrjrdYxhoKHUm8logIkiQRkC5duRH9 1sn38jwHw0s+jhgMS6VYP2AX2Po2vfnagL7t8SfhwmI5cSsrzbV8kXHGHIMWECT0zCpa 2CfQ== X-Gm-Message-State: AOAM530bgO19RwxCMD1u2ra0WX46rMc97QWebpEwfGx/uOisCHyLbUj5 rOElSsjP928Sa0/xGNkjcHhfrg== X-Google-Smtp-Source: ABdhPJwutZJE8aiWvcNxtLTRLki2GRNZHZiM+YTPOkZuGIVpCqNB5rDOXOX33qg++t0t+HfIIggPyQ== X-Received: by 2002:adf:f88a:: with SMTP id u10mr6663411wrp.1.1602889778384; Fri, 16 Oct 2020 16:09:38 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id n9sm5754512wrq.72.2020.10.16.16.09.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:37 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 2/6] refcount: Move refcount_t definition into linux/types.h Date: Sat, 17 Oct 2020 01:09:11 +0200 Message-Id: <20201016230915.1972840-3-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I want to use refcount_t in mm_struct, but if refcount_t is defined in linux/refcount.h, that header would have to be included in linux/mm_types.h; that would be wasteful. Let's move refcount_t over into linux/types.h so that includes can be written less wastefully. Signed-off-by: Jann Horn --- include/linux/refcount.h | 13 +------------ include/linux/types.h | 12 ++++++++++++ 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/include/linux/refcount.h b/include/linux/refcount.h index 0e3ee25eb156..fd8cf65e4e2f 100644 --- a/include/linux/refcount.h +++ b/include/linux/refcount.h @@ -96,22 +96,11 @@ #include #include #include +#include /* refcount_t is defined here */ #include struct mutex; -/** - * struct refcount_t - variant of atomic_t specialized for reference counts - * @refs: atomic_t counter field - * - * The counter saturates at REFCOUNT_SATURATED and will not move once - * there. This avoids wrapping the counter and causing 'spurious' - * use-after-free bugs. - */ -typedef struct refcount_struct { - atomic_t refs; -} refcount_t; - #define REFCOUNT_INIT(n) { .refs = ATOMIC_INIT(n), } #define REFCOUNT_MAX INT_MAX #define REFCOUNT_SATURATED (INT_MIN / 2) diff --git a/include/linux/types.h b/include/linux/types.h index a147977602b5..34e4e779e767 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -175,6 +175,18 @@ typedef struct { } atomic64_t; #endif +/** + * struct refcount_t - variant of atomic_t specialized for reference counts + * @refs: atomic_t counter field + * + * The counter saturates at REFCOUNT_SATURATED and will not move once + * there. This avoids wrapping the counter and causing 'spurious' + * use-after-free bugs. + */ +typedef struct refcount_struct { + atomic_t refs; +} refcount_t; + struct list_head { struct list_head *next, *prev; }; From patchwork Fri Oct 16 23:09:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842445 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D11DA1580 for ; Fri, 16 Oct 2020 23:09:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 71C0222203 for ; Fri, 16 Oct 2020 23:09:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u0P1wqPf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71C0222203 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 04F8C6B006E; Fri, 16 Oct 2020 19:09:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 02A33900002; Fri, 16 Oct 2020 19:09:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E607F6B0071; Fri, 16 Oct 2020 19:09:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id AAD266B006E for ; Fri, 16 Oct 2020 19:09:43 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 459438249980 for ; Fri, 16 Oct 2020 23:09:43 +0000 (UTC) X-FDA: 77379332646.04.year84_611773927220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 29F8D800C886 for ; Fri, 16 Oct 2020 23:09:43 +0000 (UTC) X-Spam-Summary: 1,0,0,dedb22f1deae9c5e,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:2:41:355:379:541:800:960:966:973:982:988:989:1260:1311:1314:1345:1359:1431:1437:1513:1515:1521:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3152:3165:3622:3865:3866:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4385:4605:5007:6119:6120:6261:6653:7903:9969:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12895:12986:13149:13161:13229:13230:14096:21080:21433:21444:21451:21627:21740:30034:30045:30054,0,RBL:209.85.221.65:@google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04y8t89xgwp9aq5urgfhbwap4bddbycdk3gycnodw6cd4ayaatjjn614n5qs9mh.o5wwhdooh6dxxme5ritkkp8yskgr9oqegm9hpnioihtt5463nkkz8karbxb8pyt.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: year84_611773927220 X-Filterd-Recvd-Size: 9810 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:42 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id e17so4831442wru.12 for ; Fri, 16 Oct 2020 16:09:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VB/fdMlO8ew2DF3nfC/fPT+g4jr/Z2tfK9/9xA97SGY=; b=u0P1wqPffQ2VFRsEqCRxRlBzECdbyLFxdM2oSnYV5SNagwnF1TaI9eox0pgis1JKtI QDsOyVYg0B1Lv3XU/Nq5KwjgJuZ7lAfw4YWhxnS5vIfnNaAzpQyl1kG+I613VF75rbVO h9kjCEAmpgTT1ytZcv1EDtkGrp22s0Y01E8FRiRvcykUDZmgUhrkdy/Fd+raOE9R2t4d 0ExR1r1KQjufoaomZS3Ijhib8bQsdWy+UJrXpqE4zY16JdllX6mcMbkMd8sd7/TcOZOx UnPJwhgQsPJwrOIWBNEpN3VajZWBMamc/b6AmOvWb6c5W2RdjPj/AnaoN5XYfc09fZKE da6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VB/fdMlO8ew2DF3nfC/fPT+g4jr/Z2tfK9/9xA97SGY=; b=c+Cd5xDfqXIkSWFYDwDDNaYVdB0yGAV0yqKNj937tJzl2IrxgyOOiwI6WolB06rQI7 iYcz7Skx53BzEj6Ud+1rkWv1v/9R9TBbCKvys3SHoW0ztGfmhgMUoY6bpRaBDaGgy7ze 31ITs6h3KdI+HR2Gl09fh7b/VeLC4fkI8Sk3+Yu6s8m0ORVRZTT8Zat6MtdqsCn3zibd Zf0ZSEuu3guFT1YYdb/0VVydC9u7SjdFk/yeU6wvva3qLHp9sWFHs3XwN/mcYbYEVbPk qtuJtdAySx8iaMsrGUsvnlw4MqrJIDUKAw/1TY26S5Sj3PYvgIl5nXlXKzhJ4c22PU3P SS+w== X-Gm-Message-State: AOAM530jyuIC2ws4WDPI9iTLqhwzVziTXRT1gkbLZ0tBHFugfseuLc9p lpSN/VGkHTPac6l6aqNKWYTqow== X-Google-Smtp-Source: ABdhPJzxpFy5MX6UOKoNUQh+BlIQ+0+ZXkXGZZLzQjDVXmpdQ5LM/mJzw0E+jJ7KhNjl5jrAWgc66w== X-Received: by 2002:adf:fc08:: with SMTP id i8mr7087525wrr.116.1602889779754; Fri, 16 Oct 2020 16:09:39 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id g83sm4666419wmf.15.2020.10.16.16.09.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:39 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 3/6] mm: Add refcount for preserving mm_struct without pgd Date: Sat, 17 Oct 2020 01:09:12 +0200 Message-Id: <20201016230915.1972840-4-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, mm_struct has two refcounts: - mm_users: preserves everything - the mm_struct, the page tables, the memory mappings, and so on - mm_count: preserves the mm_struct and pgd However, there are three types of users of mm_struct: 1. users that want page tables, memory mappings and so on 2. users that want to preserve the pgd (for lazy TLB) 3. users that just want to keep the mm_struct itself around (e.g. for mmget_not_zero() or __ptrace_may_access()) Dropping mm_count references can be messy because dropping mm_count to zero deletes the pgd, which takes the pgd_lock on x86, meaning it doesn't work from RCU callbacks (which run in IRQ context). In those cases, mmdrop_async() must be used to punt the invocation of __mmdrop() to workqueue context. That's fine when mmdrop_async() is a rare case, but the preceding patch "ptrace: Keep mm around after exit_mm() for __ptrace_may_access()" makes it the common case; we should probably avoid punting freeing to workqueue context all the time if we can avoid it? To resolve this, add a third refcount that just protects the mm_struct and the user_ns it points to, and which can be dropped with synchronous freeing from (almost) any context. Signed-off-by: Jann Horn --- arch/x86/kernel/tboot.c | 2 ++ drivers/firmware/efi/efi.c | 2 ++ include/linux/mm_types.h | 13 +++++++++++-- include/linux/sched/mm.h | 13 +++++++++++++ kernel/fork.c | 14 ++++++++++---- mm/init-mm.c | 2 ++ 6 files changed, 40 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c index 992fb1415c0f..b92ea1bb3bb9 100644 --- a/arch/x86/kernel/tboot.c +++ b/arch/x86/kernel/tboot.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -93,6 +94,7 @@ static struct mm_struct tboot_mm = { .pgd = swapper_pg_dir, .mm_users = ATOMIC_INIT(2), .mm_count = ATOMIC_INIT(1), + .mm_bare_refs = REFCOUNT_INIT(1), MMAP_LOCK_INITIALIZER(init_mm) .page_table_lock = __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock), .mmlist = LIST_HEAD_INIT(init_mm.mmlist), diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 3aa07c3b5136..3b73a0717c6e 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include @@ -54,6 +55,7 @@ struct mm_struct efi_mm = { .mm_rb = RB_ROOT, .mm_users = ATOMIC_INIT(2), .mm_count = ATOMIC_INIT(1), + .mm_bare_refs = REFCOUNT_INIT(1), MMAP_LOCK_INITIALIZER(efi_mm) .page_table_lock = __SPIN_LOCK_UNLOCKED(efi_mm.page_table_lock), .mmlist = LIST_HEAD_INIT(efi_mm.mmlist), diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index ed028af3cb19..764d251966c7 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -429,13 +429,22 @@ struct mm_struct { /** * @mm_count: The number of references to &struct mm_struct - * (@mm_users count as 1). + * including its pgd (@mm_users count as 1). * * Use mmgrab()/mmdrop() to modify. When this drops to 0, the - * &struct mm_struct is freed. + * pgd is freed. */ atomic_t mm_count; + /** + * @mm_bare_refs: The number of references to &struct mm_struct + * that preserve no page table state whatsoever (@mm_count + * counts as 1). + * + * When this drops to 0, the &struct mm_struct is freed. + */ + refcount_t mm_bare_refs; + /** * @has_pinned: Whether this mm has pinned any pages. This can * be either replaced in the future by @pinned_vm when it diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index f889e332912f..e5b236e15757 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -109,6 +109,19 @@ extern void mmput(struct mm_struct *); void mmput_async(struct mm_struct *); #endif +static inline void mm_ref(struct mm_struct *mm) +{ + refcount_inc(&mm->mm_bare_refs); +} + +void __mm_unref(struct mm_struct *mm); + +static inline void mm_unref(struct mm_struct *mm) +{ + if (refcount_dec_and_test(&mm->mm_bare_refs)) + __mm_unref(mm); +} + /* Grab a reference to a task's mm, if it is not already going away */ extern struct mm_struct *get_task_mm(struct task_struct *task); /* diff --git a/kernel/fork.c b/kernel/fork.c index 4942428a217c..fcdd1ace79e4 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -642,10 +642,16 @@ static void check_mm(struct mm_struct *mm) #define allocate_mm() (kmem_cache_alloc(mm_cachep, GFP_KERNEL)) #define free_mm(mm) (kmem_cache_free(mm_cachep, (mm))) +void __mm_unref(struct mm_struct *mm) +{ + put_user_ns(mm->user_ns); + free_mm(mm); +} + /* - * Called when the last reference to the mm + * Called when the last PGD-preserving reference to the mm * is dropped: either by a lazy thread or by - * mmput. Free the page directory and the mm. + * mmput. Free the page directory. */ void __mmdrop(struct mm_struct *mm) { @@ -656,8 +662,7 @@ void __mmdrop(struct mm_struct *mm) destroy_context(mm); mmu_notifier_subscriptions_destroy(mm); check_mm(mm); - put_user_ns(mm->user_ns); - free_mm(mm); + mm_unref(mm); } EXPORT_SYMBOL_GPL(__mmdrop); @@ -1007,6 +1012,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, mm->vmacache_seqnum = 0; atomic_set(&mm->mm_users, 1); atomic_set(&mm->mm_count, 1); + refcount_set(&mm->mm_bare_refs, 1); mmap_init_lock(mm); INIT_LIST_HEAD(&mm->mmlist); mm->core_state = NULL; diff --git a/mm/init-mm.c b/mm/init-mm.c index 3a613c85f9ed..3c3cd35236fd 100644 --- a/mm/init-mm.c +++ b/mm/init-mm.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -31,6 +32,7 @@ struct mm_struct init_mm = { .pgd = swapper_pg_dir, .mm_users = ATOMIC_INIT(2), .mm_count = ATOMIC_INIT(1), + .mm_bare_refs = REFCOUNT_INIT(1), MMAP_LOCK_INITIALIZER(init_mm) .page_table_lock = __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock), .arg_lock = __SPIN_LOCK_UNLOCKED(init_mm.arg_lock), From patchwork Fri Oct 16 23:09:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842447 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A83115E6 for ; Fri, 16 Oct 2020 23:09:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C785A2222B for ; Fri, 16 Oct 2020 23:09:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rClZn7/P" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C785A2222B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB1026B0070; Fri, 16 Oct 2020 19:09:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A87D66B0071; Fri, 16 Oct 2020 19:09:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A4866B0072; Fri, 16 Oct 2020 19:09:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 625486B0070 for ; Fri, 16 Oct 2020 19:09:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E95788249980 for ; Fri, 16 Oct 2020 23:09:44 +0000 (UTC) X-FDA: 77379332688.12.hook04_600acc227220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id CB8E01801A106 for ; Fri, 16 Oct 2020 23:09:44 +0000 (UTC) X-Spam-Summary: 1,0,0,36d2baceac04a3b9,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1513:1515:1521:1534:1541:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2559:2562:2693:2903:3138:3139:3140:3141:3142:3152:3352:3865:3867:3868:3871:3872:3874:4250:4321:4385:5007:6119:6120:6261:6653:7903:8660:9969:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12895:12986:13069:13148:13161:13229:13230:13255:13311:13357:14096:14181:14384:14721:21080:21433:21444:21627:21939:30054,0,RBL:209.85.221.68:@google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yfkabon747hpdm8syxz5ucfhgfrocou11nhwurz5psuipoopz7k9q673ucu6o.uu7o4914mzne3xhpgjqk17bt1qoizqybxwyedqa1ojczskj4bjjczrprxu9bbxg.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: hook04_600acc227220 X-Filterd-Recvd-Size: 4671 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:44 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id y12so4875466wrp.6 for ; Fri, 16 Oct 2020 16:09:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J4AQ8wqNpjVt1ykz0FMlQ/Hn0cAH4pxEjRmJmEVEXzM=; b=rClZn7/P2brEcbJ17hk3v5Ov20ONpnJc3N7mscK6TVtzEI4C2dZVcNCjb2UbK3GcrH KOrmC+GfSAgf5CBO5WNij6aH3a293buv1EwV41bQyMSXwzh3MJn0iVkBEv+G7SWWNczj vtzPCkqVAbbr0bjfD35ZzQpr1Lgn5CWWWEo3YXHFLlZ8tkf99ubo03jvazvBLhHOC1xP 8gQ+Xx3eR9NO0/YTyQOT7IdV6mTksrXqZTCaWg2l4hqnPa7E/QS3IEX9eLUnc7yluLWQ xj88YsUnFOCQpCiwuPuGMsh2AevEdExmakr+DPQ/A9NSXLpx5p+Kxfi/o7cfTQtApJEc NN4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J4AQ8wqNpjVt1ykz0FMlQ/Hn0cAH4pxEjRmJmEVEXzM=; b=jLSpeeLxvqnD7abj+8Dpohda7QcSbUYVwvzCyNkVUqEcpn8JXO14dfDZ2SIxpgParL B40ou3VE/JJp3p9BMTUyM2vPDrJqNPmQs8xS4cAMxKI0LlvHr1cYia52VKj9lF8E5RPN +apquSyjskZP8H4EbsW71DEdRck5QNELHnvN5agF2FKl/BwpyiT8Ff5ZdL5RYd8jR0oI hn3fVOaqsfGJoyBnusbgbr3knJEyO+Jw+mG3PTtBej2+pe/NeJxjtG5Uo918mxOlv72y srum12NP+mQBuSpoYpiklOWzUrPH1hW/CIEFOlnOSh63wt2smRndHevmzfP2TJ5m4lR0 AYUQ== X-Gm-Message-State: AOAM530be5K5SuQINUGLEsUgEaEwcPrYSW09N254um2sS7fgb7fly9br SfrdFspDo1VxLSozXddwqpLQGA== X-Google-Smtp-Source: ABdhPJzEacUrzZyN0193KfsbDyP+Nq2E/fuEoba3R3HJmfpoAO6isYjIW2vwr6uW3TdeSSNG7cYQgg== X-Received: by 2002:adf:e6c4:: with SMTP id y4mr6904873wrm.423.1602889783182; Fri, 16 Oct 2020 16:09:43 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id q6sm4718202wma.0.2020.10.16.16.09.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:42 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 4/6] mm, oom: Use mm_ref()/mm_unref() and avoid mmdrop_async() Date: Sat, 17 Oct 2020 01:09:13 +0200 Message-Id: <20201016230915.1972840-5-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The OOM killer uses MMF_OOM_SKIP to avoid running on an mm that has started __mmput(); it only uses the mmgrab() reference to ensure that the mm_struct itself stays alive. This means that we don't need a full mmgrab() reference, which will keep the pgd (and potentially also some pmd pages) alive and can't be cleaned up from RCU callback context; we can use an mm_ref() reference instead. Signed-off-by: Jann Horn --- kernel/fork.c | 6 +----- mm/oom_kill.c | 2 +- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/kernel/fork.c b/kernel/fork.c index fcdd1ace79e4..59c119b03351 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -686,12 +686,8 @@ static inline void free_signal_struct(struct signal_struct *sig) { taskstats_tgid_free(sig); sched_autogroup_exit(sig); - /* - * __mmdrop is not safe to call from softirq context on x86 due to - * pgd_dtor so postpone it to the async context - */ if (sig->oom_mm) - mmdrop_async(sig->oom_mm); + mm_unref(sig->oom_mm); kmem_cache_free(signal_cachep, sig); } diff --git a/mm/oom_kill.c b/mm/oom_kill.c index e90f25d6385d..12967f54fbcf 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -704,7 +704,7 @@ static void mark_oom_victim(struct task_struct *tsk) /* oom_mm is bound to the signal struct life time. */ if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) { - mmgrab(tsk->signal->oom_mm); + mm_ref(tsk->signal->oom_mm); set_bit(MMF_OOM_VICTIM, &mm->flags); } From patchwork Fri Oct 16 23:09:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842449 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EEC217E6 for ; Fri, 16 Oct 2020 23:09:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 015FE2222A for ; Fri, 16 Oct 2020 23:09:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LhDuPgRe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 015FE2222A Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 13DAA6B0071; Fri, 16 Oct 2020 19:09:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C5E86B0072; Fri, 16 Oct 2020 19:09:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFEBF6B0073; Fri, 16 Oct 2020 19:09:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id B50066B0071 for ; Fri, 16 Oct 2020 19:09:46 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 52391362C for ; Fri, 16 Oct 2020 23:09:46 +0000 (UTC) X-FDA: 77379332772.14.swing74_011011127220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 33F2118229835 for ; Fri, 16 Oct 2020 23:09:46 +0000 (UTC) X-Spam-Summary: 1,0,0,713e4ae39f2622c3,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1513:1515:1521:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3152:3352:3865:3866:3867:3868:3871:3872:3874:4250:4321:4385:5007:6119:6120:6261:6653:7903:9969:10004:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12895:13069:13311:13357:14096:14181:14384:14721:14877:21080:21324:21444:21627:21939:30054,0,RBL:209.85.221.66:@google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygrefk31tyjmg54ff147syrcfqgyc4qygrnss35wufphkbdcbwyxf6xzi4u8w.oidoxsdzq1938ab8yqsxhy96sttya7qnhmzm7mtwd6a9q6et6qqmiqpdoos9j8q.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: swing74_011011127220 X-Filterd-Recvd-Size: 4421 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:45 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id i1so4896015wro.1 for ; Fri, 16 Oct 2020 16:09:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZqE9YT9elxKHcuMev0eqxdSAa29MOpdvFBgbgr/Dw4c=; b=LhDuPgReRaskA4lZJUuE4meGMw+AgSOBWcN0nqcL7KFy86acDGi8um3oRgd5CDOyyW bJFG22PeRLjTWFTU0ukD0tLvCbDgdbDFYPyGitxBgX1xKbXiv598o0EuHa9yELSghqKw VA4T4kdrHVqd88Cs852iMYBnFRMpkx301IXdoi39KfAj+zcBhsCDBtvTEc8dT3mQEW7T 7cylEg7GMWupAk7aZgn16x9jAZeU8zBSiMABrBnBpXTYtig7uF8ZJ7QptnUymL6udJDm sSK7gfMVEBYedaoF17S5Y2Kt1jq2Ee3Ymywe0au8W2vWFh7Kb2xhY8Sfp9vAdq5IO04Z rTDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZqE9YT9elxKHcuMev0eqxdSAa29MOpdvFBgbgr/Dw4c=; b=Wiq+K/R90fiwhco9fesNsnVaFkpkWegrEpDMY/9leFOIdlo/tOmxrQxySJFDM+U7qf HMQSq/5BATV1jQMQbHDqtmSdoNRC4IQLoIDIc9F1CzqWo3lgNPGI1jIBoNW0eP0sULQy VPPyulhMPJt5/6U2uColnsbETE2i6tYmQfUKL2hMpuOi+PNoqUUCt8dC0wZDtXv4Ska/ UasjxF/IydzT2UuyuBZ/x/V1f3LJsRashat2UInfALeDgdQVggHMrWL62G7P7i/kJwFS +LmXckGn/0aF60hoy0IQF5T1lktRSr1hay7tIFPCl3b2IV2/Ha622aqDE1flrbZiJcZ+ X46g== X-Gm-Message-State: AOAM531x5XR33ITXtjJZ92tvwJ8X/mJB2ScIym6WEduOzYS88s/hP0Da v8bNK87GE/jRpkXsBEChvJQTfA== X-Google-Smtp-Source: ABdhPJxhjUYQmLunbOMgkbnTqMUJ/Am0yPCMEVT4RqDmKWTqbn4mCWM3hyiVNo2TJIXie6ukzbX0NA== X-Received: by 2002:a5d:4b49:: with SMTP id w9mr7015093wrs.41.1602889784661; Fri, 16 Oct 2020 16:09:44 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id v17sm6144214wrc.23.2020.10.16.16.09.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:44 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 5/6] ptrace: Use mm_ref() for ->exit_mm Date: Sat, 17 Oct 2020 01:09:14 +0200 Message-Id: <20201016230915.1972840-6-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We only use ->exit_mm to look up dumpability and the ->user_mm; we don't need to keep the PGD alive for this. mmgrab() is also inconvenient here, because it means that we need to use mmdrop_async() when dropping the reference to the mm from an RCU callback. Use mm_ref() instead of mmgrab() to make things neater. Signed-off-by: Jann Horn --- kernel/exit.c | 2 +- kernel/fork.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/exit.c b/kernel/exit.c index 97253ef33486..03ba6d13ef1e 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -476,7 +476,7 @@ static void exit_mm(void) /* more a memory barrier than a real lock */ task_lock(current); current->mm = NULL; - mmgrab(mm); /* for current->exit_mm */ + mm_ref(mm); /* for current->exit_mm */ current->exit_mm = mm; mmap_read_unlock(mm); enter_lazy_tlb(mm, current); diff --git a/kernel/fork.c b/kernel/fork.c index 59c119b03351..4383bf055b40 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -720,7 +720,7 @@ void free_task(struct task_struct *tsk) if (tsk->flags & PF_KTHREAD) free_kthread_struct(tsk); if (tsk->exit_mm) - mmdrop_async(tsk->exit_mm); + mm_unref(tsk->exit_mm); free_task_struct(tsk); } EXPORT_SYMBOL(free_task); From patchwork Fri Oct 16 23:09:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11842451 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 106B717E6 for ; Fri, 16 Oct 2020 23:09:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA33522201 for ; Fri, 16 Oct 2020 23:09:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GEabQQbY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA33522201 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F5DC6B0072; Fri, 16 Oct 2020 19:09:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5AAF76B0073; Fri, 16 Oct 2020 19:09:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BD4A6B0074; Fri, 16 Oct 2020 19:09:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 12C946B0072 for ; Fri, 16 Oct 2020 19:09:48 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ABFDC362C for ; Fri, 16 Oct 2020 23:09:47 +0000 (UTC) X-FDA: 77379332814.18.grade73_0b0c68227220 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 910A710269E77 for ; Fri, 16 Oct 2020 23:09:47 +0000 (UTC) X-Spam-Summary: 1,0,0,979ee92b861f1a21,d41d8cd98f00b204,jannh@google.com,,RULES_HIT:41:69:355:379:541:800:960:966:968:973:988:989:1260:1311:1314:1345:1359:1437:1513:1515:1521:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3152:3352:3865:3866:3870:3871:3872:3874:4321:4385:4605:5007:6120:6261:6653:7903:9592:9969:10004:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12895:12986:13069:13161:13180:13229:13311:13357:14096:14181:14384:14721:21080:21444:21627:21740:30054,0,RBL:209.85.221.66:@google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yffxcmdrqquhcqdw3n5oo9m7yc5yp3rfdcx6ba3844f5eqza5jh5n8fwfp6cd.bpbxa91xqchsrfbqwaq1k9w4tsf8n9iwqaourcabkpq6unadjg7wqbwo8p6ext8.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: grade73_0b0c68227220 X-Filterd-Recvd-Size: 4737 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 23:09:47 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id i1so4896081wro.1 for ; Fri, 16 Oct 2020 16:09:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gdMs/4BkKI2uJNTjUP69TVUzfQ5T9+m8xYXEzWMRNr4=; b=GEabQQbYPZdJBOj2BGbAAiir4cXarxTnRaOVXj4JRtZnmqkGa2CMArEAtTbi/+hATc B3cLhuNPZmMDZZJ86auETwNS7l/UKxN72uloxbt8mov62APgfpCPMibJZymXO2u+hfYY tWXoKpqkr2Qqiu4JCttGYfZAkH/ujpEvJ1ewfBPeR4HRYEvHCoGe/kRLeMDB/rCwR+td NMmO7SkZilT2N5jpR4yiruA6rC3UNfH7boHds7qX9AyQpG3q9B/gPQRxlhOAm08sEyOa 5WtVENVcOyUP7EBHec33EsP8pE9FjCvAdNR+mwZMFo//3lJ7WJyRClrGRBrrLRzn4e2R Z1Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gdMs/4BkKI2uJNTjUP69TVUzfQ5T9+m8xYXEzWMRNr4=; b=f4pjndQFnD0WRUI7RNi6AcyLu4u+FQJat4jv/3CZcDE6G4sRgfZJS/9IjHkzwnExd7 6jQlqavRp+qP4YSnm5U84wqNQ0Hqj+XbwKdzFIclGLw+5hh0qOy1ixZmRuKTnrscK1Na rk8odGayRz2yg4jxeYMaUB1accC2m8RQ66DTrprjyAS8DEHKPYeBEDQ9bYaf3oP6Y5tb JHnnI1XQbMT7DZfqZMLnCvk8D9w1lF9poWI6dqh/vfN7OMQKswO1hQrI54LMXjok3Lzo C6bNDP4sJRmWNmXS6GDhoGCGza020oEf7BJT6l+AG52wd/RiRhn+0UWGrRjxInH5FIo8 Qpbw== X-Gm-Message-State: AOAM5334iDgFDwfhGCuFfK9XKVqawQzDM/hv+68EFFNQG1FBggiayosL S8OL0PcOat0XZf/fe5eNYu9jgQ== X-Google-Smtp-Source: ABdhPJxbfO8IUNwESku/fIzsOaLeXDX/vfAWxdZkmAezDlH0QNO6awB1LAUdwuJDWyVhjwRK4WO4nQ== X-Received: by 2002:adf:f810:: with SMTP id s16mr6374268wrp.424.1602889786072; Fri, 16 Oct 2020 16:09:46 -0700 (PDT) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id t19sm5148559wmi.26.2020.10.16.16.09.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Oct 2020 16:09:45 -0700 (PDT) From: Jann Horn To: Andrew Morton , linux-mm@kvack.org, Eric Biederman , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, Will Deacon , Kees Cook , Ingo Molnar Subject: [RFC PATCH resend 6/6] mm: remove now-unused mmdrop_async() Date: Sat, 17 Oct 2020 01:09:15 +0200 Message-Id: <20201016230915.1972840-7-jannh@google.com> X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog In-Reply-To: <20201016230915.1972840-1-jannh@google.com> References: <20201016230915.1972840-1-jannh@google.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The preceding patches have removed all users of mmdrop_async(); get rid of it. Note that on MMU, we still need async_put_work because mmput_async() uses it, which in turn is used by binder's shrinker callback. We could claw back those 4 words per mm if we made mmput_async() depend on CONFIG_ANDROID_BINDER_IPC. Signed-off-by: Jann Horn --- include/linux/mm_types.h | 2 ++ kernel/fork.c | 16 ---------------- 2 files changed, 2 insertions(+), 16 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 764d251966c7..8fde2068bde1 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -560,7 +560,9 @@ struct mm_struct { #ifdef CONFIG_HUGETLB_PAGE atomic_long_t hugetlb_usage; #endif +#ifdef CONFIG_MMU struct work_struct async_put_work; +#endif } __randomize_layout; /* diff --git a/kernel/fork.c b/kernel/fork.c index 4383bf055b40..c5f2ec544933 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -666,22 +666,6 @@ void __mmdrop(struct mm_struct *mm) } EXPORT_SYMBOL_GPL(__mmdrop); -static void mmdrop_async_fn(struct work_struct *work) -{ - struct mm_struct *mm; - - mm = container_of(work, struct mm_struct, async_put_work); - __mmdrop(mm); -} - -static void mmdrop_async(struct mm_struct *mm) -{ - if (unlikely(atomic_dec_and_test(&mm->mm_count))) { - INIT_WORK(&mm->async_put_work, mmdrop_async_fn); - schedule_work(&mm->async_put_work); - } -} - static inline void free_signal_struct(struct signal_struct *sig) { taskstats_tgid_free(sig);