From patchwork Sat Feb 11 10:26:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hoeun Ryu X-Patchwork-Id: 9567947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6DA826043D for ; Sat, 11 Feb 2017 10:27:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5BED0285C3 for ; Sat, 11 Feb 2017 10:27:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4D8E5285EE; Sat, 11 Feb 2017 10:27:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 70F46285C3 for ; Sat, 11 Feb 2017 10:27:44 +0000 (UTC) Received: (qmail 18036 invoked by uid 550); 11 Feb 2017 10:27:42 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 18015 invoked from network); 11 Feb 2017 10:27:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=SpAGv1I8kD+bYdEW2vDU5n8AcOiy+JEeRkb62vhXbgI=; b=u7VwIgCdHrKt79aDXYuD8HJR6FnfQMoYDpFAXX7YahVT+M9i9IXvtMbEwM7WzyJNq8 id7dct9TBU2+gLz47+Gwk8UZseFUUnN+zHwrMrMrSX7k2km06pJbTEmzGOvHtBiHW9w+ fPZW6sDxgZN6641/yMT9cSj63iVZcnvgIwI1wBQQwaoHSync4F/mOyIJZFJsGvcepBxp a7y7XOuU2uym4JUaWCfrToQKw9QGdFkNg3QwIJkI1CYjLlFKu88EomfVqnMZfSS/qZBl vJKgquN/9Ze7SFhsku2eBg0/n8LaKKfFaHOsFYvQI7NaL624fBBiXVD1+tgKUTEitUye Wkjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=SpAGv1I8kD+bYdEW2vDU5n8AcOiy+JEeRkb62vhXbgI=; b=qj86/lnQtCI0n12IXokm9eCgtmaZTrkwsXI1OoubI8Xf1wxmJ91KHep0wDrpuUxOKY rZuXgvdbWjuJEHuVw7prcCXIUbRA3hnLaS0wd2ErczHYDv/0MohQuaXEc8UuYag+ZnvP 25Y4h2AOP10zNX4Trt5svenLKdSljAC31qerNYe+f5B5zlg03PXMO2eE5016NI3ss5n4 YrXR26fjLV/8OeUmEX3JrW26yTqrOZdzXJ0y0yCZafJCwqHNma2D3hLIIgQzUH91mAfW 9encjF8j6HTI1kEJ+hSd/2ijGeFmojmC8dhr6kFBUhNW75Hpzg8ewAVuePE64x+SS5ZX tlrg== X-Gm-Message-State: AMke39kFTpfdFe6KtrGOYkddy4ZUkS34U9nOlRAMU5jN2pRLE3A6C75F7YVHAQgcRZlsvQ== X-Received: by 10.84.133.163 with SMTP id f32mr17103476plf.64.1486808850234; Sat, 11 Feb 2017 02:27:30 -0800 (PST) From: Hoeun Ryu To: Thomas Gleixner , Andrew Morton , Michal Hocko , Ingo Molnar , Andy Lutomirski , "Eric W. Biederman" , Oleg Nesterov Cc: linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Hoeun Ryu Date: Sat, 11 Feb 2017 19:26:48 +0900 Message-Id: <1486808828-8854-1-git-send-email-hoeun.ryu@gmail.com> X-Mailer: git-send-email 2.7.4 Subject: [kernel-hardening] [PATCH v5] fork: free vmapped stacks in cache when cpus are offline X-Virus-Scanned: ClamAV using ClamSMTP Using virtually mapped stack, kernel stacks are allocated via vmalloc. In the current implementation, two stacks per cpu can be cached when tasks are freed and the cached stacks are used again in task duplications. but the cached stacks may remain unfreed even when cpus are offline. By adding a cpu hotplug callback to free the cached stacks when a cpu goes offline, the pages of the cached stacks are not wasted. Signed-off-by: Hoeun Ryu --- v5: - wrap cpuhp_setup_state() in a new function, vm_stack_cache_init() which actually do nothing when !CONFIG_VMAP_STACK - add __may_unused to free_vm_stack_cache() v4: - use CPUHP_BP_PREPARE_DYN state for cpuhp setup - fix minor coding style v3: - fix misuse of per-cpu api - fix location of function definition within CONFIG_VMAP_STACK v2: - remove cpuhp callback for `startup`, only `teardown` callback is installed. kernel/fork.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/kernel/fork.c b/kernel/fork.c index 937ba59..86a1110 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -168,6 +168,32 @@ void __weak arch_release_thread_stack(unsigned long *stack) */ #define NR_CACHED_STACKS 2 static DEFINE_PER_CPU(struct vm_struct *, cached_stacks[NR_CACHED_STACKS]); + +static int __maybe_unused free_vm_stack_cache(unsigned int cpu) +{ + struct vm_struct **cached_vm_stacks = per_cpu_ptr(cached_stacks, cpu); + int i; + + for (i = 0; i < NR_CACHED_STACKS; i++) { + struct vm_struct *vm_stack = cached_vm_stacks[i]; + + if (!vm_stack) + continue; + + vfree(vm_stack->addr); + cached_vm_stacks[i] = NULL; + } + + return 0; +} + +static inline int vm_stack_cache_init(void) +{ + return cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "fork:vm_stack_cache", + NULL, free_vm_stack_cache); +} +#else +static __always_inline int vm_stack_cache_init(void) { return 0; } #endif static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) @@ -456,6 +482,8 @@ void __init fork_init(void) for (i = 0; i < UCOUNT_COUNTS; i++) { init_user_ns.ucount_max[i] = max_threads/2; } + + vm_stack_cache_init(); } int __weak arch_dup_task_struct(struct task_struct *dst,