From patchwork Wed Aug 29 22:59:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10581037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E4B35A4 for ; Wed, 29 Aug 2018 22:59:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7ED8E2B157 for ; Wed, 29 Aug 2018 22:59:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7349F2B16D; Wed, 29 Aug 2018 22:59:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B1B82B157 for ; Wed, 29 Aug 2018 22:59:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8003C6B4E25; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 787F66B4E28; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64DA26B4E29; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 1F6856B4E25 for ; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) Received: by mail-pl1-f200.google.com with SMTP id b93-v6so2868730plb.10 for ; Wed, 29 Aug 2018 15:59:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=y9y4Vx2iOX2o/hg2UHXLgGMAdsn0DmQc363NJwvUEWs=; b=S64KCH4A6aO1lzVNx7An9GYYI6HevTecQ0vRAK3w7igZ6IZm60/pP5qr7F5K3SHJDA w7A1RUa0I6mEsP5uUBJorwldarPjnRPQVDj9FWRJSp+EpfclE2C4zj85ST8/tOkFHjSZ EnL50I4jRG69Ka2RVAPFLSHOHN2/wHV7pc0zwoq46XT3AwyR8qBfusezPtx+2hKFKjyw pzaeOfUScNLnKwhjJSDXsbaH3N7GdqIJt2Y1VXWoGSnV7a8TDHeKnFjFGlMBNKLWuHR/ DbHaEGC4EsGy+k/pmK1yW+Vz5PHTrfC8BntQQaPqY1yU8GYmLWNfHN0aGgzfjDt6ia+6 bwIw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51Cqi3VWtz4SrzPIZulSjkrqAcMj08R6SQLUq9LIPL50lmF4DcOK IcFvppivYntALFW7eE404xS88E5Z54Y9zb+q3D5UVJZK2JPK7ed0S4mPU3+6OffaNSEbNSSA+Rj Hf4xgtvQLWou7rOL3fidbcmYd7kH6sngDmjS3hlm/2rgbArBMnFvgR7Dj2gzhhn7iGA== X-Received: by 2002:a63:f60:: with SMTP id 32-v6mr7394177pgp.399.1535583586741; Wed, 29 Aug 2018 15:59:46 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYKL1ZAllK55d3FgSICNCLXigEcxSHcbvXpZKjNBNpD3OZRMtIVIa27y/kJS6Uw1Ez7gd5n X-Received: by 2002:a63:f60:: with SMTP id 32-v6mr7394153pgp.399.1535583585825; Wed, 29 Aug 2018 15:59:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535583585; cv=none; d=google.com; s=arc-20160816; b=ddGibFF27gHXJxA2le/qAgmOv29e6zQ+waKWxn7lanc61yh7SyBJf62SPjVKeS/r6R Jx0SywE4T/QEv0jQXDgjAnwv69+rrkkuCMiAibhMsfIgxnqL+PcerlN2Ke5KbTtzTc8A +njqntSgwJW6z3wXIOzZwjc2QTTfIL1DtyYvJTf1dowdq/gUB4rqncxRfRNMrkS+Hmn5 9YgP3IwfYxiT030QLQzDXWErHHApNTxnikHH19bhHrdtLaihRGrpMzEcaNcIzJf7GxfE ESR0chgGrP0rYv1L6VfMIOsY7OREBL4jYyFvdxwLe8pX1bBnTVtA7ONB8JCpR0gcQPYm K45w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=y9y4Vx2iOX2o/hg2UHXLgGMAdsn0DmQc363NJwvUEWs=; b=AdBrEpj86yNYXRWZ/WZdhrTLZHSS691qUA1W4Po+6M8GjAdRb4MI0W6qkPkr+q9lCb T2xvEK4dFghupf9wssfyQWsLPb3djRvAMJz6pe98goDdG4W3GRy2YPeMyfgVhqq6N18d mwdIc5OGYaEvbgs8+CpGQpHCDfF/ttcWrzDj+rJdHcWpThyzURtKzHcQ10xWO2pavXpb 3e5VEw/XiaU4PRKuoU3m7SUCoPsi/UbtCceJdMNtCdouTMH2NVax3YDhGzdF3AsTKsCn 6c+UhvEDUQuk2z5cK6RoTAj+v3ToElO5YsMC6wIEyWtjLixn3riwWt9CbRN8fENFujVa K1jg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id o13-v6si4876523pll.86.2018.08.29.15.59.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 15:59:45 -0700 (PDT) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 15:59:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,305,1531810800"; d="scan'208";a="85609395" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.168]) by fmsmga001.fm.intel.com with ESMTP; 29 Aug 2018 15:59:41 -0700 From: Rick Edgecombe To: tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com, x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-hardening@lists.openwall.com, daniel@iogearbox.net, jannh@google.com, keescook@chromium.org Cc: kristen@linux.intel.com, dave.hansen@intel.com, arjan@linux.intel.com, Rick Edgecombe Subject: [PATCH v4 1/3] vmalloc: Add __vmalloc_node_try_addr function Date: Wed, 29 Aug 2018 15:59:37 -0700 Message-Id: <1535583579-6138-2-git-send-email-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> References: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Create __vmalloc_node_try_addr function that tries to allocate at a specific address and supports caller specified behavior for whether any lazy purging happens if there is a collision. This new function draws from the __vmalloc_node_range implementation. Attempts to merge the two into a single allocator resulted in logic that was difficult to follow, so they are left separate. Signed-off-by: Rick Edgecombe --- include/linux/vmalloc.h | 3 + mm/vmalloc.c | 177 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 179 insertions(+), 1 deletion(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 398e9c9..c7712c8 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -82,6 +82,9 @@ extern void *__vmalloc_node_range(unsigned long size, unsigned long align, unsigned long start, unsigned long end, gfp_t gfp_mask, pgprot_t prot, unsigned long vm_flags, int node, const void *caller); +extern void *__vmalloc_node_try_addr(unsigned long addr, unsigned long size, + gfp_t gfp_mask, pgprot_t prot, unsigned long vm_flags, + int node, int try_purge, const void *caller); #ifndef CONFIG_MMU extern void *__vmalloc_node_flags(unsigned long size, int node, gfp_t flags); static inline void *__vmalloc_node_flags_caller(unsigned long size, int node, diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a728fc4..1954458 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1709,6 +1709,181 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, return NULL; } +static bool pvm_find_next_prev(unsigned long end, + struct vmap_area **pnext, + struct vmap_area **pprev); + +/* Try to allocate a region of KVA of the specified address and size. */ +static struct vmap_area *try_alloc_vmap_area(unsigned long addr, + unsigned long size, int node, gfp_t gfp_mask, + int try_purge) +{ + struct vmap_area *va; + struct vmap_area *cur_va = NULL; + struct vmap_area *first_before = NULL; + int need_purge = 0; + int blocked = 0; + int purged = 0; + unsigned long addr_end; + + WARN_ON(!size); + WARN_ON(offset_in_page(size)); + + addr_end = addr + size; + if (addr > addr_end) + return ERR_PTR(-EOVERFLOW); + + might_sleep(); + + va = kmalloc_node(sizeof(struct vmap_area), + gfp_mask & GFP_RECLAIM_MASK, node); + if (unlikely(!va)) + return ERR_PTR(-ENOMEM); + + /* + * Only scan the relevant parts containing pointers to other objects + * to avoid false negatives. + */ + kmemleak_scan_area(&va->rb_node, SIZE_MAX, gfp_mask & GFP_RECLAIM_MASK); + +retry: + spin_lock(&vmap_area_lock); + + pvm_find_next_prev(addr, &cur_va, &first_before); + + if (!cur_va) + goto found; + + /* + * If there is no VA that starts before the target address, start the + * check from the closest VA in order to cover the case where the + * allocation overlaps at the end. + */ + if (first_before && addr < first_before->va_end) + cur_va = first_before; + + /* Linearly search through to make sure there is a hole */ + while (cur_va->va_start < addr_end) { + if (cur_va->va_end > addr) { + if (cur_va->flags & VM_LAZY_FREE) { + need_purge = 1; + } else { + blocked = 1; + break; + } + } + + if (list_is_last(&cur_va->list, &vmap_area_list)) + break; + + cur_va = list_next_entry(cur_va, list); + } + + /* + * If a non-lazy free va blocks the allocation, or + * we are not supposed to purge, but we need to, the + * allocation fails. + */ + if (blocked || (need_purge && !try_purge)) + goto fail; + + if (try_purge && need_purge) { + /* if purged once before, give up */ + if (purged) + goto fail; + + /* + * If the va blocking the allocation is set to + * be purged then purge all vmap_areas that are + * set to purged since this will flush the TLBs + * anyway. + */ + spin_unlock(&vmap_area_lock); + purge_vmap_area_lazy(); + need_purge = 0; + purged = 1; + goto retry; + } + +found: + va->va_start = addr; + va->va_end = addr_end; + va->flags = 0; + __insert_vmap_area(va); + spin_unlock(&vmap_area_lock); + + return va; +fail: + spin_unlock(&vmap_area_lock); + kfree(va); + if (need_purge && !blocked) + return ERR_PTR(-EUCLEAN); + return ERR_PTR(-EBUSY); +} + +/** + * __vmalloc_try_addr - try to alloc at a specific address + * @addr: address to try + * @size: size to try + * @gfp_mask: flags for the page level allocator + * @prot: protection mask for the allocated pages + * @vm_flags: additional vm area flags (e.g. %VM_NO_GUARD) + * @node: node to use for allocation or NUMA_NO_NODE + * @try_purge: try to purge if needed to fulfill and allocation + * @caller: caller's return address + * + * Try to allocate at the specific address. If it succeeds the address is + * returned. If it fails an EBUSY ERR_PTR is returned. If try_purge is + * zero, it will return an EUCLEAN ERR_PTR if it could have allocated if it + * was allowed to purge. It may trigger TLB flushes if a purge is needed, + * and try_purge is set. + */ +void *__vmalloc_node_try_addr(unsigned long addr, unsigned long size, + gfp_t gfp_mask, pgprot_t prot, unsigned long vm_flags, + int node, int try_purge, const void *caller) +{ + struct vmap_area *va; + struct vm_struct *area; + void *alloc_addr; + unsigned long real_size = size; + + size = PAGE_ALIGN(size); + if (!size || (size >> PAGE_SHIFT) > totalram_pages) + return NULL; + + WARN_ON(in_interrupt()); + + if (!(vm_flags & VM_NO_GUARD)) + size += PAGE_SIZE; + + va = try_alloc_vmap_area(addr, size, node, gfp_mask, try_purge); + if (IS_ERR(va)) + goto fail; + + area = kzalloc_node(sizeof(*area), gfp_mask & GFP_RECLAIM_MASK, node); + if (unlikely(!area)) { + warn_alloc(gfp_mask, NULL, "kmalloc: allocation failure"); + return ERR_PTR(-ENOMEM); + } + + setup_vmalloc_vm(area, va, vm_flags, caller); + + alloc_addr = __vmalloc_area_node(area, gfp_mask, prot, node); + if (!alloc_addr) { + warn_alloc(gfp_mask, NULL, + "vmalloc: allocation failure: %lu bytes", real_size); + return ERR_PTR(-ENOMEM); + } + + clear_vm_uninitialized_flag(area); + + kmemleak_vmalloc(area, real_size, gfp_mask); + + return alloc_addr; +fail: + return va; +} + /** * __vmalloc_node_range - allocate virtually contiguous memory * @size: allocation size @@ -2355,7 +2530,6 @@ void free_vm_area(struct vm_struct *area) } EXPORT_SYMBOL_GPL(free_vm_area); -#ifdef CONFIG_SMP static struct vmap_area *node_to_va(struct rb_node *n) { return rb_entry_safe(n, struct vmap_area, rb_node); @@ -2403,6 +2577,7 @@ static bool pvm_find_next_prev(unsigned long end, return true; } +#ifdef CONFIG_SMP /** * pvm_determine_end - find the highest aligned address between two vmap_areas * @pnext: in/out arg for the next vmap_area From patchwork Wed Aug 29 22:59:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10581041 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 012F8174A for ; Wed, 29 Aug 2018 22:59:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E404F2B157 for ; Wed, 29 Aug 2018 22:59:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D85C12B16D; Wed, 29 Aug 2018 22:59:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A10E2B157 for ; Wed, 29 Aug 2018 22:59:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 102146B4E27; Wed, 29 Aug 2018 18:59:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDCE86B4E2A; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B7CA26B4E28; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 5A4D96B4E27 for ; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id t23-v6so3580456pfe.20 for ; Wed, 29 Aug 2018 15:59:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=QqwWDnXJn8z6+qpoq1dVwrfhWZlisBIiIIAv83FSdxI=; b=Q2ulsukenn84HnhNzlFqLUPZo24sKDUpdmNFQFlm3gSW5BMzJiUuJotk/HcWE5xeFW wYvb9XPNK63L9yYnVYBJpcG5kzmbSyjQD+FzjzaQIeJACkwuqsCoNR794EqtZnzzK+FT 8pTtz0iC2w9+Iwhea3fmMlgZlLAdP/G+Nwrr/V8Lc3rp2d6mG6/wBqxzpAoYTF4hpD9v FhUN+mddu+MnKZlGlZjmC9Zo/kulr/L25Qf0Si1ITkbAcHQgvZ2JSRuE6TEwYSm79iDe TWmpGWRQdHxekX4AiMY88m0/ggwzbTgzy6XAnb/affaRaW4+K8siriU3GHjq7f2JVNDT M+XA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51CE3WVSLF00Kss2mq0TX9blYHNjDrc0u3L5xYOx7NxPbJspsUNg yGdisHNP+exz6QaFp9NC9c7FhO57Ip9PWoj+dUUIAgHlh8wUf3naIWpUq4g0rxks2DzgNY9cOOf sJH3Wye8en6paSDd8rnFHb7VCI+0noEyLbJUBJLm/fG+jhhGcQl1kh4u+/dZ7vAcUMg== X-Received: by 2002:a63:5904:: with SMTP id n4-v6mr7328755pgb.134.1535583587012; Wed, 29 Aug 2018 15:59:47 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZsz2VUDtIKPP/Elvn2/Q5GgyyeEl/KxUC8qCnBpMuZ1BCox6pfm6+bopnNwIykOivRrao6 X-Received: by 2002:a63:5904:: with SMTP id n4-v6mr7328728pgb.134.1535583586012; Wed, 29 Aug 2018 15:59:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535583585; cv=none; d=google.com; s=arc-20160816; b=Dk7xsxZiUzJP7JZqAr0g5C6aQFAUXtxwTAK0JaT2epwsSRhrVqFcXHicud0WLVlroP bfnXIXte3PWMwpFNkNdR/3atuAlfILrbTFx31+u89wN2omo74i4QJq/MwMz7rwozVouM w3CXXBm6/T8mdRgEqz7XFwFjZiEk1g2hHINQUxCRdwYfK9SEymnlSnEy0Yl5RfLPVL46 X4AerXgfEPuFnFd0+TqFd7F67UV2Amc48JJxDMnfV6QJRg3x1lvA+UguGk8jS4P+wflz Pfr8rCwYuteYiF5AgKz1Ft/DRj8uK0gJOULUGClNt0RvDzAF5hDRuxS3BbhKlFrAS/uM wiIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=QqwWDnXJn8z6+qpoq1dVwrfhWZlisBIiIIAv83FSdxI=; b=Oq0qXgrHnJhmipqBYn6Jbt/MnM9cUUr8e+sR/wnB6d8uYWepZaTr2ls8+n8VJOM9F4 Fv8fiJiPdWpjX864asnHm+wgJqOeT3hLriuk01+EqJlDjRvw9I/69RKVy1e0TLsZve5M m7t6j58l9e4cnrqh+X84e6Xa/MrPatByfZF5v8CEqvnP0p3M8UXsTblts1dk5zrqAjdc ykCgjfzXcnaW1LSvzyfZ3xAHHyHwxrAZju5EJCtFQjlX66gn96mSUbGpoKVwkhNyTa41 Au17H29pwrRW2mTwdI8vUWknhBbkvBe/OKyaAnOj2xfqc2fQO9lI4fOr32BAUxp+4iQc 0xzQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id o13-v6si4876523pll.86.2018.08.29.15.59.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 15:59:45 -0700 (PDT) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 15:59:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,305,1531810800"; d="scan'208";a="85609398" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.168]) by fmsmga001.fm.intel.com with ESMTP; 29 Aug 2018 15:59:42 -0700 From: Rick Edgecombe To: tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com, x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-hardening@lists.openwall.com, daniel@iogearbox.net, jannh@google.com, keescook@chromium.org Cc: kristen@linux.intel.com, dave.hansen@intel.com, arjan@linux.intel.com, Rick Edgecombe Subject: [PATCH v4 2/3] x86/modules: Increase randomization for modules Date: Wed, 29 Aug 2018 15:59:38 -0700 Message-Id: <1535583579-6138-3-git-send-email-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> References: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This changes the behavior of the KASLR logic for allocating memory for the text sections of loadable modules. It randomizes the location of each module text section with about 17 bits of entropy in typical use. This is enabled on X86_64 only. For 32 bit, the behavior is unchanged. It refactors existing code around module randomization somewhat. There are now three different behaviors for x86 module_alloc depending on config. RANDOMIZE_BASE=n, and RANDOMIZE_BASE=y ARCH=x86_64, and RANDOMIZE_BASE=y ARCH=i386. The refactor of the existing code is to try to clearly show what those behaviors are without having three separate versions or threading the behaviors in a bunch of little spots. The reason it is not enabled on 32 bit yet is because the module space is much smaller and simulations haven't been run to see how it performs. The new algorithm breaks the module space in two, a random area and a backup area. It first tries to allocate at a number of randomly located starting pages inside the random section without purging any lazy free vmap areas and triggering the associated TLB flush. If this fails, it will try again a number of times allowing for purges if needed. It also saves any position that could have succeeded if it was allowed to purge, which doubles the chances of finding a spot that would fit. Finally if those both fail to find a position it will allocate in the backup area. The backup area base will be offset in the same way as the current algorithm does for the base area, 1024 possible locations. Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/pgtable_64_types.h | 7 ++ arch/x86/kernel/module.c | 165 +++++++++++++++++++++++++++----- 2 files changed, 149 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 04edd2d..5e26369 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -143,6 +143,13 @@ extern unsigned int ptrs_per_p4d; #define MODULES_END _AC(0xffffffffff000000, UL) #define MODULES_LEN (MODULES_END - MODULES_VADDR) +/* + * Dedicate the first part of the module space to a randomized area when KASLR + * is in use. Leave the remaining part for a fallback if we are unable to + * allocate in the random area. + */ +#define MODULES_RAND_LEN PAGE_ALIGN((MODULES_LEN/3)*2) + #define ESPFIX_PGD_ENTRY _AC(-2, UL) #define ESPFIX_BASE_ADDR (ESPFIX_PGD_ENTRY << P4D_SHIFT) diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index f58336a..d50a0a0 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -48,34 +48,151 @@ do { \ } while (0) #endif -#ifdef CONFIG_RANDOMIZE_BASE +#if defined(CONFIG_X86_64) && defined(CONFIG_RANDOMIZE_BASE) +static inline unsigned long get_modules_rand_len(void) +{ + return MODULES_RAND_LEN; +} +#else +static inline unsigned long get_modules_rand_len(void) +{ + BUILD_BUG(); + return 0; +} + +inline bool kaslr_enabled(void); +#endif + +static inline int kaslr_randomize_each_module(void) +{ + return IS_ENABLED(CONFIG_RANDOMIZE_BASE) + && IS_ENABLED(CONFIG_X86_64) + && kaslr_enabled(); +} + +static inline int kaslr_randomize_base(void) +{ + return IS_ENABLED(CONFIG_RANDOMIZE_BASE) + && !IS_ENABLED(CONFIG_X86_64) + && kaslr_enabled(); +} + static unsigned long module_load_offset; +static const unsigned long NR_NO_PURGE = 5000; +static const unsigned long NR_TRY_PURGE = 5000; /* Mutex protects the module_load_offset. */ static DEFINE_MUTEX(module_kaslr_mutex); static unsigned long int get_module_load_offset(void) { - if (kaslr_enabled()) { - mutex_lock(&module_kaslr_mutex); - /* - * Calculate the module_load_offset the first time this - * code is called. Once calculated it stays the same until - * reboot. - */ - if (module_load_offset == 0) - module_load_offset = - (get_random_int() % 1024 + 1) * PAGE_SIZE; - mutex_unlock(&module_kaslr_mutex); - } + mutex_lock(&module_kaslr_mutex); + /* + * Calculate the module_load_offset the first time this + * code is called. Once calculated it stays the same until + * reboot. + */ + if (module_load_offset == 0) + module_load_offset = (get_random_int() % 1024 + 1) * PAGE_SIZE; + mutex_unlock(&module_kaslr_mutex); + return module_load_offset; } -#else -static unsigned long int get_module_load_offset(void) + +static unsigned long get_module_vmalloc_start(void) { - return 0; + if (kaslr_randomize_each_module()) + return MODULES_VADDR + get_modules_rand_len() + + get_module_load_offset(); + else if (kaslr_randomize_base()) + return MODULES_VADDR + get_module_load_offset(); + + return MODULES_VADDR; +} + +static void *try_module_alloc(unsigned long addr, unsigned long size, + int try_purge) +{ + const unsigned long vm_flags = 0; + + return __vmalloc_node_try_addr(addr, size, GFP_KERNEL, PAGE_KERNEL_EXEC, + vm_flags, NUMA_NO_NODE, try_purge, + __builtin_return_address(0)); +} + +/* + * Find a random address to try that won't obviously not fit. Random areas are + * allowed to overflow into the backup area + */ +static unsigned long get_rand_module_addr(unsigned long size) +{ + unsigned long nr_max_pos = (MODULES_LEN - size) / MODULE_ALIGN + 1; + unsigned long nr_rnd_pos = get_modules_rand_len() / MODULE_ALIGN; + unsigned long nr_pos = min(nr_max_pos, nr_rnd_pos); + + unsigned long module_position_nr = get_random_long() % nr_pos; + unsigned long offset = module_position_nr * MODULE_ALIGN; + + return MODULES_VADDR + offset; +} + +/* + * Try to allocate in the random area. First 5000 times without purging, then + * 5000 times with purging. If these fail, return NULL. + */ +static void *try_module_randomize_each(unsigned long size) +{ + void *p = NULL; + unsigned int i; + unsigned long last_lazy_free_blocked = 0; + + /* This will have a guard page */ + unsigned long va_size = PAGE_ALIGN(size) + PAGE_SIZE; + + if (!kaslr_randomize_each_module()) + return NULL; + + /* Make sure there is at least one address that might fit. */ + if (va_size < PAGE_ALIGN(size) || va_size > MODULES_LEN) + return NULL; + + /* Try to find a spot that doesn't need a lazy purge */ + for (i = 0; i < NR_NO_PURGE; i++) { + unsigned long addr = get_rand_module_addr(va_size); + + /* First try to avoid having to purge */ + p = try_module_alloc(addr, size, 0); + + /* + * Save the last value that was blocked by a + * lazy purge area. + */ + if (IS_ERR(p) && PTR_ERR(p) == -EUCLEAN) + last_lazy_free_blocked = addr; + else if (!IS_ERR(p)) + return p; + } + + /* Try the most recent spot that could be used after a lazy purge */ + if (last_lazy_free_blocked) { + p = try_module_alloc(last_lazy_free_blocked, size, 1); + + if (!IS_ERR(p)) + return p; + } + + /* Look for more spots and allow lazy purges */ + for (i = 0; i < NR_TRY_PURGE; i++) { + unsigned long addr = get_rand_module_addr(va_size); + + /* Give up and allow for purges */ + p = try_module_alloc(addr, size, 1); + + if (!IS_ERR(p)) + return p; + } + return NULL; } -#endif void *module_alloc(unsigned long size) { @@ -84,16 +201,18 @@ void *module_alloc(unsigned long size) if (PAGE_ALIGN(size) > MODULES_LEN) return NULL; - p = __vmalloc_node_range(size, MODULE_ALIGN, - MODULES_VADDR + get_module_load_offset(), - MODULES_END, GFP_KERNEL, - PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, - __builtin_return_address(0)); + p = try_module_randomize_each(size); + + if (!p) + p = __vmalloc_node_range(size, MODULE_ALIGN, + get_module_vmalloc_start(), MODULES_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, 0, + NUMA_NO_NODE, __builtin_return_address(0)); + if (p && (kasan_module_alloc(p, size) < 0)) { vfree(p); return NULL; } - return p; } From patchwork Wed Aug 29 22:59:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10581039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4CB335A4 for ; Wed, 29 Aug 2018 22:59:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C8FA2B157 for ; Wed, 29 Aug 2018 22:59:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 30D9C2B16D; Wed, 29 Aug 2018 22:59:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE9162B157 for ; Wed, 29 Aug 2018 22:59:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEC236B4E26; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C42396B4E27; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE44F6B4E29; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 46A186B4E26 for ; Wed, 29 Aug 2018 18:59:47 -0400 (EDT) Received: by mail-pf1-f199.google.com with SMTP id e15-v6so3618910pfi.5 for ; Wed, 29 Aug 2018 15:59:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Lmk7RJ4HAcUTrVlTM0qUi0H/bFfvBm4kuPmUSOALSlM=; b=hLc0RbD+q8QLHs+36mLDa6fqJ94HASWeBJPswc4ajRcU5e0maqRb9hdAbaSjLTdQmK MhwOrgH+mzwQ571rkhDqfAVdp29ZuLJ6qffjrhxS64ZlV0ZRkreHaB0QS6x4Ib+bCxWb ju3YLAxb4Zax3e5frqj5//1HzjetZYld2PCQh2l5GuHuLgRXAg1QAzC/+exI/kv9ZwXd 1J/3Qsc9tjZdC0STto3LHlWyGxPRM5h5nQrAPC9obJ7vZRjNSia3xmS/JQF1afRdE9iV 7cZMfC4P3efl6I8A3zlLqLij0fsubGq9IaAfdQ4qECNuxRN1VzJWoS9aHaAzfpoqcdyZ 40RQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51AExnQYqUh3QtzwL9flUZI2W1s/ypQH7KGAq1IgWsVZRkGiGiLw jp376SVhHoSIpotfj7Y75DLl+gtLjW72UUDvOrDmgjUZZniSqL/8LVU9zVbO/SzpdV3KvUO1yj2 OAMerCNG0Iutya4g6lcWhNaoIipgq/OodtO9jk6mBHwbi90vz7maxhS2LM4rz2s0F3w== X-Received: by 2002:a17:902:1121:: with SMTP id d30-v6mr7481180pla.250.1535583586940; Wed, 29 Aug 2018 15:59:46 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYhWmBUbgIlSxyCLODvwY1IM4m2WNJs2CGtZ/dV72sOp7yrzeD7Fg4VfFRu4wrHNjeUyuha X-Received: by 2002:a17:902:1121:: with SMTP id d30-v6mr7481128pla.250.1535583585188; Wed, 29 Aug 2018 15:59:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535583585; cv=none; d=google.com; s=arc-20160816; b=WVCZnpQrOL7ezrWZTMGxt3ZDd4TWKySuNm5LQ1pNQV4vxQJXCLn/jW141amHdVeV4b HHyvSAR7hWQucL6TDcQdPf5Os5bTBW18/CnHP7OGRvL9kBvMdrMsTbxbbZyYpJhDb2YQ tdLA3jSd92X3tICpNywsO1IVnnaB6DaPuQOO42CwgqGIxuwAhkao3hgntQbnSo1k+p1f 0ubpavuxz9Q4IAmtzuTv8t2DOupTr1BdoN1hVWbFIY7i5QNcwF9W+SQP0JtBFjbkCmCS KeGRgXmufSZRY0PJ3ElYOo7yohntlnFaL056vwrhCWAcUe41WeKfUuOeMmDGk6NAFcAi BnJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=Lmk7RJ4HAcUTrVlTM0qUi0H/bFfvBm4kuPmUSOALSlM=; b=HAI3rQjc8DwDu/TmI7Ghrs7ULHCq9SFqdTes0TAyeFTWLu5+GI7fKhN0NY9GpICHgs IM3MKlKJLS45yNREynsK6+GFishSh4jz6K4b9pNzS5pp1+xVn1cWHuT+9oSxMNCPHVUu AYFhCqh16oSCqefo9R6tKIq4ufL0kLMkqT9qXUPLDh1SZKf/MGB5H6LayZ4jHtoDKMKJ saD49E2d1B5sHtTdBVLqy6Kge7zgDJV++ngxl3ucxSxM2pEJsaP6X8g0ZYOPimNAzGRC wCtZSTXb5OxQcIRGthZILZyD/uxRKCCfxrRe8iXLwLOX0WLUTG1h6nKvOe7AoKfmjpIe Wr/w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga06.intel.com (mga06.intel.com. [134.134.136.31]) by mx.google.com with ESMTPS id v11-v6si5026798pgg.303.2018.08.29.15.59.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 15:59:45 -0700 (PDT) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.31 as permitted sender) client-ip=134.134.136.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 15:59:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,305,1531810800"; d="scan'208";a="85609402" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.168]) by fmsmga001.fm.intel.com with ESMTP; 29 Aug 2018 15:59:42 -0700 From: Rick Edgecombe To: tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com, x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-hardening@lists.openwall.com, daniel@iogearbox.net, jannh@google.com, keescook@chromium.org Cc: kristen@linux.intel.com, dave.hansen@intel.com, arjan@linux.intel.com, Rick Edgecombe Subject: [PATCH v4 3/3] vmalloc: Add debugfs modfraginfo Date: Wed, 29 Aug 2018 15:59:39 -0700 Message-Id: <1535583579-6138-4-git-send-email-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> References: <1535583579-6138-1-git-send-email-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Add debugfs file "modfraginfo" for providing info on module space fragmentation. This can be used for determining if loadable module randomization is causing any problems for extreme module loading situations, like huge numbers of modules or extremely large modules. Sample output when KASLR is enabled and X86_64 is configured: Largest free space: 897912 kB Total free space: 1025424 kB Allocations in backup area: 0 Sample output when just X86_64: Largest free space: 897912 kB Total free space: 1025424 kB Signed-off-by: Rick Edgecombe --- mm/vmalloc.c | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 1954458..a44b902 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -33,6 +34,7 @@ #include #include +#include #include #include @@ -2919,7 +2921,105 @@ static int __init proc_vmalloc_init(void) proc_create_seq("vmallocinfo", 0400, NULL, &vmalloc_op); return 0; } -module_init(proc_vmalloc_init); +#else +static int __init proc_vmalloc_init(void) +{ + return 0; +} +#endif + +#if defined(CONFIG_RANDOMIZE_BASE) && defined(CONFIG_X86_64) +static inline unsigned long is_in_backup(unsigned long addr) +{ + return addr >= MODULES_VADDR + MODULES_RAND_LEN; +} +#else +static inline unsigned long is_in_backup(unsigned long addr) +{ + return 0; +} +inline bool kaslr_enabled(void); #endif + +#if defined(CONFIG_DEBUG_FS) && defined(CONFIG_X86_64) +static int modulefraginfo_debug_show(struct seq_file *m, void *v) +{ + unsigned long last_end = MODULES_VADDR; + unsigned long total_free = 0; + unsigned long largest_free = 0; + unsigned long backup_cnt = 0; + unsigned long gap; + struct vmap_area *prev, *cur = NULL; + + spin_lock(&vmap_area_lock); + + if (!pvm_find_next_prev(MODULES_VADDR, &cur, &prev) || !cur) + goto done; + + for (; cur->va_end <= MODULES_END; cur = list_next_entry(cur, list)) { + /* Don't count areas that are marked to be lazily freed */ + if (!(cur->flags & VM_LAZY_FREE)) { + backup_cnt += is_in_backup(cur->va_start); + gap = cur->va_start - last_end; + if (gap > largest_free) + largest_free = gap; + total_free += gap; + last_end = cur->va_end; + } + + if (list_is_last(&cur->list, &vmap_area_list)) + break; + } + +done: + gap = (MODULES_END - last_end); + if (gap > largest_free) + largest_free = gap; + total_free += gap; + + spin_unlock(&vmap_area_lock); + + seq_printf(m, "\tLargest free space:\t%lu kB\n", largest_free / 1024); + seq_printf(m, "\t Total free space:\t%lu kB\n", total_free / 1024); + + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && kaslr_enabled()) + seq_printf(m, "Allocations in backup area:\t%lu\n", backup_cnt); + + return 0; +} + +static int proc_module_frag_debug_open(struct inode *inode, struct file *file) +{ + return single_open(file, modulefraginfo_debug_show, NULL); +} + +static const struct file_operations debug_module_frag_operations = { + .open = proc_module_frag_debug_open, + .read = seq_read, + .llseek = seq_lseek, + .release = single_release, +}; + +static void __init debug_modfrag_init(void) +{ + debugfs_create_file("modfraginfo", 0400, NULL, NULL, + &debug_module_frag_operations); +} +#else /* defined(CONFIG_DEBUG_FS) && defined(CONFIG_X86_64) */ +static void __init debug_modfrag_init(void) +{ +} +#endif + +#if defined(CONFIG_DEBUG_FS) || defined(CONFIG_PROC_FS) +static int __init info_vmalloc_init(void) +{ + proc_vmalloc_init(); + debug_modfrag_init(); + return 0; +} + +module_init(info_vmalloc_init); +#endif