From patchwork Fri May 28 17:35:50 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Magenheimer X-Patchwork-Id: 102931 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.3/8.14.3) with ESMTP id o4SHcEod029556 for ; Fri, 28 May 2010 17:38:15 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932927Ab0E1RiC (ORCPT ); Fri, 28 May 2010 13:38:02 -0400 Received: from rcsinet10.oracle.com ([148.87.113.121]:54272 "EHLO rcsinet10.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758638Ab0E1RhN (ORCPT ); Fri, 28 May 2010 13:37:13 -0400 Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by rcsinet10.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4SHachF003324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 28 May 2010 17:36:40 GMT Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4SHaakg032708; Fri, 28 May 2010 17:36:36 GMT Received: from ca-server1.us.oracle.com by acsmt354.oracle.com with ESMTP id 278043731275068152; Fri, 28 May 2010 10:35:52 -0700 Received: from dmagenhe by ca-server1.us.oracle.com with local (Exim 4.69) (envelope-from ) id 1OI3TK-0003BB-NQ; Fri, 28 May 2010 10:35:50 -0700 Date: Fri, 28 May 2010 10:35:50 -0700 From: Dan Magenheimer To: chris.mason@oracle.com, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, adilger@sun.com, tytso@mit.edu, mfasheh@suse.com, joel.becker@oracle.com, matthew@wil.cx, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, ocfs2-devel@oss.oracle.com, linux-mm@kvack.org, ngupta@vflare.org, jeremy@goop.org, JBeulich@novell.com, kurt.hackel@oracle.com, npiggin@suse.de, dave.mccracken@oracle.com, riel@redhat.com, avi@redhat.com, konrad.wilk@oracle.com, dan.magenheimer@oracle.com Subject: [PATCH V2 2/7] Cleancache (was Transcendent Memory): core files Message-ID: <20100528173550.GA12219@ca-server1.us.oracle.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-11) X-Auth-Type: Internal IP X-Source-IP: rcsinet13.oracle.com [148.87.113.125] X-CT-RefId: str=0001.0A090203.4BFFFF2B.00A1:SCFMA4539811,ss=1,fgs=0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter.kernel.org [140.211.167.41]); Fri, 28 May 2010 17:38:15 +0000 (UTC) --- linux-2.6.34/include/linux/cleancache.h 1969-12-31 17:00:00.000000000 -0700 +++ linux-2.6.34-cleancache/include/linux/cleancache.h 2010-05-24 18:14:33.000000000 -0600 @@ -0,0 +1,90 @@ +#ifndef _LINUX_CLEANCACHE_H +#define _LINUX_CLEANCACHE_H + +#include +#include + +#define CLEANCACHE_GET_PAGE_SUCCESS 1 + +struct cleancache_ops { + int (*init_fs)(size_t); + int (*init_shared_fs)(char *uuid, size_t); + int (*get_page)(int, ino_t, pgoff_t, struct page *); + int (*put_page)(int, ino_t, pgoff_t, struct page *); + int (*flush_page)(int, ino_t, pgoff_t); + int (*flush_inode)(int, ino_t); + void (*flush_fs)(int); +}; + +extern struct cleancache_ops *cleancache_ops; +extern int __cleancache_get_page(struct page *); +extern int __cleancache_put_page(struct page *); +extern int __cleancache_flush_page(struct address_space *, struct page *); +extern int __cleancache_flush_inode(struct address_space *); + +#ifndef CONFIG_CLEANCACHE +#define cleancache_ops ((struct cleancache_ops *)NULL) +#endif + +static inline int cleancache_init_fs(size_t pagesize) +{ + int ret = -1; + + if (cleancache_ops) + ret = (*cleancache_ops->init_fs)(pagesize); + return ret; +} + +static inline int cleancache_init_shared_fs(char *uuid, size_t pagesize) +{ + int ret = -1; + + if (cleancache_ops) + ret = (*cleancache_ops->init_shared_fs)(uuid, pagesize); + return ret; +} + +static inline int cleancache_get_page(struct page *page) +{ + int ret = 0; + + if (cleancache_ops) + ret = __cleancache_get_page(page); + return ret; +} + +static inline int cleancache_put_page(struct page *page) +{ + int ret = 0; + + if (cleancache_ops) + ret = __cleancache_put_page(page); + return ret; +} + +static inline int cleancache_flush_page(struct address_space *mapping, + struct page *page) +{ + int ret = 0; + + if (cleancache_ops) + ret = __cleancache_flush_page(mapping, page); + return ret; +} + +static inline int cleancache_flush_inode(struct address_space *mapping) +{ + int ret = 0; + + if (cleancache_ops) + ret = __cleancache_flush_inode(mapping); + return ret; +} + +static inline void cleancache_flush_fs(int pool_id) +{ + if (cleancache_ops && pool_id >= 0) + (*cleancache_ops->flush_fs)(pool_id); +} + +#endif /* _LINUX_CLEANCACHE_H */ --- linux-2.6.34/mm/cleancache.c 1969-12-31 17:00:00.000000000 -0700 +++ linux-2.6.34-cleancache/mm/cleancache.c 2010-05-24 18:07:11.000000000 -0600 @@ -0,0 +1,203 @@ +/* mm/cleancache.c + + Copyright (C) 2009-2010 Oracle Corp. All rights reserved. + Author: Dan Magenheimer + + Cleancache can be thought of as a page-granularity victim cache for clean + pages that the kernel's pageframe replacement algorithm (PFRA) would like + to keep around, but can't since there isn't enough memory. So when the + PFRA "evicts" a page, it first attempts to put it into a synchronous + concurrency-safe page-oriented pseudo-RAM device (such as Xen's Transcendent + Memory, aka "tmem", or in-kernel compressed memory, aka "zmem", or other + RAM-like devices) which is not directly accessible or addressable by the + kernel and is of unknown and possibly time-varying size. And when a + cleancache-enabled filesystem wishes to access a page in a file on disk, + it first checks cleancache to see if it already contains it; if it does, + the page is copied into the kernel and a disk access is avoided. + This pseudo-RAM device links itself to cleancache by setting the + cleancache_ops pointer appropriately and the functions it provides must + conform to certain semantics as follows: + + Most important, cleancache is "ephemeral". Pages which are copied into + cleancache have an indefinite lifetime which is completely unknowable + by the kernel and so may or may not still be in cleancache at any later time. + Thus, as its name implies, cleancache is not suitable for dirty pages. The + pseudo-RAM has complete discretion over what pages to preserve and what + pages to discard and when. + + A filesystem calls "init_fs" to obtain a pool id which, if positive, must be + saved in the filesystem's superblock; a negative return value indicates + failure. A "put_page" will copy a (presumably about-to-be-evicted) page into + pseudo-RAM and associate it with the pool id, the file inode, and a page + index into the file. (The combination of a pool id, an inode, and an index + is called a "handle".) A "get_page" will copy the page, if found, from + pseudo-RAM into kernel memory. A "flush_page" will ensure the page no longer + is present in pseudo-RAM; a "flush_inode" will flush all pages associated + with the specified inode; and a "flush_fs" will flush all pages in all + inodes specified by the given pool id. + + A "init_shared_fs", like init, obtains a pool id but tells the pseudo-RAM + to treat the pool as shared using a 128-bit UUID as a key. On systems + that may run multiple kernels (such as hard partitioned or virtualized + systems) that may share a clustered filesystem, and where the pseudo-RAM + may be shared among those kernels, calls to init_shared_fs that specify the + same UUID will receive the same pool id, thus allowing the pages to + be shared. Note that any security requirements must be imposed outside + of the kernel (e.g. by "tools" that control the pseudo-RAM). Or a + pseudo-RAM implementation can simply disable shared_init by always + returning a negative value. + + If a get_page is successful on a non-shared pool, the page is flushed (thus + making cleancache an "exclusive" cache). On a shared pool, the page + is NOT flushed on a successful get_page so that it remains accessible to + other sharers. The kernel is responsible for ensuring coherency between + cleancache (shared or not), the page cache, and the filesystem, using + cleancache flush operations as required. + + Note that the pseudo-RAM must enforce put-put-get coherency and get-get + coherency. For the former, if two puts are made to the same handle but + with different data, say AAA by the first put and BBB by the second, a + subsequent get can never return the stale data (AAA). For get-get coherency, + if a get for a given handle fails, subsequent gets for that handle will + never succeed unless preceded by a successful put with that handle. + + Last, pseudo-RAM provides no SMP serialization guarantees; if two + different Linux threads are putting an flushing a page with the same + handle, the results are indeterminate. + + */ + +#include +#include +#include +#include + +struct cleancache_ops *cleancache_ops; +EXPORT_SYMBOL(cleancache_ops); + +/* useful stats available via /sys/kernel/mm/frontswap */ +static unsigned long succ_gets; +static unsigned long failed_gets; +static unsigned long puts; +static unsigned long flushes; + +int __cleancache_get_page(struct page *page) +{ + int ret = 0; + int pool_id = page->mapping->host->i_sb->cleancache_poolid; + + if (pool_id >= 0) { + ret = (*cleancache_ops->get_page)(pool_id, + page->mapping->host->i_ino, + page->index, + page); + if (ret == CLEANCACHE_GET_PAGE_SUCCESS) + succ_gets++; + else + failed_gets++; + } + return ret; +} +EXPORT_SYMBOL(__cleancache_get_page); + +int __cleancache_put_page(struct page *page) +{ + int ret = 0; + int pool_id = page->mapping->host->i_sb->cleancache_poolid; + + if (pool_id >= 0) { + ret = (*cleancache_ops->put_page)(pool_id, + page->mapping->host->i_ino, + page->index, + page); + puts++; + } + return ret; +} + +int __cleancache_flush_page(struct address_space *mapping, struct page *page) +{ + int ret = 0; + int pool_id = mapping->host->i_sb->cleancache_poolid; + + if (pool_id >= 0) { + ret = (*cleancache_ops->flush_page)(pool_id, + mapping->host->i_ino, + page->index); + flushes++; + } + return ret; +} +EXPORT_SYMBOL(__cleancache_flush_page); + +int __cleancache_flush_inode(struct address_space *mapping) +{ + int ret = 0; + int pool_id = mapping->host->i_sb->cleancache_poolid; + + if (pool_id >= 0) { + ret = (*cleancache_ops->flush_inode)(pool_id, + mapping->host->i_ino); + } + return ret; +} +EXPORT_SYMBOL(__cleancache_flush_inode); + +#ifdef CONFIG_SYSFS + +#define CLEANCACHE_ATTR_RO(_name) \ + static struct kobj_attribute _name##_attr = __ATTR_RO(_name) + +static ssize_t succ_gets_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sprintf(buf, "%lu\n", succ_gets); +} +CLEANCACHE_ATTR_RO(succ_gets); + +static ssize_t failed_gets_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sprintf(buf, "%lu\n", failed_gets); +} +CLEANCACHE_ATTR_RO(failed_gets); + +static ssize_t puts_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sprintf(buf, "%lu\n", puts); +} +CLEANCACHE_ATTR_RO(puts); + +static ssize_t flushes_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sprintf(buf, "%lu\n", flushes); +} +CLEANCACHE_ATTR_RO(flushes); + +static struct attribute *cleancache_attrs[] = { + &succ_gets_attr.attr, + &failed_gets_attr.attr, + &puts_attr.attr, + &flushes_attr.attr, + NULL, +}; + +static struct attribute_group cleancache_attr_group = { + .attrs = cleancache_attrs, + .name = "cleancache", +}; + +#endif /* CONFIG_SYSFS */ + +static int __init init_cleancache(void) +{ +#ifdef CONFIG_SYSFS + int err; + + err = sysfs_create_group(mm_kobj, &cleancache_attr_group); +#endif /* CONFIG_SYSFS */ + return 0; +} +module_init(init_cleancache) --- linux-2.6.34/mm/Kconfig 2010-05-16 15:17:36.000000000 -0600 +++ linux-2.6.34-cleancache/mm/Kconfig 2010-05-24 12:14:44.000000000 -0600 @@ -287,3 +287,25 @@ config NOMMU_INITIAL_TRIM_EXCESS of 1 says that all excess pages should be trimmed. See Documentation/nommu-mmap.txt for more information. + +config CLEANCACHE + bool "Enable cleancache pseudo-RAM driver to cache clean pages" + default y + help + Cleancache can be thought of as a page-granularity victim cache + for clean pages that the kernel's pageframe replacement algorithm + (PFRA) would like to keep around, but can't since there isn't enough + memory. So when the PFRA "evicts" a page, it first attempts to put + it into a synchronous concurrency-safe page-oriented pseudo-RAM + device (such as Xen's Transcendent Memory, aka "tmem") which is not + directly accessible or addressable by the kernel and is of unknown + (and possibly time-varying) size. And when a cleancache-enabled + filesystem wishes to access a page in a file on disk, it first + checks cleancache to see if it already contains it; if it does, + the page is copied into the kernel and a disk access is avoided. + When a pseudo-RAM device is available, a significant I/O reduction + may be achieved. When none is available, all cleancache calls + are reduced to a single pointer-compare-against-NULL resulting + in a negligible performance hit. + + If unsure, say Y to enable cleancache --- linux-2.6.34/mm/Makefile 2010-05-16 15:17:36.000000000 -0600 +++ linux-2.6.34-cleancache/mm/Makefile 2010-05-24 12:14:44.000000000 -0600 @@ -44,3 +44,4 @@ obj-$(CONFIG_MEMORY_FAILURE) += memory-f obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak-test.o +obj-$(CONFIG_CLEANCACHE) += cleancache.o