From patchwork Mon Aug 29 17:10:06 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Butsykin X-Patchwork-Id: 9304449 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B7C7C60756 for ; Mon, 29 Aug 2016 20:45:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8A602876F for ; Mon, 29 Aug 2016 20:45:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B24B28942; Mon, 29 Aug 2016 20:45:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8DD522876F for ; Mon, 29 Aug 2016 20:45:45 +0000 (UTC) Received: from localhost ([::1]:45695 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTR9-0008GM-HR for patchwork-qemu-devel@patchwork.kernel.org; Mon, 29 Aug 2016 16:45:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59694) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTQl-0008EA-2O for qemu-devel@nongnu.org; Mon, 29 Aug 2016 16:45:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1beTQj-00076O-Po for qemu-devel@nongnu.org; Mon, 29 Aug 2016 16:45:19 -0400 Received: from mail-eopbgr30122.outbound.protection.outlook.com ([40.107.3.122]:29504 helo=EUR03-AM5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTQg-00075w-Kj; Mon, 29 Aug 2016 16:45:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=D92OGGPdxuGUANo6vq8CIcitX0tIPe7qcpNxU7FzOI8=; b=FuigeDIMHe8tNf1eaaOT8nIPdJk3VQuPlp/iL6ARS6WIisTbsS/n9i5rqe6pg5vRaMDiBWAogupVs8sRGLBr13Z4kiL+CK1RpELQxfB1/KzqJvb2rE3gkRuZwlznDFtyVSd7iE1set2npiQhvisym26pN5qnxQAPBsYGk4U9YII= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=pbutsykin@virtuozzo.com; Received: from pavelb-Z68P-DS3.sw.ru (195.214.232.10) by AM5PR0802MB2547.eurprd08.prod.outlook.com (10.175.45.23) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.13; Mon, 29 Aug 2016 17:11:20 +0000 From: Pavel Butsykin To: , Date: Mon, 29 Aug 2016 20:10:06 +0300 Message-ID: <20160829171021.4902-8-pbutsykin@virtuozzo.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20160829171021.4902-1-pbutsykin@virtuozzo.com> References: <20160829171021.4902-1-pbutsykin@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM5PR0901CA0011.eurprd09.prod.outlook.com (10.164.186.149) To AM5PR0802MB2547.eurprd08.prod.outlook.com (10.175.45.23) X-MS-Office365-Filtering-Correlation-Id: 988fc5df-a17e-4d7f-9de4-08d3d02f7f8c X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 2:dRDFxacd90li89Tf706smpW75vtUYPONnRFKrphZojfdkBwQrNxuB46yW0sECHjABZxn8I16EAcEiZTpASsfpyZhA7fN3YxbVQJSsuOSvu9l9OjfE4zenqUZrXcyzO2vh9mxklJbHXxH9EhYVRjIHmlpctKY79/lyrkOAfrt6QtUC97LembDyE0zfk04eHEB; 3:dW2+NzyFiCvv/iBiza6dNaEnJc7CV0xmbjgAdIGH75upCQvN5MxXLRze3r3fVhGkMk6VuveiRZTYAopB6MvH7kOf1dyAygilg1MRR4sQH7d9leiN05LCl/ymGf1E64oS; 25:/JTPjPDzvyxlfkwmceZ80h+576ZVdBaAaGB4gOlGVgLurFQUiTyOt04T7vNg1LEaSfj9UKVfat2azcCLuOkoZ68FZQgvXAFv1y24A1v21eBFZdwCVaXOBpHCep7Ezzg0DNJfe5HLUme8caYeaauvldCbcWjZZYm3cL0cmfZWq009twkQMwX5494+XkKL8tu3JXT25dqMw/8776d4dXuRAnRSJfnmU11EBp2m5DJy1HMsCBb/QFcdqSATxe5ZGtclsU/45GMlT9NaihhpeuObugnunDrBVPbMWljmOVRaDGLmSlBQtP0Tnt7I6zUEZ2W3KmnS8pbfH5kZNGVsVDIzIk1iQIp1ZX0Rzdl+ifHsOpV+kon8fNgIrXBUdz5QbExot1Mie2m6cBQHrUrXb1RjDfFwnfv889x1BfvAOqFP24U= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM5PR0802MB2547; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 31:bdK7pcyObHg2HTassIBY9/pB0XOO/py5IO9PB1aWmYYdOWdkejg4+dNvRwQXfD8XaRS+olZLb42KFH5mBs7s1cRTQh8QnyO9imcVVMigiDbuQXLKMT69AvPiMNpO6JWYyKydesUlyttg1kb8BXWaGybfcjXOUdRmuoXDtlnpKm6dfA9VmcR/elpuPbj8uWA8u1N6l/YJuI3ndEQZp3swhNC26CTFjp3r8LkZJOa7KKM=; 4:zqtKRlNYWnpOGrZ+xlVxhMWj8u74pyDut6LMOMoSIyE20lSf7TixFQoMD8O1dT4Jtli7LnE3ZzskiAMqnJgEIq8H3fwpvhrr/I4LpZsGCosU5FZczeXNHoN0s9+yNcb/516CRgkuqebHdKRlDPJutM/Hja03XD3bpXUoi5tVCGq1XFcAn0W5NiA4iq+76CPbmqLBENvEwNZUUr1qRa+S/fnYN1hTo+5Pd3SSUhoR/wHNrbIJtXNDoqQ1dzCNYSEndOQXpCr0rd4nVTyvzuyxOyCN8/WqtoJWILkxq0pl7l6BzywSK0noRmzrR2qKUXUN+eK8jFBE4nEY8vq4M/inR2x4uHOVEd89cHYdcTDrDDkD7L4wD8toP37/GBvBvMePby7UqQzCWXPhr+Bs9QCiBR0GiFx0GlNbJ35I7f/cHIY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6042046)(6043046); SRVR:AM5PR0802MB2547; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0802MB2547; X-Forefront-PRVS: 0049B3F387 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(7916002)(199003)(189002)(3846002)(305945005)(77096005)(86362001)(1076002)(53416004)(92566002)(68736007)(42186005)(97736004)(5001770100001)(47776003)(105586002)(2906002)(6116002)(189998001)(50986999)(4326007)(48376002)(586003)(2950100001)(5003940100001)(50466002)(66066001)(19580405001)(19580395003)(76176999)(69596002)(5660300001)(229853001)(101416001)(36756003)(7736002)(8676002)(106356001)(50226002)(7846002)(81156014)(33646002)(81166006)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM5PR0802MB2547; H:pavelb-Z68P-DS3.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0802MB2547; 23:Lig0tz5XGaBYY3eoOTMVqA7GBA0Y+TfpUo0pDa4?= =?us-ascii?Q?0TQ762/H+pLcFCHWhSQKfy7ckU29FFDEkFvnZ1vxmNVVL2u4EjDuPgocNvkj?= =?us-ascii?Q?eOOA3qWLaSTwCRMK4/pR15RuRYkzIyEpveWHePc02kfiWn4Sba4v19TDHr3m?= =?us-ascii?Q?4Yy5/XuEg1Za6Xn2eJnrMVIk2UNnyc8+wyPAfBeIfhflyycxLjEBO/+/9Vt1?= =?us-ascii?Q?lvv1XPOoxyST6OIh41HNB91lMGn4sfJAFtPoX8YEVrvFYyn3PNKYn45QTopO?= =?us-ascii?Q?KAf6p0WSBX/GYBj6NqgLWjzM48dQSe1dWGx43Tf7zus6a1ASwVTlWXJDbhgK?= =?us-ascii?Q?b9q91sHFDsKz6RvIg8PMo46Uu5xzLx8QkHqZto6Cz8JT6sCYXZve8J3XqCPW?= =?us-ascii?Q?7kIxl9M4MFpXJXnMPws98TYKpW9gORK7r3ZKJXLB3zun0zxkgaduxhnaBGtW?= =?us-ascii?Q?4B5CGWqrRPriI/68vVDMI60SWr1zXvod0P9m+Oc/B+XoySGJsr+MLsiNBEp/?= =?us-ascii?Q?JpbwicNxvGMJl9oYppnOzzSwDDb6zSmWxgyL5HhoETEcQHHYmkfyDXat+LAf?= =?us-ascii?Q?jBzngaiiTdBTrYf5zF6lyZ1Yms/XWGIayjcKixII/YpOHJJy6kTX4aH0EPC7?= =?us-ascii?Q?Nz54hNxhoupVgDxNWqj7gL11nWSm4Nti8Bqxvtzyg3PWq+BzbsU/CVUUKzBx?= =?us-ascii?Q?JeKo6ntLDq/uA41+EBC/WdmvNxy7WWKb8LxCvDlouuL555lhZAezrd4ZQcoS?= =?us-ascii?Q?ywYetHdwGc/cINI6jGYHctwkLg06Q/MNwTEerQx2HZmLkLarDWvJQTtQm6Jh?= =?us-ascii?Q?KQvKZ8VnGRGZHg/UAAWDaM8MKX/DYAOVZIxm0keQIQ0/6uw/opw4nhUoTZ4l?= =?us-ascii?Q?8kq5KZSChyM//3qqyaNePP+ojc2OTAQ6Ag5pWYxT5zpvpQKP4trl7z6yMNhi?= =?us-ascii?Q?5KDxfMPahAyBG7B4Khl/zsqyAnjhvUq50MF5tnWVfukOZw7EbM/mLEeVaUvu?= =?us-ascii?Q?aWM+ZjmPVhP0vlsluga5Y22jhuuNiir/F2LZNHOa67YtVP9YErME3w5s6RZO?= =?us-ascii?Q?cOBk9yq6oRuVohyK4ty5s1nNhNCeRZArDCA7SxUWjBz32b7sJo4lk7eocBOa?= =?us-ascii?Q?HxsB+wJIB2W2GCvcwLDfj24dmwJ3tfuguaR8p407dAPCW8LLQJEUlEw=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 6:457nzd1213cOtFDE0Zb6GqO3C1rh3vhE5TBw2VNPjDbSnL4VC/JUdLyRmN/yTTSQP704kpcp6YHEbXIl2pR2riDZKkXxlRiE8WPU6RZ2Ug+EF6uSOYyxRByln3R+hWgq1IOl8VShY+gCnBtVLguJGhQtKqmEHya/GuYrFYiFzpOrGAmbvaGkkbHpOGufbfWDI+YSAylEU6WBEQLCKaQzOIFUtKp4xkNxeCz+xPW+1Xz3flKQTgIQ6VFes8JZ/9De4FDHGSawlkso7hUGJtja2KVQsNpN7cX9P1WeBKMOToqEESoCtxFklESogkez75Ap; 5:YLoqqneUJNALjdcSyqdB34XM5hOMQl2v11LakTr0gv0OIoa4Ja4XXWpaz+whBupoTlyXmQDl3JibKJWHSU0Z1tJJMIkhZre0cJ8kj6HF1nX3LSmMlHc0ERbuAFU7JsaD8iLwXvJ5iY4gnlAzyNUBbg==; 24:gD/LG4jhN96IFpuWTYNvZNDXVwGJnaVvEXngTQiafFgsdVwnD9UkKcPwJuxJuFdre+dO9kqIclJl4FJ2ixZhSGHCLhwLIJbamL88IbqLvm4=; 7:+DbAi38MWal6MlI7ulOAcnHHwcldDRwgkGf7V2N8HZLbJC6yhem/3Veuule71PUO7QZfih4TXZq9gMKz10hcv4foFOnj/ZhMzT7cjh/RD9CiO7oy+Se5tLqzrG8erOaPsXr9DU/qOq9NhBcqwR0cBQ+3elz4oFHzctGVKl6OlERwTnDz7wcS2luK549OfKYDwicyPiGvbnjnZFMppE7aa1r74XekNwQS74x+c9hjB5Frmkpi7vHQ9CLFVFFxdkNo SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 20:vvihYQaYleAdf27x72yNoE+FjRFNrO7yYK5RdqjN8q2eq/ccFEFt7EBm0NxYmeX+PY1VM57XSJD7nYcxMbW9wT93OzMIuEDc4EFdDpNEJFZAYpbRTbCrNsCZjlnH6JHnh+HZaTA28NNnYMpMEob8h5b5rVHPq68hI+V1o2vdLBE= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2016 17:11:20.5929 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2547 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.3.122 Subject: [Qemu-devel] [PATCH RFC v2 07/22] block/pcache: introduce LRU as method of memory X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is a simple solution to the problem of displacement of cache memory. The LRU can be useful to avoid the displacement of the nodes, which have been partially read. Signed-off-by: Pavel Butsykin --- block/pcache.c | 74 +++++++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 58 insertions(+), 16 deletions(-) diff --git a/block/pcache.c b/block/pcache.c index 54d4526..7504db8 100644 --- a/block/pcache.c +++ b/block/pcache.c @@ -67,6 +67,11 @@ typedef struct ReqStor { CoMutex lock; } tree; + struct { + QTAILQ_HEAD(lru_head, BlockNode) list; + CoMutex lock; + } lru; + uint32_t curr_size; } ReqStor; @@ -75,12 +80,11 @@ typedef struct BDRVPCacheState { ReqStor pcache; - struct { - QTAILQ_HEAD(pcache_head, BlockNode) head; - CoMutex lock; - } list; - uint32_t cfg_cache_size; + +#ifdef PCACHE_DEBUG + uint64_t shrink_cnt_node; +#endif } BDRVPCacheState; typedef struct PrefCacheAIOCB { @@ -182,6 +186,44 @@ static inline void *pcache_node_alloc(RbNodeKey* key) return node; } +static void pcache_node_drop(BDRVPCacheState *s, PCNode *node) +{ + atomic_sub(&s->pcache.curr_size, node->cm.nb_sectors); + + qemu_co_mutex_lock(&s->pcache.lru.lock); + QTAILQ_REMOVE(&s->pcache.lru.list, &node->cm, entry); + qemu_co_mutex_unlock(&s->pcache.lru.lock); + + qemu_co_mutex_lock(&s->pcache.tree.lock); + rb_erase(&node->cm.rb_node, &s->pcache.tree.root); + qemu_co_mutex_unlock(&s->pcache.tree.lock); + + pcache_node_free(node); +} + +static void pcache_try_shrink(BDRVPCacheState *s) +{ + while (s->pcache.curr_size > s->cfg_cache_size) { + qemu_co_mutex_lock(&s->pcache.lru.lock); + assert(!QTAILQ_EMPTY(&s->pcache.lru.list)); + PCNode *rmv_node = PCNODE(QTAILQ_LAST(&s->pcache.lru.list, lru_head)); + qemu_co_mutex_unlock(&s->pcache.lru.lock); + + pcache_node_drop(s, rmv_node); +#ifdef PCACHE_DEBUG + atomic_inc(&s->shrink_cnt_node); +#endif + } +} + +static inline void pcache_lru_node_up(BDRVPCacheState *s, PCNode *node) +{ + qemu_co_mutex_lock(&s->pcache.lru.lock); + QTAILQ_REMOVE(&s->pcache.lru.list, &node->cm, entry); + QTAILQ_INSERT_HEAD(&s->pcache.lru.list, &node->cm, entry); + qemu_co_mutex_unlock(&s->pcache.lru.lock); +} + static bool pcache_node_find_and_create(PrefCacheAIOCB *acb, RbNodeKey *key, PCNode **out_node) { @@ -194,14 +236,17 @@ static bool pcache_node_find_and_create(PrefCacheAIOCB *acb, RbNodeKey *key, qemu_co_mutex_unlock(&s->pcache.tree.lock); if (found != new_node) { pcache_node_free(new_node); + pcache_lru_node_up(s, found); *out_node = found; return false; } atomic_add(&s->pcache.curr_size, new_node->cm.nb_sectors); - qemu_co_mutex_lock(&s->list.lock); - QTAILQ_INSERT_HEAD(&s->list.head, &new_node->cm, entry); - qemu_co_mutex_unlock(&s->list.lock); + qemu_co_mutex_lock(&s->pcache.lru.lock); + QTAILQ_INSERT_HEAD(&s->pcache.lru.list, &new_node->cm, entry); + qemu_co_mutex_unlock(&s->pcache.lru.lock); + + pcache_try_shrink(s); *out_node = new_node; return true; @@ -275,10 +320,7 @@ static BlockAIOCB *pcache_aio_readv(BlockDriverState *bs, { PrefCacheAIOCB *acb = pcache_aio_get(bs, sector_num, qiov, nb_sectors, cb, opaque, QEMU_AIO_READ); - - if (acb->s->pcache.curr_size < acb->s->cfg_cache_size) { - pcache_prefetch(acb); - } + pcache_prefetch(acb); bdrv_aio_readv(bs->file, sector_num, qiov, nb_sectors, pcache_aio_cb, acb); @@ -309,8 +351,8 @@ static void pcache_state_init(QemuOpts *opts, BDRVPCacheState *s) s->pcache.tree.root = RB_ROOT; qemu_co_mutex_init(&s->pcache.tree.lock); - QTAILQ_INIT(&s->list.head); - qemu_co_mutex_init(&s->list.lock); + QTAILQ_INIT(&s->pcache.lru.list); + qemu_co_mutex_init(&s->pcache.lru.lock); s->pcache.curr_size = 0; s->cfg_cache_size = cache_size >> BDRV_SECTOR_BITS; @@ -350,8 +392,8 @@ static void pcache_close(BlockDriverState *bs) uint32_t cnt = 0; BDRVPCacheState *s = bs->opaque; BlockNode *node, *next; - QTAILQ_FOREACH_SAFE(node, &s->list.head, entry, next) { - QTAILQ_REMOVE(&s->list.head, node, entry); + QTAILQ_FOREACH_SAFE(node, &s->pcache.lru.list, entry, next) { + QTAILQ_REMOVE(&s->pcache.lru.list, node, entry); pcache_node_free(PCNODE(node)); cnt++; }