From patchwork Fri Aug 2 20:30:22 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sage Weil X-Patchwork-Id: 2838171 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id D68A6BF535 for ; Fri, 2 Aug 2013 20:30:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 57B2120326 for ; Fri, 2 Aug 2013 20:30:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2B62B20304 for ; Fri, 2 Aug 2013 20:30:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754178Ab3HBUaY (ORCPT ); Fri, 2 Aug 2013 16:30:24 -0400 Received: from cobra.newdream.net ([66.33.216.30]:49975 "EHLO cobra.newdream.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753701Ab3HBUaX (ORCPT ); Fri, 2 Aug 2013 16:30:23 -0400 Received: from cobra.newdream.net (localhost [127.0.0.1]) by cobra.newdream.net (Postfix) with ESMTP id 1E0D980060; Fri, 2 Aug 2013 13:30:23 -0700 (PDT) Received: by cobra.newdream.net (Postfix, from userid 1031) id F140F80068; Fri, 2 Aug 2013 13:30:22 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by cobra.newdream.net (Postfix) with ESMTP id E2B5280060; Fri, 2 Aug 2013 13:30:22 -0700 (PDT) Date: Fri, 2 Aug 2013 13:30:22 -0700 (PDT) From: Sage Weil X-X-Sender: sage@cobra.newdream.net To: Sha Zhengju cc: "Yan, Zheng" , "linux-fsdevel@vger.kernel.org" , ceph-devel , linux-mm , Cgroups , Michal Hocko , KAMEZAWA Hiroyuki , Glauber Costa , Greg Thelen , Wu Fengguang , Andrew Morton , Sha Zhengju Subject: Re: [PATCH V5 2/8] fs/ceph: vfs __set_page_dirty_nobuffers interface instead of doing it inside filesystem In-Reply-To: Message-ID: References: <1375357402-9811-1-git-send-email-handai.szj@taobao.com> <1375357892-10188-1-git-send-email-handai.szj@taobao.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00,KHOP_BIG_TO_CC, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, 2 Aug 2013, Sha Zhengju wrote: > On Fri, Aug 2, 2013 at 2:27 AM, Sage Weil wrote: > > On Thu, 1 Aug 2013, Yan, Zheng wrote: > >> On Thu, Aug 1, 2013 at 7:51 PM, Sha Zhengju wrote: > >> > From: Sha Zhengju > >> > > >> > Following we will begin to add memcg dirty page accounting around > >> __set_page_dirty_ > >> > {buffers,nobuffers} in vfs layer, so we'd better use vfs interface to > >> avoid exporting > >> > those details to filesystems. > >> > > >> > Signed-off-by: Sha Zhengju > >> > --- > >> > fs/ceph/addr.c | 13 +------------ > >> > 1 file changed, 1 insertion(+), 12 deletions(-) > >> > > >> > diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c > >> > index 3e68ac1..1445bf1 100644 > >> > --- a/fs/ceph/addr.c > >> > +++ b/fs/ceph/addr.c > >> > @@ -76,7 +76,7 @@ static int ceph_set_page_dirty(struct page *page) > >> > if (unlikely(!mapping)) > >> > return !TestSetPageDirty(page); > >> > > >> > - if (TestSetPageDirty(page)) { > >> > + if (!__set_page_dirty_nobuffers(page)) { > >> it's too early to set the radix tree tag here. We should set page's snapshot > >> context and increase the i_wrbuffer_ref first. This is because once the tag > >> is set, writeback thread can find and start flushing the page. > > > > Unfortunately I only remember being frustrated by this code. :) Looking > > at it now, though, it seems like the minimum fix is to set the > > page->private before marking the page dirty. I don't know the locking > > rules around that, though. If that is potentially racy, maybe the safest > > thing would be if __set_page_dirty_nobuffers() took a void* to set > > page->private to atomically while holding the tree_lock. > > > > Sorry, I don't catch the point of your last sentence... Could you > please explain it again? It didn't make much sense. :) I was worried about multiple callers to set_page_dirty, but as understand it, this all happens under page->lock, right? (There is a mention of other special cases in mm/page-writeback.c, but I'm hoping we don't need to worry about that.) In any case, I suspect what we actually want is something like the below (untested) patch. The snapc accounting can be ignored here because invalidatepage will clean it up... sage the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index afb2fc2..7602e46 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -76,9 +76,10 @@ static int ceph_set_page_dirty(struct page *page) if (unlikely(!mapping)) return !TestSetPageDirty(page); - if (TestSetPageDirty(page)) { + if (PageDirty(page)) { dout("%p set_page_dirty %p idx %lu -- already dirty\n", mapping->host, page, page->index); + BUG_ON(!PagePrivate(page)); return 0; } @@ -107,35 +108,16 @@ static int ceph_set_page_dirty(struct page *page) snapc, snapc->seq, snapc->num_snaps); spin_unlock(&ci->i_ceph_lock); - /* now adjust page */ - spin_lock_irq(&mapping->tree_lock); - if (page->mapping) { /* Race with truncate? */ - WARN_ON_ONCE(!PageUptodate(page)); - account_page_dirtied(page, page->mapping); - radix_tree_tag_set(&mapping->page_tree, - page_index(page), PAGECACHE_TAG_DIRTY); - - /* - * Reference snap context in page->private. Also set - * PagePrivate so that we get invalidatepage callback. - */ - page->private = (unsigned long)snapc; - SetPagePrivate(page); - } else { - dout("ANON set_page_dirty %p (raced truncate?)\n", page); - undo = 1; - } - - spin_unlock_irq(&mapping->tree_lock); - - if (undo) - /* whoops, we failed to dirty the page */ - ceph_put_wrbuffer_cap_refs(ci, 1, snapc); - - __mark_inode_dirty(mapping->host, I_DIRTY_PAGES); + /* + * Reference snap context in page->private. Also set + * PagePrivate so that we get invalidatepage callback. + */ + BUG_ON(PagePrivate(page)); + page->private = (unsigned long)snapc; + SetPagePrivate(page); - BUG_ON(!PageDirty(page)); - return 1; + return __set_page_dirty_nobuffers(page); } /* -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in