From patchwork Fri Jul 26 02:07:03 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: majianpeng X-Patchwork-Id: 2833766 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id DE183C0319 for ; Fri, 26 Jul 2013 02:07:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1F4A2201DA for ; Fri, 26 Jul 2013 02:07:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 78DA4201D5 for ; Fri, 26 Jul 2013 02:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758244Ab3GZCHS (ORCPT ); Thu, 25 Jul 2013 22:07:18 -0400 Received: from mail-pa0-f46.google.com ([209.85.220.46]:49154 "EHLO mail-pa0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757980Ab3GZCHR (ORCPT ); Thu, 25 Jul 2013 22:07:17 -0400 Received: by mail-pa0-f46.google.com with SMTP id fa1so2692803pad.5 for ; Thu, 25 Jul 2013 19:07:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:reply-to:subject:references:x-priority:x-guid :x-has-attach:x-mailer:mime-version:message-id:content-type :content-transfer-encoding; bh=4529YRJhdNuQbx2JRVAahlDxXSvyzy7S75STthC0viU=; b=Cak2ck1GTFehyhp+POpjLGkR6wvt+zQMQoROuhobeBo8SydTQjUkbnB1rDpo4+lLOY hcI/HF6/VTKlWF460QzEypeSilUYY4yDk9bR3znqF9yJAY29z+xBjIiL2r569yY6fAKT hs8+vwffu690Y7FCD1BsBxB4By8xinC+cItOBgdnZLOBGBtTIRa2PN7fe6OYbnXzzKx7 zofI5BAQYOrm0rIH7cuYYi8ddimsNYZXpN1eUdF4Kext7L/k6eaBhIE8yWf4zI6Dekf5 pr7fcn3a23GwA3pRTxN6maPfXt85sV35euNybh1TSxH7PlB66zxNxwcAFNurr6B6S49B yO1g== X-Received: by 10.68.29.2 with SMTP id f2mr51219346pbh.184.1374804436518; Thu, 25 Jul 2013 19:07:16 -0700 (PDT) Received: from majianpeng ([218.242.10.182]) by mx.google.com with ESMTPSA id yj2sm56904874pbb.40.2013.07.25.19.07.05 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 25 Jul 2013 19:07:15 -0700 (PDT) Date: Fri, 26 Jul 2013 10:07:03 +0800 From: majianpeng To: =?gb2312?B?WWFuLCBaaGVuZw==?= Cc: sage , ceph-devel Reply-To: majianpeng Subject: Re: Re: question about striped_read References: <201307250852310158703@gmail.com> <201307251455102701172@gmail.com> <201307260848241340451@gmail.com> <201307260922285205344@gmail.com> <201307260938095935695@gmail.com>, X-Priority: 3 X-GUID: FBBC5261-1A82-4F95-AEB4-E7F4C888E0DF X-Has-Attach: no X-Mailer: Foxmail 7.0.1.90[en] Mime-Version: 1.0 Message-ID: <201307261007003577701@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP >On Fri, Jul 26, 2013 at 9:38 AM, majianpeng wrote: >>>On Fri, Jul 26, 2013 at 9:22 AM, majianpeng wrote: >>>>>On Fri, Jul 26, 2013 at 8:48 AM, majianpeng wrote: >>>>>>>On Thu, 25 Jul 2013, Yan, Zheng wrote: >>>>>>>> On Thu, Jul 25, 2013 at 2:55 PM, majianpeng wrote: >>>>>>>> >>On Thu, 25 Jul 2013, majianpeng wrote: >>>>>>>> >>> Hi all, >>>>>>>> >>> I met a problem and ask somebody could help me. >>>>>>>> >>> In func striped_read() >>>>>>>> >>> > if (ret > 0) { >>>>>>>> >>> > int didpages = (page_align + ret) >> PAGE_CACHE_SHIFT; >>>>>>>> >>> >>>>>>>> >>> > if (read < pos - off) { >>>>>>>> >>> > dout(" zero gap %llu to %llu\n", off + read, pos); >>>>>>>> >>> > ceph_zero_page_vector_range(page_align + read, >>>>>>>> >>> > pos - off - read, pages); >>>>>>>> >>> > } >>>>>>>> >>> > pos += ret; >>>>>>>> >>>>>>>> I think you are right. probably above line should be 'pos += this_len' >>>>>>> >>>>>>>It should be easy to construct a simple test for this. E.g., something >>>>>>>like >>>>>>> >>>>>>> pwrite(fd, buf, 0, 3000000); >>>>>>> pwrite(fd, buf, 4194304, 1000); >>>>>>> pread(fd, buf, 0, 6000000); >>>>>>> ... >>>>>>> >>>>>>>and whatever else to verify that pos was in fact advanced properly? >>>>>>> >>>>>> The following is my test code: >>>>>> void hole_test() >>>>>> { >>>>>> char buf[4194304]; >>>>>> ssize_t ret; >>>>>> int fd = open("/media/ceph/test", O_RDWR|O_CREAT|O_DIRECT|O_TRUNC); >>>>>> if (fd < 0) { >>>>>> printf("open error %s\n", strerror(errno)); >>>>>> return; >>>>>> } >>>>>> >>>>>> ret = pwrite(fd, buf, 0, 3000000); >>>>>> ret = pwrite(fd, buf, 4194304, 1000); >>>>>> ret = pread(fd, buf , 0, 6000000); >>>>>> close(fd); >>>>>> } >>>>>> >>>>>> The debug message from striped_read are: >>>>>> [ 267.530266] ceph: file.c:356 : striped_read 6000000~0 (read 0) got 0 >>>>>> [ 267.530270] ceph: file.c:396 : striped_read returns 0 >>>>>> >>>>>> The result isn't what's your said. >>>>>> Am i missing something? >>>>>> >>>>>> BTW, i think Yan is ok, >>>>>> The code >>>>>> pos += ret; should pos += this_len. >>>>>> Because this_len can larger than ret. >>>>>> But the question is what's condition can cause this? >>>>> >>>>>by default, ceph strips file to 4M objects. In above example, the >>>>>first object only has >>>>>3M data, so 'ret = 3M' and 'this_len = 4M' >>>> actually, in func calc_layout: the read length is the smaller between object and left. >>>>> >>>>>> And can the short_read operation handle this situation? >>>>> >>>>>I don't think it's good idea to short read unless we really reach EOF. >>>> Can you explain in detail? >>>> >>> >>>because some user program interpret short read as EOF reached. If we >>>return short >>>read they get confused. >> The reason cause short-read is only EOF? or other reason? >> > >I think yes, (at least for reading from FS and read size is not very large) we can remove those code: Becase in fun striped_read, the short-read have two reasons:eof or met a hole. In either case ,the later was_short handler can handle. Is it ok? Thanks Jianpeng Ma diff --git a/fs/ceph/file.c b/fs/ceph/file.c index 2ddf061..94fa378 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -352,11 +352,6 @@ more: if (ret > 0) { int didpages = (page_align + ret) >> PAGE_CACHE_SHIFT; - if (read < pos - off) { - dout(" zero gap %llu to %llu\n", off + read, pos); - ceph_zero_page_vector_range(page_align + read, - pos - off - read, pages); - } pos += ret; read = pos - off; left -= ret;