From patchwork Fri Jul 28 13:13:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?UTF-8?q?Javier=20Gonz=C3=A1lez?= X-Patchwork-Id: 9868755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C5F286038F for ; Fri, 28 Jul 2017 13:14:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADF06288DA for ; Fri, 28 Jul 2017 13:14:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3064288D2; Fri, 28 Jul 2017 13:14:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C64B288CB for ; Fri, 28 Jul 2017 13:14:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751958AbdG1NN4 (ORCPT ); Fri, 28 Jul 2017 09:13:56 -0400 Received: from mail-wm0-f52.google.com ([74.125.82.52]:38010 "EHLO mail-wm0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751964AbdG1NN3 (ORCPT ); Fri, 28 Jul 2017 09:13:29 -0400 Received: by mail-wm0-f52.google.com with SMTP id m85so117194244wma.1 for ; Fri, 28 Jul 2017 06:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lightnvm-io.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+Ebd5b8ooWKo4L20W2e+HMOnm3zcJYbIrhqjh+pNp0o=; b=0ABV7cobcZ+ret68LGdaG6jcdM/Cbz9wTpbwgzLv8TZn7ClBwYtCwQF94D9+fiXLjt J9l63dbjipyh0+uXO1U9IEM0uMQW95JpJO72PnIY7N26y4lJ2n/MftCX/eEkKO1UOAJA suCq1AxzoihB8+fkV2ovzegLWdnoem8QoadrHAdD98ahHRQdGqAYaikYP6NcobzzspKw qDMiLTRmmYB7aRHpKdNIfxv2ygqiMT8mtehU9IVdk9fFVaNsweG6nBIlimkzEeCdpVd7 ZEvnFNPRhfB01vv8Mk3S7MXzOoms/vuJAlvPQWkoyYqoC5cPcwGx5yfXzLh5+vZAWThd L5wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+Ebd5b8ooWKo4L20W2e+HMOnm3zcJYbIrhqjh+pNp0o=; b=Heds+exTQlVdou1Y/BMN8wO0PCmiZlvxGR7uk37J7QGF78NxSTeEEkcmjXC/Go2Er4 2uqjU4HYCDvyB4UOWasovSrbXtOverlV9pyGjOYMykJFp8M/uYd7fUqVvvwm5GGX4Fon Zm8uydeGbENuXeqbGKdpjXfi6uu/4k5yp6dWxQlnxVVjOEcldcIq06IUxMSBySh+0le2 CANYxq8839ZMYYyqehahqdJFlLbgf49fjG3DEidMWMCNLDsqp6VDHxqSb0YDmPrNI9dt BjjdAwaQdvtL5zI+jl3oNciwnJrom12JAJMpfXkN6O2eTXoQJK+2/Y8yvjuJzA8nkMmD Nkqg== X-Gm-Message-State: AIVw1129pZbE4Mvhcss66Mj2IMYTY1FkI1LZH42i3AqoSEP41dcKJvEH ksoYM84omZSL07nl X-Received: by 10.80.174.6 with SMTP id c6mr6405047edd.235.1501247608412; Fri, 28 Jul 2017 06:13:28 -0700 (PDT) Received: from uHalley.cnexlabs.com (6164211-cl69.boa.fiberby.dk. [193.106.164.211]) by smtp.gmail.com with ESMTPSA id h28sm10573417ede.56.2017.07.28.06.13.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 28 Jul 2017 06:13:27 -0700 (PDT) From: "=?UTF-8?q?Javier=20Gonz=C3=A1lez?=" X-Google-Original-From: =?UTF-8?q?Javier=20Gonz=C3=A1lez?= To: mb@lightnvm.io, axboe@fb.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, =?UTF-8?q?Javier=20Gonz=C3=A1lez?= Subject: [PATCH] lightnvm: pblk: advance bio according to lba index Date: Fri, 28 Jul 2017 15:13:16 +0200 Message-Id: <1501247596-23999-2-git-send-email-javier@javigon.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1501247596-23999-1-git-send-email-javier@javigon.com> References: <1501247596-23999-1-git-send-email-javier@javigon.com> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When a lba either hits the cache or corresponds to an empty entry in the L2P table, we need to advance the bio according to the position in which the lba is located. Otherwise, we will copy data in the wrong page, thus causing data corruption for the application. In case of a cache hit, we assumed that bio->bi_iter.bi_idx would contain the correct index, but this is no necessarily true. Instead, use the local bio advance counter and iterator. This guarantees that lbas hitting the cache are copied into the right bv_page. In case of an empty L2P entry, we omitted to advance the bio. In the cases when the same I/O also contains a cache hit, data corresponding to this lba will be copied to the wrong bv_page. Fix this by advancing the bio as we do in the case of a cache hit. Fixes: a4bd217b4326 lightnvm: physical block device (pblk) target Signed-off-by: Javier González --- drivers/lightnvm/pblk-rb.c | 4 ++-- drivers/lightnvm/pblk-read.c | 23 ++++++++++++++++------- drivers/lightnvm/pblk.h | 2 +- 3 files changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/lightnvm/pblk-rb.c b/drivers/lightnvm/pblk-rb.c index 5ecc154f6831..9bc32578a766 100644 --- a/drivers/lightnvm/pblk-rb.c +++ b/drivers/lightnvm/pblk-rb.c @@ -657,7 +657,7 @@ unsigned int pblk_rb_read_to_bio(struct pblk_rb *rb, struct nvm_rq *rqd, * be directed to disk. */ int pblk_rb_copy_to_bio(struct pblk_rb *rb, struct bio *bio, sector_t lba, - struct ppa_addr ppa, int bio_iter) + struct ppa_addr ppa, int bio_iter, bool advanced_bio) { struct pblk *pblk = container_of(rb, struct pblk, rwb); struct pblk_rb_entry *entry; @@ -694,7 +694,7 @@ int pblk_rb_copy_to_bio(struct pblk_rb *rb, struct bio *bio, sector_t lba, * filled with data from the cache). If part of the data resides on the * media, we will read later on */ - if (unlikely(!bio->bi_iter.bi_idx)) + if (unlikely(!advanced_bio)) bio_advance(bio, bio_iter * PBLK_EXPOSED_PAGE_SIZE); data = bio_data(bio); diff --git a/drivers/lightnvm/pblk-read.c b/drivers/lightnvm/pblk-read.c index 4e5c48f3de62..d682e89e6493 100644 --- a/drivers/lightnvm/pblk-read.c +++ b/drivers/lightnvm/pblk-read.c @@ -26,7 +26,7 @@ */ static int pblk_read_from_cache(struct pblk *pblk, struct bio *bio, sector_t lba, struct ppa_addr ppa, - int bio_iter) + int bio_iter, bool advanced_bio) { #ifdef CONFIG_NVM_DEBUG /* Callers must ensure that the ppa points to a cache address */ @@ -34,7 +34,8 @@ static int pblk_read_from_cache(struct pblk *pblk, struct bio *bio, BUG_ON(!pblk_addr_in_cache(ppa)); #endif - return pblk_rb_copy_to_bio(&pblk->rwb, bio, lba, ppa, bio_iter); + return pblk_rb_copy_to_bio(&pblk->rwb, bio, lba, ppa, + bio_iter, advanced_bio); } static void pblk_read_ppalist_rq(struct pblk *pblk, struct nvm_rq *rqd, @@ -44,7 +45,7 @@ static void pblk_read_ppalist_rq(struct pblk *pblk, struct nvm_rq *rqd, struct ppa_addr ppas[PBLK_MAX_REQ_ADDRS]; sector_t blba = pblk_get_lba(bio); int nr_secs = rqd->nr_ppas; - int advanced_bio = 0; + bool advanced_bio = false; int i, j = 0; /* logic error: lba out-of-bounds. Ignore read request */ @@ -62,19 +63,26 @@ static void pblk_read_ppalist_rq(struct pblk *pblk, struct nvm_rq *rqd, retry: if (pblk_ppa_empty(p)) { WARN_ON(test_and_set_bit(i, read_bitmap)); - continue; + + if (unlikely(!advanced_bio)) { + bio_advance(bio, (i) * PBLK_EXPOSED_PAGE_SIZE); + advanced_bio = true; + } + + goto next; } /* Try to read from write buffer. The address is later checked * on the write buffer to prevent retrieving overwritten data. */ if (pblk_addr_in_cache(p)) { - if (!pblk_read_from_cache(pblk, bio, lba, p, i)) { + if (!pblk_read_from_cache(pblk, bio, lba, p, i, + advanced_bio)) { pblk_lookup_l2p_seq(pblk, &p, lba, 1); goto retry; } WARN_ON(test_and_set_bit(i, read_bitmap)); - advanced_bio = 1; + advanced_bio = true; #ifdef CONFIG_NVM_DEBUG atomic_long_inc(&pblk->cache_reads); #endif @@ -83,6 +91,7 @@ static void pblk_read_ppalist_rq(struct pblk *pblk, struct nvm_rq *rqd, rqd->ppa_list[j++] = p; } +next: if (advanced_bio) bio_advance(bio, PBLK_EXPOSED_PAGE_SIZE); } @@ -282,7 +291,7 @@ static void pblk_read_rq(struct pblk *pblk, struct nvm_rq *rqd, * write buffer to prevent retrieving overwritten data. */ if (pblk_addr_in_cache(ppa)) { - if (!pblk_read_from_cache(pblk, bio, lba, ppa, 0)) { + if (!pblk_read_from_cache(pblk, bio, lba, ppa, 0, 1)) { pblk_lookup_l2p_seq(pblk, &ppa, lba, 1); goto retry; } diff --git a/drivers/lightnvm/pblk.h b/drivers/lightnvm/pblk.h index 0c5692cc2f60..67e623bd5c2d 100644 --- a/drivers/lightnvm/pblk.h +++ b/drivers/lightnvm/pblk.h @@ -670,7 +670,7 @@ unsigned int pblk_rb_read_to_bio_list(struct pblk_rb *rb, struct bio *bio, struct list_head *list, unsigned int max); int pblk_rb_copy_to_bio(struct pblk_rb *rb, struct bio *bio, sector_t lba, - struct ppa_addr ppa, int bio_iter); + struct ppa_addr ppa, int bio_iter, bool advanced_bio); unsigned int pblk_rb_read_commit(struct pblk_rb *rb, unsigned int entries); unsigned int pblk_rb_sync_init(struct pblk_rb *rb, unsigned long *flags);