From patchwork Fri Aug 3 13:30:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Javier_Gonz=C3=A1lez?= X-Patchwork-Id: 10555111 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C79C615A6 for ; Fri, 3 Aug 2018 13:30:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5AAA2BCC0 for ; Fri, 3 Aug 2018 13:30:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8FA02BCC5; Fri, 3 Aug 2018 13:30:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A5F82BCC0 for ; Fri, 3 Aug 2018 13:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729849AbeHCP0x (ORCPT ); Fri, 3 Aug 2018 11:26:53 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:38183 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729796AbeHCP0x (ORCPT ); Fri, 3 Aug 2018 11:26:53 -0400 Received: by mail-ed1-f68.google.com with SMTP id t2-v6so2162099edr.5 for ; Fri, 03 Aug 2018 06:30:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=javigon-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=IUvsjD9m1QX7o5WGCRuXIk0TAKa5VmkAaQ+JpCUQt64=; b=LL9cim/Q+heT2mDJlQC5BmWMW+LDPU3kypF4dOm4w+ZkhBKPjSPAXFIxt7e04YwKbK IJCcjuBxLVdXP/S2grcm0aHga9IPP1ujXKbnHTu87OHfHt2Dlek2Z/O7si09lWJjWkR6 FSF1FZCyNDcw4d5e0ERISJFkF+LcjdLJc+lSyAkvzRM5fjNuasmuPy3IMdxkj+caVapD fQ4e7mx4t7WS7mbhyOdQycFr48BIM1yzrnhUy/HRIIQByPf5BA93SMQlbU2+FyvEIEUj kbPsJL8p9jc/e72RUNqArP+SVV3GcWuCGXxgLmbmZy5R/6PfUMwylxvJo+n3kvjMPquU bsOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=IUvsjD9m1QX7o5WGCRuXIk0TAKa5VmkAaQ+JpCUQt64=; b=jOeQkrFFY/dWVbRCXd7EvNmdKsZvMuUjSPKp+8fsB8Q5lwcoV24EBP711gqa9IpUHp NWToN9G5jDKU0aq8/TKu8KdOCLQbq9qVAxGXrFN2p4idDZ+GW7BAS96ay5w39fCcAfl7 NHNNDkPVWOrrHtDz1cDUvzbOaX54nA0h6rRVBnAR/jhMkpoNTR5fkCkIIEj5HMLjFwNQ McWtE1uw/Ej7+//lx1Pg4I/JPZmOYZ3SEM+4t1X+j7MRIkcVFNg6K7bP5V3kiF6OvN0P uiO7mwJXnGqp8gen3+gvknBt9WfDL0ZPHSxhw6RgqfzWfxwihGx60DPUNFNiyk9HQfFh 8LhQ== X-Gm-Message-State: AOUpUlFTcroduTna/feGfm9DEKxSYcAI2qPKvXJPFwaX460rUvCV6O8I ZN7cNjTKgdldquWhLNz2Q8ZJFg== X-Google-Smtp-Source: AAOMgpc9douFSS0U/WyPeyLjVpFIPxpRuEjJK7MqvrlczwnexExKo2leOPZRFQEI6itddvBZo0rRVQ== X-Received: by 2002:a50:9286:: with SMTP id k6-v6mr7560592eda.66.1533303029923; Fri, 03 Aug 2018 06:30:29 -0700 (PDT) Received: from ch-wrk-javier.cnexlabs.com (6164211-cl69.boa.fiberby.dk. [193.106.164.211]) by smtp.gmail.com with ESMTPSA id b44-v6sm5998532edd.93.2018.08.03.06.30.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 03 Aug 2018 06:30:29 -0700 (PDT) From: " =?utf-8?q?Javier_Gonz=C3=A1lez?= " X-Google-Original-From: =?utf-8?q?Javier_Gonz=C3=A1lez?= To: mb@lightnvm.io Cc: axboe@kernel.dk, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, =?utf-8?q?Javier_Gonz=C3=A1lez?= Subject: [PATCH V2] lightnvm: pblk: take write semaphore on metadata Date: Fri, 3 Aug 2018 15:30:11 +0200 Message-Id: <1533303011-2855-1-git-send-email-javier@cnexlabs.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP # Changes singe V1: - Fix double I/O on the read path (by Matias) - Improve commit message (by Jens) pblk guarantees write ordering at a chunk level through a per open chunk semaphore. At this point, since we only have an open I/O stream for both user and GC data, the semaphore is per parallel unit. Since metadata I/O is synchronous, the semaphore is not needed as ordering is guaranteed. However, if the metadata scheme changes or multiple streams are open, this guarantee might not be preserved. This patch makes sure that all writes go through the semaphore, even for synchronous I/O. This is consistent with pblk's write I/O model. It also simplifies maintenance since changes in the metdatada scheme could cause ordering issues. Signed-off-by: Javier González --- drivers/lightnvm/pblk-core.c | 18 ++++++++++++++++-- drivers/lightnvm/pblk.h | 1 + 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c index 00984b486fea..6432faf5b19c 100644 --- a/drivers/lightnvm/pblk-core.c +++ b/drivers/lightnvm/pblk-core.c @@ -493,6 +493,20 @@ int pblk_submit_io_sync(struct pblk *pblk, struct nvm_rq *rqd) return nvm_submit_io_sync(dev, rqd); } +int pblk_submit_io_sync_sem(struct pblk *pblk, struct nvm_rq *rqd) +{ + int ret; + + if (rqd->opcode != NVM_OP_PWRITE) + return pblk_submit_io_sync(pblk, rqd); + + pblk_down_page(pblk, rqd->ppa_list, rqd->nr_ppas); + ret = pblk_submit_io_sync(pblk, rqd); + pblk_up_page(pblk, rqd->ppa_list, rqd->nr_ppas); + + return ret; +} + static void pblk_bio_map_addr_endio(struct bio *bio) { bio_put(bio); @@ -737,7 +751,7 @@ static int pblk_line_submit_emeta_io(struct pblk *pblk, struct pblk_line *line, } } - ret = pblk_submit_io_sync(pblk, &rqd); + ret = pblk_submit_io_sync_sem(pblk, &rqd); if (ret) { pblk_err(pblk, "emeta I/O submission failed: %d\n", ret); bio_put(bio); @@ -842,7 +856,7 @@ static int pblk_line_submit_smeta_io(struct pblk *pblk, struct pblk_line *line, * the write thread is the only one sending write and erase commands, * there is no need to take the LUN semaphore. */ - ret = pblk_submit_io_sync(pblk, &rqd); + ret = pblk_submit_io_sync_sem(pblk, &rqd); if (ret) { pblk_err(pblk, "smeta I/O submission failed: %d\n", ret); bio_put(bio); diff --git a/drivers/lightnvm/pblk.h b/drivers/lightnvm/pblk.h index 4760af7b6499..6ccc6ad8e1ce 100644 --- a/drivers/lightnvm/pblk.h +++ b/drivers/lightnvm/pblk.h @@ -782,6 +782,7 @@ void pblk_log_write_err(struct pblk *pblk, struct nvm_rq *rqd); void pblk_log_read_err(struct pblk *pblk, struct nvm_rq *rqd); int pblk_submit_io(struct pblk *pblk, struct nvm_rq *rqd); int pblk_submit_io_sync(struct pblk *pblk, struct nvm_rq *rqd); +int pblk_submit_io_sync_sem(struct pblk *pblk, struct nvm_rq *rqd); int pblk_submit_meta_io(struct pblk *pblk, struct pblk_line *meta_line); struct bio *pblk_bio_map_addr(struct pblk *pblk, void *data, unsigned int nr_secs, unsigned int len,