From patchwork Mon Nov 16 00:59:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907003 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6F99B138B for ; Mon, 16 Nov 2020 01:00:12 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F0F322223C for ; Mon, 16 Nov 2020 01:00:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F0F322223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E4FF821F9C9; Sun, 15 Nov 2020 17:00:09 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 17CF821F895 for ; Sun, 15 Nov 2020 17:00:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 47FEE222C; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3C60B2C7E6; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:34 -0500 Message-Id: <1605488401-981-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 01/28] llite: remove splice_read handling for PCC X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" For older kernels with PCC we handled splice_read ourself but this is no longer needed since generic_file_splice_read() is just a wrapper around the read_iter() operation. We can safely remove pcc_file_splice_read(). Use the read_iter() / write_iter() functions for pccf_file instead of the default file of the passed in kiocb. WC-bug-id: https://jira.whamcloud.com/browse/LU-13745 Lustre-commit: 1635dc9de0bc1d ("LU-13745 llite: switch generic_file_splice_read() to use of ->read_iter()") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/39272 Reviewed-by: Wang Shilong Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin --- fs/lustre/llite/pcc.c | 33 ++------------------------------- fs/lustre/llite/pcc.h | 5 ----- 2 files changed, 2 insertions(+), 36 deletions(-) diff --git a/fs/lustre/llite/pcc.c b/fs/lustre/llite/pcc.c index 5a4bb33..28ca9cb 100644 --- a/fs/lustre/llite/pcc.c +++ b/fs/lustre/llite/pcc.c @@ -1592,7 +1592,7 @@ ssize_t pcc_file_read_iter(struct kiocb *iocb, * filp->f_ops->read_iter uses ->aio_read hook directly * to add support for ext4-dax. */ - result = file->f_op->read_iter(iocb, iter); + result = iocb->ki_filp->f_op->read_iter(iocb, iter); iocb->ki_filp = file; pcc_io_fini(inode); @@ -1633,7 +1633,7 @@ ssize_t pcc_file_write_iter(struct kiocb *iocb, * the normal vfs interface to the local PCC file system, * the inode lock is not needed. */ - result = file->f_op->write_iter(iocb, iter); + result = iocb->ki_filp->f_op->write_iter(iocb, iter); iocb->ki_filp = file; out: pcc_io_fini(inode); @@ -1734,35 +1734,6 @@ int pcc_inode_getattr(struct inode *inode, u32 request_mask, return rc; } -ssize_t pcc_file_splice_read(struct file *in_file, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t count, unsigned int flags, - bool *cached) -{ - struct inode *inode = file_inode(in_file); - struct ll_file_data *fd = in_file->private_data; - struct file *pcc_file = fd->fd_pcc_file.pccf_file; - ssize_t result; - - *cached = false; - if (!pcc_file) - return 0; - - if (!file_inode(pcc_file)->i_fop->splice_read) - return -ENOTSUPP; - - pcc_io_init(inode, PIT_SPLICE_READ, cached); - if (!*cached) - return 0; - - result = file_inode(pcc_file)->i_fop->splice_read(pcc_file, - ppos, pipe, count, - flags); - - pcc_io_fini(inode); - return result; -} - int pcc_fsync(struct file *file, loff_t start, loff_t end, int datasync, bool *cached) { diff --git a/fs/lustre/llite/pcc.h b/fs/lustre/llite/pcc.h index b13f9da8..d3aae9b 100644 --- a/fs/lustre/llite/pcc.h +++ b/fs/lustre/llite/pcc.h @@ -184,8 +184,6 @@ enum pcc_io_type { PIT_FAULT, /* fsync system call handling */ PIT_FSYNC, - /* splice_read system call */ - PIT_SPLICE_READ, /* open system call */ PIT_OPEN, }; @@ -241,9 +239,6 @@ ssize_t pcc_file_write_iter(struct kiocb *iocb, struct iov_iter *iter, int pcc_inode_getattr(struct inode *inode, u32 request_mask, unsigned int flags, bool *cached); int pcc_inode_setattr(struct inode *inode, struct iattr *attr, bool *cached); -ssize_t pcc_file_splice_read(struct file *in_file, loff_t *ppos, - struct pipe_inode_info *pipe, size_t count, - unsigned int flags, bool *cached); int pcc_fsync(struct file *file, loff_t start, loff_t end, int datasync, bool *cached); int pcc_file_mmap(struct file *file, struct vm_area_struct *vma, bool *cached); From patchwork Mon Nov 16 00:59:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907007 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85BEF138B for ; Mon, 16 Nov 2020 01:00:18 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1C2A2223C for ; Mon, 16 Nov 2020 01:00:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1C2A2223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CEECC21FDA2; Sun, 15 Nov 2020 17:00:12 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6A2AC21F895 for ; Sun, 15 Nov 2020 17:00:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4B346222D; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3EF581D8E; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:35 -0500 Message-Id: <1605488401-981-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 02/28] lustre: llite: disable statahead_agl for sanity test_56ra X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown The sanity test_56ra can fail because statahead_agl can cause extra glimpse request. If a stat() systemcall is made after an AGL glimpse request is sent, but before the reply has been received, the code handling the stat cannot see that glimpse request and so will send another. This elevates the number of requests counted. There is a parameter (statahead_agl) which make it easy to disable the AGL, but it isn't implemented properly. Specifically, inodes can still be added to the sai_agls list when agl is disabled. They will never be removed, which causes an assertion to fail. To clean this up, remove the sai_agl_valid flag, and use a test on sai_task being non-NULL instead. Also check agl_should_run() while locked against ->sai_task changing, and before adding anything to lli_agl_list. We don't need the 'added' variable. It is perfectly OK to wake_up the sai_agl_task *before* adding to the list as long is that is all done under the lock. The task will wait for the lock before checking the list, so it won't see it being empty. WC-bug-id: https://jira.whamcloud.com/browse/LU-13017 Lustre-commit: 3e04c4f0757c22 ("LU-13017 tests: disable statahead_agl for sanity test_56ra") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/39667 Reviewed-by: Lai Siyao Reviewed-by: Yingjin Qian Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_internal.h | 1 - fs/lustre/llite/statahead.c | 31 +++++++++++++------------------ 2 files changed, 13 insertions(+), 19 deletions(-) diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 0bd6795..9d988aac 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -1398,7 +1398,6 @@ struct ll_statahead_info { unsigned int sai_ls_all:1, /* "ls -al", do stat-ahead for * hidden entries */ - sai_agl_valid:1,/* AGL is valid for the dir */ sai_in_readpage:1;/* statahead in readdir() */ wait_queue_head_t sai_waitq; /* stat-ahead wait queue */ struct task_struct *sai_task; /* stat-ahead thread */ diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index 895e496..a7d3a43 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -129,7 +129,7 @@ static inline int sa_hash(int val) static inline int agl_should_run(struct ll_statahead_info *sai, struct inode *inode) { - return (inode && S_ISREG(inode->i_mode) && sai->sai_agl_valid); + return inode && S_ISREG(inode->i_mode) && sai->sai_agl_task; } /* statahead window is full */ @@ -424,7 +424,6 @@ static void ll_agl_add(struct ll_statahead_info *sai, { struct ll_inode_info *child = ll_i2info(inode); struct ll_inode_info *parent = ll_i2info(sai->sai_dentry->d_inode); - int added = 0; spin_lock(&child->lli_agl_lock); if (child->lli_agl_index == 0) { @@ -433,18 +432,20 @@ static void ll_agl_add(struct ll_statahead_info *sai, LASSERT(list_empty(&child->lli_agl_list)); - igrab(inode); spin_lock(&parent->lli_agl_lock); - if (list_empty(&sai->sai_agls)) - added = 1; - list_add_tail(&child->lli_agl_list, &sai->sai_agls); + /* Re-check under the lock */ + if (agl_should_run(sai, inode)) { + if (list_empty(&sai->sai_agls)) + wake_up_process(sai->sai_agl_task); + igrab(inode); + list_add_tail(&child->lli_agl_list, &sai->sai_agls); + } else { + child->lli_agl_index = 0; + } spin_unlock(&parent->lli_agl_lock); } else { spin_unlock(&child->lli_agl_lock); } - - if (added > 0) - wake_up_process(sai->sai_agl_task); } /* allocate sai */ @@ -936,7 +937,6 @@ static void ll_stop_agl(struct ll_statahead_info *sai) sai->sai_agl_task = NULL; spin_lock(&plli->lli_agl_lock); - sai->sai_agl_valid = 0; while ((clli = list_first_entry_or_null(&sai->sai_agls, struct ll_inode_info, lli_agl_list)) != NULL) { @@ -967,16 +967,11 @@ static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) plli->lli_opendir_pid); if (IS_ERR(task)) { CERROR("can't start ll_agl thread, rc: %ld\n", PTR_ERR(task)); - sai->sai_agl_valid = 0; return; } sai->sai_agl_task = task; - LASSERT(sai->sai_agl_valid == 1); atomic_inc(&ll_i2sbi(d_inode(parent))->ll_agl_total); - spin_lock(&plli->lli_agl_lock); - sai->sai_agl_valid = 1; - spin_unlock(&plli->lli_agl_lock); /* Get an extra reference that the thread holds */ ll_sai_get(d_inode(parent)); @@ -1569,7 +1564,6 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry, } sai->sai_ls_all = (first == LS_FIRST_DOT_DE); - sai->sai_agl_valid = agl; /* * if current lli_opendir_key was deauthorized, or dir re-opened by @@ -1643,10 +1637,11 @@ static inline bool ll_statahead_started(struct inode *dir, bool agl) spin_lock(&lli->lli_sa_lock); sai = lli->lli_sai; - if (sai && sai->sai_agl_valid != agl) + if (sai && (sai->sai_agl_task != NULL) != agl) CDEBUG(D_READA, "%s: Statahead AGL hint changed from %d to %d\n", - ll_i2sbi(dir)->ll_fsname, sai->sai_agl_valid, agl); + ll_i2sbi(dir)->ll_fsname, + sai->sai_agl_task != NULL, agl); spin_unlock(&lli->lli_sa_lock); return !!sai; From patchwork Mon Nov 16 00:59:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907039 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7101314B3 for ; Mon, 16 Nov 2020 01:01:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4FB6320E65 for ; Mon, 16 Nov 2020 01:01:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FB6320E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1E322306ECD; Sun, 15 Nov 2020 17:01:25 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BD5D721F895 for ; Sun, 15 Nov 2020 17:00:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4D7CE222E; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 434FA2C7FB; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:36 -0500 Message-Id: <1605488401-981-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 03/28] lustre: seq_file .next functions must update *pos X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown A seq_file .next function must update *pos on EOF to a value which will cause a subsequent ->start to also return EOF. If it doesn't the last record of the file can be returned twice to a 'read()'. Also the seq_file code will generate a warning. This patch fixes various ->next functions to always update *pos. WC-bug-id: https://jira.whamcloud.com/browse/LU-13985 Lustre-commit: 817d6c11659963 ("LU-13985 lustre: seq_file .next functions must update *pos") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/40035 Reviewed-by: Andreas Dilger Reviewed-by: Chris Horn Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lov_pool.c | 2 +- fs/lustre/ptlrpc/lproc_ptlrpc.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/lustre/lov/lov_pool.c b/fs/lustre/lov/lov_pool.c index 9e937ac..f8f14f9 100644 --- a/fs/lustre/lov/lov_pool.c +++ b/fs/lustre/lov/lov_pool.c @@ -111,6 +111,7 @@ static void *pool_proc_next(struct seq_file *s, void *v, loff_t *pos) LASSERTF(iter->magic == POOL_IT_MAGIC, "%08X\n", iter->magic); + (*pos)++; /* test if end of file */ if (*pos >= pool_tgt_count(iter->pool)) return NULL; @@ -122,7 +123,6 @@ static void *pool_proc_next(struct seq_file *s, void *v, loff_t *pos) iter->idx = prev_idx; /* we stay on the last entry */ return NULL; } - (*pos)++; /* return != NULL to continue */ return iter; } diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 4d2ae14..7276f81 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -932,6 +932,7 @@ struct ptlrpc_srh_iterator { } kfree(srhi); + ++*pos; return NULL; } From patchwork Mon Nov 16 00:59:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6268138B for ; Mon, 16 Nov 2020 01:01:23 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 984BF20578 for ; Mon, 16 Nov 2020 01:01:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 984BF20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 97001306D8C; Sun, 15 Nov 2020 17:01:05 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0DA5821F895 for ; Sun, 15 Nov 2020 17:00:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4F1AE222F; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 45F772C802; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:37 -0500 Message-Id: <1605488401-981-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 04/28] lustre: llite: ASSERTION( last_oap_count > 0 ) failed X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andriy Skulysh , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andriy Skulysh Punch uses o_blocks to send end of a region. So it can be mixed with real blocks count on error. Update blocks count only on success. HPE-bug-id: LUS-7407 WC-bug-id: https://jira.whamcloud.com/browse/LU-13992 Lustre-commit: a56fefc535677b ("LU-13992 llite: ASSERTION( last_oap_count > 0 ) failed") Signed-off-by: Andriy Skulysh Reviewed-by: Andrew Perepechko Reviewed-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/40050 Reviewed-by: Alexander Boyko Reviewed-by: Andreas Dilger Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/osc/osc_io.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 7ec059a..6121f39 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -691,14 +691,16 @@ void osc_io_setattr_end(const struct lu_env *env, if (cl_io_is_trunc(io)) { u64 size = io->u.ci_setattr.sa_attr.lvb_size; - cl_object_attr_lock(obj); - if (oa->o_valid & OBD_MD_FLBLOCKS) { - attr->cat_blocks = oa->o_blocks; - cl_valid |= CAT_BLOCKS; - } + if (result == 0) { + cl_object_attr_lock(obj); + if (oa->o_valid & OBD_MD_FLBLOCKS) { + attr->cat_blocks = oa->o_blocks; + cl_valid |= CAT_BLOCKS; + } - cl_object_attr_update(env, obj, attr, cl_valid); - cl_object_attr_unlock(obj); + cl_object_attr_update(env, obj, attr, cl_valid); + cl_object_attr_unlock(obj); + } osc_trunc_check(env, io, oio, size); osc_cache_truncate_end(env, oio->oi_trunc); oio->oi_trunc = NULL; From patchwork Mon Nov 16 00:59:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907055 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1487414B3 for ; Mon, 16 Nov 2020 01:02:16 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E9CB820E65 for ; Mon, 16 Nov 2020 01:02:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9CB820E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 18C5D21F928; Sun, 15 Nov 2020 17:01:40 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0938221F9D3 for ; Sun, 15 Nov 2020 17:00:10 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 560E62230; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4961A2C803; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:38 -0500 Message-Id: <1605488401-981-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 05/28] lnet: o2ib: raise bind cap before resolving address X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: "John L. Hammond" In kiblnd_resolve_addr(), ensure that the current task has CAP_NET_BIND_SERVICE before calling rdma_resolve_addr() with a protected source port. WC-bug-id: https://jira.whamcloud.com/browse/LU-14006 Lustre-commit: 1e4bd16acfa26a ("LU-14006 o2ib: raise bind cap before resolving address") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/40127 Reviewed-by: Amir Shehata Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 38 +++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index ba2f46f..b642162 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -1219,14 +1219,17 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, spin_unlock(&conn->ibc_lock); } -static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, - struct sockaddr_in *srcaddr, - struct sockaddr_in *dstaddr, - int timeout_ms) +static int +kiblnd_resolve_addr_cap(struct rdma_cm_id *cmid, + struct sockaddr_in *srcaddr, + struct sockaddr_in *dstaddr, + int timeout_ms) { unsigned short port; int rc; + LASSERT(capable(CAP_NET_BIND_SERVICE)); + /* allow the port to be reused */ rc = rdma_set_reuseaddr(cmid, 1); if (rc) { @@ -1256,6 +1259,33 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, return rc; } +static int +kiblnd_resolve_addr(struct rdma_cm_id *cmid, + struct sockaddr_in *srcaddr, + struct sockaddr_in *dstaddr, + int timeout_ms) +{ + const struct cred *old_creds = NULL; + struct cred *new_creds; + int rc; + + if (!capable(CAP_NET_BIND_SERVICE)) { + new_creds = prepare_creds(); + if (!new_creds) + return -ENOMEM; + + cap_raise(new_creds->cap_effective, CAP_NET_BIND_SERVICE); + old_creds = override_creds(new_creds); + } + + rc = kiblnd_resolve_addr_cap(cmid, srcaddr, dstaddr, timeout_ms); + + if (old_creds) + revert_creds(old_creds); + + return rc; +} + static void kiblnd_connect_peer(struct kib_peer_ni *peer_ni) { From patchwork Mon Nov 16 00:59:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907051 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82996138B for ; Mon, 16 Nov 2020 01:02:10 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 664D420E65 for ; Mon, 16 Nov 2020 01:02:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 664D420E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 501B2308191; Sun, 15 Nov 2020 17:01:36 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5107321F895 for ; Sun, 15 Nov 2020 17:00:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 563AC2239; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4C3F48AE3; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:39 -0500 Message-Id: <1605488401-981-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 06/28] lustre: use memalloc_nofs_save() for GFP_NOFS kvmalloc allocations. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: NeilBrown The allocation of lo_sub should be GFP_NOFS as it can happen in support of write-out, and should allow vmalloc as the array can be as much as 2000 pointers (16K). So change it to kvmalloc_array() and use memalloc_nofs_save() as GFP_NOFS doesn't work in kvmalloc_array(). The allocation in echo_client passes GFP_NOFS to kvmalloc_array() which causes it to map directly to kmalloc_array(). So use memalloc_nofs_save() there too. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- fs/lustre/lov/lov_object.c | 13 +++++++------ fs/lustre/obdecho/echo_client.c | 7 ++++++- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/fs/lustre/lov/lov_object.c b/fs/lustre/lov/lov_object.c index 7285276..0762cc5 100644 --- a/fs/lustre/lov/lov_object.c +++ b/fs/lustre/lov/lov_object.c @@ -38,6 +38,7 @@ #define DEBUG_SUBSYSTEM S_LOV +#include #include "lov_cl_internal.h" static inline struct lov_device *lov_object_dev(struct lov_object *obj) @@ -207,6 +208,7 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, struct cl_object_conf *subconf = <i->lti_stripe_conf; struct lu_fid *ofid = <i->lti_fid; struct cl_object *stripe; + unsigned int flags; int result; int psz, sz; int i; @@ -214,8 +216,10 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, spin_lock_init(&r0->lo_sub_lock); r0->lo_nr = lse->lsme_stripe_count; - r0->lo_sub = kcalloc(r0->lo_nr, sizeof(r0->lo_sub[0]), - GFP_KERNEL); + flags = memalloc_nofs_save(); + r0->lo_sub = kvmalloc_array(r0->lo_nr, sizeof(r0->lo_sub[0]), + GFP_KERNEL | __GFP_ZERO); + memalloc_nofs_restore(flags); if (!r0->lo_sub) return -ENOMEM; @@ -335,10 +339,7 @@ static void lov_fini_raid0(const struct lu_env *env, { struct lov_layout_raid0 *r0 = &lle->lle_raid0; - if (r0->lo_sub) { - kvfree(r0->lo_sub); - r0->lo_sub = NULL; - } + kvfree(r0->lo_sub); } static int lov_print_raid0(const struct lu_env *env, void *cookie, diff --git a/fs/lustre/obdecho/echo_client.c b/fs/lustre/obdecho/echo_client.c index a52e0362..3bee0c2 100644 --- a/fs/lustre/obdecho/echo_client.c +++ b/fs/lustre/obdecho/echo_client.c @@ -34,6 +34,7 @@ #define DEBUG_SUBSYSTEM S_ECHO #include +#include #include #include #include @@ -1369,6 +1370,7 @@ static int echo_client_prep_commit(const struct lu_env *env, struct niobuf_remote rnb; u64 off; u64 npages, tot_pages; + unsigned int flags; int i, ret = 0, brw_flags = 0; if (count <= 0 || (count & (~PAGE_MASK)) != 0) @@ -1377,8 +1379,11 @@ static int echo_client_prep_commit(const struct lu_env *env, npages = batch >> PAGE_SHIFT; tot_pages = count >> PAGE_SHIFT; + flags = memalloc_nofs_save(); lnb = kvmalloc_array(npages, sizeof(*lnb), - GFP_NOFS | __GFP_ZERO); + GFP_KERNEL | __GFP_ZERO); + memalloc_nofs_restore(flags); + if (!lnb) { ret = -ENOMEM; goto out; From patchwork Mon Nov 16 00:59:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907009 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A22B14B3 for ; Mon, 16 Nov 2020 01:00:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D24E82223C for ; Mon, 16 Nov 2020 01:00:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D24E82223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 71BA721F9C9; Sun, 15 Nov 2020 17:00:32 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ACE3C21F8DB for ; Sun, 15 Nov 2020 17:00:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 57773223A; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4F3668AEB; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:40 -0500 Message-Id: <1605488401-981-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 07/28] lnet: o2iblnd: Don't retry indefinitely X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata If peer is down don't retry indefinitely. Use the retry_count parameter to restrict the number of retries. After which the connection fails and error is propagated up. This prevents long timeouts when mounting a file system with nodes which might have their NIDs configured in the FS, but the nodes have been taken offline. WC-bug-id: https://jira.whamcloud.com/browse/LU-13972 Lustre-commit: 7c8ad11ef08f0f ("LU-13972 o2iblnd: Don't retry indefinitely") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/39981 Reviewed-by: Serguei Smirnov Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd.h | 2 ++ net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 9 +++++++++ net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 2 +- 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.h b/net/lnet/klnds/o2iblnd/o2iblnd.h index 9a2fb42..9b2f043 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.h +++ b/net/lnet/klnds/o2iblnd/o2iblnd.h @@ -626,6 +626,8 @@ struct kib_peer_ni { unsigned char ibp_races; /* # consecutive reconnection attempts to this peer_ni */ unsigned int ibp_reconnected; + /* number of total active retries */ + unsigned int ibp_retries; /* errno on closing this peer_ni */ int ibp_error; /* max map_on_demand */ diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index b642162..9de733d 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -2181,6 +2181,9 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, /* connection established */ write_lock_irqsave(&kiblnd_data.kib_global_lock, flags); + /* reset retry count */ + peer_ni->ibp_retries = 0; + conn->ibc_last_send = ktime_get(); kiblnd_set_conn_state(conn, IBLND_CONN_ESTABLISHED); kiblnd_peer_alive(peer_ni); @@ -2631,6 +2634,11 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, goto out; } + if (peer_ni->ibp_retries > *kiblnd_tunables.kib_retry_count) { + reason = "retry count exceeded due to no listener"; + goto out; + } + switch (why) { default: reason = "Unknown"; @@ -2688,6 +2696,7 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, break; case IBLND_REJECT_INVALID_SRV_ID: + peer_ni->ibp_retries++; reason = "invalid service id"; break; } diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c index 73ad22d..029c9fb 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -93,7 +93,7 @@ static int retry_count = 5; module_param(retry_count, int, 0644); -MODULE_PARM_DESC(retry_count, "Retransmissions when no ACK received"); +MODULE_PARM_DESC(retry_count, "Number of times to retry connection operations"); static int rnr_retry_count = 6; module_param(rnr_retry_count, int, 0644); From patchwork Mon Nov 16 00:59:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 159FF14B3 for ; Mon, 16 Nov 2020 01:00:59 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E6EFA2223C for ; Mon, 16 Nov 2020 01:00:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6EFA2223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7FE0C21F325; Sun, 15 Nov 2020 17:00:58 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4660F21FA27 for ; Sun, 15 Nov 2020 17:00:10 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5A1CC223B; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 532A6B5; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:41 -0500 Message-Id: <1605488401-981-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 08/28] lustre: llite: rmdir releases inode on client X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Same as file unlink, rmdir should release inode on client, to achieve this, ll_rmdir() update inode i_nlink after rmdir, then the last iput() will release the inode. WC-bug-id: https://jira.whamcloud.com/browse/LU-13983 Lustre-commit: 4a4794364eb05f ("LU-13983 llite: rmdir releases inode on client") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/40011 Reviewed-by: Andreas Dilger Reviewed-by: Yingjin Qian Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/namei.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index f9c10d0..da6b729 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -1659,12 +1659,23 @@ static int ll_rmdir(struct inode *dir, struct dentry *dchild) rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request); ll_finish_md_op_data(op_data); if (rc == 0) { + struct mdt_body *body; + ll_update_times(request, dir); ll_stats_ops_tally(ll_i2sbi(dir), LPROC_LL_RMDIR, ktime_us_delta(ktime_get(), kstart)); + /* + * The server puts attributes in on the last unlink, use them + * to update the link count so the inode can be freed + * immediately. + */ + body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY); + if (body->mbo_valid & OBD_MD_FLNLINK) + set_nlink(dchild->d_inode, body->mbo_nlink); } ptlrpc_req_finished(request); + return rc; } From patchwork Mon Nov 16 00:59:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E899A138B for ; Mon, 16 Nov 2020 01:01:55 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9CC820E65 for ; Mon, 16 Nov 2020 01:01:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9CC820E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4DAA7306EF1; Sun, 15 Nov 2020 17:01:28 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8421F21FA27 for ; Sun, 15 Nov 2020 17:00:10 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5B1332245; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 55EAC8B09; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:42 -0500 Message-Id: <1605488401-981-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 09/28] lustre: gss: update sequence in case of target disconnect X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Client to OST connections can go idle, leading to target disconnect. In this event, maintaining correct sequence number ensures that GSS does not erroneously consider requests as replays. Sequence is normally updated on export destroy, but this can occur too late, ie after a new target connect request has been processed. So explicitly update sec context at disconnect time. WC-bug-id: https://jira.whamcloud.com/browse/LU-13498 Lustre-commit: 1275857c178fdf ("LU-13498 gss: update sequence in case of target disconnect") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/40122 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/sec.c | 4 ++-- fs/lustre/ptlrpc/sec_null.c | 8 -------- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index ca8a646..44c15e6 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -626,8 +626,8 @@ int sptlrpc_req_refresh_ctx(struct ptlrpc_request *req, long timeout) return 0; if (unlikely(test_bit(PTLRPC_CTX_NEW_BIT, &ctx->cc_flags))) { - LASSERT(ctx->cc_ops->refresh); - ctx->cc_ops->refresh(ctx); + if (ctx->cc_ops->refresh) + ctx->cc_ops->refresh(ctx); } LASSERT(test_bit(PTLRPC_CTX_NEW_BIT, &ctx->cc_flags) == 0); diff --git a/fs/lustre/ptlrpc/sec_null.c b/fs/lustre/ptlrpc/sec_null.c index 14058bf..97c4e19 100644 --- a/fs/lustre/ptlrpc/sec_null.c +++ b/fs/lustre/ptlrpc/sec_null.c @@ -66,13 +66,6 @@ enum lustre_sec_part null_decode_sec_part(struct lustre_msg *msg) return (msg->lm_secflvr >> 24) & 0xFF; } -static int null_ctx_refresh(struct ptlrpc_cli_ctx *ctx) -{ - /* should never reach here */ - LBUG(); - return 0; -} - static int null_ctx_sign(struct ptlrpc_cli_ctx *ctx, struct ptlrpc_request *req) { @@ -374,7 +367,6 @@ int null_authorize(struct ptlrpc_request *req) } static struct ptlrpc_ctx_ops null_ctx_ops = { - .refresh = null_ctx_refresh, .sign = null_ctx_sign, .verify = null_ctx_verify, }; From patchwork Mon Nov 16 00:59:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7965138B for ; Mon, 16 Nov 2020 01:01:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5C6B20578 for ; Mon, 16 Nov 2020 01:01:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5C6B20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1EFA6306E83; Sun, 15 Nov 2020 17:01:19 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C0FC121FA27 for ; Sun, 15 Nov 2020 17:00:10 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5CB4B2246; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5929D8B17; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:43 -0500 Message-Id: <1605488401-981-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 10/28] lustre: lov: doesn't check lov_refcount X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Hongchao Zhang , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Hongchao Zhang In lov_cleanup, the check of each OSC is protected by lov_tgt_getrefs, which will increment the "lov_refcount", so the "lov_refcount" shouldn't be checked inside because it is always larger than 0. WC-bug-id: https://jira.whamcloud.com/browse/LU-13719 Lustre-commit: 6ae92a7f1bd94c ("LU-13719 lov: doesn't check lov_refcount") Signed-off-by: Hongchao Zhang Reviewed-on: https://review.whamcloud.com/39702 Reviewed-by: Lai Siyao Reviewed-by: Bobi Jam Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/lov/lov_obd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/lustre/lov/lov_obd.c b/fs/lustre/lov/lov_obd.c index d88d325..c8654bd 100644 --- a/fs/lustre/lov/lov_obd.c +++ b/fs/lustre/lov/lov_obd.c @@ -820,8 +820,7 @@ static int lov_cleanup(struct obd_device *obd) continue; /* Inactive targets may never have connected */ - if (lov->lov_tgts[i]->ltd_active || - atomic_read(&lov->lov_refcount)) + if (lov->lov_tgts[i]->ltd_active) /* We should never get here - these * should have been removed in the * disconnect. From patchwork Mon Nov 16 00:59:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4876138B for ; Mon, 16 Nov 2020 01:01:21 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5103D2054F for ; Mon, 16 Nov 2020 01:01:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5103D2054F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5AFF9306D5D; Sun, 15 Nov 2020 17:01:04 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0A11E21FA27 for ; Sun, 15 Nov 2020 17:00:11 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5F5E82247; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5C1492C7E6; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:44 -0500 Message-Id: <1605488401-981-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 11/28] lustre: ptlrpc: remove unused code at pinger X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Boyko , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexander Boyko The timeout_list was previously used for grant shrinking, but right now is dead code. HPE-bug-id: LUS-8520 Fixes: abc88e83673c ("lustre: osc: depart grant shrinking from pinger") WC-bug-id: https://jira.whamcloud.com/browse/LU-14031 Lustre-commit: f022663059414 ("LU-14031 ptlrpc: remove unused code at pinger") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/40243 Reviewed-by: Aurelien Degremont Reviewed-by: Alexey Lyashkov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 5 -- fs/lustre/ptlrpc/pinger.c | 137 ----------------------------------------- 2 files changed, 142 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index 1e7fe03..61be05c 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -2369,11 +2369,6 @@ enum timeout_event { typedef int (*timeout_cb_t)(struct timeout_item *, void *); int ptlrpc_pinger_add_import(struct obd_import *imp); int ptlrpc_pinger_del_import(struct obd_import *imp); -int ptlrpc_add_timeout_client(time64_t time, enum timeout_event event, - timeout_cb_t cb, void *data, - struct list_head *obd_list); -int ptlrpc_del_timeout_client(struct list_head *obd_list, - enum timeout_event event); struct ptlrpc_request *ptlrpc_prep_ping(struct obd_import *imp); int ptlrpc_obd_ping(struct obd_device *obd); void ptlrpc_pinger_ir_up(void); diff --git a/fs/lustre/ptlrpc/pinger.c b/fs/lustre/ptlrpc/pinger.c index 9f57c61..e23ba3c 100644 --- a/fs/lustre/ptlrpc/pinger.c +++ b/fs/lustre/ptlrpc/pinger.c @@ -51,7 +51,6 @@ struct mutex pinger_mutex; static LIST_HEAD(pinger_imports); -static LIST_HEAD(timeout_list); struct ptlrpc_request * ptlrpc_prep_ping(struct obd_import *imp) @@ -162,20 +161,8 @@ static inline time64_t ptlrpc_next_reconnect(struct obd_import *imp) static time64_t pinger_check_timeout(time64_t time) { - struct timeout_item *item; time64_t timeout = PING_INTERVAL; - /* This list is sorted in increasing timeout order */ - mutex_lock(&pinger_mutex); - list_for_each_entry(item, &timeout_list, ti_chain) { - time64_t ti_timeout = item->ti_timeout; - - if (timeout > ti_timeout) - timeout = ti_timeout; - break; - } - mutex_unlock(&pinger_mutex); - return time + timeout - ktime_get_seconds(); } @@ -259,16 +246,12 @@ static void ptlrpc_pinger_process_import(struct obd_import *imp, static void ptlrpc_pinger_main(struct work_struct *ws) { time64_t this_ping, time_after_ping, time_to_next_wake; - struct timeout_item *item; struct obd_import *imp; do { this_ping = ktime_get_seconds(); mutex_lock(&pinger_mutex); - list_for_each_entry(item, &timeout_list, ti_chain) { - item->ti_cb(item, item->ti_cb_data); - } list_for_each_entry(imp, &pinger_imports, imp_pinger_chain) { ptlrpc_pinger_process_import(imp, this_ping); /* obd_timeout might have changed */ @@ -323,15 +306,12 @@ int ptlrpc_start_pinger(void) return 0; } -static int ptlrpc_pinger_remove_timeouts(void); - int ptlrpc_stop_pinger(void) { #ifdef CONFIG_LUSTRE_FS_PINGER if (!pinger_wq) return -EALREADY; - ptlrpc_pinger_remove_timeouts(); cancel_delayed_work_sync(&ping_work); destroy_workqueue(pinger_wq); pinger_wq = NULL; @@ -398,123 +378,6 @@ int ptlrpc_pinger_del_import(struct obd_import *imp) } EXPORT_SYMBOL(ptlrpc_pinger_del_import); -/** - * Register a timeout callback to the pinger list, and the callback will - * be called when timeout happens. - */ -static struct timeout_item *ptlrpc_new_timeout(time64_t time, - enum timeout_event event, - timeout_cb_t cb, void *data) -{ - struct timeout_item *ti; - - ti = kzalloc(sizeof(*ti), GFP_NOFS); - if (!ti) - return NULL; - - INIT_LIST_HEAD(&ti->ti_obd_list); - INIT_LIST_HEAD(&ti->ti_chain); - ti->ti_timeout = time; - ti->ti_event = event; - ti->ti_cb = cb; - ti->ti_cb_data = data; - - return ti; -} - -/** - * Register timeout event on the pinger thread. - * Note: the timeout list is an sorted list with increased timeout value. - */ -static struct timeout_item* -ptlrpc_pinger_register_timeout(time64_t time, enum timeout_event event, - timeout_cb_t cb, void *data) -{ - struct timeout_item *item, *tmp; - - LASSERT(mutex_is_locked(&pinger_mutex)); - - list_for_each_entry(item, &timeout_list, ti_chain) - if (item->ti_event == event) - goto out; - - item = ptlrpc_new_timeout(time, event, cb, data); - if (item) { - list_for_each_entry_reverse(tmp, &timeout_list, ti_chain) { - if (tmp->ti_timeout < time) { - list_add(&item->ti_chain, &tmp->ti_chain); - goto out; - } - } - list_add(&item->ti_chain, &timeout_list); - } -out: - return item; -} - -/* Add a client_obd to the timeout event list, when timeout(@time) - * happens, the callback(@cb) will be called. - */ -int ptlrpc_add_timeout_client(time64_t time, enum timeout_event event, - timeout_cb_t cb, void *data, - struct list_head *obd_list) -{ - struct timeout_item *ti; - - mutex_lock(&pinger_mutex); - ti = ptlrpc_pinger_register_timeout(time, event, cb, data); - if (!ti) { - mutex_unlock(&pinger_mutex); - return -EINVAL; - } - list_add(obd_list, &ti->ti_obd_list); - mutex_unlock(&pinger_mutex); - return 0; -} -EXPORT_SYMBOL(ptlrpc_add_timeout_client); - -int ptlrpc_del_timeout_client(struct list_head *obd_list, - enum timeout_event event) -{ - struct timeout_item *ti = NULL, *item; - - if (list_empty(obd_list)) - return 0; - mutex_lock(&pinger_mutex); - list_del_init(obd_list); - /** - * If there are no obd attached to the timeout event - * list, remove this timeout event from the pinger - */ - list_for_each_entry(item, &timeout_list, ti_chain) { - if (item->ti_event == event) { - ti = item; - break; - } - } - if (list_empty(&ti->ti_obd_list)) { - list_del(&ti->ti_chain); - kfree(ti); - } - mutex_unlock(&pinger_mutex); - return 0; -} -EXPORT_SYMBOL(ptlrpc_del_timeout_client); - -static int ptlrpc_pinger_remove_timeouts(void) -{ - struct timeout_item *item, *tmp; - - mutex_lock(&pinger_mutex); - list_for_each_entry_safe(item, tmp, &timeout_list, ti_chain) { - LASSERT(list_empty(&item->ti_obd_list)); - list_del(&item->ti_chain); - kfree(item); - } - mutex_unlock(&pinger_mutex); - return 0; -} - void ptlrpc_pinger_wake_up(void) { #ifdef CONFIG_LUSTRE_FS_PINGER From patchwork Mon Nov 16 00:59:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4762114B3 for ; Mon, 16 Nov 2020 01:01:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2498B20578 for ; Mon, 16 Nov 2020 01:01:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2498B20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F2D1E306DBB; Sun, 15 Nov 2020 17:01:06 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5A30621FA27 for ; Sun, 15 Nov 2020 17:00:11 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6239B2249; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5F01C2C7FB; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:45 -0500 Message-Id: <1605488401-981-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 12/28] lustre: mdc: remote object support getattr from cache X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao For historical reason, IT_GETATTR lock revalidate matches LOOKUP|UPDATE|PERM lock bits because for MDS < 2.4, permission is protected by LOOKUP lock, but this will cause remote object not able to match the cached lock because LOOKUP and UPDATE lock are fetched separately. Add sanity 803b, and rename 803 to 803a. WC-bug-id: https://jira.whamcloud.com/browse/LU-13437 Lustre-commit: 72a1ca996e3a35 ("LU-13437 mdc: remote object support getattr from cache") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/40218 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_locks.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 72ee070..a82e8ca 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -1235,7 +1235,6 @@ int mdc_revalidate_lock(struct obd_export *exp, struct lookup_intent *it, * by LOOKUP lock, so it needs to match all bits here. */ policy.l_inodebits.bits = MDS_INODELOCK_UPDATE | - MDS_INODELOCK_LOOKUP | MDS_INODELOCK_PERM; break; case IT_READDIR: From patchwork Mon Nov 16 00:59:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44634138B for ; Mon, 16 Nov 2020 01:01:00 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 211952223C for ; Mon, 16 Nov 2020 01:01:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 211952223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2F7BA21F55D; Sun, 15 Nov 2020 17:00:59 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 99AF021FB8B for ; Sun, 15 Nov 2020 17:00:11 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 645E7224A; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6231D2C802; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:46 -0500 Message-Id: <1605488401-981-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 13/28] lustre: llite: pass name in getattr by FID X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao Now parent FID is packed in getattr_by_FID request (see https://review.whamcloud.com/39290), it should also pass in name from llite, so that lmv can replace fid1 with stripe FID, otherwise MDS may treat sub files under striped directory as remote object. Note, the name is not packed in request, because if it's packed, MDS will getattr by name instead of FID. Fixes: 3a3a9eeaa ("lustre: llite: pack parent FID in getattr") WC-bug-id: https://jira.whamcloud.com/browse/LU-13437 Lustre-commit: 90ebab5833007d ("LU-13437 llite: pass name in getattr by FID") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/40219 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd.h | 1 + fs/lustre/llite/file.c | 15 +++++++++++---- fs/lustre/llite/llite_lib.c | 4 +++- fs/lustre/lmv/lmv_intent.c | 22 +++++++++++++++++++--- fs/lustre/lmv/lmv_obd.c | 8 ++++++-- 5 files changed, 40 insertions(+), 10 deletions(-) diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index 39e3d51..a017997 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -704,6 +704,7 @@ enum md_op_flags { MF_MDC_CANCEL_FID3 = BIT(2), MF_MDC_CANCEL_FID4 = BIT(3), MF_GET_MDT_IDX = BIT(4), + MF_GETATTR_BY_FID = BIT(5), }; enum md_cli_flags { diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 0a9a689..3ba9152 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -4587,22 +4587,29 @@ static int ll_inode_revalidate(struct dentry *dentry, enum ldlm_intent_flags op) }; struct ptlrpc_request *req = NULL; struct md_op_data *op_data; + const char *name = NULL; + size_t namelen = 0; int rc = 0; CDEBUG(D_VFSTRACE, "VFS Op:inode=" DFID "(%p),name=%pd\n", PFID(ll_inode2fid(inode)), inode, dentry); - if (exp_connect_flags2(exp) & OBD_CONNECT2_GETATTR_PFID) + if (exp_connect_flags2(exp) & OBD_CONNECT2_GETATTR_PFID) { parent = dentry->d_parent->d_inode; - else + name = dentry->d_name.name; + namelen = dentry->d_name.len; + } else { parent = inode; + } - /* Call getattr by fid, so do not provide name at all. */ - op_data = ll_prep_md_op_data(NULL, parent, inode, NULL, 0, 0, + op_data = ll_prep_md_op_data(NULL, parent, inode, name, namelen, 0, LUSTRE_OPC_ANY, NULL); if (IS_ERR(op_data)) return PTR_ERR(op_data); + /* Call getattr by fid */ + if (exp_connect_flags2(exp) & OBD_CONNECT2_GETATTR_PFID) + op_data->op_flags = MF_GETATTR_BY_FID; rc = md_intent_lock(exp, op_data, &oit, &req, &ll_md_blocking_ast, 0); ll_finish_md_op_data(op_data); if (rc < 0) { diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 8ef2437..d94c6ca 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -2811,7 +2811,9 @@ struct md_op_data *ll_prep_md_op_data(struct md_op_data *op_data, if (namelen > ll_i2sbi(i1)->ll_namelen) return ERR_PTR(-ENAMETOOLONG); - if (!lu_name_is_valid_2(name, namelen)) + /* "/" is not valid name, but it's allowed */ + if (!lu_name_is_valid_2(name, namelen) && + strncmp("/", name, namelen) != 0) return ERR_PTR(-EINVAL); } diff --git a/fs/lustre/lmv/lmv_intent.c b/fs/lustre/lmv/lmv_intent.c index 3960b93..ad59b64 100644 --- a/fs/lustre/lmv/lmv_intent.c +++ b/fs/lustre/lmv/lmv_intent.c @@ -448,13 +448,29 @@ static int lmv_intent_lookup(struct obd_export *exp, } retry: - if (op_data->op_name) { + if (op_data->op_flags & MF_GETATTR_BY_FID) { + /* getattr by FID, replace fid1 with stripe FID */ + LASSERT(op_data->op_name); + tgt = lmv_locate_tgt(lmv, op_data); + if (IS_ERR(tgt)) + return PTR_ERR(tgt); + + /* name is used to locate stripe target, clear it here + * to avoid packing name in request, so that MDS knows + * it's getattr by FID. + */ + op_data->op_name = NULL; + op_data->op_namelen = 0; + + /* getattr request is sent to MDT where fid2 inode is */ + tgt = lmv_fid2tgt(lmv, &op_data->op_fid2); + } else if (op_data->op_name) { + /* getattr by name */ tgt = lmv_locate_tgt(lmv, op_data); if (!fid_is_sane(&op_data->op_fid2)) fid_zero(&op_data->op_fid2); - } else if (fid_is_sane(&op_data->op_fid2)) { - tgt = lmv_fid2tgt(lmv, &op_data->op_fid2); } else { + /* old way to getattr by FID, parent FID not packed */ tgt = lmv_fid2tgt(lmv, &op_data->op_fid1); } if (IS_ERR(tgt)) diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 5a75c69..fa1dae5 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -1513,6 +1513,9 @@ static struct lu_tgt_desc *lmv_locate_tgt_rr(struct lmv_obd *lmv, u32 *mdt) return ERR_PTR(-ENODEV); } +/* locate MDT by file name, for striped directory, the file name hash decides + * which stripe its dirent is stored. + */ static struct lmv_tgt_desc * lmv_locate_tgt_by_name(struct lmv_obd *lmv, struct lmv_stripe_md *lsm, const char *name, int namelen, struct lu_fid *fid, @@ -1564,8 +1567,9 @@ static struct lu_tgt_desc *lmv_locate_tgt_rr(struct lmv_obd *lmv, u32 *mdt) * For plain direcotry, it just locate the MDT of op_data->op_fid1. * * @lmv: LMV device - * @op_data: client MD stack parameters, name, namelen - * mds_num etc. + * @op_data: client MD stack parameters, name, namelen etc, + * op_mds and op_fid1 will be updated if op_mea1 + * indicates fid1 represents a striped directory. * * Returns: pointer to the lmv_tgt_desc if succeed. * ERR_PTR(errno) if failed. From patchwork Mon Nov 16 00:59:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 044F5138B for ; Mon, 16 Nov 2020 01:01:15 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A1B692223C for ; Mon, 16 Nov 2020 01:01:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A1B692223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EDC4E21FBBB; Sun, 15 Nov 2020 17:01:01 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EA92721FC00 for ; Sun, 15 Nov 2020 17:00:11 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 669F6224B; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 651F3B5; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:47 -0500 Message-Id: <1605488401-981-15-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 14/28] lnet: o2iblnd: 'Timed out tx' error message X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Brian Behlendorf Trivial fix to report the total RDMA time outstanding rather than the number of seconds past the deadline. WC-bug-id: https://jira.whamcloud.com/browse/LU-1742 Lustre-commit: 2be289a2b1f12b ("LU-1742 o2iblnd: 'Timed out tx' error message") Signed-off-by: Brian Behlendorf Reviewed-on: https://review.whamcloud.com/3622 Reviewed-by: Chris Horn Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index 9de733d..3d7026b 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -3219,6 +3219,7 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, if (ktime_compare(ktime_get(), tx->tx_deadline) >= 0) { CERROR("Timed out tx: %s, %lld seconds\n", kiblnd_queue2str(conn, txs), + kiblnd_timeout() + ktime_ms_delta(ktime_get(), tx->tx_deadline) / MSEC_PER_SEC); return 1; From patchwork Mon Nov 16 00:59:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7697C14B3 for ; Mon, 16 Nov 2020 01:02:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5830320E65 for ; Mon, 16 Nov 2020 01:02:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5830320E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id ECE5B306FBB; Sun, 15 Nov 2020 17:01:38 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3543921FC00 for ; Sun, 15 Nov 2020 17:00:12 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6B46E224C; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 681331511A; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:48 -0500 Message-Id: <1605488401-981-16-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 15/28] lustre: ldlm: Fix unbounded OBD_FAIL_LDLM_CANCEL_BL_CB_RACE wait X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin in ldlm_handle_cp_callback the while loop is clearly supposed to be limited by the "to" value of 1 second, but is not. Seems to have been broken by all the Solaris porting in HEAD all the way back in 2008. Restore the to assignment to make it not hang indefinitely. Fixes: adde80ff ("Land b_head_libcfs onto OpenSFS tree HEAD") WC-bug-id: https://jira.whamcloud.com/browse/LU-14069 Lustre-commit: 5da99051e58b9e ("LU-14069 ldlm: Fix unbounded OBD_FAIL_LDLM_CANCEL_BL_CB_RACE wait") Signed-off-by: Oleg Drokin Reviewed-on: https://review.whamcloud.com/40375 Reviewed-by: Mike Pershin Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_lockd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 4a91a7f..1ae65b829 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -223,7 +223,7 @@ static int ldlm_handle_cp_callback(struct ptlrpc_request *req, ldlm_callback_reply(req, 0); while (to > 0) { - schedule_timeout_interruptible(to); + to = schedule_timeout_interruptible(to); if (ldlm_is_granted(lock) || ldlm_is_destroyed(lock)) break; From patchwork Mon Nov 16 00:59:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 685E1138B for ; Mon, 16 Nov 2020 01:01:47 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A68F20578 for ; Mon, 16 Nov 2020 01:01:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A68F20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 02A27306EB0; Sun, 15 Nov 2020 17:01:22 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7271221FDA2 for ; Sun, 15 Nov 2020 17:00:12 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6CEA7224D; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6B2111511D; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:49 -0500 Message-Id: <1605488401-981-17-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 16/28] lustre: ldlm: group locks for DOM IBIT lock X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vitaly Fertman , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Vitaly Fertman Group lock is supposed to be taken on such operations as layout swap used for e.g. HSM, and is to be taken for DOM locks as well. HPE-bug-id: LUS-8987 WC-bug-id: https://jira.whamcloud.com/browse/LU-13645 Lustre-commit: 06740440363424 (LU-13645 ldlm: group locks for DOM IBIT lock") Signed-off-by: Vitaly Fertman Reviewed-on: https://review.whamcloud.com/39406 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_inodebits.c | 2 ++ fs/lustre/ldlm/ldlm_lock.c | 12 ++++++++++-- fs/lustre/mdc/mdc_dev.c | 10 ++++++++-- fs/lustre/ptlrpc/wiretest.c | 6 +++++- include/uapi/linux/lustre/lustre_idl.h | 1 + 5 files changed, 26 insertions(+), 5 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_inodebits.c b/fs/lustre/ldlm/ldlm_inodebits.c index 2288eb5..a367ff1 100644 --- a/fs/lustre/ldlm/ldlm_inodebits.c +++ b/fs/lustre/ldlm/ldlm_inodebits.c @@ -59,6 +59,7 @@ void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits; + lpolicy->l_inodebits.li_gid = wpolicy->l_inodebits.li_gid; } void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, @@ -66,6 +67,7 @@ void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, { memset(wpolicy, 0, sizeof(*wpolicy)); wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits; + wpolicy->l_inodebits.li_gid = lpolicy->l_inodebits.li_gid; } /** diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index 0dbd4f3..56f1550 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -1111,6 +1111,12 @@ static bool lock_matches(struct ldlm_lock *lock, void *vdata) data->lmd_policy->l_inodebits.bits) != data->lmd_policy->l_inodebits.bits) return false; + + if (unlikely(match == LCK_GROUP) && + data->lmd_policy->l_inodebits.li_gid != LDLM_GID_ANY && + lpol->l_inodebits.li_gid != + data->lmd_policy->l_inodebits.li_gid) + return false; break; default: break; @@ -1903,7 +1909,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, switch (resource->lr_type) { case LDLM_EXTENT: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) gid %llu flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, @@ -1918,6 +1924,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, lock->l_policy_data.l_extent.end, lock->l_req_extent.start, lock->l_req_extent.end, + lock->l_req_extent.gid, lock->l_flags, nid, lock->l_remote_handle.cookie, exp ? refcount_read(&exp->exp_handle.h_ref) : -99, @@ -1949,7 +1956,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, case LDLM_IBITS: libcfs_debug_msg(msgdata, - "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", + "%pV ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s gid %llu flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lld lvb_type: %d\n", &vaf, ldlm_lock_to_ns_name(lock), lock, lock->l_handle.h_cookie, @@ -1961,6 +1968,7 @@ void _ldlm_lock_debug(struct ldlm_lock *lock, lock->l_policy_data.l_inodebits.bits, atomic_read(&resource->lr_refcount), ldlm_typename[resource->lr_type], + lock->l_policy_data.l_inodebits.li_gid, lock->l_flags, nid, lock->l_remote_handle.cookie, exp ? refcount_read(&exp->exp_handle.h_ref) : -99, diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 329371b..90b60f5 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -40,10 +40,13 @@ #include "mdc_internal.h" static void mdc_lock_build_policy(const struct lu_env *env, + const struct cl_lock *lock, union ldlm_policy_data *policy) { memset(policy, 0, sizeof(*policy)); policy->l_inodebits.bits = MDS_INODELOCK_DOM; + if (lock) + policy->l_inodebits.li_gid = lock->cll_descr.cld_gid; } int mdc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) @@ -144,7 +147,8 @@ struct ldlm_lock *mdc_dlmlock_at_pgoff(const struct lu_env *env, enum ldlm_match_flags match_flags = 0; fid_build_reg_res_name(lu_object_fid(osc2lu(obj)), resname); - mdc_lock_build_policy(env, policy); + mdc_lock_build_policy(env, NULL, policy); + policy->l_inodebits.li_gid = LDLM_GID_ANY; flags = LDLM_FL_BLOCK_GRANTED | LDLM_FL_CBPENDING; if (dap_flags & OSC_DAP_FL_TEST_LOCK) @@ -867,7 +871,7 @@ static int mdc_lock_enqueue(const struct lu_env *env, * osc_lock. */ fid_build_reg_res_name(lu_object_fid(osc2lu(osc)), resname); - mdc_lock_build_policy(env, policy); + mdc_lock_build_policy(env, lock, policy); LASSERT(!oscl->ols_speculative); result = mdc_enqueue_send(env, osc_export(osc), resname, &oscl->ols_flags, policy, @@ -931,6 +935,8 @@ int mdc_lock_init(const struct lu_env *env, struct cl_object *obj, ols->ols_flags = flags; ols->ols_speculative = !!(enqflags & CEF_SPECULATIVE); + if (lock->cll_descr.cld_mode == CLM_GROUP) + ols->ols_flags |= LDLM_FL_ATOMIC_CB; if (ols->ols_flags & LDLM_FL_HAS_INTENT) { ols->ols_flags |= LDLM_FL_BLOCK_GRANTED; diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 556aaff..9b1caf4 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -3249,12 +3249,16 @@ void lustre_assert_wire_constants(void) (long long)(int)sizeof(((struct ldlm_extent *)0)->gid)); /* Checks for struct ldlm_inodebits */ - LASSERTF((int)sizeof(struct ldlm_inodebits) == 16, "found %lld\n", + LASSERTF((int)sizeof(struct ldlm_inodebits) == 24, "found %lld\n", (long long)(int)sizeof(struct ldlm_inodebits)); LASSERTF((int)offsetof(struct ldlm_inodebits, bits) == 0, "found %lld\n", (long long)(int)offsetof(struct ldlm_inodebits, bits)); LASSERTF((int)sizeof(((struct ldlm_inodebits *)0)->bits) == 8, "found %lld\n", (long long)(int)sizeof(((struct ldlm_inodebits *)0)->bits)); + LASSERTF((int)offsetof(struct ldlm_inodebits, li_gid) == 16, "found %lld\n", + (long long)(int)offsetof(struct ldlm_inodebits, li_gid)); + LASSERTF((int)sizeof(((struct ldlm_inodebits *)0)->li_gid) == 8, "found %lld\n", + (long long)(int)sizeof(((struct ldlm_inodebits *)0)->li_gid)); /* Checks for struct ldlm_flock_wire */ LASSERTF((int)sizeof(struct ldlm_flock_wire) == 32, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index fda56d8..34b2367 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -2206,6 +2206,7 @@ static inline bool ldlm_extent_equal(const struct ldlm_extent *ex1, struct ldlm_inodebits { __u64 bits; __u64 cancel_bits; /* for lock convert */ + __u64 li_gid; }; struct ldlm_flock_wire { From patchwork Mon Nov 16 00:59:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B12FA14B3 for ; Mon, 16 Nov 2020 01:01:52 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 946CD20E65 for ; Mon, 16 Nov 2020 01:01:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 946CD20E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8DE82306ED9; Sun, 15 Nov 2020 17:01:26 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C294421FFA9 for ; Sun, 15 Nov 2020 17:00:12 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6FD8B224E; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6E0771512A; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:50 -0500 Message-Id: <1605488401-981-18-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 17/28] lustre: ptlrpc: decrease time between reconnection X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Boyko , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Alexander Boyko When a connection get a timeout or get an error reply from a sever, the next attempt happens after PING_INTERVAL. It is equal to obd_timeout/4. When a first reconnection fails, a second go to failover pair. And a third connection go to a original server. Only 3 reconnection before server evicts client base on blocking ast timeout. Some times a first failed and the last is a bit late, so client is evicted. It is better to try reconnect with a timeout equal to a connection request deadline, it would increase a number of attempts in 5 times for a large obd_timeout. For example, obd_timeout=200 - [ 1597902357, CONNECTING ] - [ 1597902357, FULL ] - [ 1597902422, DISCONN ] - [ 1597902422, CONNECTING ] - [ 1597902433, DISCONN ] - [ 1597902473, CONNECTING ] - [ 1597902473, DISCONN ] <- ENODEV from a failover pair - [ 1597902523, CONNECTING ] - [ 1597902539, DISCONN ] The patch adds a logic to wakeup pinger for failed connection request with ETIMEDOUT or ENODEV. It adds imp_next_ping processing for ptlrpc_pinger_main() time_to_next_wake calculation, and fixes setting of imp_next_ping value. HPE-bug-id: LUS-8520 WC-bug-id: https://jira.whamcloud.com/browse/LU-14031 Lustre-commit: de8ed5f19f0413 ("LU-14031 ptlrpc: decrease time between reconnection") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/40244 Reviewed-by: Andreas Dilger Reviewed-by: Alexey Lyashkov Reviewed-by: Vitaly Fertman Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/events.c | 5 ++++ fs/lustre/ptlrpc/import.c | 36 ++++++++++++++++++++++- fs/lustre/ptlrpc/niobuf.c | 2 -- fs/lustre/ptlrpc/pinger.c | 73 ++++++++++++++++++++++++++++++----------------- 4 files changed, 87 insertions(+), 29 deletions(-) diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 0943612..fe33600 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -59,6 +59,11 @@ void request_out_callback(struct lnet_event *ev) DEBUG_REQ(D_NET, req, "type %d, status %d", ev->type, ev->status); + /* Do not update imp_next_ping for connection request */ + if (lustre_msg_get_opc(req->rq_reqmsg) != + req->rq_import->imp_connect_op) + ptlrpc_pinger_sending_on_import(req->rq_import); + sptlrpc_request_out_callback(req); spin_lock(&req->rq_lock); diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 4e573cd..21ce593 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -1037,7 +1037,6 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, */ imp->imp_force_reconnect = ptlrpc_busy_reconnect(rc); spin_unlock(&imp->imp_lock); - ptlrpc_maybe_ping_import_soon(imp); goto out; } @@ -1303,6 +1302,8 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, if (rc) { bool inact = false; + time64_t now = ktime_get_seconds(); + time64_t next_connect; import_set_state_nolock(imp, LUSTRE_IMP_DISCON); if (rc == -EACCES) { @@ -1344,7 +1345,28 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, import_set_state_nolock(imp, LUSTRE_IMP_CLOSED); inact = true; } + } else if (rc == -ENODEV || rc == -ETIMEDOUT) { + /* ENODEV means there is no service, force reconnection + * to a pair if attempt happen ptlrpc_next_reconnect + * before now. ETIMEDOUT could be set during network + * error and do not guarantee request deadline happened. + */ + struct obd_import_conn *conn; + time64_t reconnect_time; + + /* Same as ptlrpc_next_reconnect, but in past */ + reconnect_time = now - INITIAL_CONNECT_TIMEOUT; + list_for_each_entry(conn, &imp->imp_conn_list, + oic_item) { + if (conn->oic_last_attempt <= reconnect_time) { + imp->imp_force_verify = 1; + break; + } + } } + + next_connect = imp->imp_conn_current->oic_last_attempt + + (request->rq_deadline - request->rq_sent); spin_unlock(&imp->imp_lock); if (inact) @@ -1353,6 +1375,18 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, if (rc == -EPROTO) return rc; + /* adjust imp_next_ping to request deadline + 1 and reschedule + * a pinger if import lost processing during CONNECTING or far + * away from request deadline. It could happen when connection + * was initiated outside of pinger, like + * ptlrpc_set_import_discon(). + */ + if (!imp->imp_force_verify && (imp->imp_next_ping <= now || + imp->imp_next_ping > next_connect)) { + imp->imp_next_ping = max(now, next_connect) + 1; + ptlrpc_pinger_wake_up(); + } + ptlrpc_maybe_ping_import_soon(imp); CDEBUG(D_HA, "recovery of %s on %s failed (%d)\n", diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index 924b9c4..a1e6581 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -701,8 +701,6 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) request->rq_deadline = request->rq_sent + request->rq_timeout + ptlrpc_at_get_net_latency(request); - ptlrpc_pinger_sending_on_import(imp); - DEBUG_REQ(D_INFO, request, "send flags=%x", lustre_msg_get_flags(request->rq_reqmsg)); rc = ptl_send_buf(&request->rq_req_md_h, diff --git a/fs/lustre/ptlrpc/pinger.c b/fs/lustre/ptlrpc/pinger.c index e23ba3c..178153c 100644 --- a/fs/lustre/ptlrpc/pinger.c +++ b/fs/lustre/ptlrpc/pinger.c @@ -108,6 +108,21 @@ static bool ptlrpc_check_import_is_idle(struct obd_import *imp) return true; } +static void ptlrpc_update_next_ping(struct obd_import *imp, int soon) +{ +#ifdef CONFIG_LUSTRE_FS_PINGER + time64_t time = soon ? PING_INTERVAL_SHORT : PING_INTERVAL; + + if (imp->imp_state == LUSTRE_IMP_DISCON) { + time64_t dtime = max_t(time64_t, CONNECTION_SWITCH_MIN, + AT_OFF ? 0 : + at_get(&imp->imp_at.iat_net_latency)); + time = min(time, dtime); + } + imp->imp_next_ping = ktime_get_seconds() + time; +#endif +} + static int ptlrpc_ping(struct obd_import *imp) { struct ptlrpc_request *req; @@ -125,26 +140,17 @@ static int ptlrpc_ping(struct obd_import *imp) DEBUG_REQ(D_INFO, req, "pinging %s->%s", imp->imp_obd->obd_uuid.uuid, obd2cli_tgt(imp->imp_obd)); + /* Updating imp_next_ping early, it allows pinger_check_timeout to + * see an actual time for next awake. request_out_callback update + * happens at another thread, and ptlrpc_pinger_main may sleep + * already. + */ + ptlrpc_update_next_ping(imp, 0); ptlrpcd_add_req(req); return 0; } -static void ptlrpc_update_next_ping(struct obd_import *imp, int soon) -{ -#ifdef CONFIG_LUSTRE_FS_PINGER - time64_t time = soon ? PING_INTERVAL_SHORT : PING_INTERVAL; - - if (imp->imp_state == LUSTRE_IMP_DISCON) { - time64_t dtime = max_t(time64_t, CONNECTION_SWITCH_MIN, - AT_OFF ? 0 : - at_get(&imp->imp_at.iat_net_latency)); - time = min(time, dtime); - } - imp->imp_next_ping = ktime_get_seconds() + time; -#endif -} - static inline int imp_is_deactive(struct obd_import *imp) { return (imp->imp_deactive || @@ -153,17 +159,32 @@ static inline int imp_is_deactive(struct obd_import *imp) static inline time64_t ptlrpc_next_reconnect(struct obd_import *imp) { - if (imp->imp_server_timeout) - return ktime_get_seconds() + (obd_timeout >> 1); - else - return ktime_get_seconds() + obd_timeout; + return ktime_get_seconds() + INITIAL_CONNECT_TIMEOUT; } -static time64_t pinger_check_timeout(time64_t time) +static timeout_t pinger_check_timeout(time64_t time) { - time64_t timeout = PING_INTERVAL; + timeout_t timeout = PING_INTERVAL; + timeout_t next_timeout; + time64_t now; + struct list_head *iter; + struct obd_import *imp; + + mutex_lock(&pinger_mutex); + now = ktime_get_seconds(); + /* Process imports to find a nearest next ping */ + list_for_each(iter, &pinger_imports) { + imp = list_entry(iter, struct obd_import, imp_pinger_chain); + if (!imp->imp_pingable || imp->imp_next_ping < now) + continue; + next_timeout = imp->imp_next_ping - now; + /* make sure imp_next_ping in the future from time */ + if (next_timeout > (now - time) && timeout > next_timeout) + timeout = next_timeout; + } + mutex_unlock(&pinger_mutex); - return time + timeout - ktime_get_seconds(); + return timeout - (now - time); } static bool ir_up; @@ -245,7 +266,8 @@ static void ptlrpc_pinger_process_import(struct obd_import *imp, static void ptlrpc_pinger_main(struct work_struct *ws) { - time64_t this_ping, time_after_ping, time_to_next_wake; + time64_t this_ping, time_after_ping; + timeout_t time_to_next_wake; struct obd_import *imp; do { @@ -276,9 +298,8 @@ static void ptlrpc_pinger_main(struct work_struct *ws) * we will SKIP the next ping at next_ping, and the * ping will get sent 2 timeouts from now! Beware. */ - CDEBUG(D_INFO, "next wakeup in %lld (%lld)\n", - time_to_next_wake, - this_ping + PING_INTERVAL); + CDEBUG(D_INFO, "next wakeup in %d (%lld)\n", + time_to_next_wake, this_ping + PING_INTERVAL); } while (time_to_next_wake <= 0); queue_delayed_work(pinger_wq, &ping_work, From patchwork Mon Nov 16 00:59:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5ADA9138B for ; Mon, 16 Nov 2020 01:01:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E35120578 for ; Mon, 16 Nov 2020 01:01:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E35120578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A9B57306E0E; Sun, 15 Nov 2020 17:01:09 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1E666308152 for ; Sun, 15 Nov 2020 17:00:13 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 72F93224F; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 710A12C7E6; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:51 -0500 Message-Id: <1605488401-981-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 18/28] lustre: ptlrpc: throttle RPC resend if network error X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Aurelien Degremont When sending a callback AST to a non-responding client, the server retries endlessly until the client is eventually evicted. When using ksocklnd, it will retry after each AST timeout, until the socket is eventually closed, after sock_timeout sec, where the retry will fail immediately, returning -110, as no socket could be established. The thread will spin on retrying and failing, until eventual client eviction. This will cause high thread CPU usage and possible resource denial. To workaround that, this patch avoids re-trying callback resend if: - the request is flagged with network error and timeout - last try was less than 1 sec ago In worst case, retry will happen after a timeout based on req->rq_deadline. If there is nothing else to handle, thread will be sleeping during that time, removing CPU overhead. WC-bug-id: https://jira.whamcloud.com/browse/LU-13984 Lustre-commit: 4103527c1c9b38 ("LU-13984 ptlrpc: throttle RPC resend if network error") Signed-off-by: Aurelien Degremont Reviewed-on: https://review.whamcloud.com/40020 Reviewed-by: Andreas Dilger Reviewed-by: Alexander Boyko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/client.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index c9d9fe9..0e01ab33 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1900,6 +1900,26 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) goto interpret; } + /* don't resend too fast in case of network + * errors. + */ + if (ktime_get_real_seconds() < (req->rq_sent + 1) + && req->rq_net_err && req->rq_timedout) { + DEBUG_REQ(D_INFO, req, + "throttle request"); + /* Don't try to resend RPC right away + * as it is likely it will fail again + * and ptlrpc_check_set() will be + * called again, keeping this thread + * busy. Instead, wait for the next + * timeout. Flag it as resend to + * ensure we don't wait to long. + */ + req->rq_resend = 1; + spin_unlock(&imp->imp_lock); + continue; + } + list_move_tail(&req->rq_list, &imp->imp_sending_list); From patchwork Mon Nov 16 00:59:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38B7C14B3 for ; Mon, 16 Nov 2020 01:01:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1925720578 for ; Mon, 16 Nov 2020 01:01:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1925720578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 46E2521FA08; Sun, 15 Nov 2020 17:01:08 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5DC5C306CFB for ; Sun, 15 Nov 2020 17:00:13 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 760872250; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 74199B5; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:52 -0500 Message-Id: <1605488401-981-20-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 19/28] lustre: ldlm: BL AST vs failed lock enqueue race X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andriy Skulysh , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andriy Skulysh failed_lock_cleanup() marks the lock with LDLM_FL_LOCAL_ONLY, so cancel request isn't sent. Mark failed lock with LDLM_FL_LOCAL_ONLY only if BL AST wasn't received. Add server's lock handle to BL AST RPC. So client will be able to cancel the lock even if enqueue fails. HPE-bug-id: LUS-8493, LUS-8830 WC-bug-id: https://jira.whamcloud.com/browse/LU-13989 Lustre-commit: c1be044913dde3 ("LU-13989 ldlm: BL AST vs failed lock enqueue race") Signed-off-by: Andriy Skulysh Reviewed-on: https://review.whamcloud.com/40046 Reviewed-by: Vladimir Saveliev Reviewed-by: Alexander Boyko Reviewed-by: Vitaly Fertman Reviewed-by: Alexander Boyko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_lockd.c | 2 ++ fs/lustre/ldlm/ldlm_request.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/lustre/ldlm/ldlm_lockd.c b/fs/lustre/ldlm/ldlm_lockd.c index 1ae65b829..6f498cc 100644 --- a/fs/lustre/ldlm/ldlm_lockd.c +++ b/fs/lustre/ldlm/ldlm_lockd.c @@ -698,6 +698,8 @@ static int ldlm_callback_handler(struct ptlrpc_request *req) ldlm_lock_remove_from_lru(lock); ldlm_set_bl_ast(lock); } + if (lock->l_remote_handle.cookie == 0) + lock->l_remote_handle = dlm_req->lock_handle[1]; unlock_res_and_lock(lock); /* diff --git a/fs/lustre/ldlm/ldlm_request.c b/fs/lustre/ldlm/ldlm_request.c index dd897ec..74bcba2 100644 --- a/fs/lustre/ldlm/ldlm_request.c +++ b/fs/lustre/ldlm/ldlm_request.c @@ -317,8 +317,11 @@ static void failed_lock_cleanup(struct ldlm_namespace *ns, * bl_ast and -EINVAL reply is sent to server anyways. * b=17645 */ - lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_FAILED | + lock->l_flags |= LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING; + if (!(ldlm_is_bl_ast(lock) && + lock->l_remote_handle.cookie != 0)) + lock->l_flags |= LDLM_FL_LOCAL_ONLY; need_cancel = 1; } unlock_res_and_lock(lock); From patchwork Mon Nov 16 00:59:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58F7F14B3 for ; Mon, 16 Nov 2020 01:01:58 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A13A20E65 for ; Mon, 16 Nov 2020 01:01:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A13A20E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 92461306F01; Sun, 15 Nov 2020 17:01:29 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9BCDC306D0B for ; Sun, 15 Nov 2020 17:00:13 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 791282255; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 76FFC2C7FB; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:53 -0500 Message-Id: <1605488401-981-21-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 20/28] lustre: ptlrpc: don't log connection 'restored' inappropriately X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Aurelien Degremont Reverse imports maintain a target->client connection which does not support recovery as client don't run a recovery. At every connection, the reverse import state goes from NEW to RECOVER to FULL which triggers a `Connection restored` log message, even if this is the first connection from this client. Suppress this log message for reverse import to avoid this wrong logging. WC-bug-id: https://jira.whamcloud.com/browse/LU-14057 Lustre-commit: 2135f46b816223 ("LU-14057 ptlrpc: don't log connection 'restored' inappropriately") Signed-off-by: Aurelien Degremont Reviewed-on: https://review.whamcloud.com/40331 Reviewed-by: Andreas Dilger Reviewed-by: Alexander Boyko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/import.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 21ce593..35c4f83 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -1590,12 +1590,16 @@ int ptlrpc_import_recovery_state_machine(struct obd_import *imp) goto out; ptlrpc_activate_import(imp, true); - CDEBUG_LIMIT(imp->imp_was_idle ? - imp->imp_idle_debug : D_CONSOLE, - "%s: Connection restored to %s (at %s)\n", - imp->imp_obd->obd_name, - obd_uuid2str(&conn->c_remote_uuid), - obd_import_nid2str(imp)); + /* Reverse import are flagged with dlm_fake == 1. + * They do not do recovery and connection are not "restored". + */ + if (!imp->imp_dlm_fake) + CDEBUG_LIMIT(imp->imp_was_idle ? + imp->imp_idle_debug : D_CONSOLE, + "%s: Connection restored to %s (at %s)\n", + imp->imp_obd->obd_name, + obd_uuid2str(&conn->c_remote_uuid), + obd_import_nid2str(imp)); spin_lock(&imp->imp_lock); imp->imp_was_idle = 0; spin_unlock(&imp->imp_lock); From patchwork Mon Nov 16 00:59:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907057 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EE48138B for ; Mon, 16 Nov 2020 01:02:20 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D93E22245 for ; Mon, 16 Nov 2020 01:02:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D93E22245 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5AF6E306FE0; Sun, 15 Nov 2020 17:01:42 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DA483306D0B for ; Sun, 15 Nov 2020 17:00:13 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 7BB592256; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 79F4921DA3; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:54 -0500 Message-Id: <1605488401-981-22-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 21/28] lustre: llite: Avoid eternel retry loops with MAP_POPULATE X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin Kernels 5.4+ have an infinite retry loop from MAP_POPULATE mmap option. Use the FAULT_FLAG_RETRY_NOWAIT to instruct filemap_fault to not drop the mmap_sem so if the call fails, we could use the slow path and break the loop from forming. (Idea by Neil Brown) WC-bug-id: https://jira.whamcloud.com/browse/LU-13182 Lustre-commit: bb50c62c6f4cdd ("LU-13182 llite: Avoid eternel retry loops with MAP_POPULATE") Signed-off-by: Oleg Drokin Reviewed-on: https://review.whamcloud.com/40221 Reviewed-by: Neil Brown Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/llite/llite_mmap.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/lustre/llite/llite_mmap.c b/fs/lustre/llite/llite_mmap.c index f77b8f9..f0be7ba 100644 --- a/fs/lustre/llite/llite_mmap.c +++ b/fs/lustre/llite/llite_mmap.c @@ -288,18 +288,24 @@ static vm_fault_t __ll_fault(struct vm_area_struct *vma, struct vm_fault *vmf) if (ll_sbi_has_fast_read(ll_i2sbi(file_inode(vma->vm_file)))) { /* do fast fault */ + bool has_retry = vmf->flags & FAULT_FLAG_RETRY_NOWAIT; + + /* To avoid loops, instruct downstream to not drop mmap_sem */ + vmf->flags |= FAULT_FLAG_RETRY_NOWAIT; ll_cl_add(vma->vm_file, env, NULL, LCC_MMAP); fault_ret = filemap_fault(vmf); ll_cl_remove(vma->vm_file, env); + if (has_retry) + vmf->flags &= ~FAULT_FLAG_RETRY_NOWAIT; /* * - If there is no error, then the page was found in cache and * uptodate; * - If VM_FAULT_RETRY is set, the page existed but failed to - * lock. It will return to kernel and retry; + * lock. We will try slow path to avoid loops. * - Otherwise, it should try normal fault under DLM lock. */ - if ((fault_ret & VM_FAULT_RETRY) || + if (!(fault_ret & VM_FAULT_RETRY) && !(fault_ret & VM_FAULT_ERROR)) { result = 0; goto out; From patchwork Mon Nov 16 00:59:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7502138B for ; Mon, 16 Nov 2020 01:01:40 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A924620578 for ; Mon, 16 Nov 2020 01:01:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A924620578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3F00430817E; Sun, 15 Nov 2020 17:01:18 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2571D306D0B for ; Sun, 15 Nov 2020 17:00:14 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 7F5FB2257; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7CEFE21DCA; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:55 -0500 Message-Id: <1605488401-981-23-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 22/28] lustre: ptlrpc: introduce OST_SEEK RPC X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhail Pershin For the purposes of SEEK_HOLE/SEEK_DATA support introduce new OST_SEEK RPC. Patch add RPC layout, unified handler and connect flag for compatibility needs. WC-bug-id: https://jira.whamcloud.com/browse/LU-10810 Lustre-commit: 6d5fe29066af5f ("LU-10810 ptlrpc: introduce OST_SEEK RPC") Signed-off-by: Mikhail Pershin Reviewed-on: https://review.whamcloud.com/39707 Reviewed-by: Sebastien Buisson Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/obdclass/lprocfs_status.c | 1 + fs/lustre/ptlrpc/layout.c | 5 +++++ fs/lustre/ptlrpc/lproc_ptlrpc.c | 3 ++- fs/lustre/ptlrpc/wiretest.c | 6 +++++- include/uapi/linux/lustre/lustre_idl.h | 2 ++ 6 files changed, 16 insertions(+), 2 deletions(-) diff --git a/fs/lustre/include/lustre_req_layout.h b/fs/lustre/include/lustre_req_layout.h index 54041b0..434bb08 100644 --- a/fs/lustre/include/lustre_req_layout.h +++ b/fs/lustre/include/lustre_req_layout.h @@ -199,6 +199,7 @@ void req_capsule_shrink(struct req_capsule *pill, extern struct req_format RQF_OST_SET_INFO_LAST_FID; extern struct req_format RQF_OST_GET_INFO_FIEMAP; extern struct req_format RQF_OST_LADVISE; +extern struct req_format RQF_OST_SEEK; /* LDLM req_format */ extern struct req_format RQF_LDLM_ENQUEUE; diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index a3c5657..6ce0a5d 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -129,6 +129,7 @@ "client_encryption", /* 0x8000 */ "fidmap", /* 0x10000 */ "getattr_pfid", /* 0x20000 */ + "lseek", /* 0x40000 */ NULL }; diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index 4e37bbb..11c0d50 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -777,6 +777,7 @@ &RQF_OST_SET_INFO_LAST_FID, &RQF_OST_GET_INFO_FIEMAP, &RQF_OST_LADVISE, + &RQF_OST_SEEK, &RQF_LDLM_ENQUEUE, &RQF_LDLM_ENQUEUE_LVB, &RQF_LDLM_CONVERT, @@ -1611,6 +1612,10 @@ struct req_format RQF_OST_FALLOCATE = DEFINE_REQ_FMT0("OST_FALLOCATE", ost_body_capa, ost_body_only); EXPORT_SYMBOL(RQF_OST_FALLOCATE); +struct req_format RQF_OST_SEEK = + DEFINE_REQ_FMT0("OST_SEEK", ost_body_only, ost_body_only); +EXPORT_SYMBOL(RQF_OST_SEEK); + struct req_format RQF_OST_SYNC = DEFINE_REQ_FMT0("OST_SYNC", ost_body_capa, ost_body_only); EXPORT_SYMBOL(RQF_OST_SYNC); diff --git a/fs/lustre/ptlrpc/lproc_ptlrpc.c b/fs/lustre/ptlrpc/lproc_ptlrpc.c index 7276f81..26ca55e 100644 --- a/fs/lustre/ptlrpc/lproc_ptlrpc.c +++ b/fs/lustre/ptlrpc/lproc_ptlrpc.c @@ -67,7 +67,8 @@ { OST_QUOTACTL, "ost_quotactl" }, { OST_QUOTA_ADJUST_QUNIT, "ost_quota_adjust_qunit" }, { OST_LADVISE, "ost_ladvise" }, - { OST_FALLOCATE, "ost_fallocate"}, + { OST_FALLOCATE, "ost_fallocate" }, + { OST_SEEK, "ost_seek" }, { MDS_GETATTR, "mds_getattr" }, { MDS_GETATTR_NAME, "mds_getattr_lock" }, { MDS_CLOSE, "mds_close" }, diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 9b1caf4..ba19b78 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -108,7 +108,9 @@ void lustre_assert_wire_constants(void) (long long)OST_LADVISE); LASSERTF(OST_FALLOCATE == 22, "found %lld\n", (long long)OST_FALLOCATE); - LASSERTF(OST_LAST_OPC == 23, "found %lld\n", + LASSERTF(OST_SEEK == 23, "found %lld\n", + (long long)OST_SEEK); + LASSERTF(OST_LAST_OPC == 24, "found %lld\n", (long long)OST_LAST_OPC); LASSERTF(OBD_OBJECT_EOF == 0xffffffffffffffffULL, "found 0x%.16llxULL\n", OBD_OBJECT_EOF); @@ -1245,6 +1247,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT2_FIDMAP); LASSERTF(OBD_CONNECT2_GETATTR_PFID == 0x20000ULL, "found 0x%.16llxULL\n", OBD_CONNECT2_GETATTR_PFID); + LASSERTF(OBD_CONNECT2_LSEEK == 0x40000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT2_LSEEK); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 34b2367..f56b3c5 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -838,6 +838,7 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT2_ENCRYPT 0x8000ULL /* client-to-disk encrypt */ #define OBD_CONNECT2_FIDMAP 0x10000ULL /* FID map */ #define OBD_CONNECT2_GETATTR_PFID 0x20000ULL /* pack parent FID in getattr */ +#define OBD_CONNECT2_LSEEK 0x40000ULL /* SEEK_HOLE/DATA RPC */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same * flag value is not in use on some other branch. Please clear any such @@ -972,6 +973,7 @@ enum ost_cmd { OST_QUOTA_ADJUST_QUNIT = 20, /* not used since 2.4 */ OST_LADVISE = 21, OST_FALLOCATE = 22, + OST_SEEK = 23, OST_LAST_OPC /* must be < 33 to avoid MDS_GETATTR */ }; #define OST_FIRST_OPC OST_REPLY From patchwork Mon Nov 16 00:59:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82362138B for ; Mon, 16 Nov 2020 01:01:37 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F27D20578 for ; Mon, 16 Nov 2020 01:01:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F27D20578 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 88F17306C52; Sun, 15 Nov 2020 17:01:12 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 76CBA306D0B for ; Sun, 15 Nov 2020 17:00:14 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 81FA62258; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 800A22C7E6; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:56 -0500 Message-Id: <1605488401-981-24-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 23/28] lustre: clio: SEEK_HOLE/SEEK_DATA on client side X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Pershin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mikhail Pershin Patch introduces basic support for lseek SEEK_HOLE/SEEK_DATA parameters in lustre client. - introduce new IO type CIT_LSEEK in CLIO stack - LOV splits request to all stripes involved and merges results back. - OSC sends OST LSEEK RPC asynchronously - if target doesn't support LSEEK RPC then OSC assumes whole related object is data with virtual hole at the end - lseek restores released files assuming it is done prior the file copying. - tool is added to request needed lseek on file - basic tests are added in sanity, sanityn and sanity-hsm WC-bug-id: https://jira.whamcloud.com/browse/LU-10810 Lustre-commit: cda353e6efae50 ("LU-10810 clio: SEEK_HOLE/SEEK_DATA on client side") Signed-off-by: Mikhail Pershin Reviewed-on: https://review.whamcloud.com/39708 Reviewed-by: Oleg Drokin Tested-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 10 +++ fs/lustre/include/lustre_export.h | 5 ++ fs/lustre/include/lustre_osc.h | 4 ++ fs/lustre/llite/file.c | 61 ++++++++++++++++-- fs/lustre/llite/llite_lib.c | 4 +- fs/lustre/llite/vvp_io.c | 53 +++++++++++++++- fs/lustre/lov/lov_io.c | 99 ++++++++++++++++++++++++++++- fs/lustre/mdc/mdc_dev.c | 4 ++ fs/lustre/obdclass/cl_io.c | 1 + fs/lustre/osc/osc_io.c | 127 ++++++++++++++++++++++++++++++++++++++ 10 files changed, 358 insertions(+), 10 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index 56200d2..e17385c0 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1415,6 +1415,11 @@ enum cl_io_type { * To give advice about access of a file */ CIT_LADVISE, + /** + * SEEK_HOLE/SEEK_DATA handling to search holes or data + * across all file objects + */ + CIT_LSEEK, CIT_OP_NR }; @@ -1892,6 +1897,11 @@ struct cl_io { enum lu_ladvise_type li_advice; u64 li_flags; } ci_ladvise; + struct cl_lseek_io { + loff_t ls_start; + loff_t ls_result; + int ls_whence; + } ci_lseek; } u; struct cl_2queue ci_queue; size_t ci_nob; diff --git a/fs/lustre/include/lustre_export.h b/fs/lustre/include/lustre_export.h index b5fdf8a..ed49a97 100644 --- a/fs/lustre/include/lustre_export.h +++ b/fs/lustre/include/lustre_export.h @@ -285,6 +285,11 @@ static inline int exp_connect_encrypt(struct obd_export *exp) return !!(exp_connect_flags2(exp) & OBD_CONNECT2_ENCRYPT); } +static inline int exp_connect_lseek(struct obd_export *exp) +{ + return !!(exp_connect_flags2(exp) & OBD_CONNECT2_LSEEK); +} + enum { /* archive_ids in array format */ KKUC_CT_DATA_ARRAY_MAGIC = 0x092013cea, diff --git a/fs/lustre/include/lustre_osc.h b/fs/lustre/include/lustre_osc.h index 24cfec8..ef5237b 100644 --- a/fs/lustre/include/lustre_osc.h +++ b/fs/lustre/include/lustre_osc.h @@ -704,6 +704,10 @@ int osc_fsync_ost(const struct lu_env *env, struct osc_object *obj, void osc_io_fsync_end(const struct lu_env *env, const struct cl_io_slice *slice); void osc_read_ahead_release(const struct lu_env *env, void *cbdata); +int osc_io_lseek_start(const struct lu_env *env, + const struct cl_io_slice *slice); +void osc_io_lseek_end(const struct lu_env *env, + const struct cl_io_slice *slice); /* osc_lock.c */ void osc_lock_to_lockless(const struct lu_env *env, struct osc_lock *ols, diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 3ba9152..4a3c534 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -3984,26 +3984,75 @@ static int ll_heat_set(struct inode *inode, enum lu_heat_flag flags) } } +loff_t ll_lseek(struct inode *inode, loff_t offset, int whence) +{ + struct lu_env *env; + struct cl_io *io; + struct cl_lseek_io *lsio; + u16 refcheck; + int rc; + loff_t retval; + + env = cl_env_get(&refcheck); + if (IS_ERR(env)) + return PTR_ERR(env); + + io = vvp_env_thread_io(env); + io->ci_obj = ll_i2info(inode)->lli_clob; + + lsio = &io->u.ci_lseek; + lsio->ls_start = offset; + lsio->ls_whence = whence; + lsio->ls_result = -ENXIO; + + do { + rc = cl_io_init(env, io, CIT_LSEEK, io->ci_obj); + if (!rc) + rc = cl_io_loop(env, io); + else + rc = io->ci_result; + retval = rc ? : lsio->ls_result; + cl_io_fini(env, io); + } while (unlikely(io->ci_need_restart)); + + cl_env_put(env, &refcheck); + + return retval; +} + static loff_t ll_file_seek(struct file *file, loff_t offset, int origin) { struct inode *inode = file_inode(file); - loff_t retval, eof = 0; + loff_t retval = offset, eof = 0; ktime_t kstart = ktime_get(); - retval = offset + ((origin == SEEK_END) ? i_size_read(inode) : - (origin == SEEK_CUR) ? file->f_pos : 0); CDEBUG(D_VFSTRACE, "VFS Op:inode=" DFID "(%p), to=%llu=%#llx(%d)\n", PFID(ll_inode2fid(inode)), inode, retval, retval, origin); - if (origin == SEEK_END || origin == SEEK_HOLE || origin == SEEK_DATA) { + if (origin == SEEK_END) { retval = ll_glimpse_size(inode); if (retval != 0) return retval; eof = i_size_read(inode); } - retval = generic_file_llseek_size(file, offset, origin, - ll_file_maxbytes(inode), eof); + if (origin == SEEK_HOLE || origin == SEEK_DATA) { + if (offset < 0) + return -ENXIO; + + /* flush local cache first if any */ + cl_sync_file_range(inode, offset, OBD_OBJECT_EOF, + CL_FSYNC_LOCAL, 0); + + retval = ll_lseek(inode, offset, origin); + if (retval < 0) + return retval; + + retval = vfs_setpos(file, retval, ll_file_maxbytes(inode)); + } else { + retval = generic_file_llseek_size(file, offset, origin, + ll_file_maxbytes(inode), eof); + } if (retval >= 0) ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_LLSEEK, ktime_us_delta(ktime_get(), kstart)); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index d94c6ca..a4042b8 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -263,7 +263,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT2_LSOM | OBD_CONNECT2_ASYNC_DISCARD | OBD_CONNECT2_PCC | - OBD_CONNECT2_CRUSH | + OBD_CONNECT2_CRUSH | OBD_CONNECT2_LSEEK | OBD_CONNECT2_GETATTR_PFID; if (sbi->ll_flags & LL_SBI_LRU_RESIZE) @@ -473,7 +473,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT_FLAGS2 | OBD_CONNECT_GRANT_SHRINK; data->ocd_connect_flags2 = OBD_CONNECT2_LOCKAHEAD | - OBD_CONNECT2_INC_XID; + OBD_CONNECT2_INC_XID | OBD_CONNECT2_LSEEK; if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_GRANT_PARAM)) data->ocd_connect_flags |= OBD_CONNECT_GRANT_PARAM; diff --git a/fs/lustre/llite/vvp_io.c b/fs/lustre/llite/vvp_io.c index 3a2e1cc..d6ca267 100644 --- a/fs/lustre/llite/vvp_io.c +++ b/fs/lustre/llite/vvp_io.c @@ -1531,6 +1531,51 @@ static int vvp_io_read_ahead(const struct lu_env *env, return result; } +static int vvp_io_lseek_lock(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct cl_io *io = ios->cis_io; + u64 lock_start = io->u.ci_lseek.ls_start; + u64 lock_end = OBD_OBJECT_EOF; + u32 enqflags = CEF_MUST; /* always take client lock */ + + return vvp_io_one_lock(env, io, enqflags, CLM_READ, + lock_start, lock_end); +} + +static int vvp_io_lseek_start(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct cl_io *io = ios->cis_io; + struct inode *inode = vvp_object_inode(io->ci_obj); + u64 start = io->u.ci_lseek.ls_start; + + inode_lock(inode); + inode_dio_wait(inode); + + /* At the moment we have DLM lock so just update inode + * to know the file size. + */ + ll_merge_attr(env, inode); + if (start >= i_size_read(inode)) { + io->u.ci_lseek.ls_result = -ENXIO; + return -ENXIO; + } + return 0; +} + +static void vvp_io_lseek_end(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct cl_io *io = ios->cis_io; + struct inode *inode = vvp_object_inode(io->ci_obj); + + if (io->u.ci_lseek.ls_result > i_size_read(inode)) + io->u.ci_lseek.ls_result = -ENXIO; + + inode_unlock(inode); +} + static const struct cl_io_operations vvp_io_ops = { .op = { [CIT_READ] = { @@ -1576,7 +1621,13 @@ static int vvp_io_read_ahead(const struct lu_env *env, }, [CIT_LADVISE] = { .cio_fini = vvp_io_fini - } + }, + [CIT_LSEEK] = { + .cio_fini = vvp_io_fini, + .cio_lock = vvp_io_lseek_lock, + .cio_start = vvp_io_lseek_start, + .cio_end = vvp_io_lseek_end, + }, }, .cio_read_ahead = vvp_io_read_ahead, }; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index af79d20..20fcde1 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -529,6 +529,12 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj, break; } + case CIT_LSEEK: { + lio->lis_pos = io->u.ci_lseek.ls_start; + lio->lis_endpos = OBD_OBJECT_EOF; + break; + } + case CIT_GLIMPSE: lio->lis_pos = 0; lio->lis_endpos = OBD_OBJECT_EOF; @@ -715,6 +721,12 @@ static void lov_io_sub_inherit(struct lov_io_sub *sub, struct lov_io *lio, io->u.ci_ladvise.li_flags = parent->u.ci_ladvise.li_flags; break; } + case CIT_LSEEK: { + io->u.ci_lseek.ls_start = start; + io->u.ci_lseek.ls_whence = parent->u.ci_lseek.ls_whence; + io->u.ci_lseek.ls_result = parent->u.ci_lseek.ls_result; + break; + } case CIT_GLIMPSE: case CIT_MISC: default: @@ -1265,6 +1277,80 @@ static void lov_io_fsync_end(const struct lu_env *env, } } +static void lov_io_lseek_end(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct lov_io *lio = cl2lov_io(env, ios); + struct cl_io *io = lio->lis_cl.cis_io; + struct lov_stripe_md *lsm = lio->lis_object->lo_lsm; + struct lov_io_sub *sub; + loff_t offset = -ENXIO; + bool seek_hole = io->u.ci_lseek.ls_whence == SEEK_HOLE; + + list_for_each_entry(sub, &lio->lis_active, sub_linkage) { + struct cl_io *subio = &sub->sub_io; + int index = lov_comp_entry(sub->sub_subio_index); + int stripe = lov_comp_stripe(sub->sub_subio_index); + loff_t sub_off, lov_off; + + lov_io_end_wrapper(sub->sub_env, subio); + + if (io->ci_result == 0) + io->ci_result = sub->sub_io.ci_result; + + if (io->ci_result) + continue; + + CDEBUG(D_INFO, DFID ": entry %x stripe %u: SEEK_%s from %lld\n", + PFID(lu_object_fid(lov2lu(lio->lis_object))), + index, stripe, seek_hole ? "HOLE" : "DATA", + subio->u.ci_lseek.ls_start); + + /* first subio with positive result is what we need */ + sub_off = subio->u.ci_lseek.ls_result; + /* Expected error, offset is out of stripe file size */ + if (sub_off == -ENXIO) + continue; + /* Any other errors are not expected with ci_result == 0 */ + if (sub_off < 0) { + CDEBUG(D_INFO, "unexpected error: rc = %lld\n", + sub_off); + io->ci_result = sub_off; + continue; + } + lov_off = lov_stripe_size(lsm, index, sub_off + 1, stripe) - 1; + if (lov_off < 0) { + /* the only way to get negatove lov_off here is too big + * result. Return -EOVERFLOW then. + */ + io->ci_result = -EOVERFLOW; + CDEBUG(D_INFO, "offset %llu is too big: rc = %d\n", + (u64)lov_off, io->ci_result); + continue; + } + if (lov_off < io->u.ci_lseek.ls_start) { + io->ci_result = -EINVAL; + CDEBUG(D_INFO, "offset %lld < start %lld: rc = %d\n", + sub_off, io->u.ci_lseek.ls_start, io->ci_result); + continue; + } + /* resulting offset can be out of component range if stripe + * object is full and its file size was returned as virtual + * hole start. Skip this result, the next component will give + * us correct lseek result. + */ + if (lov_off >= lsm->lsm_entries[index]->lsme_extent.e_end) + continue; + + CDEBUG(D_INFO, "SEEK_%s: %lld->%lld/%lld: rc = %d\n", + seek_hole ? "HOLE" : "DATA", + subio->u.ci_lseek.ls_start, sub_off, lov_off, + sub->sub_io.ci_result); + offset = min_t(u64, offset, lov_off); + } + io->u.ci_lseek.ls_result = offset; +} + static const struct cl_io_operations lov_io_ops = { .op = { [CIT_READ] = { @@ -1330,8 +1416,17 @@ static void lov_io_fsync_end(const struct lu_env *env, .cio_start = lov_io_start, .cio_end = lov_io_end }, + [CIT_LSEEK] = { + .cio_fini = lov_io_fini, + .cio_iter_init = lov_io_iter_init, + .cio_iter_fini = lov_io_iter_fini, + .cio_lock = lov_io_lock, + .cio_unlock = lov_io_unlock, + .cio_start = lov_io_start, + .cio_end = lov_io_lseek_end + }, [CIT_GLIMPSE] = { - .cio_fini = lov_io_fini, + .cio_fini = lov_io_fini, }, [CIT_MISC] = { .cio_fini = lov_io_fini @@ -1459,6 +1554,7 @@ int lov_io_init_empty(const struct lu_env *env, struct cl_object *obj, break; case CIT_FSYNC: case CIT_LADVISE: + case CIT_LSEEK: case CIT_SETATTR: case CIT_DATA_VERSION: result = 1; @@ -1522,6 +1618,7 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj, case CIT_READ: case CIT_WRITE: case CIT_FAULT: + case CIT_LSEEK: io->ci_restore_needed = 1; result = -ENODATA; break; diff --git a/fs/lustre/mdc/mdc_dev.c b/fs/lustre/mdc/mdc_dev.c index 90b60f5..214fd31 100644 --- a/fs/lustre/mdc/mdc_dev.c +++ b/fs/lustre/mdc/mdc_dev.c @@ -1297,6 +1297,10 @@ static void mdc_io_data_version_end(const struct lu_env *env, .cio_start = mdc_io_fsync_start, .cio_end = osc_io_fsync_end, }, + [CIT_LSEEK] = { + .cio_start = osc_io_lseek_start, + .cio_end = osc_io_lseek_end, + }, }, .cio_read_ahead = mdc_io_read_ahead, .cio_submit = osc_io_submit, diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index aa3cb17..c57a3766 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -127,6 +127,7 @@ void cl_io_fini(const struct lu_env *env, struct cl_io *io) case CIT_GLIMPSE: break; case CIT_LADVISE: + case CIT_LSEEK: break; default: LBUG(); diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 6121f39..a0537b8 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -1042,6 +1042,128 @@ void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice) } EXPORT_SYMBOL(osc_io_end); +struct osc_lseek_args { + struct osc_io *lsa_oio; +}; + +static int osc_lseek_interpret(const struct lu_env *env, + struct ptlrpc_request *req, + void *arg, int rc) +{ + struct ost_body *reply; + struct osc_lseek_args *lsa = arg; + struct osc_io *oio = lsa->lsa_oio; + struct cl_io *io = oio->oi_cl.cis_io; + struct cl_lseek_io *lsio = &io->u.ci_lseek; + + if (rc != 0) + goto out; + + reply = req_capsule_server_get(&req->rq_pill, &RMF_OST_BODY); + if (!reply) { + rc = -EPROTO; + goto out; + } + + lsio->ls_result = reply->oa.o_size; +out: + osc_async_upcall(&oio->oi_cbarg, rc); + return rc; +} + +int osc_io_lseek_start(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct cl_io *io = slice->cis_io; + struct osc_io *oio = cl2osc_io(env, slice); + struct cl_object *obj = slice->cis_obj; + struct lov_oinfo *loi = cl2osc(obj)->oo_oinfo; + struct cl_lseek_io *lsio = &io->u.ci_lseek; + struct obdo *oa = &oio->oi_oa; + struct osc_async_cbargs *cbargs = &oio->oi_cbarg; + struct obd_export *exp = osc_export(cl2osc(obj)); + struct ptlrpc_request *req; + struct ost_body *body; + struct osc_lseek_args *lsa; + int rc = 0; + + /* No negative values at this point */ + LASSERT(lsio->ls_start >= 0); + LASSERT(lsio->ls_whence == SEEK_HOLE || lsio->ls_whence == SEEK_DATA); + + /* with IO lock taken we have object size in LVB and can check + * boundaries prior sending LSEEK RPC + */ + if (lsio->ls_start >= loi->loi_lvb.lvb_size) { + /* consider area beyond end of object as hole */ + if (lsio->ls_whence == SEEK_HOLE) + lsio->ls_result = lsio->ls_start; + else + lsio->ls_result = -ENXIO; + return 0; + } + + /* if LSEEK RPC is not supported by server, consider whole stripe + * object is data with hole after end of object + */ + if (!exp_connect_lseek(exp)) { + if (lsio->ls_whence == SEEK_HOLE) + lsio->ls_result = loi->loi_lvb.lvb_size; + else + lsio->ls_result = lsio->ls_start; + return 0; + } + + memset(oa, 0, sizeof(*oa)); + oa->o_oi = loi->loi_oi; + oa->o_valid = OBD_MD_FLID | OBD_MD_FLGROUP; + oa->o_size = lsio->ls_start; + oa->o_mode = lsio->ls_whence; + if (oio->oi_lockless) { + oa->o_flags = OBD_FL_SRVLOCK; + oa->o_valid |= OBD_MD_FLFLAGS; + } + + init_completion(&cbargs->opc_sync); + req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_OST_SEEK); + if (!req) + return -ENOMEM; + + rc = ptlrpc_request_pack(req, LUSTRE_OST_VERSION, OST_SEEK); + if (rc < 0) { + ptlrpc_request_free(req); + return rc; + } + + body = req_capsule_client_get(&req->rq_pill, &RMF_OST_BODY); + lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa); + ptlrpc_request_set_replen(req); + req->rq_interpret_reply = osc_lseek_interpret; + lsa = ptlrpc_req_async_args(lsa, req); + lsa->lsa_oio = oio; + + ptlrpcd_add_req(req); + cbargs->opc_rpc_sent = 1; + + return 0; +} +EXPORT_SYMBOL(osc_io_lseek_start); + +void osc_io_lseek_end(const struct lu_env *env, + const struct cl_io_slice *slice) +{ + struct osc_io *oio = cl2osc_io(env, slice); + struct osc_async_cbargs *cbargs = &oio->oi_cbarg; + int rc = 0; + + if (cbargs->opc_rpc_sent) { + wait_for_completion(&cbargs->opc_sync); + rc = cbargs->opc_rc; + } + slice->cis_io->ci_result = rc; +} +EXPORT_SYMBOL(osc_io_lseek_end); + static const struct cl_io_operations osc_io_ops = { .op = { [CIT_READ] = { @@ -1084,6 +1206,11 @@ void osc_io_end(const struct lu_env *env, const struct cl_io_slice *slice) .cio_end = osc_io_ladvise_end, .cio_fini = osc_io_fini }, + [CIT_LSEEK] = { + .cio_start = osc_io_lseek_start, + .cio_end = osc_io_lseek_end, + .cio_fini = osc_io_fini + }, [CIT_MISC] = { .cio_fini = osc_io_fini } From patchwork Mon Nov 16 00:59:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907015 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B459C14B3 for ; Mon, 16 Nov 2020 01:01:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D61852223C for ; Mon, 16 Nov 2020 01:01:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D61852223C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8842F21FB61; Sun, 15 Nov 2020 17:01:01 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C6FD2306D0B for ; Sun, 15 Nov 2020 17:00:14 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 85484225A; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 830EDB5; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:57 -0500 Message-Id: <1605488401-981-25-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 24/28] lustre: sec: O_DIRECT for encrypted file X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Add O_DIRECT support for encrypted files. By default, fscrypt does not support O_DIRECT because it needs pagecache pages to proceed. With Lustre, we can make use of pages being used for sending RPCs. They can be twisted so that they have a proper mapping and index, suitable for encryption/decryption. One of the benefits of O_DIRECT support for encrypted files is that we get support for mirroring at the same time. WC-bug-id: https://jira.whamcloud.com/browse/LU-12275 Lustre-commit: 728036f25635a ("LU-12275 sec: O_DIRECT for encrypted file") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/38967 Reviewed-by: Andreas Dilger Reviewed-by: Wang Shilong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../client_side_encryption/access_semantics.txt | 3 -- fs/lustre/llite/dir.c | 1 - fs/lustre/llite/llite_lib.c | 11 ++++++- fs/lustre/llite/rw26.c | 27 +++++++++++++--- fs/lustre/llite/super25.c | 11 +++++++ fs/lustre/obdclass/cl_io.c | 11 +++++++ fs/lustre/osc/osc_request.c | 37 ++++++++++++++++++---- fs/lustre/ptlrpc/wiretest.c | 2 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/Documentation/lustre/client_side_encryption/access_semantics.txt b/Documentation/lustre/client_side_encryption/access_semantics.txt index fe2c28d..7ed0bc7 100644 --- a/Documentation/lustre/client_side_encryption/access_semantics.txt +++ b/Documentation/lustre/client_side_encryption/access_semantics.txt @@ -42,9 +42,6 @@ astute users may notice some differences in behavior: may be used to overwrite the source files but isn't guaranteed to be effective on all filesystems and storage devices. -- Direct I/O is not supported on encrypted files. Attempts to use - direct I/O on such files will fall back to buffered I/O. - - The fallocate operations FALLOC_FL_COLLAPSE_RANGE, FALLOC_FL_INSERT_RANGE, and FALLOC_FL_ZERO_RANGE are not supported on encrypted files and will fail with EOPNOTSUPP. diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 262aea0..6bc95d9 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -481,7 +481,6 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, goto out_op_data; } - op_data->op_cli_flags |= CLI_SET_MEA; err = md_create(sbi->ll_md_exp, op_data, lump, len, mode, from_kuid(&init_user_ns, current_fsuid()), diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index a4042b8..e4036af 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -1759,6 +1759,7 @@ int ll_io_zero_page(struct inode *inode, pgoff_t index, pgoff_t offset, * file, we must not zero and write as below. Subsequent * server-side truncate will handle things correctly. */ + rc = 0; goto clpfini; ClearPagePrivate2(vmpage); if (rc) @@ -1960,7 +1961,15 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, attr->ia_valid & ATTR_SIZE) { xvalid |= OP_XVALID_FLAGS; flags = LUSTRE_ENCRYPT_FL; - if (attr->ia_size & ~PAGE_MASK) { + /* Call to ll_io_zero_page is not necessary if + * truncating on PAGE_SIZE boundary, because + * whole pages will be wiped. + * In case of Direct IO, all we need is to set + * new size. + */ + if (attr->ia_size & ~PAGE_MASK && + !(attr->ia_valid & ATTR_FILE && + attr->ia_file->f_flags & O_DIRECT)) { pgoff_t offset; offset = attr->ia_size & (PAGE_SIZE - 1); diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index a4ae211..1736e9a 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -207,6 +207,7 @@ struct ll_dio_pages { int io_pages = 0; size_t page_size = cl_page_size(obj); int i; + pgoff_t index = offset >> PAGE_SHIFT; ssize_t rc = 0; cl_2queue_init(queue); @@ -226,6 +227,28 @@ struct ll_dio_pages { } page->cp_sync_io = anchor; + if (inode && IS_ENCRYPTED(inode)) { + struct page *vmpage = cl_page_vmpage(page); + + /* In case of Direct IO on encrypted file, we need to + * set the correct page index, and add a reference to + * the mapping. This is required by llcrypt to proceed + * to encryption/decryption, because each block is + * encrypted independently, and each block's IV is set + * to the logical block number within the file. + * This is safe because we know these pages are private + * to the thread doing the Direct IO, and despite + * setting a mapping on the pages, cached lookups will + * not find them. + * Set PageChecked to detect special case of Direct IO + * in osc_brw_fini_request(). + * Reference to the mapping and PageChecked flag are + * removed in cl_aio_end(). + */ + vmpage->index = index++; + vmpage->mapping = inode->i_mapping; + SetPageChecked(vmpage); + } cl_page_list_add(&queue->c2_qin, page); /* * Set page clip to tell transfer formation engine @@ -297,10 +320,6 @@ static ssize_t ll_direct_IO(struct kiocb *iocb, struct iov_iter *iter) int rw = iov_iter_rw(iter); struct vvp_io *vio; - /* if file is encrypted, return 0 so that we fall back to buffered IO */ - if (IS_ENCRYPTED(inode)) - return 0; - /* Check EOF by ourselves */ if (rw == READ && file_offset >= i_size_read(inode)) return 0; diff --git a/fs/lustre/llite/super25.c b/fs/lustre/llite/super25.c index 8eb3fc3..d02c8cf 100644 --- a/fs/lustre/llite/super25.c +++ b/fs/lustre/llite/super25.c @@ -72,10 +72,21 @@ static void ll_destroy_inode(struct inode *inode) call_rcu(&inode->i_rcu, ll_inode_destroy_callback); } +static int ll_drop_inode(struct inode *inode) +{ + int drop = generic_drop_inode(inode); + + if (!drop) + drop = llcrypt_drop_inode(inode); + + return drop; +} + /* exported operations */ struct super_operations lustre_super_operations = { .alloc_inode = ll_alloc_inode, .destroy_inode = ll_destroy_inode, + .drop_inode = ll_drop_inode, .evict_inode = ll_delete_inode, .put_super = ll_put_super, .statfs = ll_statfs, diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index c57a3766..37b0828 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -1081,8 +1081,19 @@ static void cl_aio_end(const struct lu_env *env, struct cl_sync_io *anchor) /* release pages */ while (aio->cda_pages.pl_nr > 0) { struct cl_page *page = cl_page_list_first(&aio->cda_pages); + struct page *vmpage = cl_page_vmpage(page); + struct inode *inode = vmpage ? page2inode(vmpage) : NULL; cl_page_get(page); + /* We end up here in case of Direct IO only. For encrypted file, + * mapping was set on pages in ll_direct_rw_pages(), so it has + * to be cleared now before page cleanup. + * PageChecked flag was also set there, so we clean up here. + */ + if (inode && IS_ENCRYPTED(inode)) { + vmpage->mapping = NULL; + ClearPageChecked(vmpage); + } cl_page_list_del(env, &aio->cda_pages, page); cl_page_delete(env, page); cl_page_put(env, page); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 8a8a624..bf9ce44 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1369,13 +1369,9 @@ static inline void osc_release_bounce_pages(struct brw_page **pga, int i; for (i = 0; i < page_count; i++) { - if (pga[i]->pg->mapping) + if (!pga[i]->pg->mapping) /* bounce pages are unmapped */ - continue; - if (pga[i]->flag & OBD_BRW_SYNC) - /* sync transfer cannot have encrypted pages */ - continue; - llcrypt_finalize_bounce_page(&pga[i]->pg); + llcrypt_finalize_bounce_page(&pga[i]->pg); pga[i]->count -= pga[i]->bp_count_diff; pga[i]->off += pga[i]->bp_off_diff; } @@ -1470,6 +1466,19 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, pg->bp_off_diff = pg->off & ~PAGE_MASK; pg->off = pg->off & PAGE_MASK; } + } else if (opc == OST_READ && inode && IS_ENCRYPTED(inode)) { + for (i = 0; i < page_count; i++) { + struct brw_page *pg = pga[i]; + + /* count/off are forced to cover the whole page so that + * all encrypted data is stored on the OST, so adjust + * bp_{count,off}_diff for the size of the clear text. + */ + pg->bp_count_diff = PAGE_SIZE - pg->count; + pg->count = PAGE_SIZE; + pg->bp_off_diff = pg->off & ~PAGE_MASK; + pg->off = pg->off & PAGE_MASK; + } } for (niocount = i = 1; i < page_count; i++) { @@ -1483,8 +1492,13 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, req_capsule_set_size(pill, &RMF_NIOBUF_REMOTE, RCL_CLIENT, niocount * sizeof(*niobuf)); - for (i = 0; i < page_count; i++) + for (i = 0; i < page_count; i++) { short_io_size += pga[i]->count; + if (!inode || !IS_ENCRYPTED(inode)) { + pga[i]->bp_count_diff = 0; + pga[i]->bp_off_diff = 0; + } + } /* Check if read/write is small enough to be a short io. */ if (short_io_size > cli->cl_max_short_io_bytes || niocount > 1 || @@ -2093,8 +2107,17 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) continue; } + /* The page is already locked when we arrive here, + * except when we deal with a twisted page for + * specific Direct IO support, in which case + * PageChecked flag is set on page. + */ + if (PageChecked(pg->pg)) + lock_page(pg->pg); rc = llcrypt_decrypt_pagecache_blocks(pg->pg, PAGE_SIZE, 0); + if (PageChecked(pg->pg)) + unlock_page(pg->pg); if (rc) goto out; } diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index ba19b78..c8b97fa 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -2307,6 +2307,8 @@ void lustre_assert_wire_constants(void) LUSTRE_TOPDIR_FL); LASSERTF(LUSTRE_INLINE_DATA_FL == 0x10000000, "found 0x%.8x\n", LUSTRE_INLINE_DATA_FL); + LASSERTF(LUSTRE_ENCRYPT_FL == 0x00800000UL, "found 0x%.8x\n", + LUSTRE_ENCRYPT_FL); LASSERTF(MDS_INODELOCK_LOOKUP == 0x00000001UL, "found 0x%.8x\n", MDS_INODELOCK_LOOKUP); LASSERTF(MDS_INODELOCK_UPDATE == 0x00000002UL, "found 0x%.8x\n", From patchwork Mon Nov 16 00:59:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907035 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 942A114B3 for ; Mon, 16 Nov 2020 01:01:45 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74EF9206B7 for ; Mon, 16 Nov 2020 01:01:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74EF9206B7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CA146306E9A; Sun, 15 Nov 2020 17:01:21 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 23549306D0B for ; Sun, 15 Nov 2020 17:00:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 87C0A225B; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8629A2C7FB; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:58 -0500 Message-Id: <1605488401-981-26-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 25/28] lustre: sec: restrict fallocate on encrypted files X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson For now, ll_fallocate only supports standard preallocation. Anyway, encrypted inodes can't handle collapse range or zero range or insert range since we would need to re-encrypt blocks with a different IV or XTS tweak (which are based on the logical block number). So make sure we return -EOPNOTSUPP in this case, like what ext4 does. WC-bug-id: https://jira.whamcloud.com/browse/LU-12275 Lustre-commit: a7870fb9568bf ("LU-12275 sec: restrict fallocate on encrypted files") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/39220 Reviewed-by: Andreas Dilger Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 4a3c534..02cc2d6 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -4927,6 +4927,17 @@ long ll_fallocate(struct file *filp, int mode, loff_t offset, loff_t len) struct inode *inode = filp->f_path.dentry->d_inode; /* + * Encrypted inodes can't handle collapse range or zero range or insert + * range since we would need to re-encrypt blocks with a different IV or + * XTS tweak (which are based on the logical block number). + * Similar to what ext4 does. + */ + if (IS_ENCRYPTED(inode) && + (mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE | + FALLOC_FL_ZERO_RANGE))) + return -EOPNOTSUPP; + + /* * Only mode == 0 (which is standard prealloc) is supported now. * Punch is not supported yet. */ From patchwork Mon Nov 16 00:59:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0D6F138B for ; Mon, 16 Nov 2020 01:02:24 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A379222245 for ; Mon, 16 Nov 2020 01:02:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A379222245 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AD62021FDD0; Sun, 15 Nov 2020 17:01:45 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 61941308154 for ; Sun, 15 Nov 2020 17:00:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 8AF32236C; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 891E82C802; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 19:59:59 -0500 Message-Id: <1605488401-981-27-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 26/28] lustre: sec: encryption with different client PAGE_SIZE X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson In order to properly handle encryption/decryption on clients that have a PAGE_SIZE != LUSTRE_ENCRYPTION_UNIT_SIZE (typically aarch64/ppc64), a few adjustements are necessary: - when encrypting, do not proceed with PAGE_SIZE as encryption length. Instead, round up to a multiple of LUSTRE_ENCRYPTION_UNIT_SIZE. On aarch64/ppc64, it avoids encrypting way beyond LUSTRE_ENCRYPTION_UNIT_SIZE when the page is not full. - when decrypting, do not proceed with PAGE_SIZE as decryption length. Instead, do LUSTRE_ENCRYPTION_UNIT_SIZE length at a time. It enables proper detection of 'all 0s' sent by servers for content beyond file size. Regarding tests, add sanity-sec test_53 to exercise encryption from clients with different PAGE_SIZE. The trick to achieve this with AT is to expect the client to have 64KB PAGE_SIZE, and the servers to have 4KB PAGE_SIZE, and then mount a client from the MDS node. This also means code running on server side needs to have client encryption support enabled, so CentOS/RHEL 8 at least. WC-bug-id: https://jira.whamcloud.com/browse/LU-12275 Lustre-commit: ac5fcdce025b4 ("LU-12275 sec: encryption with different client PAGE_SIZE") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/39315 Reviewed-by: Andreas Dilger Reviewed-by: Wang Shilong Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 28 +++++++++++----- fs/lustre/osc/osc_request.c | 79 +++++++++++++++++++++++++++------------------ 2 files changed, 68 insertions(+), 39 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 02cc2d6..f7f917b 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -444,16 +444,28 @@ static inline int ll_dom_readpage(void *data, struct page *page) kunmap_atomic(kaddr); if (inode && IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) { - if (!llcrypt_has_encryption_key(inode)) + if (!llcrypt_has_encryption_key(inode)) { CDEBUG(D_SEC, "no enc key for " DFID "\n", PFID(ll_inode2fid(inode))); - /* decrypt only if page is not empty */ - else if (memcmp(page_address(page), - page_address(ZERO_PAGE(0)), - PAGE_SIZE) != 0) - rc = llcrypt_decrypt_pagecache_blocks(page, - PAGE_SIZE, - 0); + } else { + unsigned int offs = 0; + + while (offs < PAGE_SIZE) { + /* decrypt only if page is not empty */ + if (memcmp(page_address(page) + offs, + page_address(ZERO_PAGE(0)), + LUSTRE_ENCRYPTION_UNIT_SIZE) == 0) + break; + + rc = llcrypt_decrypt_pagecache_blocks(page, + LUSTRE_ENCRYPTION_UNIT_SIZE, + 0); + if (rc) + break; + + offs += LUSTRE_ENCRYPTION_UNIT_SIZE; + } + } } unlock_page(page); diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index bf9ce44..746b695 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1421,8 +1421,12 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, struct page *data_page = NULL; bool retried = false; bool lockedbymyself; + u32 nunits = (pg->off & ~PAGE_MASK) + pg->count; retry_encrypt: + if (nunits & ~LUSTRE_ENCRYPTION_MASK) + nunits = (nunits & LUSTRE_ENCRYPTION_MASK) + + LUSTRE_ENCRYPTION_UNIT_SIZE; /* The page can already be locked when we arrive here. * This is possible when cl_page_assume/vvp_page_assume * is stuck on wait_on_page_writeback with page lock @@ -1435,7 +1439,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, lockedbymyself = trylock_page(pg->pg); data_page = llcrypt_encrypt_pagecache_blocks(pg->pg, - PAGE_SIZE, 0, + nunits, 0, GFP_NOFS); if (lockedbymyself) unlock_page(pg->pg); @@ -1458,24 +1462,29 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, oap->oap_obj_off + oap->oap_page_off; } - /* len is forced to PAGE_SIZE, and poff to 0 + /* len is forced to nunits, and relative offset to 0 * so store the old, clear text info */ - pg->bp_count_diff = PAGE_SIZE - pg->count; - pg->count = PAGE_SIZE; + pg->bp_count_diff = nunits - pg->count; + pg->count = nunits; pg->bp_off_diff = pg->off & ~PAGE_MASK; pg->off = pg->off & PAGE_MASK; } } else if (opc == OST_READ && inode && IS_ENCRYPTED(inode)) { for (i = 0; i < page_count; i++) { struct brw_page *pg = pga[i]; - - /* count/off are forced to cover the whole page so that - * all encrypted data is stored on the OST, so adjust - * bp_{count,off}_diff for the size of the clear text. + u32 nunits = (pg->off & ~PAGE_MASK) + pg->count; + + if (nunits & ~LUSTRE_ENCRYPTION_MASK) + nunits = (nunits & LUSTRE_ENCRYPTION_MASK) + + LUSTRE_ENCRYPTION_UNIT_SIZE; + /* count/off are forced to cover the whole encryption + * unit size so that all encrypted data is stored on the + * OST, so adjust bp_{count,off}_diff for the size of + * the clear text. */ - pg->bp_count_diff = PAGE_SIZE - pg->count; - pg->count = PAGE_SIZE; + pg->bp_count_diff = nunits - pg->count; + pg->count = nunits; pg->bp_off_diff = pg->off & ~PAGE_MASK; pg->off = pg->off & PAGE_MASK; } @@ -2096,30 +2105,38 @@ static int osc_brw_fini_request(struct ptlrpc_request *req, int rc) } for (idx = 0; idx < aa->aa_page_count; idx++) { struct brw_page *pg = aa->aa_ppga[idx]; + unsigned int offs = 0; + + while (offs < PAGE_SIZE) { + /* do not decrypt if page is all 0s */ + if (memchr_inv(page_address(pg->pg) + offs, 0, + LUSTRE_ENCRYPTION_UNIT_SIZE) == NULL) { + /* if page is empty forward info to + * upper layers (ll_io_zero_page) by + * clearing PagePrivate2 + */ + if (!offs) + ClearPagePrivate2(pg->pg); + break; + } - /* do not decrypt if page is all 0s */ - if (memchr_inv(page_address(pg->pg), 0, - PAGE_SIZE) == NULL) { - /* if page is empty forward info to upper layers - * (ll_io_zero_page) by clearing PagePrivate2 + /* The page is already locked when we arrive + * here, except when we deal with a twisted + * page for specific Direct IO support, in + * which case PageChecked flag is set on page. */ - ClearPagePrivate2(pg->pg); - continue; + if (PageChecked(pg->pg)) + lock_page(pg->pg); + rc = llcrypt_decrypt_pagecache_blocks(pg->pg, + LUSTRE_ENCRYPTION_UNIT_SIZE, + offs); + if (PageChecked(pg->pg)) + unlock_page(pg->pg); + if (rc) + goto out; + + offs += LUSTRE_ENCRYPTION_UNIT_SIZE; } - - /* The page is already locked when we arrive here, - * except when we deal with a twisted page for - * specific Direct IO support, in which case - * PageChecked flag is set on page. - */ - if (PageChecked(pg->pg)) - lock_page(pg->pg); - rc = llcrypt_decrypt_pagecache_blocks(pg->pg, - PAGE_SIZE, 0); - if (PageChecked(pg->pg)) - unlock_page(pg->pg); - if (rc) - goto out; } } From patchwork Mon Nov 16 01:00:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907049 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84626138B for ; Mon, 16 Nov 2020 01:02:04 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 668D020E65 for ; Mon, 16 Nov 2020 01:02:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 668D020E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B3F58306F1C; Sun, 15 Nov 2020 17:01:32 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B21E5306D2E for ; Sun, 15 Nov 2020 17:00:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 8DF12236D; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8C04D2C803; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 20:00:00 -0500 Message-Id: <1605488401-981-28-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 27/28] lustre: sec: require enc key in case of O_CREAT only X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson In ll_atomic_open(), do not return -ENOKEY when trying to open either a directory or a file without the encryption key, unless O_CREAT flag is specified. Indeed, listing directory content is allowed even without the key. And in case of regular file, ll_file_open() already checks for the presence of an encryption key. Improve sanity-sec test_54 to verify this is working properly. WC-bug-id: https://jira.whamcloud.com/browse/LU-13975 Lustre-commit: f6daee15b2c8ec ("LU-13975 sec: require enc key in case of O_CREAT only") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/39983 Reviewed-by: John L. Hammond Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/namei.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index da6b729..b24f097 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -1113,18 +1113,19 @@ static int ll_atomic_open(struct inode *dir, struct dentry *dentry, it->it_flags &= ~MDS_OPEN_FL_INTERNAL; if (ll_sbi_has_encrypt(ll_i2sbi(dir)) && IS_ENCRYPTED(dir)) { - /* we know that we are going to create a regular file because + /* in case of create, this is going to be a regular file because * we set S_IFREG bit on it->it_create_mode above */ rc = llcrypt_get_encryption_info(dir); if (rc) goto out_release; - if (!llcrypt_has_encryption_key(dir)) { - rc = -ENOKEY; - goto out_release; + if (open_flags & O_CREAT) { + if (!llcrypt_has_encryption_key(dir)) { + rc = -ENOKEY; + goto out_release; + } + encrypt = true; } - encrypt = true; - rc = 0; } OBD_FAIL_TIMEOUT(OBD_FAIL_LLITE_CREATE_FILE_PAUSE2, cfs_fail_val); From patchwork Mon Nov 16 01:00:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11907047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99B8814B3 for ; Mon, 16 Nov 2020 01:02:01 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A33220E65 for ; Mon, 16 Nov 2020 01:02:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A33220E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7385F306F11; Sun, 15 Nov 2020 17:01:31 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EF86121FA38 for ; Sun, 15 Nov 2020 17:00:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 90C692375; Sun, 15 Nov 2020 20:00:06 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8EE902C7E6; Sun, 15 Nov 2020 20:00:06 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 15 Nov 2020 20:00:01 -0500 Message-Id: <1605488401-981-29-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1605488401-981-1-git-send-email-jsimmons@infradead.org> References: <1605488401-981-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 28/28] lustre: sec: fix O_DIRECT and encrypted files X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Sometimes, we can end up in a situation where osc_release_bounce_pages() mistakenly consider pages as fscrypt bounce pages, and tries to free them. Fix the way we consider bounce pages by always setting the PageChecked flag on them. Fixes: feca6b62a6 ("lustre: sec: O_DIRECT for encrypted file") WC-bug-id: https://jira.whamcloud.com/browse/LU-14045 Lustre-commit: e07d0516dcde4b ("LU-14045 sec: fix O_DIRECT and encrypted files") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/40295 Reviewed-by: Andreas Dilger Reviewed-by: Neil Brown Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/osc/osc_request.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 746b695..f225ccd 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1369,8 +1369,11 @@ static inline void osc_release_bounce_pages(struct brw_page **pga, int i; for (i = 0; i < page_count; i++) { - if (!pga[i]->pg->mapping) - /* bounce pages are unmapped */ + /* Bounce pages allocated by a call to + * llcrypt_encrypt_pagecache_blocks() in osc_brw_prep_request() + * are identified thanks to the PageChecked flag. + */ + if (PageChecked(pga[i]->pg)) llcrypt_finalize_bounce_page(&pga[i]->pg); pga[i]->count -= pga[i]->bp_count_diff; pga[i]->off += pga[i]->bp_off_diff; @@ -1453,6 +1456,10 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, ptlrpc_request_free(req); return rc; } + /* Set PageChecked flag on bounce page for + * disambiguation in osc_release_bounce_pages(). + */ + SetPageChecked(data_page); pg->pg = data_page; /* there should be no gap in the middle of page array */ if (i == page_count - 1) {