From patchwork Thu Mar 14 00:11:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 10851953 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00C411575 for ; Thu, 14 Mar 2019 00:12:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE01A2A03F for ; Thu, 14 Mar 2019 00:12:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D19F02A063; Thu, 14 Mar 2019 00:12:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 540282A03F for ; Thu, 14 Mar 2019 00:12:51 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id AC18C21F9FE; Wed, 13 Mar 2019 17:12:50 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2F29121F9D1 for ; Wed, 13 Mar 2019 17:12:49 -0700 (PDT) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 4635EAEB3; Thu, 14 Mar 2019 00:12:47 +0000 (UTC) From: NeilBrown To: Andreas Dilger , James Simmons , Oleg Drokin Date: Thu, 14 Mar 2019 11:11:48 +1100 Message-ID: <155252182126.26912.1842463462595601611.stgit@noble.brown> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 00/32] Another bunch of lustre patches. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" X-Virus-Scanned: ClamAV using ClamSMTP There are two parts of this series worth highlighting. Lustre's "memory descriptors" allow a payload to be described as either a list of iov - virtual address based - or bvec - page based. This results in lots of duplication at various levels of the stack. At the bottom of the stack, the virtual addresses get mapped to pages. So it is cleaner to do that mapping at the top of the stack and pass only pages down. So this patchset results in lnet_libmd and lnet_msg only supporting bvec. ksock_tx still supports a single iov for the header, but otherwise only handles bvec. lnet_md supports a single virtual address or a list of bvec, as that is all the client needs. When the server is added we can add back support for a list of iov if it is still needed. The other change involves CONFIG_FS_POSIX_ACL. This is a config option that enables a library of support software. While lustre is out-of-tree and being built to fit a pre-existing config it makes sense to test that config option. When lustre is in-tree, it does not. lustre can select that config option if it wants to. The approach I have taken for now is to introduce CONFIG_LUSTRE_FS_POSIX_ACL, and select CONFIG_FS_POSIX_ACL if the new config is selected. Possibly we should make Posix ACL support non-optional. Some filesystems do that, but most seem to have a config option. I'd be interested in other people's thoughts on the pros and cons. Thanks, NeilBrown --- NeilBrown (32): lustre: remove outdated comments about ->ap_* functions. lustre: ptlrpc: remove ptlrpc_prep_bulk_frag lustre: ptlrpc: make ptlrpc_bulk_frag_ops always const. lustre: ptlrpc: remove inline on non-inlined functions. lustre: ptlrpc: drop support for KVEC bulk descriptors. lustre: ptlrpc: discard BULK_BUF types lustre: ptlrpc: remove *GET*KIOV macros and fields. lustre: ptlrpc: simplify bd_vec access. lustre: lnet: discard LNET_MD_PHYS lustre: lnet: discard LNET_MD_IOVEC lustre: lnet: don't embed whole lnet_md in lnet_event lustre: lnet: merge lnet_md_alloc into lnet_md_build. lustre: lnet: always put a page list into struct lnet_libmd. lustre: lnet: discard kvec option from lnet_libmd. lustre: lnet: remove msg_iov from lnet_msg. lustre: lnet: simplify ksock_tx. lustre: socklnd: discard tx_iov. lustre: socklnd: don't fall-back to tcp_sendpage. lustre: socklnd: discard kiblnd_setup_rd_iov lustre: lnet: discard lnet_kvaddr_to_page lustre: ptlrpc: discard a server-only waitq. lustre: ptlrpc: simplify locking in ptlrpc_add_rqs_to_pool() lustre: ptlrpc: make ptlrpc_last_xid an atomic64_t lustre: ptlrpc: discard cb_list and ptlrpc_set_cbdata; lustre: incorporate BUILD_BUG_ON into ptlrpc_req_async_args() lustre: ptlrpc: don't use list_for_each_entry_safe unnecessarily. lustre: don't call unshare_fs_struct() lustre: don't declare extern variables in C files. lustre: introduce CONFIG_LUSTRE_FS_POSIX_ACL lustre: remove some "#ifdef CONFIG*" from .c files. lustre: mdc: create mdc_acl.c lustre: mgc: remove llog_process_lock .../staging/lustre/include/linux/lnet/lib-lnet.h | 29 --- .../staging/lustre/include/linux/lnet/lib-types.h | 6 - .../lustre/include/uapi/linux/lnet/lnet-types.h | 25 +- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 123 +---------- .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 17 -- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 69 ++---- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 28 +-- .../lustre/lnet/klnds/socklnd/socklnd_proto.c | 17 +- drivers/staging/lustre/lnet/lnet/api-ni.c | 4 drivers/staging/lustre/lnet/lnet/lib-md.c | 220 +++++++------------- drivers/staging/lustre/lnet/lnet/lib-move.c | 21 -- drivers/staging/lustre/lnet/lnet/lib-msg.c | 4 drivers/staging/lustre/lnet/lnet/lo.c | 17 -- drivers/staging/lustre/lnet/lnet/peer.c | 10 - drivers/staging/lustre/lnet/lnet/router.c | 2 drivers/staging/lustre/lnet/selftest/rpc.c | 5 drivers/staging/lustre/lustre/Kconfig | 11 + drivers/staging/lustre/lustre/include/lustre_acl.h | 6 - drivers/staging/lustre/lustre/include/lustre_net.h | 96 +-------- drivers/staging/lustre/lustre/include/obd.h | 32 ++- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 3 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 2 drivers/staging/lustre/lustre/llite/Makefile | 2 .../staging/lustre/lustre/llite/llite_internal.h | 33 +++ drivers/staging/lustre/lustre/llite/llite_lib.c | 39 +--- drivers/staging/lustre/lustre/llite/xattr.c | 4 drivers/staging/lustre/lustre/mdc/Makefile | 1 drivers/staging/lustre/lustre/mdc/mdc_acl.c | 48 ++++ drivers/staging/lustre/lustre/mdc/mdc_internal.h | 10 + drivers/staging/lustre/lustre/mdc/mdc_locks.c | 7 - drivers/staging/lustre/lustre/mdc/mdc_request.c | 67 +----- drivers/staging/lustre/lustre/mgc/mgc_request.c | 8 - drivers/staging/lustre/lustre/obdclass/llog.c | 3 drivers/staging/lustre/lustre/osc/osc_internal.h | 4 drivers/staging/lustre/lustre/osc/osc_io.c | 3 drivers/staging/lustre/lustre/osc/osc_page.c | 4 drivers/staging/lustre/lustre/osc/osc_request.c | 31 +-- drivers/staging/lustre/lustre/ptlrpc/client.c | 151 ++++---------- drivers/staging/lustre/lustre/ptlrpc/events.c | 22 +- drivers/staging/lustre/lustre/ptlrpc/import.c | 21 +- drivers/staging/lustre/lustre/ptlrpc/pers.c | 21 +- .../staging/lustre/lustre/ptlrpc/ptlrpc_module.c | 5 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 6 - drivers/staging/lustre/lustre/ptlrpc/recover.c | 14 + drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 18 +- drivers/staging/lustre/lustre/ptlrpc/sec_plain.c | 12 - drivers/staging/lustre/lustre/ptlrpc/service.c | 4 47 files changed, 450 insertions(+), 835 deletions(-) create mode 100644 drivers/staging/lustre/lustre/mdc/mdc_acl.c -- Signature