From patchwork Thu Feb 27 21:08:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11409673 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5084514BC for ; Thu, 27 Feb 2020 21:18:58 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 361E9246A1 for ; Thu, 27 Feb 2020 21:18:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 361E9246A1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 12EC221FB5A; Thu, 27 Feb 2020 13:18:49 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6D0EE21FA55 for ; Thu, 27 Feb 2020 13:18:20 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 61F5D909; Thu, 27 Feb 2020 16:18:13 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 604C3468; Thu, 27 Feb 2020 16:18:13 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Thu, 27 Feb 2020 16:08:06 -0500 Message-Id: <1582838290-17243-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> References: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 018/622] lustre: ptlrpc: Add QoS for uid and gid in NRS-TBF X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Shilong , Li Xi , Qian Yingjin , Teddy Chan , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Teddy Chan This patch add a new QoS feature in TBF policy which could limits the rate based on uid or gid. The policy is able to limit the rate both on MDT and OSS site. The command for this feature is like: Start the tbf uid QoS on OST: lctl set_param ost.OSS.*.nrs_policies="tbf uid" Limit the rate of ptlrpc requests of the uid 500 lctl set_param ost.OSS.*.nrs_tbf_rule= "start tbf_name uid={500} rate=100" Start the tbf gid QoS on OST: lctl set_param ost.OSS.*.nrs_policies="tbf gid" Limit the rate of ptlrpc requests of the gid 500 lctl set_param ost.OSS.*.nrs_tbf_rule= "start tbf_name gid={500} rate=100" or use generic tbf rule to mix them on OST: lctl set_param ost.OSS.*.nrs_policies="tbf" Limit the rate of ptlrpc requests of the uid 500 gid 500 lctl set_param ost.OSS.*.nrs_tbf_rule= "start tbf_name uid={500}&gid={500} rate=100" Also, you can use the following rule to control all reqs to mds: Start the tbf uid QoS on MDS: lctl set_param mds.MDS.*.nrs_policies="tbf uid" Limit the rate of ptlrpc requests of the uid 500 lctl set_param mds.MDS.*.nrs_tbf_rule= "start tbf_name uid={500} rate=100" For the linux client we need to send the uid and gid information to the NRS-TBF handling on the servers. WC-bug-id: https://jira.whamcloud.com/browse/LU-9658 Lustre-commit: e0cdde123c14 ("LU-9658 ptlrpc: Add QoS for uid and gid in NRS-TBF") Signed-off-by: Teddy Chan Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Qian Yingjin Reviewed-on: https://review.whamcloud.com/27608 Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/vvp_object.c | 5 ++--- fs/lustre/obdclass/obdo.c | 5 +++++ fs/lustre/osc/osc_request.c | 10 ++++++++++ 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 24cde0d..eeb8823 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -196,7 +196,7 @@ static int vvp_object_glimpse(const struct lu_env *env, static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj, struct cl_req_attr *attr) { - u64 valid_flags = OBD_MD_FLTYPE; + u64 valid_flags = OBD_MD_FLTYPE | OBD_MD_FLUID | OBD_MD_FLGID; struct inode *inode; struct obdo *oa; @@ -204,8 +204,7 @@ static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj, inode = vvp_object_inode(obj); if (attr->cra_type == CRT_WRITE) { - valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME | - OBD_MD_FLUID | OBD_MD_FLGID; + valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME; obdo_set_o_projid(oa, ll_i2info(inode)->lli_projid); } obdo_from_inode(oa, inode, valid_flags & attr->cra_flags); diff --git a/fs/lustre/obdclass/obdo.c b/fs/lustre/obdclass/obdo.c index 1926896..e5475f1 100644 --- a/fs/lustre/obdclass/obdo.c +++ b/fs/lustre/obdclass/obdo.c @@ -144,6 +144,11 @@ void lustre_set_wire_obdo(const struct obd_connect_data *ocd, if (!ocd) return; + if (!(wobdo->o_valid & OBD_MD_FLUID)) + wobdo->o_uid = from_kuid(&init_user_ns, current_uid()); + if (!(wobdo->o_valid & OBD_MD_FLGID)) + wobdo->o_gid = from_kgid(&init_user_ns, current_gid()); + if (unlikely(!(ocd->ocd_connect_flags & OBD_CONNECT_FID)) && fid_seq_is_echo(ostid_seq(&lobdo->o_oi))) { /* diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 300dee5..99c9620 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1184,6 +1184,16 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa, oa); + /* For READ and WRITE, we can't fill o_uid and o_gid using from_kuid() + * and from_kgid(), because they are asynchronous. Fortunately, variable + * oa contains valid o_uid and o_gid in these two operations. + * Besides, filling o_uid and o_gid is enough for nrs-tbf, see LU-9658. + * OBD_MD_FLUID and OBD_MD_FLUID is not set in order to avoid breaking + * other process logic + */ + body->oa.o_uid = oa->o_uid; + body->oa.o_gid = oa->o_gid; + obdo_to_ioobj(oa, ioobj); ioobj->ioo_bufcnt = niocount; /* The high bits of ioo_max_brw tells server _maximum_ number of bulks