From patchwork Fri Nov 17 17:12:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459164 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="SAvVAj92" Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C4B5D57 for ; Fri, 17 Nov 2023 09:12:31 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1cc3bb4c307so20165535ad.0 for ; Fri, 17 Nov 2023 09:12:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241151; x=1700845951; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zsXp4akWhByUsbhRVOnU84CPXghQ4aI2lN+tCctcdH0=; b=SAvVAj92HZ1HS6L78aWxMpa9vL0HjaFNGoE/xzkniHjbft2fBCxGAKVC3fDWIMtz8T 0Q8Gdwus425xXVjB/oaD3Vk+F9eHq9Zb5inaJy64cF4HA0wppnlNMlW9zZxjJnEsF0KJ GeOcb3Slo1SOYuGq4uM3N1p7yVi6nIDt9e63G/i4iu3QXq0pYDrIabjgXhMGa81+QCQP c0TO+NMNcWbkhocxVTyxNByNQQSe1q97mq+BCJIUuPSOHforYwoteZPVEaqxkN0fCp2h Qqj8obZYP9b+Ax2JoRO37LE1HJ44tJmJsLwwGsa421F8OaQl8lHfeQePbB7XOUAF0Nlw 4RIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241151; x=1700845951; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zsXp4akWhByUsbhRVOnU84CPXghQ4aI2lN+tCctcdH0=; b=LJhy8c2Uuoqv1GgR+tGdJ9XwRmPXmI1A5JS5XAk2cFM/lz5Ya7skuM39gTLV9ji5tq c06n/0gW/dHzYJelighYzc0YwK8ufXHxja+Byq8ppWFvy6WHNRrUn3LthBigSY6PtLSm VOLfwwsl6fcKBpN4UApFkwh4SgoX3RA96DBFNI/cBkk5Ma04zixZj91vgHDADLAtDbpw zQjglemvIhoXK061MNPTziFviHmgfQ/wO+aEXUwmhuS4K3vr8GvRqltZ9U/YnqvSBzCN zb+5v0ov+4CzRk87AgC5Q+BeHCvMgg3OAkEeQl/BolTaxlWSgrfGAygHpHQBU+TanavI 61xA== X-Gm-Message-State: AOJu0Yy6j6seICyf3lZW/S3oGCCz/G3wKINZYYxwP503p2f5Z81O7oHi ejErjviltUgeKlZvn1fcWQidjU61L1tIlefRTuw= X-Google-Smtp-Source: AGHT+IEzO7OOtRz7VaeGDR7n3NTuNYAzXgo6/Y08x7WKenl0GXzaR+ddYY2iPY3JBetDqAL8dTTk6w== X-Received: by 2002:a17:902:ecc1:b0:1cc:6dd4:5955 with SMTP id a1-20020a170902ecc100b001cc6dd45955mr287585plh.19.1700241150708; Fri, 17 Nov 2023 09:12:30 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:30 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com Subject: [PATCH net-next 1/6] selftests: tc-testing: cap parallel tdc to 4 cores Date: Fri, 17 Nov 2023 14:12:03 -0300 Message-Id: <20231117171208.2066136-2-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org We have observed a lot of lock contention and test instability when running with >8 cores. Enough to actually make the tests run slower than with fewer cores. Cap the maximum cores of parallel tdc to 4 which showed in testing to be a reasonable number for efficiency and stability in different kernel config scenarios. Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- tools/testing/selftests/tc-testing/tdc.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py index a6718192aff3..f764b43f112b 100755 --- a/tools/testing/selftests/tc-testing/tdc.py +++ b/tools/testing/selftests/tc-testing/tdc.py @@ -1017,6 +1017,7 @@ def main(): parser = pm.call_add_args(parser) (args, remaining) = parser.parse_known_args() args.NAMES = NAMES + args.mp = min(args.mp, 4) pm.set_args(args) check_default_settings(args, remaining, pm) if args.verbose > 2: From patchwork Fri Nov 17 17:12:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459165 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="KQya1Tm0" Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1C3CD57 for ; Fri, 17 Nov 2023 09:12:34 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1cc131e52f1so25272165ad.0 for ; Fri, 17 Nov 2023 09:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241154; x=1700845954; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZxDQzw6ad5px6yFhBl5tnkJGwPfE9XAaXdVR53d4yv4=; b=KQya1Tm00LfHfngzCT8W6zFsg72buOrdaooLPVGLw+CuiuHQU07VfmODOQAZFlKPp2 93vL4eYRvE5JmMdODROKFhXlpmvZMc5x2GXc8rXDvPG/CB9gCaiOZa/OEQoAVZGMhbZe 5gBknA0I9nTA0buiutw2y64k1cFs5qx7dnF4YanoCgeMsIiI5sPdPDDgcF8/K3FeiDAx BPdu4hJCfsadyNG41JWwljN/FmTuCdAr61b8wXSnGFQKLM0UWlzBzUh9dUrCy9aOoerS JnJtAnmqEnqNSlfR+CM2tTsrM4cBx3AqaYIsAJh447a4miQzrAjeZ5snPZoOb+TW1TGa 5ZjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241154; x=1700845954; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZxDQzw6ad5px6yFhBl5tnkJGwPfE9XAaXdVR53d4yv4=; b=h70mnp7zL2EgMqyZ3Bi2ZdT3P9KyXy5VXxNaAH590hNEtBYNBzFAcSKsrVu9hDRHSX ISduiXi2yyHkftdOqJowL8aCVzCJXHOKeCd/YnTxfx1XhdwGZ6ODEqmu5C0OUhRbPmB/ V77irszw7l3FctMYjSWOUGFjmsoW0m54OSBLU9DRpDzxPDh/qk2B5bPBCJl6GWTL7AGw NK5c3ZgVDcmnpk6XLBAgWapLv55lQw3ZuyyuRKNO1NklwEkFArIWP+4SBni8xruwOD/E ANLLyT8xNV+M/3PD3K1NBC0KaivBjZqAbLkMMJ4SGX+z4daZLgz06i4dnJXkk3zhbiZV q6Gw== X-Gm-Message-State: AOJu0Yz9R6fJMw53/NKFJa8bXaUF4A2ryIG66JHrNeGW0vyHogDmgr44 DtN2jlQADBcT7R2a6QY52hfqd1BrqEVzevWfyQI= X-Google-Smtp-Source: AGHT+IF2cH447d6VEGuB1MVbsnaSO4m7AUa+vAiCkF0qQHPup8CjAbliC+QXGDaav3HeTShdQU8t8g== X-Received: by 2002:a17:902:f7cf:b0:1cc:5f5a:5d3 with SMTP id h15-20020a170902f7cf00b001cc5f5a05d3mr6474902plw.22.1700241154151; Fri, 17 Nov 2023 09:12:34 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:33 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com, kernel test robot Subject: [PATCH net-next 2/6] selftests: tc-testing: move back to per test ns setup Date: Fri, 17 Nov 2023 14:12:04 -0300 Message-Id: <20231117171208.2066136-3-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Surprisingly in kernel configs with most of the debug knobs turned on, pre-allocating the test resources makes tdc run much slower overall than when allocating resources on a per test basis. As these knobs are used in kselftests in downstream CIs, let's go back to the old way of doing things to avoid kselftests timeouts. Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-lkp/202311161129.3b45ed53-oliver.sang@intel.com Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- .../tc-testing/plugin-lib/nsPlugin.py | 68 +++++++------------ 1 file changed, 25 insertions(+), 43 deletions(-) diff --git a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py index 62974bd3a4a5..2b8cbfdf1083 100644 --- a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py +++ b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py @@ -17,44 +17,6 @@ except ImportError: netlink = False print("!!! Consider installing pyroute2 !!!") -def prepare_suite(obj, test): - original = obj.args.NAMES - - if 'skip' in test and test['skip'] == 'yes': - return - - if 'nsPlugin' not in test['plugins']: - return - - shadow = {} - shadow['IP'] = original['IP'] - shadow['TC'] = original['TC'] - shadow['NS'] = '{}-{}'.format(original['NS'], test['random']) - shadow['DEV0'] = '{}id{}'.format(original['DEV0'], test['id']) - shadow['DEV1'] = '{}id{}'.format(original['DEV1'], test['id']) - shadow['DUMMY'] = '{}id{}'.format(original['DUMMY'], test['id']) - shadow['DEV2'] = original['DEV2'] - obj.args.NAMES = shadow - - if netlink == True: - obj._nl_ns_create() - else: - obj._ns_create() - - # Make sure the netns is visible in the fs - while True: - obj._proc_check() - try: - ns = obj.args.NAMES['NS'] - f = open('/run/netns/{}'.format(ns)) - f.close() - break - except: - time.sleep(0.1) - continue - - obj.args.NAMES = original - class SubPlugin(TdcPlugin): def __init__(self): self.sub_class = 'ns/SubPlugin' @@ -65,19 +27,39 @@ class SubPlugin(TdcPlugin): super().pre_suite(testcount, testlist) - print("Setting up namespaces and devices...") + def prepare_test(self, test): + if 'skip' in test and test['skip'] == 'yes': + return - with Pool(self.args.mp) as p: - it = zip(cycle([self]), testlist) - p.starmap(prepare_suite, it) + if 'nsPlugin' not in test['plugins']: + return - def pre_case(self, caseinfo, test_skip): + if netlink == True: + self._nl_ns_create() + else: + self._ns_create() + + # Make sure the netns is visible in the fs + while True: + self._proc_check() + try: + ns = self.args.NAMES['NS'] + f = open('/run/netns/{}'.format(ns)) + f.close() + break + except: + time.sleep(0.1) + continue + + def pre_case(self, test, test_skip): if self.args.verbose: print('{}.pre_case'.format(self.sub_class)) if test_skip: return + self.prepare_test(test) + def post_case(self): if self.args.verbose: print('{}.post_case'.format(self.sub_class)) From patchwork Fri Nov 17 17:12:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459166 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="sd5eOrNm" Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B230D57 for ; Fri, 17 Nov 2023 09:12:38 -0800 (PST) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1cc3388621cso25046265ad.1 for ; Fri, 17 Nov 2023 09:12:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241157; x=1700845957; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4sSM2YYcJWYCrctxv7+5tEvoUWWBO1kFQOKrhYqc9ds=; b=sd5eOrNmGRO4dX7PnMsRQ8Mu3TOMnr0N1bF1uo9fuyFsrSnYIg8b7S+RaQF9dyzQLL WdIyNWjx2BaQ9kZYGGDKiKLkOVn0+Onun6BBuQNPDROT73jQp3Wy6v6bgslMyccpMLzU 1VVj8kc1QnyHLovx9s2Axdqbs5WxmlOrqb2ATneRmX7cH6NZoaYW5mMIYING4GaaZKGh NUgV5N7nFAy0y+15M3erdcOx19AZKnrPQ+/ilWaGggHE0HHizNJv9TTmydVx/RlmRpXl c+UX/TPdjHZZLjeUqxZQyuVuQrh2MmS7hGWQkZlYjiQguHHN1ZbTkz2sRuyOaGiRnQ9h VMwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241157; x=1700845957; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4sSM2YYcJWYCrctxv7+5tEvoUWWBO1kFQOKrhYqc9ds=; b=KeBiflu6ErnuTeBGrFZ3764uxbKAbysymouk1+hm9fnVW+fJSkL8+Wh33Rb1lSJX6H sdedBhvbPFRyJvDyzSHF63vY9n6LmJXRQ5t38xVcpJ1/W+R8+6Q3KMb4FPIbELCwQdkV stxgzAF1fzkhRoQSevF6oEXds4AlKGB5w3ZxIbfqk9DMImvn6qo3r0klgivnBN7AxTEg /8o8bb6RHKRGt6oAqSsV3pZV7a7yuviXoCD/OmsZp0a6xCSiDQFj6tLY4kIWQ+YXezbw Y5c1yozSjGoDT9ZlihvAxD/L5gQUpHThvrt+Lop0gbKYI1NjPazSv7ZgeMOIowHEAmCL 0P0g== X-Gm-Message-State: AOJu0YyL4AkZpSJZcfpmGZOrQNJZD1zAlYlJuxxSco4ZA7E8gx3qW98P AkwB+Bhk78qWo0A4/c/v9tt6r0w6UArGugMLx6g= X-Google-Smtp-Source: AGHT+IGFk6zQcJi6Wrn5fa+btHvWP9izBoZjX6bY2ibKD4OLNn1EImDdOkNlxHAS5eO+XWSzWSxgVA== X-Received: by 2002:a17:902:e845:b0:1cc:6078:52ff with SMTP id t5-20020a170902e84500b001cc607852ffmr7283744plg.26.1700241157386; Fri, 17 Nov 2023 09:12:37 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:37 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com Subject: [PATCH net-next 3/6] selftests: tc-testing: use netns delete from pyroute2 Date: Fri, 17 Nov 2023 14:12:05 -0300 Message-Id: <20231117171208.2066136-4-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org When pyroute2 is available, use the native netns delete routine instead of calling iproute2 to do it. As forks are expensive with some kernel configs, minimize its usage to avoid kselftests timeouts. Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- .../testing/selftests/tc-testing/plugin-lib/nsPlugin.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py index 2b8cbfdf1083..920dcbedc395 100644 --- a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py +++ b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py @@ -64,7 +64,10 @@ class SubPlugin(TdcPlugin): if self.args.verbose: print('{}.post_case'.format(self.sub_class)) - self._ns_destroy() + if netlink == True: + self._nl_ns_destroy() + else: + self._ns_destroy() def post_suite(self, index): if self.args.verbose: @@ -174,6 +177,10 @@ class SubPlugin(TdcPlugin): ''' self._exec_cmd_batched('pre', self._ns_create_cmds()) + def _nl_ns_destroy(self): + ns = self.args.NAMES['NS'] + netns.remove(ns) + def _ns_destroy_cmd(self): return self._replace_keywords('netns delete {}'.format(self.args.NAMES['NS'])) From patchwork Fri Nov 17 17:12:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459167 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="wD8d0+5F" Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7453CD79 for ; Fri, 17 Nov 2023 09:12:41 -0800 (PST) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1cc3c51f830so18222065ad.1 for ; Fri, 17 Nov 2023 09:12:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241161; x=1700845961; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5sFgMe+AuXutIDLD0oR35V1Wr0GcNbXboHMMBFDV8h8=; b=wD8d0+5FYKfc+rbNtjZx2XazYiI70/fA2K/9LqgJ3dYGEDwRRjeJJB1VuwDuyffiG3 PuER6UvOL8qkCsuhRW3pXoy4lxyeMDKzKkdyyBimqYX7BlFtcupZOa2vGKuG/WOnjp41 +xWcArbTZyyahiZzgWAnOLvvAx0uhENp9JjxKAIaGgqHBYUh3cSipoZ9Evno88/QExFV ZaXyG6KgMbB3h7dYDV+GvJeYLPrHrDBE7ScwZEHAg1YqqpNKEZ7aeTd7pLNfIVLz8Nz5 lpgGU+Dq+qBC93CdtC4U9duiXAEnkj55ElF5pBR7E1iZj/pqzjzZVLJHdMc2l5G1t4yZ OaAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241161; x=1700845961; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5sFgMe+AuXutIDLD0oR35V1Wr0GcNbXboHMMBFDV8h8=; b=b7xMWmN6u1yFHT2+iAOTnXubuTnyiJE5qTlO0n1XWCHsthHQA9Ph/8V8cJHklYcfxF QJstTAaaJuAe2ofiPmlPfvl+oLx1nK/j7C1ic2wjbbfEpm6+A3dMnlcbBYxNlVvc1iWr m35krh3ycd1d1F2jZbeLF5XefTjQbhwHbS7/fCydcmH3JlQl67JGjPHtFs9wC+0pMFBA RFVU543UrCeUQlkaHuww+BhjIIKsJyvW9IolcDFxcQ7IVApwi4njYQyvo2dGy4crqdc7 8kOdbOVoDNOat8SOIXCwaFjHFlJ6E5m+7/AXfgzL3BtD6bTz/1h5E1Xw2Uba+j2rKYRO A4+A== X-Gm-Message-State: AOJu0Yz+tLAOg3ZQU+BZ1KkYG156VouR0o8f3FHGjlFOiOyqBJOE6/Hp Y6SVbDQ9+lYypsQMKzfJoc/36nQJTgauGMQW1Go= X-Google-Smtp-Source: AGHT+IFB2SxTHkLQnD+QJwN2OjwQGluzURrQNK3Z2ZoSDnFCcVJl1zvHnuzFDpo9Ds3k0ahrZ0tIGQ== X-Received: by 2002:a17:902:d50d:b0:1ce:1892:2fa6 with SMTP id b13-20020a170902d50d00b001ce18922fa6mr385321plg.0.1700241160635; Fri, 17 Nov 2023 09:12:40 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:40 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com Subject: [PATCH net-next 4/6] selftests: tc-testing: leverage -all in suite ns teardown Date: Fri, 17 Nov 2023 14:12:06 -0300 Message-Id: <20231117171208.2066136-5-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Instead of listing lingering ns pinned files and delete them one by one, leverage '-all' from iproute2 to do it in a single process fork. Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- .../testing/selftests/tc-testing/plugin-lib/nsPlugin.py | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py index 920dcbedc395..7b674befceec 100644 --- a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py +++ b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py @@ -74,13 +74,12 @@ class SubPlugin(TdcPlugin): print('{}.post_suite'.format(self.sub_class)) # Make sure we don't leak resources - for f in os.listdir('/run/netns/'): - cmd = self._replace_keywords("$IP netns del {}".format(f)) + cmd = "$IP -a netns del" - if self.args.verbose > 3: - print('_exec_cmd: command "{}"'.format(cmd)) + if self.args.verbose > 3: + print('_exec_cmd: command "{}"'.format(cmd)) - subprocess.run(cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + subprocess.run(cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) def adjust_command(self, stage, command): super().adjust_command(stage, command) From patchwork Fri Nov 17 17:12:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459168 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="EO8PhBys" Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DDB8D68 for ; Fri, 17 Nov 2023 09:12:44 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1ce616f12e4so5503255ad.3 for ; Fri, 17 Nov 2023 09:12:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241164; x=1700845964; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N+qyZzDWnVznUNHIiUa56n6EFtwnakHDNjxkjPjN2zA=; b=EO8PhBysquLp+pei+G1gZ5kS0pyrLlF2pONSAYcdKsE40ETiBtQD19DwJy4FYMq0Vr P4rHbQ/xyqN+bO9yn9viqkQc2jm/5ZNWqWmFVHatSGmEhBhgliV7oi+9pwBsoDsGYN1i daIkAo5tOczMEizlghqEAhFD0Kgp2uXcuQ6HmCxj2oSfFOWie9S3mlJgKw/ldi40H0kN IBbZg/a2RWHnsPg8kTBin+mG1i4RW8J/qVpUTceoZNGLY2brS/2cqx3yRVg2JwYvC7WB J0ngLnDqcZU97NhGDnWygEaAd+ts3BguNJpeC+1bxZp3ABPvyt9isSFdHqGZXcituoiu EKxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241164; x=1700845964; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N+qyZzDWnVznUNHIiUa56n6EFtwnakHDNjxkjPjN2zA=; b=f3r3ZZ90rTc0PnRaH4uCnyZq/c4PPkwxXJxe7Oy+F5y5bFQ349MEv9quSYwG21Mwxc sA9LT2OoaRsp02uCgwEDMLlswG8SwdTIFDorFx3qLy8GdKujm5geXkRJDyFujJWT5hSZ OROYZpb04wmiZd4OMKtvWjyoS6E1NlWXaHlHrhOisTwqshgORcS6mpHdZ7L3TeAWKKKx 5X51aTHw3ClyKDcm7xiHBsoZkx7BEKxgHiBWwWLPtWH2kZE4VBOuJHppLsp8N38F9JrB xdJva6JC/rZy8sMis6OX+zTreClrf7bABYjgNh1jCc/lkIueL+2pHuyX274cKHqrBoR/ FWgw== X-Gm-Message-State: AOJu0Ywgc1Vkg1aK6kcdgCe8pwgAABEi1dA2IfZZE4Yj9yaMUJPmxuPI HpEKDmm2mHVEdJLxYlpTOrmqggNyUxB/g3sPmtw= X-Google-Smtp-Source: AGHT+IERm5+OG9eMPjoC9I+G/uPDt9UpN3eCVk41LuZggvbnJkO+cT0Vu+Vg+ti2BgbmunR0ZvraAw== X-Received: by 2002:a17:902:7682:b0:1b8:954c:1f6 with SMTP id m2-20020a170902768200b001b8954c01f6mr230223pll.36.1700241163825; Fri, 17 Nov 2023 09:12:43 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:43 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com Subject: [PATCH net-next 5/6] selftests: tc-testing: timeout on unbounded loops Date: Fri, 17 Nov 2023 14:12:07 -0300 Message-Id: <20231117171208.2066136-6-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org In the spirit of failing early, timeout on unbounded loops that take longer than 20 ticks to complete. Such loops are to ensure that objects created are already visible so tests can proceed without any issues. If a test setup takes more than 20 ticks to see an object, there's definetely something wrong. Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- .../selftests/tc-testing/plugin-lib/nsPlugin.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py index 7b674befceec..65c8f3f983b9 100644 --- a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py +++ b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py @@ -40,7 +40,10 @@ class SubPlugin(TdcPlugin): self._ns_create() # Make sure the netns is visible in the fs + ticks = 20 while True: + if ticks == 0: + raise TimeoutError self._proc_check() try: ns = self.args.NAMES['NS'] @@ -49,6 +52,7 @@ class SubPlugin(TdcPlugin): break except: time.sleep(0.1) + ticks -= 1 continue def pre_case(self, test, test_skip): @@ -127,7 +131,10 @@ class SubPlugin(TdcPlugin): with IPRoute() as ip: ip.link('add', ifname=dev1, kind='veth', peer={'ifname': dev0, 'net_ns_fd':'/proc/1/ns/net'}) ip.link('add', ifname=dummy, kind='dummy') + ticks = 20 while True: + if ticks == 0: + raise TimeoutError try: dev1_idx = ip.link_lookup(ifname=dev1)[0] dummy_idx = ip.link_lookup(ifname=dummy)[0] @@ -136,17 +143,22 @@ class SubPlugin(TdcPlugin): break except: time.sleep(0.1) + ticks -= 1 continue netns.popns() with IPRoute() as ip: + ticks = 20 while True: + if ticks == 0: + raise TimeoutError try: dev0_idx = ip.link_lookup(ifname=dev0)[0] ip.link('set', index=dev0_idx, state='up') break except: time.sleep(0.1) + ticks -= 1 continue def _ns_create_cmds(self): From patchwork Fri Nov 17 17:12:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13459169 X-Patchwork-Delegate: kuba@kernel.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=mojatatu-com.20230601.gappssmtp.com header.i=@mojatatu-com.20230601.gappssmtp.com header.b="03yixsoL" Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B283DD71 for ; Fri, 17 Nov 2023 09:12:47 -0800 (PST) Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1cc5916d578so19905675ad.2 for ; Fri, 17 Nov 2023 09:12:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20230601.gappssmtp.com; s=20230601; t=1700241167; x=1700845967; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ywcsv0Vot4tIP0btu2oA5BeANdc32gnbECR0tYYRLJY=; b=03yixsoLV6YElKp238rKpO3bjuTETnmFx5roIb43dzgPbeaY6HBaCOCcJ4Dp2C6Z3r piGeTLPMVucFuCZ78H19pvB5iHoc2+fA/4dJ/jgguCyoB7wit8synKI7h9fgroiR8U2E sFT4A9SrRGcxWqA/HTtzMMA3Q9ntQVqynaanIBEJQSa3m6qFabKKkDCKGNQZn87VIuu5 jaUQrpeM9jETseo0cyTXNRtUUkCaWVhey9qig+ph2L6t0GM8PhugppUe5RDs5FBwyrmr YbV88LwMU0wTds8cSLCIpPcEDSvpd0Nt1TMi9vIbtE8Mkh72cXgUkbWK0PjTWZl/FQg8 tRHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700241167; x=1700845967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ywcsv0Vot4tIP0btu2oA5BeANdc32gnbECR0tYYRLJY=; b=d8CsWL3tt75PcDlkFFWa+dTnEhHs+/pWRnD7qzsq/Otyaq1fn+Zy6xn9LkqaTb3H36 ez9QKRlAIVwlY4AaTeXCl5b+MBGrj0RX4Jwaq1K14CpZ5tr40g2ym9NtugJjkoda7+Rd q9SOcgrucAOxA2g98nDTCTlVKAwGd3HovlN8II5yWVJu443V+1n0U/GW0IrO6zUzetAR sMCuG1Tro4yEyDX8dorhTrPKO758gio8x30+XuNuAWJofa7Rud6hAlSr+CV4ElW0fbIm eoLc0zccubnkr28CYM4CWr+tYSgv57uzxakhb6JSbyNufuJ3RD+5B9TDnHU/9Uqz/ByR rZlQ== X-Gm-Message-State: AOJu0YySYH6vDalxu1+UYBZcYYy+NEXug4Gtvnk1DL5yvO6SJcHKhzcy mG82rGyilgSZiIQ4hiQqZSD6r/gVopnfAVXfkf4= X-Google-Smtp-Source: AGHT+IFffJepZH1l3sxSYQy3bL7eQJMnUKwB9jZ3xIDrXY+g0dOJMo+lTpujh1VWCaoV7RQGpZ7RmQ== X-Received: by 2002:a17:903:32c8:b0:1c2:1068:1f4f with SMTP id i8-20020a17090332c800b001c210681f4fmr293382plr.17.1700241167006; Fri, 17 Nov 2023 09:12:47 -0800 (PST) Received: from rogue-one.tail33bf8.ts.net ([2804:14d:5c5e:44fb:49f6:37e1:cbd9:76d]) by smtp.gmail.com with ESMTPSA id c4-20020a170902c1c400b001ce5f0de726sm1343979plc.70.2023.11.17.09.12.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 09:12:46 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, pctammela@mojatatu.com, victor@mojatatu.com Subject: [PATCH net-next 6/6] selftests: tc-testing: report number of workers in use Date: Fri, 17 Nov 2023 14:12:08 -0300 Message-Id: <20231117171208.2066136-7-pctammela@mojatatu.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231117171208.2066136-1-pctammela@mojatatu.com> References: <20231117171208.2066136-1-pctammela@mojatatu.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Report the number of workers in use to process the test batches. Since the number is now subject to a limit, avoid users getting confused. Signed-off-by: Pedro Tammela Reviewed-by: Simon Horman --- tools/testing/selftests/tc-testing/tdc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py index f764b43f112b..669ec89ebfe1 100755 --- a/tools/testing/selftests/tc-testing/tdc.py +++ b/tools/testing/selftests/tc-testing/tdc.py @@ -616,7 +616,7 @@ def test_runner_mp(pm, args, alltests): batches.insert(0, serial) print("Executing {} tests in parallel and {} in serial".format(len(parallel), len(serial))) - print("Using {} batches".format(len(batches))) + print("Using {} batches and {} workers".format(len(batches), args.mp)) # We can't pickle these objects so workaround them global mp_pm