From patchwork Tue Feb 2 18:57:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Gardon X-Patchwork-Id: 12062675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.2 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCF2BC433E9 for ; Tue, 2 Feb 2021 19:02:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 82E5164F5F for ; Tue, 2 Feb 2021 19:02:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239208AbhBBTBz (ORCPT ); Tue, 2 Feb 2021 14:01:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239262AbhBBS7d (ORCPT ); Tue, 2 Feb 2021 13:59:33 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F09F4C0617AA for ; Tue, 2 Feb 2021 10:57:49 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id ds13so2869025pjb.4 for ; Tue, 02 Feb 2021 10:57:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=2jALOnR02DelYQsUJSUDQ9Zj6pptUDPmbQP0oTGAN5k=; b=Iev1zX72MfK7mD5itgi9KXbRFo17KVmLsFSv3pkVFuJA56XKPonwgx4fyJLFQFFuPQ 1qqgfd5EZyJ8faqztnNf/X34uP3WvAVpLYuKv9yoShz8iwOBTAwHzQDi5WzTnVveprJs RrpoX6LFPZfYUIZ7tqxPpOYwi3i2hMR3gtYumreFEWQLAGsIlKQK7wULg3yjsIVQ/t1v a/kR6kdfBC7uLLm3D7koCOuPg+I5Tbsy00wlJ50eCxZtvliRdbiYFuT22Xro5nrXnJpt newSqu6vAiwQB6Zj3k4hWYYfNYE0vRdzBp5rYeG2ENS1o+/mXCiWd660JKRKVumOrNpK sbLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2jALOnR02DelYQsUJSUDQ9Zj6pptUDPmbQP0oTGAN5k=; b=KXQ4oMjAz3SWLDJHMLCUJwXIoieMD+/v9cwOMXVticyNSBlLo5hhHPgtxn9TLWo8M4 AV+3mAbFNmR2m7Wk65QOGao6p98c7AvnP4u6q5hCf17OncbzbOc81KkwOV6yGjyYiWMh TokM0vRm8ZtYLfET96sRC1elyieF0/LshjtYvG5PiRtyXrjoU4LMHGV1GRNxHLgqnajE SMLvbxPV0D+tHkwZhLfYH6fXvukfa3YNodbEoZ+wSE+sbrPgMTRYR/Yt/s0GhYZEzzId jF1VkIf6js6Q7YSLG6J1kMzHBbHJ2Rb4n9l81OGzxI+uctuvh3k03NA1lowxGjslIJya /Sug== X-Gm-Message-State: AOAM532mpmq4KEzRh8+r4sznDeb4QFvtn7t41GeQ7LnEgUwiRv8w61b1 FNrrf0dcl+YzXOqrjbiBv5WGswlSiflj X-Google-Smtp-Source: ABdhPJxT7zsP4FMf12L3RkWtR7DMzVl1GMgVhCc5fj2RYAmrngkxUabKWYjAkS0v/TGl4HDYLG4nU9xuFPXy Sender: "bgardon via sendgmr" X-Received: from bgardon.sea.corp.google.com ([2620:15c:100:202:9090:561:5a98:6d47]) (user=bgardon job=sendgmr) by 2002:a17:902:7c88:b029:e1:c7d:c271 with SMTP id y8-20020a1709027c88b02900e10c7dc271mr24095510pll.74.1612292269427; Tue, 02 Feb 2021 10:57:49 -0800 (PST) Date: Tue, 2 Feb 2021 10:57:12 -0800 In-Reply-To: <20210202185734.1680553-1-bgardon@google.com> Message-Id: <20210202185734.1680553-7-bgardon@google.com> Mime-Version: 1.0 References: <20210202185734.1680553-1-bgardon@google.com> X-Mailer: git-send-email 2.30.0.365.g02bc693789-goog Subject: [PATCH v2 06/28] locking/rwlocks: Add contention detection for rwlocks From: Ben Gardon To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Peter Xu , Sean Christopherson , Peter Shier , Peter Feiner , Junaid Shahid , Jim Mattson , Yulei Zhang , Wanpeng Li , Vitaly Kuznetsov , Xiao Guangrong , Ben Gardon , Ingo Molnar , Will Deacon , Peter Zijlstra , Davidlohr Bueso , Waiman Long Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org rwlocks do not currently have any facility to detect contention like spinlocks do. In order to allow users of rwlocks to better manage latency, add contention detection for queued rwlocks. CC: Ingo Molnar CC: Will Deacon Acked-by: Peter Zijlstra Acked-by: Davidlohr Bueso Acked-by: Waiman Long Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- include/asm-generic/qrwlock.h | 24 ++++++++++++++++++------ include/linux/rwlock.h | 7 +++++++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index 84ce841ce735..0020d3b820a7 100644 --- a/include/asm-generic/qrwlock.h +++ b/include/asm-generic/qrwlock.h @@ -14,6 +14,7 @@ #include #include +#include /* * Writer states & reader shift and bias. @@ -116,15 +117,26 @@ static inline void queued_write_unlock(struct qrwlock *lock) smp_store_release(&lock->wlocked, 0); } +/** + * queued_rwlock_is_contended - check if the lock is contended + * @lock : Pointer to queue rwlock structure + * Return: 1 if lock contended, 0 otherwise + */ +static inline int queued_rwlock_is_contended(struct qrwlock *lock) +{ + return arch_spin_is_locked(&lock->wait_lock); +} + /* * Remapping rwlock architecture specific functions to the corresponding * queue rwlock functions. */ -#define arch_read_lock(l) queued_read_lock(l) -#define arch_write_lock(l) queued_write_lock(l) -#define arch_read_trylock(l) queued_read_trylock(l) -#define arch_write_trylock(l) queued_write_trylock(l) -#define arch_read_unlock(l) queued_read_unlock(l) -#define arch_write_unlock(l) queued_write_unlock(l) +#define arch_read_lock(l) queued_read_lock(l) +#define arch_write_lock(l) queued_write_lock(l) +#define arch_read_trylock(l) queued_read_trylock(l) +#define arch_write_trylock(l) queued_write_trylock(l) +#define arch_read_unlock(l) queued_read_unlock(l) +#define arch_write_unlock(l) queued_write_unlock(l) +#define arch_rwlock_is_contended(l) queued_rwlock_is_contended(l) #endif /* __ASM_GENERIC_QRWLOCK_H */ diff --git a/include/linux/rwlock.h b/include/linux/rwlock.h index 3dcd617e65ae..7ce9a51ae5c0 100644 --- a/include/linux/rwlock.h +++ b/include/linux/rwlock.h @@ -128,4 +128,11 @@ do { \ 1 : ({ local_irq_restore(flags); 0; }); \ }) +#ifdef arch_rwlock_is_contended +#define rwlock_is_contended(lock) \ + arch_rwlock_is_contended(&(lock)->raw_lock) +#else +#define rwlock_is_contended(lock) ((void)(lock), 0) +#endif /* arch_rwlock_is_contended */ + #endif /* __LINUX_RWLOCK_H */