From patchwork Fri Feb 1 07:29:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Budankov X-Patchwork-Id: 10791873 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EDEC6746 for ; Fri, 1 Feb 2019 07:29:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA97731365 for ; Fri, 1 Feb 2019 07:29:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDEB431387; Fri, 1 Feb 2019 07:29:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id E92B531365 for ; Fri, 1 Feb 2019 07:29:30 +0000 (UTC) Received: (qmail 22247 invoked by uid 550); 1 Feb 2019 07:29:29 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 22229 invoked from network); 1 Feb 2019 07:29:28 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,547,1539673200"; d="scan'208";a="143307554" Subject: [PATCH v1 1/3] perf-security: document perf_events/Perf resource control From: Alexey Budankov To: Jonatan Corbet , Kees Cook , Peter Zijlstra , Thomas Gleixner , Ingo Molnar Cc: Jann Horn , Arnaldo Carvalho de Melo , Jiri Olsa , Namhyung Kim , Alexander Shishkin , Mark Rutland , Andi Kleen , Tvrtko Ursulin , "kernel-hardening@lists.openwall.com" , "linux-doc@vger.kernel.org" , linux-kernel References: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Organization: Intel Corp. Message-ID: Date: Fri, 1 Feb 2019 10:29:11 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Content-Language: en-US X-Virus-Scanned: ClamAV using ClamSMTP Extend perf-security.rst file with perf_events/Perf resource control section describing RLIMIT_NOFILE and perf_event_mlock_kb settings for performance monitoring user processes. Signed-off-by: Alexey Budankov --- Documentation/admin-guide/perf-security.rst | 36 +++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/Documentation/admin-guide/perf-security.rst b/Documentation/admin-guide/perf-security.rst index f73ebfe9bfe2..ff6832191577 100644 --- a/Documentation/admin-guide/perf-security.rst +++ b/Documentation/admin-guide/perf-security.rst @@ -84,6 +84,40 @@ governed by perf_event_paranoid [2]_ setting: locking limit is imposed but ignored for unprivileged processes with CAP_IPC_LOCK capability. +perf_events/Perf resource control +--------------------------------- + +perf_events system call API [2]_ allocates file descriptors for every configured +PMU event. Open file descriptors are a per-process accountable *resource* governed +by RLIMIT_NOFILE [11]_ limit (ulimit -n), which is usually derived from the login +shell process. When configuring Perf collection for a long list of events on a +large server system, this limit can be easily hit preventing required monitoring +configuration. RLIMIT_NOFILE limit can be increased on per-user basis modifying +content of limits.conf file [12]_ on some systems. Ordinary Perf sampling session +(perf record) requires an amount of open perf_event file descriptors that is not +less than a number of monitored events multiplied by a number of monitored CPUs. + +An amount of memory available to user processes for capturing performance monitoring +data is governed by perf_event_mlock_kb [2]_ setting. This perf_event specific +*resource* setting defines overall per-cpu limits of memory allowed for mapping +by the user processes to execute performance monitoring. The setting essentially +extends RLIMIT_MEMLOCK [11]_ limit but only for memory regions mapped specially +for capturing monitored performance events and related data. + +For example, if a machine has eight cores and perf_event_mlock_kb limit is set +to 516 KiB then a user process is provided with 516 KiB * 8 = 4128 KiB of memory +above RLIMIT_MEMLOCK limit (ulimit -l) for perf_event mmap buffers. In particular +this means that if the user wants to start two or more performance monitoring +processes, it is required to manually distribute available 4128 KiB between the +monitoring processes, for example, using --mmap-pages Perf record mode option. +Otherwise, the first started performance monitoring process allocates all available +4128 KiB and the other processes will fail to proceed due to the lack of memory. + +RLIMIT_MEMLOCK and perf_event_mlock_kb *resource* constraints are ignored for +processes with CAP_IPC_LOCK capability. Thus, perf_events/Perf privileged users +can be provided with memory above the constraints for perf_events/Perf performance +monitoring purpose by providing the Perf executable with CAP_IPC_LOCK capability. + Bibliography ------------ @@ -94,4 +128,6 @@ Bibliography .. [5] ``_ .. [6] ``_ .. [7] ``_ +.. [11] ``_ +.. [12] ``_ From patchwork Fri Feb 1 07:30:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Budankov X-Patchwork-Id: 10791875 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9324E6C2 for ; Fri, 1 Feb 2019 07:30:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EDDB3198A for ; Fri, 1 Feb 2019 07:30:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 632953148E; Fri, 1 Feb 2019 07:30:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 54CBD3198A for ; Fri, 1 Feb 2019 07:30:22 +0000 (UTC) Received: (qmail 24141 invoked by uid 550); 1 Feb 2019 07:30:21 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 24123 invoked from network); 1 Feb 2019 07:30:20 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,547,1539673200"; d="scan'208";a="130680853" Subject: [PATCH v1 2/3] perf-security: document collected perf_events/Perf data categories From: Alexey Budankov To: Jonatan Corbet , Kees Cook , Peter Zijlstra , Thomas Gleixner , Ingo Molnar Cc: Jann Horn , Arnaldo Carvalho de Melo , Jiri Olsa , Namhyung Kim , Alexander Shishkin , Mark Rutland , Andi Kleen , Tvrtko Ursulin , "kernel-hardening@lists.openwall.com" , "linux-doc@vger.kernel.org" , linux-kernel References: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Organization: Intel Corp. Message-ID: <2af12b3e-3003-1b30-8e8e-fcdf6a1dcf57@linux.intel.com> Date: Fri, 1 Feb 2019 10:30:02 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Content-Language: en-US X-Virus-Scanned: ClamAV using ClamSMTP Document and categorize system and performance data into groups that can be captured by perf_events/Perf and explicitly indicate the group that can contain process sensitive data. Signed-off-by: Alexey Budankov --- Documentation/admin-guide/perf-security.rst | 32 +++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/perf-security.rst b/Documentation/admin-guide/perf-security.rst index ff6832191577..7da7fa459718 100644 --- a/Documentation/admin-guide/perf-security.rst +++ b/Documentation/admin-guide/perf-security.rst @@ -11,8 +11,34 @@ impose a considerable risk of leaking sensitive data accessed by monitored processes. The data leakage is possible both in scenarios of direct usage of perf_events system call API [2]_ and over data files generated by Perf tool user mode utility (Perf) [3]_ , [4]_ . The risk depends on the nature of data that -perf_events performance monitoring units (PMU) [2]_ collect and expose for -performance analysis. Having that said perf_events/Perf performance monitoring +perf_events performance monitoring units (PMU) [2]_ and Perf collect and expose +for performance analysis. Collected system and performance data may be split into +several categories: + +1. System hardware and software configuration data, for example: a CPU model and + its cache configuration, an amount of available memory and its topology, used + kernel and Perf versions, performance monitoring setup including experiment + time, events configuration, Perf command line parameters, etc. + +2. User and kernel module paths and their load addresses with sizes, process and + thread names with their PIDs and TIDs, timestamps for captured hardware and + software events. + +3. Content of kernel software counters (e.g., for context switches, page faults, + CPU migrations), architectural hardware performance counters (PMC) [8]_ and + machine specific registers (MSR) [9]_ that provide execution metrics for + various monitored parts of the system (e.g., memory controller (IMC), interconnect + (QPI/UPI) or peripheral (PCIe) uncore counters) without direct attribution to any + execution context state. + +4. Content of architectural execution context registers (e.g., RIP, RSP, RBP on + x86_64), process user and kernel space memory addresses and data, content of + various architectural MSRs that capture data from this category. + +Data that belong to the fourth category can potentially contain sensitive process +data. If PMUs in some monitoring modes capture values of execution context registers +or data from process memory then access to such monitoring capabilities requires +to be ordered and secured properly. So, perf_events/Perf performance monitoring is the subject for security access control management [5]_ . perf_events/Perf access control @@ -128,6 +154,8 @@ Bibliography .. [5] ``_ .. [6] ``_ .. [7] ``_ +.. [8] ``_ +.. [9] ``_ .. [11] ``_ .. [12] ``_ From patchwork Fri Feb 1 07:30:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Budankov X-Patchwork-Id: 10791877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A3EA16C2 for ; Fri, 1 Feb 2019 07:31:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CDAD31ED1 for ; Fri, 1 Feb 2019 07:31:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A31831EE2; Fri, 1 Feb 2019 07:31:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 9E4F231ED1 for ; Fri, 1 Feb 2019 07:31:18 +0000 (UTC) Received: (qmail 26183 invoked by uid 550); 1 Feb 2019 07:31:17 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 26161 invoked from network); 1 Feb 2019 07:31:16 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,547,1539673200"; d="scan'208";a="143307941" Subject: [PATCH v1 3/3] perf-security: document perf_events/Perf resource control From: Alexey Budankov To: Jonatan Corbet , Kees Cook , Peter Zijlstra , Thomas Gleixner , Ingo Molnar Cc: Jann Horn , Arnaldo Carvalho de Melo , Jiri Olsa , Namhyung Kim , Alexander Shishkin , Mark Rutland , Andi Kleen , Tvrtko Ursulin , "kernel-hardening@lists.openwall.com" , "linux-doc@vger.kernel.org" , linux-kernel References: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Organization: Intel Corp. Message-ID: <5ac4b7aa-b640-d8a4-1926-cf612fada974@linux.intel.com> Date: Fri, 1 Feb 2019 10:30:58 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <9cfbf7a1-72dd-f9d0-8137-0f120fa74d21@linux.intel.com> Content-Language: en-US X-Virus-Scanned: ClamAV using ClamSMTP Elaborate on possible perf_event/Perf privileged users groups and document steps about creating such groups. Signed-off-by: Alexey Budankov --- Documentation/admin-guide/perf-security.rst | 43 +++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/Documentation/admin-guide/perf-security.rst b/Documentation/admin-guide/perf-security.rst index 7da7fa459718..fe90f8952be9 100644 --- a/Documentation/admin-guide/perf-security.rst +++ b/Documentation/admin-guide/perf-security.rst @@ -73,6 +73,48 @@ enable capturing of additional data required for later performance analysis of monitored processes or a system. For example, CAP_SYSLOG capability permits reading kernel space memory addresses from /proc/kallsyms file. +perf_events/Perf privileged users +--------------------------------- + +Mechanisms of capabilities, privileged capability-dumb files [6]_ and file system +ACLs [10]_ can be used to create a dedicated group of perf_events/Perf privileged +users who are permitted to execute performance monitoring without *scope* limits. +The following steps can be taken to create such a group of privileged Perf users. + +1. Create perf_users group of privileged Perf users, assign perf_users group to + Perf tool executable and limit *access* to the executable for other users in + the system: + +:: + + # groupadd perf_users + # ls -alhF + -rwxr-xr-x 2 root root 11M Oct 19 15:12 perf + # chgrp perf_users perf + # ls -alhF + -rwxr-xr-x 2 root perf_users 11M Oct 19 15:12 perf + # chmod o-rwx perf + # ls -alhF + -rwxr-x--- 2 root perf_users 11M Oct 19 15:12 perf + +2. Assign required capabilities to the Perf tool executable file and enable + members of perf_users group with performance monitoring privileges [6]_ : + +:: + + # setcap "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf + # setcap -v "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf + perf: OK + # getcap perf + perf = cap_sys_ptrace,cap_sys_admin,cap_syslog+ep + +As a result, members of perf_users group are capable of conducting performance +monitoring by using functionality of the configured Perf tool executable that, +when executes, passes perf_events subsystem *scope* checks. + +This specific *access* control management is only available to superuser or root +running processes with CAP_SETPCAP, CAP_SETFCAP [6]_ capabilities. + perf_events/Perf unprivileged users ----------------------------------- @@ -156,6 +198,7 @@ Bibliography .. [7] ``_ .. [8] ``_ .. [9] ``_ +.. [10] ``_ .. [11] ``_ .. [12] ``_