From patchwork Tue Jun 14 23:10:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Delyan Kratunov X-Patchwork-Id: 12881695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3E56C433EF for ; Tue, 14 Jun 2022 23:10:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235852AbiFNXKy (ORCPT ); Tue, 14 Jun 2022 19:10:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240106AbiFNXKv (ORCPT ); Tue, 14 Jun 2022 19:10:51 -0400 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C13052E65 for ; Tue, 14 Jun 2022 16:10:48 -0700 (PDT) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25EMd1rY002772 for ; Tue, 14 Jun 2022 16:10:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : subject : date : message-id : content-type : content-transfer-encoding : mime-version; s=facebook; bh=e5hgjKMWM18b5zqxpyY+I7kffWVOC76p6uHNMKcJm/8=; b=C2pFDdaBW3G9x9RyxV+S5fKNWwfgSF3eh3qqdMQ1k6JRN4zVlMHZPbuS9Aebw9Mo3Lsj cBTiedaXtMd7vS2xW2aoMffonq1Tyrpi4g+32ac9HOJOmgOyOvNPP202Oe2/2T4ljkhn OMHo6/ac/zsPHkFLZuIdtcDriu75It6bYY4= Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2173.outbound.protection.outlook.com [104.47.59.173]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3gp8aw1uet-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Jun 2022 16:10:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=igZSUTiHotlCASE4C3jidbUNODWDougMMdst41+uZVmy3dRU706sQV15gLep1jV7aViC+5NhSHNaYbzgGu+B4RHN8YnT69JtNEVn1gYM/85EEmz4W2jCA67AgaZlzt04ITZNrzVvUokDwJnmailspA4/cBURxxgPQLWzCs8Ndy5ir7BXo+IgHlkPazskRkL6kS06JLXBnHDh0jizJ4GZKYhsNaTtmT6gJDlTtgXXIPZE3CsP5BG6ZD+opfo3As3CpCJynVj0b/m/mz+qq9oGlhu5bZcQYYV2zDYN58ZPzE9MpYCxzwyDVQcgGOnyYgQXG+Vgx7sZ39a3jTApD+dj4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LfH8x7mPkjMpqyh2MhosrbdB6oHwfgb/+82/gOfmJSY=; b=NqBxaX/CyTDEsIdX4NyVMuJfidKE9szzyngXKZ6pjnPcRQ0umNhfNOfMkxRpO4WZ8fFS0/J8A4DCiUgx/pm76ItFtfs2AZ92AI6bmTJzBBXSoMxVSNn7zzmlB0uhShsAsRlnWkU873IC01z8KZzBD8elu4/Ev8lNvPfDafMFiii2mNrCqo+f6JxmWide08rFi41/qwi2qTmWNwRKzYIvnGQSekQh4gR5sRpGbz6DzLPpWcmsLM2YoCuTNyQH2hCaEzYth1zPDEbuSnZuZ9TINFDlKqe6ptmJEmODznrR8n8NLoS8oEn1CsOHbw+4Oql3ej+HSaiYpim8slEgrnsKEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fb.com; dmarc=pass action=none header.from=fb.com; dkim=pass header.d=fb.com; arc=none Received: from SJ0PR15MB5154.namprd15.prod.outlook.com (2603:10b6:a03:423::6) by MW5PR15MB5220.namprd15.prod.outlook.com (2603:10b6:303:1a0::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Tue, 14 Jun 2022 23:10:45 +0000 Received: from SJ0PR15MB5154.namprd15.prod.outlook.com ([fe80::8910:e73d:9868:600b]) by SJ0PR15MB5154.namprd15.prod.outlook.com ([fe80::8910:e73d:9868:600b%9]) with mapi id 15.20.5332.016; Tue, 14 Jun 2022 23:10:45 +0000 From: Delyan Kratunov To: "daniel@iogearbox.net" , "ast@kernel.org" , "andrii@kernel.org" , "bpf@vger.kernel.org" Subject: [PATCH bpf-next v4 0/5] sleepable uprobe support Thread-Topic: [PATCH bpf-next v4 0/5] sleepable uprobe support Thread-Index: AQHYgEP6HIJ8fqTs3kql16+mzmQJ+Q== Date: Tue, 14 Jun 2022 23:10:45 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 141285f8-98c6-4ea9-8a22-08da4e5b1cbe x-ms-traffictypediagnostic: MW5PR15MB5220:EE_ x-microsoft-antispam-prvs: x-fb-source: Internal x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: FqL3VicfhSVP52aFnE3a6/RX16uKG57TwBPgaY5RzLfL60V+76a5uaOMOTsEOxhZIHG/9qnBDtsqyLsOaTh6qEjXc6ujR+pdflbHlTvDklIazD3nIw3YWZy4o3pQBootopOjrmBV5dlY1TetHnNqQ5ES1pAbYsqiltznjBi3h7FbZDi+7irlpwN1oUAhsGHpnZU20+SgMcJV06/jS4xy0gE2whQDp38kKPVk/nFGG18k0vPhwtCK5yHcVatIHhLJj/KmLfUNnv6geyakAVBMa9IpBJa1NHU5A0hVWp1eVWxPWjungIPgWLgMTFWWfFKNBXg/KpQnmZBpMYQqtUnxGUfCxy9wcLk+ro9yL48VjMwyGwes6Gmtq8f+lgBYPGi5aDHowAr8dRNQI0fKkkyxcmcyMkftimfwAx80jpzuxypCnw5MLC/Rnvh6SvP/fDSVQRCnubzfDRpm/YSc1NYVKtNc/pvjDwtgU6F6xEvg3HkN10nkM3IgKJwn9vXs0OpU6BdzFCP3YnnTFme9pSEIcK/AHqg08znkGo3AD3gtUte1+Fi8SLEtAXCbwDgb2Q7uf6EFmC1jtUmlFOLcpxxlVRjU7Ny7gYtVzNuxyRvhsMcMvyhB8Fro0aXC8gr93YSw4qFQPd3J6rHEseQ/edMds7LgUI5TZ/kYc3Dq7/cLl11YMfVGPRw/yBgwFxucXGARLTa3+UhvZ0zGQWcqTsbHZhG9fVhNgqZTjW6Sj6dBF6v/+J7efF8EtQkgxnJzxrC6bvMDDX3CnWYoTXofAiQuCc+iasUn+4ej+UovwcTdMvI= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR15MB5154.namprd15.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(366004)(186003)(8936002)(5660300002)(122000001)(38100700002)(83380400001)(6506007)(6486002)(36756003)(966005)(508600001)(38070700005)(6512007)(71200400001)(66946007)(316002)(66476007)(2616005)(2906002)(64756008)(66446008)(76116006)(8676002)(91956017)(66556008)(86362001)(110136005);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?PkYO+kHt0y7hLIDhtSeLT40?= =?iso-8859-1?q?uKGolA7QrlrKRiLcROxMdMxbFhtDMAuXaHG0Jzeh6FP8YvhBXpgS/1d/I/G1?= =?iso-8859-1?q?dQcA9PwHe4SUK8fsmr1d9NyDdG2abWOLOfIzRKa4R0X9zlB80uAep/qh0j2f?= =?iso-8859-1?q?kokKsq1s83MiXH23Jshzu+8IHd2KoKxU0czDminupsYcsfZeJXci+YrYy8Us?= =?iso-8859-1?q?JdzRb1SDSVduvbrl03amr6BcNsPKiuZ+CaDjADSeF97m8/Dq2SbMHJO4JcHU?= =?iso-8859-1?q?yt2YtDrZMl3/oIm/WRn+TTyD6RqVwjd08dHLe7rHGdN6Ld1ERTxvlvCA88gc?= =?iso-8859-1?q?4o5ee2BrqjelQdsLNM8YQ3/cLk/51B4ikFTMr/nz2LV23CKPbR9yG1RvEgf7?= =?iso-8859-1?q?xKSfFtG38LWz+vgmk6wmGReFejuQojJIbX1I+rtys3ZG4cYcISDke2aO/His?= =?iso-8859-1?q?LqOASpWSrJpDsehcgilIVNYtGtWJ/LJoVGhXyctRFRGBw9ZucMb5fKXtoltF?= =?iso-8859-1?q?rmlu8OgjPV3I0vZIQfufYU+pPtgX5oVx8St6SD4o48JGftVNT5grdnTX7561?= =?iso-8859-1?q?DO2KSr3wTYBnju2r7BRR4k4RSvZoQa+/oLYp52RUV3UPEsQO6kp62tAa6qLh?= =?iso-8859-1?q?27/0GPTvv19v/KeQYwPg57xw4/kevmCNByVC/LEu/VleuZlRK2D9x7SpkE1i?= =?iso-8859-1?q?p9s7cA292Lwc223KWJrYKYlt7brjM8fhkSOXGeuWIKviBslO27tVfDCr9cKt?= =?iso-8859-1?q?YqC9ivH8SXwiePD2Oe4Sd6CnONpChBoTQc2TgRYPmWhVSsiOlVZiAnQ8kzqL?= =?iso-8859-1?q?eWy9rpfe5SzbVEfiFMYFrJLjflNyribK+mA03s1zcihL8YNSKUf+gx5f9sET?= =?iso-8859-1?q?48WFYAAJyUUh+/BmDi9/dbn6WPa1150o+yzuAn4+MeHhifaD1qJQa15Yjezd?= =?iso-8859-1?q?Hlx4vAPDiCaiIrGy2n9UhGu46OASwiqRmiNZ3X82klfL0/QDsLwOtK0CgFat?= =?iso-8859-1?q?eCV9cgj7KTOLzp4OvJXqD9vsbtMZ0hvNw13lhXpsaE8ppg3hQr11g+H1804C?= =?iso-8859-1?q?Lz64vviwc/+3UcYFIi8F1T7K9EMgXSxCnP4e2Ala8vdW0XFsw3tACl/LDxn5?= =?iso-8859-1?q?H8abtv5LKXQSp0L8IujWFnw5oJWsMG2ME3jXrWCPG+ZnZRXRNYcAY/7UAM2+?= =?iso-8859-1?q?VBiVWUYuJuktrDGD40mT0fDpQmtbOF9DRletyw13j8tYzzMVCqvkp2Eds8nJ?= =?iso-8859-1?q?QIyK+A9LUEXvPZ310oEAdRG3Sr5TDzhqugJ0o9jLZC+IJBiU9b1qBvI8ryNT?= =?iso-8859-1?q?cV9XSFQTTamByOv640lgYOOzyVOxUiryaDPJlaVQ1sYDbdGYAL/Sllh3OoZW?= =?iso-8859-1?q?J0oI1bnSiEJYJTFNL9wxEyjoP43kV/BW3bxgP4dR6A/G1DHkWRc2KV4McdFc?= =?iso-8859-1?q?Y5C+VuYJYeWiIWik6jrzNUlZDn64enzmGtmQqHTSrwwWrnwFg7dsveryhnse?= =?iso-8859-1?q?hPiUkcOYW2M0JLOKkTr1FxZ85BRA4sSuNuOppnEdVgLvOSai7ME5cwKcoNm+?= =?iso-8859-1?q?9d8Ob1/0Os/GvJGNVPxofQk1y3JEtrKoTizFlu35rcom7SgfVHIFJyomsTEc?= =?iso-8859-1?q?d1s2Dm+kvZZ40dDSqSDs4rN+Hq8qDifiaaCNTkTvsk9hdsNx4CVCv7LOr9F9?= =?iso-8859-1?q?4Y4/FNERvM9YqAqzzVJYVVznTD7UtgNNKKmRTDfuuyvfPjprVN0ZWgD97nIs?= =?iso-8859-1?q?77JBauPqZR8VHnUKFSvqA5QHgbq5EPGuT2TbjbCyo0LCpt8cxUTnDJnAKRJj?= =?iso-8859-1?q?8F1UMd6vDaG4AtMosFN94dedpS7iX?= X-OriginatorOrg: fb.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR15MB5154.namprd15.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 141285f8-98c6-4ea9-8a22-08da4e5b1cbe X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jun 2022 23:10:45.7429 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ISuSiVam0R8IB4sgxyuUH3k+9YP0dwyxaMy4rAf5hfb6ts4x3z3cwySOqbw3qWtx X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR15MB5220 X-Proofpoint-GUID: PxmMMt-pqRDWOr25EaY4y6Fvn2F38ylb X-Proofpoint-ORIG-GUID: PxmMMt-pqRDWOr25EaY4y6Fvn2F38ylb X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.517,FMLib:17.11.64.514 definitions=2022-06-14_10,2022-06-13_01,2022-02-23_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This series implements support for sleepable uprobe programs. Key work is in patches 2 and 3, the rest is plumbing and tests. The main observation is that the only obstacle in the way of sleepable uprobe programs is not the uprobe infrastructure, which already runs in a user-like context, but the rcu usage around bpf_prog_array. Details are in patch 2 but the tl;dr is that we chain trace_tasks and normal rcu grace periods when releasing to array to accommodate users of either rcu type. This introduces latency for non-sleepable users (kprobe, tp) but that's deemed acceptable, given recent benchmarks by Andrii [1]. We're a couple of orders of magnitude under the rate of bpf_prog_array churn that would raise flags (~1MM/s per Paul). [1]: https://lore.kernel.org/bpf/CAEf4BzbpjN6ca7D9KOTiFPOoBYkciYvTz0UJNp5c-_3ptm=Mrg@mail.gmail.com/ v3 -> v4: * Fix kdoc and inline issues * Rebase v2 -> v3: * Inline uprobe_call_bpf into trace_uprobe.c, it's just a bpf_prog_run_array_sleepable call now. * Do not disable preemption for uprobe non-sleepable programs. * Add acks. v1 -> v2: * Fix lockdep annotations in bpf_prog_run_array_sleepable * Chain rcu grace periods only for perf_event-attached programs. This limits the additional latency on the free path to use cases where we know it won't be a problem. * Add tests calling helpers only available in sleepable programs. * Remove kprobe.s support from libbpf. Delyan Kratunov (5): bpf: move bpf_prog to bpf.h bpf: implement sleepable uprobes by chaining gps bpf: allow sleepable uprobe programs to attach libbpf: add support for sleepable uprobe programs selftests/bpf: add tests for sleepable (uk)probes include/linux/bpf.h | 88 +++++++++++++++++++ include/linux/filter.h | 34 ------- kernel/bpf/core.c | 15 ++++ kernel/bpf/verifier.c | 4 +- kernel/events/core.c | 16 ++-- kernel/trace/bpf_trace.c | 4 +- kernel/trace/trace_uprobe.c | 5 +- tools/lib/bpf/libbpf.c | 5 +- .../selftests/bpf/prog_tests/attach_probe.c | 49 ++++++++++- .../selftests/bpf/progs/test_attach_probe.c | 60 +++++++++++++ 10 files changed, 231 insertions(+), 49 deletions(-) --- 2.36.1