From patchwork Thu May 24 11:43:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10424447 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B3DF76019D for ; Thu, 24 May 2018 11:43:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A574B29448 for ; Thu, 24 May 2018 11:43:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 993B52944E; Thu, 24 May 2018 11:43:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD18429448 for ; Thu, 24 May 2018 11:43:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A648F6B0005; Thu, 24 May 2018 07:43:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A3D826B000C; Thu, 24 May 2018 07:43:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 952466B0269; Thu, 24 May 2018 07:43:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr0-f197.google.com (mail-wr0-f197.google.com [209.85.128.197]) by kanga.kvack.org (Postfix) with ESMTP id 353086B0005 for ; Thu, 24 May 2018 07:43:53 -0400 (EDT) Received: by mail-wr0-f197.google.com with SMTP id y9-v6so1121856wrg.22 for ; Thu, 24 May 2018 04:43:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=0gv1KvEttAMhp3x5iMsUAqDgGia06km8bt4LW3UV8mg=; b=I+fLqpo76G5gzBHGqCCgH6+sCIJWnG5Krgby1SXVf9s/g6HuZmsIxpfTYuYEZT3IU1 fmWz+U7IPg3Li/cxX8qR57wEzFAYhxB7I/rcgralvVoI99IHLpVWguEY3G+7UwBauawA p9NYdsPev63a5vENfbzZDjg207tOKp0AwZqPQ2tVSsjkvw3sXcg1u/a5dLJtdERjkrFp RiaRHQaFtVRBshdr0Sn+M01YhCEZcDBzPT79hEGN2LABzdClUkRr/RBmAh389L9483nF KGUjnbBn/aR5q4ya7kQs+dMWN6oGMt7a0vWQDP9GhgTvEWUV7ozx4OJofScYQGXHqVG7 OHAw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: ALKqPwc4BrrAWmDr+bpeDQKxS89SDq6iWu/5WAd++RZYDShYCFBhuTM+ lH/ERe7pxMbMP47ekA86i7xuYWchXcX5O/X2pXc7ItO+H59CUw7XA7AtFRYeQ9aGsFXLJ/tRBSF PDwWaAyZpU+qd/KqPQxP4f8m9bdUEH77zyj5cylSy2Yup1O3Vd0ymEKTjcxhHBivo+ck1P1J8aV wQ+TRLr6s3t/LsSpG2M4l/65qyfUgbJ3A05qaYOepfIx9xUEtolJxvrgbf28OGZ8QoRloN3PoW6 6bqx7J4WMHmf/sfiJoh20utGmsexJZby1mf7pAKRMGoPQ1pGrUUHkee0B2B3rh5KBDN8glLF8T2 qTeBWZF7iIWYsgJujDyxOSGUo9lpiNGntToagLxVSESIPzJ+r9awKyGUimk+QwtglX4OD+jG4Q= = X-Received: by 2002:a50:9129:: with SMTP id e38-v6mr11607030eda.191.1527162232753; Thu, 24 May 2018 04:43:52 -0700 (PDT) X-Received: by 2002:a50:9129:: with SMTP id e38-v6mr11606969eda.191.1527162231720; Thu, 24 May 2018 04:43:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527162231; cv=none; d=google.com; s=arc-20160816; b=s2ZIsZnzfUlC6Y0r/n49ucNMSPcmX9RlQNbb1CUep0NDnr8v2cj/J54PO2zvAzIkSy cqeBz5Nc//9/j1s8hiahBUNYBaB/Hav4VfonTEdzSK4x+daOjk89BBDmdNjupjmLFYYD pXrwVfYytu1h3T1r1NmcEsnAYfPGUuU4IZ0Bti6uaE5Od16PvL3BpmX48aEo6cuqlS+P PTI4OCd8TjKozY00X9QNRvQr7vSVzgu28ZBaCwwy71x6uk5h5dsu9KuvpbAYC40fOxih SchSZWlYOHCZkg6nKqD2hUZABLrMPpWvFtf7T8SMQCshMrl7ALYSmDC+i3EekDzfkmQY YW/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=0gv1KvEttAMhp3x5iMsUAqDgGia06km8bt4LW3UV8mg=; b=aUPqxP+FhU2nUhGsu8dBVJa/vJxRG+nO9Xjp+6tHi5sM14MgLEZPXibmKIkv3sGHFh pAODeBL8zOW2jCEP9HBigyS3/1UaJFE4YcUVXqzo9hfaFv2v/aXatOko42wwQViTzAti rYUfSYVgEs7f3s9UHWCRvfjFEqhH1HrxbUrHZo5teDPaZ2cpHoHXTMH4bT26z0FSbuFp LRs+KMFsTpABAVMX/febjdrKehTHTGDPmzWQo16m3MEuQVZxL6w5wl39tkahCs6zRmdJ vbAIy8c+TuoWmHIaCc7XpJ9C88AzGckJfPuYxPR2ZCNJ2/iMTYzXltwXZBm0Fc2SWqLE iEOQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c21-v6sor11193485eds.24.2018.05.24.04.43.51 for (Google Transport Security); Thu, 24 May 2018 04:43:51 -0700 (PDT) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AB8JxZrYcuhW/209zdlb73zcY+yOmK+OLvbsG2hKkvIkol40+DroJs1D2s3gPDJ7U5fZ+6Fb81/0vA== X-Received: by 2002:a50:83a5:: with SMTP id 34-v6mr11718283edi.276.1527162231211; Thu, 24 May 2018 04:43:51 -0700 (PDT) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id f16-v6sm11159981edr.32.2018.05.24.04.43.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 May 2018 04:43:50 -0700 (PDT) From: Michal Hocko To: Jonathan Corbet Cc: LKML , , , Michal Hocko , "Darrick J. Wong" , David Sterba Subject: [PATCH] doc: document scope NOFS, NOIO APIs Date: Thu, 24 May 2018 13:43:41 +0200 Message-Id: <20180524114341.1101-1-mhocko@kernel.org> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180424183536.GF30619@thunk.org> References: <20180424183536.GF30619@thunk.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko Although the api is documented in the source code Ted has pointed out that there is no mention in the core-api Documentation and there are people looking there to find answers how to use a specific API. Cc: "Darrick J. Wong" Cc: David Sterba Requested-by: "Theodore Y. Ts'o" Signed-off-by: Michal Hocko --- Hi Johnatan, Ted has proposed this at LSFMM and then we discussed that briefly on the mailing list [1]. I received some useful feedback from Darrick and Dave which has been (hopefully) integrated. Then the thing fall off my radar rediscovering it now when doing some cleanup. Could you take the patch please? [1] http://lkml.kernel.org/r/20180424183536.GF30619@thunk.org .../core-api/gfp_mask-from-fs-io.rst | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/core-api/gfp_mask-from-fs-io.rst diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst new file mode 100644 index 000000000000..e8b2678e959b --- /dev/null +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst @@ -0,0 +1,55 @@ +================================= +GFP masks used from FS/IO context +================================= + +:Date: Mapy, 2018 +:Author: Michal Hocko + +Introduction +============ + +Code paths in the filesystem and IO stacks must be careful when +allocating memory to prevent recursion deadlocks caused by direct +memory reclaim calling back into the FS or IO paths and blocking on +already held resources (e.g. locks - most commonly those used for the +transaction context). + +The traditional way to avoid this deadlock problem is to clear __GFP_FS +resp. __GFP_IO (note the later implies clearing the first as well) in +the gfp mask when calling an allocator. GFP_NOFS resp. GFP_NOIO can be +used as shortcut. It turned out though that above approach has led to +abuses when the restricted gfp mask is used "just in case" without a +deeper consideration which leads to problems because an excessive use +of GFP_NOFS/GFP_NOIO can lead to memory over-reclaim or other memory +reclaim issues. + +New API +======== + +Since 4.12 we do have a generic scope API for both NOFS and NOIO context +``memalloc_nofs_save``, ``memalloc_nofs_restore`` resp. ``memalloc_noio_save``, +``memalloc_noio_restore`` which allow to mark a scope to be a critical +section from the memory reclaim recursion into FS/IO POV. Any allocation +from that scope will inherently drop __GFP_FS resp. __GFP_IO from the given +mask so no memory allocation can recurse back in the FS/IO. + +FS/IO code then simply calls the appropriate save function right at the +layer where a lock taken from the reclaim context (e.g. shrinker) and +the corresponding restore function when the lock is released. All that +ideally along with an explanation what is the reclaim context for easier +maintenance. + +What about __vmalloc(GFP_NOFS) +============================== + +vmalloc doesn't support GFP_NOFS semantic because there are hardcoded +GFP_KERNEL allocations deep inside the allocator which are quite non-trivial +to fix up. That means that calling ``vmalloc`` with GFP_NOFS/GFP_NOIO is +almost always a bug. The good news is that the NOFS/NOIO semantic can be +achieved by the scope api. + +In the ideal world, upper layers should already mark dangerous contexts +and so no special care is required and vmalloc should be called without +any problems. Sometimes if the context is not really clear or there are +layering violations then the recommended way around that is to wrap ``vmalloc`` +by the scope API with a comment explaining the problem.