From patchwork Tue Mar 12 06:17:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 13589539 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B52459B5F for ; Tue, 12 Mar 2024 06:17:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710224279; cv=none; b=RedGYXuDYoRJmU5nCJ/oHzif0Yty38FBXOe6kn/Q+GFAeycTRATATcKsxDtQsqO0rveQZ0yZIQhSVLV4KwRQyMru/EQC4gVNR3onCwiDgcEw3JkMU1EPzZPhpnNj0R7XAIR8fBDUyPOKtjNtGZRcgksjP09iq0OJZDTZ4fClRJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710224279; c=relaxed/simple; bh=nmhtM2hCPPzOA3PI1UXfFRGQ2/QNiXyNuDJau9M3uBI=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=qVEvquWTJPHvRSkIerCHh23e58cGnx+3eV/7iLP8V5BchM202FoJiVFu6Ixh/YAqGiuc+gZa8VoOQi2hAeOWXfFxl+4sZrsGVZBFzl+Ewm1Z4lt2wsrDvTQdVNETcDAKqkRtXhXyezLbDQCHr+bwSc88uCtSXtgyaX3s6DQpkuw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=N87hEi6d; arc=none smtp.client-ip=209.85.222.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="N87hEi6d" Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-78831914027so153681385a.1 for ; Mon, 11 Mar 2024 23:17:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1710224276; x=1710829076; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=fgGl/GezxVhRISLyx8tZ9PTH8m4h7yDB3m04Y7JMT5Y=; b=N87hEi6diJng+w7+kr1gATEIjESXvtw+Akc9f6rIK63qn+rl+fDI9YzT3pZIvLm2AS OabDFtRqCET9CeXl9TW1mvwAw+q5Iu2x8e+dI/ZOY1Cfo4Pvg4LUdE98CjZm50RoeSQJ 9ymy2k0sf7H86rm0Dysxp5qPoueOXY3sgFMAgpvPmX1ruTqp0SBnhC3ADYXksDFcVFCq ZjtzN2DQqaPe4+byUBjTn/bvgLNg6yJ3WQPxYdioeQdXirc3fdG+iDg8VDKRhhoWNeIh 4zsxKVOOpG+K9JAD1D45Uhd8/QurjMHKsCswPJIeKzSDVNdZWisjkRAUW3KIRBJtxEC2 y21Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710224276; x=1710829076; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fgGl/GezxVhRISLyx8tZ9PTH8m4h7yDB3m04Y7JMT5Y=; b=h1UmtcrH3jNhZ/mMXc5xCb/tMFo9DlupCiMxiegzsS+JmLnG2o8zVDZb01orvjySul lN9yMh9wBSsDy1sAM0i8I13CNuNaVVqAsEjuQJG8IHltrjw/rFSDfkgo2UOgZuf+GLbY R76HS/xboB5CZeCdQFbTBe+dsHNIUlOUGceEhnO6wvVl0z04Cjsv4NfgyIDoxtcyqCol LY2MLwsoIyPZaFVC/0kW8O2yAmCtX84S9OY/8kjzPMxnARbwtoGxq+yKgdVQ3KbR3/+r XQlfkr31SrT8injv8l9n5Ocj/wvA5br7newjwiuiMgOcF/1DNo8JnMAYEWtqAcFgZCLa VR5Q== X-Forwarded-Encrypted: i=1; AJvYcCX8mZojkFNCACare4OPOObjea3qctASiXK2GG1v72cCc6niQMRSlihoYJYHIvm8/7Vg2V529k+s4r8aDwLlOHoXHsyZkR34nvGZ X-Gm-Message-State: AOJu0YzfihyRmZ4nSpUJRGL0SlYzLSQR8OIfTdWNziYKLEz/AUFznApc xtP2CyHh1PoP632QUGz9T78t8UNhFaFx11O1vyKZFil6IzjKfo+nmLxao6H09vo= X-Google-Smtp-Source: AGHT+IGQVjf55XaSaA7QRv11XERHDa0ikizCRigRPLUcQWIpU3ODAYNJ7IKwN9aaMW4bZrAoONZoqw== X-Received: by 2002:a05:620a:4494:b0:788:7dc5:cf8f with SMTP id x20-20020a05620a449400b007887dc5cf8fmr499319qkp.35.1710224276179; Mon, 11 Mar 2024 23:17:56 -0700 (PDT) Received: from n231-228-171.byted.org ([147.160.184.133]) by smtp.gmail.com with ESMTPSA id m18-20020a05620a221200b00787b93d8df1sm3394396qkh.99.2024.03.11.23.17.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Mar 2024 23:17:55 -0700 (PDT) From: "Ho-Ren (Jack) Chuang" To: "Gregory Price" , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, "Eishan Mirakhur" , "Vinicius Tavares Petrucci" , "Ravis OpenSrc" , "Alistair Popple" , "Rafael J. Wysocki" , Len Brown , Dan Williams , Vishal Verma , Dave Jiang , Andrew Morton , Jonathan Cameron , Huang Ying , "Ho-Ren (Jack) Chuang" , linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-mm@kvack.org Cc: "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , qemu-devel@nongnu.org Subject: [PATCH v2 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes Date: Tue, 12 Mar 2024 06:17:26 +0000 Message-Id: <20240312061729.1997111-1-horenchuang@bytedance.com> X-Mailer: git-send-email 2.20.1 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When a memory device, such as CXL1.1 type3 memory, is emulated as normal memory (E820_TYPE_RAM), the memory device is indistinguishable from normal DRAM in terms of memory tiering with the current implementation. The current memory tiering assigns all detected normal memory nodes to the same DRAM tier. This results in normal memory devices with different attributions being unable to be assigned to the correct memory tier, leading to the inability to migrate pages between different types of memory. https://lore.kernel.org/linux-mm/PH0PR08MB7955E9F08CCB64F23963B5C3A860A@PH0PR08MB7955.namprd08.prod.outlook.com/T/ This patchset automatically resolves the issues. It delays the initialization of memory tiers for CPUless NUMA nodes until they obtain HMAT information at boot time, eliminating the need for user intervention. If no HMAT is specified, it falls back to using `default_dram_type`. Example usecase: We have CXL memory on the host, and we create VMs with a new system memory device backed by host CXL memory. We inject CXL memory performance attributes through QEMU, and the guest now sees memory nodes with performance attributes in HMAT. With this change, we enable the guest kernel to construct the correct memory tiering for the memory nodes. -v2: Thanks to Ying's comments, * Rewrite cover letter & patch description * Rename functions, don't use _hmat * Abstract common functions into find_alloc_memory_type() * Use the expected way to use set_node_memory_tier instead of modifying it -v1: * https://lore.kernel.org/linux-mm/20240301082248.3456086-1-horenchuang@bytedance.com/T/ Ho-Ren (Jack) Chuang (1): memory tier: acpi/hmat: create CPUless memory tiers after obtaining HMAT info drivers/acpi/numa/hmat.c | 11 ++++++ drivers/dax/kmem.c | 13 +------ include/linux/acpi.h | 6 ++++ include/linux/memory-tiers.h | 8 +++++ mm/memory-tiers.c | 70 +++++++++++++++++++++++++++++++++--- 5 files changed, 92 insertions(+), 16 deletions(-)