From patchwork Sat Jan 6 08:36:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Shavit X-Patchwork-Id: 13512664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C49A1C46CD2 for ; Sat, 6 Jan 2024 08:37:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2JP0rq7gsNliJQH3Ox+5sxfQIR3BYONFci4HTknIqH8=; b=FGEd57aNE0T3fAcURQkQG6afgO dJuFCOOCcBt06LcWIFyRs8R5uXPCsfIOQwV3VAz3I674vhX/7BAFFGr5PioGbcdhMMopITvsbmW+L G1r4jfkuoxyAYZFYm6pgiJZZTLPIwZohxolOv7t8OtV9Mg6+wAgW32BdrSb/zdXillNf1LvIn0bR4 bs9du+HndA9HVwsGkp+ooOnH1YZMONeIc8qIU5ZpvckwBKh2RSu3PeZjVGA/AjVfT0VAmsWt7z+Zt WO6aFyPWr2qCfdSWNJyJTSVJXYZnlD1DDalLjri4NJUz9RGyPnU82ITcruy7afCOzcTVN7Fs8wyK+ wit6/CqA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rM2AT-000w8v-0Q; Sat, 06 Jan 2024 08:36:33 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rM2AM-000w7J-1j for linux-arm-kernel@lists.infradead.org; Sat, 06 Jan 2024 08:36:28 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5e744f7ca3bso4207147b3.2 for ; Sat, 06 Jan 2024 00:36:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704530185; x=1705134985; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wFcu+shEN12cusNKClL1jBe96Kqnp4aCk1C9WW0GLoA=; b=fuo9nCKKQ0kYON7XvNbu/c99K1ywVSzlRjumFq1+smwv7oQqsCGKJDIEjNzCKAtIs4 jyT9MAXDqrUq5R4qY9tVjQZ1SrVsJmTcn5fA8hN5t0D8Fa0NEV27PBZF9Hm0mMLC0DI+ iwynTROq7B2AYAUHwwEIcX9URTQmth6F7s8q9Npicl3aHuTk1zhMdf2S5CMoqweKrE6Z oytQ9POS5flv8A9W+qk3tk6aGvzy/xsAIjnR1meCGeNYAxStGALjBcQvtHNKLsu9gJMD u64OxUPvHBg7D7x/DDqG2gAyxaF9Ft+bI7kWDm92DUXECCsar3u7oEWy6nxj9B1akq4s L4/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704530185; x=1705134985; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wFcu+shEN12cusNKClL1jBe96Kqnp4aCk1C9WW0GLoA=; b=mnxJnna+jyP4wfdKTWmKYze4+MAmSfSyH3O9WS8CGm8HFp7PXkq+OxG12o3wN1EhzE W5VXwXEFTLnNAixqFbR/PbZmCCWbfH2JMp4LyqQxESkCuw2v9i3mjZ7wfKBqOHR1SNaz YNn5de4DDO8u/Hsj0LTh/jTdqf8I7fZitIIxh3HLzRHKE6XmRpLlqyoHlem6ULb0zCGI /jeAtwRWnb6xBvcXMSXJz8qZTk7brlBsP1oCMXB9n5BOv5OHq1GpoQVPdC4ngMgNZz/7 0ByFmFfIWIsB4jR4LekO5Cwit6LpJE0zmwFbLmqqBScThpAgC9KJZaSVRgOfVVCQs43s NQVA== X-Gm-Message-State: AOJu0YwF/Q6zny7fOczNZBeAmpORMQu8wxHNZKuS0lhe8XhbLq70Ysiy /uMFs0dPCs1YLp/K3JzxDCdshqOn8FKO1uYnaqI= X-Google-Smtp-Source: AGHT+IHp2WmPP2IitN5Ccv3crSok9AZ0SycoqOutjTRb7XCDCwSUQIMXYQziOB/HgiuQvpFlm90m8xTZEoHz X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:356c:2731:54ab:8e9]) (user=mshavit job=sendgmr) by 2002:a05:690c:a94:b0:5d0:a744:719a with SMTP id ci20-20020a05690c0a9400b005d0a744719amr300195ywb.2.1704530185535; Sat, 06 Jan 2024 00:36:25 -0800 (PST) Date: Sat, 6 Jan 2024 16:36:15 +0800 In-Reply-To: <20240106083617.1173871-1-mshavit@google.com> Mime-Version: 1.0 References: <20240103175043.GS50406@nvidia.com> <20240106083617.1173871-1-mshavit@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20240106083617.1173871-2-mshavit@google.com> Subject: [PATCH] iommu/arm-smmu-v3: Make CD programming use arm_smmu_write_entry_step() From: Michael Shavit To: jgg@nvidia.com Cc: iommu@lists.linux.dev, joro@8bytes.org, linux-arm-kernel@lists.infradead.org, robin.murphy@arm.com, will@kernel.org, nicolinc@nvidia.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240106_003626_593478_88D015BE X-CRM114-Status: GOOD ( 18.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Jason Gunthorpe CD table entries and STE's have the same essential programming sequence, just with different types and sizes. Have arm_smmu_write_ctx_desc() generate a target CD and call arm_smmu_write_entry_step() to do the programming. Due to the way the target CD is generated by modifying the existing CD this alone is not enough for the CD callers to be freed of the ordering requirements. The following patches will make the rest of the CD flow mirror the STE flow with precise CD contents generated in all cases. Signed-off-by: Jason Gunthorpe Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 90 +++++++++++++++------ 1 file changed, 67 insertions(+), 23 deletions(-) base-commit: 2cc14f52aeb78ce3f29677c2de1f06c0e91471ab prerequisite-patch-id: 3bc3d332ed043fbe64543bda7c7e734e19ba46aa prerequisite-patch-id: bb900133a10e40d3136e104b19c430442c4e2647 prerequisite-patch-id: 9ec5907dd0348b00f9341a63490bdafd99a403ca prerequisite-patch-id: dc50ec47974c35de431b80b83b501c4ca63758a3 prerequisite-patch-id: 371b31533a5abf8e1b8dc8568ffa455d16b611c6 prerequisite-patch-id: 0000000000000000000000000000000000000000 prerequisite-patch-id: 7743327071a8d8fb04cc43887fe61432f42eb60d prerequisite-patch-id: c74e8e54bd5391ef40e0a92f25db0822b421dd6a prerequisite-patch-id: 3ce8237727e2ce08261352c6b492a9bcf73651c4 prerequisite-patch-id: d6342ff93ec8850ce76e45f1e22d143208bfa13c prerequisite-patch-id: 6d2c59c2fdb9ae9e09fb042148f57b12d5058c9e prerequisite-patch-id: f86746e1c19fba223fe2e559fc0f3ecf6fc7cc47 prerequisite-patch-id: 2d43b690a831e369547d10cf08a8e785fc4c1b69 prerequisite-patch-id: ae154d0d43beba4483f29747aecceae853657561 prerequisite-patch-id: 1ac7f3a4007a4ff64813e1a117ee6f16c28695bc prerequisite-patch-id: ed34d0ebe0b56869508698367a26bd9e913394eb prerequisite-patch-id: 658bad2b9692a0f959ee73e2d3798a34f16c9f11 prerequisite-patch-id: 4d83a8451a41ee3d597f1e6be1457f695b738b76 prerequisite-patch-id: d3b421dc985d58dbaaef46ec6d16b4a2764424ea prerequisite-patch-id: ac7aab762dcd10fcc241be07503abae66f5912c8 prerequisite-patch-id: 34877d560c1c74de6e6875bdd719dafebb620732 prerequisite-patch-id: 9864c8f72ae9de7d6caf90096cf015ad0199ea7e prerequisite-patch-id: fa730102c85dc93ce0c9e7b4128d08dc09306192 prerequisite-patch-id: 8c1a8a32e9be9b282727985a542afe4766c4afd5 prerequisite-patch-id: ac25e540981c4015261293bd5502ab39f0b6d9e6 prerequisite-patch-id: 0000000000000000000000000000000000000000 prerequisite-patch-id: 245dbf34f0d60634846534ce846baa39ff91f6dc prerequisite-patch-id: 879c03c00f0023fcddfc8194692cd5706be4b893 prerequisite-patch-id: 6aa6a678f8c0d9ff3ce278d27342742ec352e95d prerequisite-patch-id: ccb225b386bb12bf442a8ac9096aabc4b2c6058c diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index c9559c4075b4b..5a598500b5c6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -994,7 +995,9 @@ static bool entry_set(const struct arm_smmu_entry_writer_ops *ops, return changed; } -#define NUM_ENTRY_QWORDS (sizeof_field(struct arm_smmu_ste, data) / sizeof(u64)) +#define NUM_ENTRY_QWORDS (max(sizeof_field(struct arm_smmu_ste, data), \ + sizeof_field(struct arm_smmu_cd, data)) \ + / sizeof(u64)) /* * Figure out if we can do a hitless update of entry to become target. Returns a @@ -1187,6 +1190,61 @@ static struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, return &l1_desc->l2ptr[idx]; } +static void arm_smmu_get_cd_used(const struct arm_smmu_entry_writer_ops *ops, + const __le64 *ent, __le64 *used_bits) +{ + memset(used_bits, 0, ops->num_entry_qwords * sizeof(*used_bits)); + + used_bits[0] = cpu_to_le64(CTXDESC_CD_0_V); + if (!(ent[0] & cpu_to_le64(CTXDESC_CD_0_V))) + return; + memset(used_bits, 0xFF, sizeof(*used_bits)); + + /* EPD0 means T0SZ/TG0/IR0/OR0/SH0/TTB0 are IGNORED */ + if (ent[0] & cpu_to_le64(CTXDESC_CD_0_TCR_EPD0)) { + used_bits[0] &= ~cpu_to_le64( + CTXDESC_CD_0_TCR_T0SZ | CTXDESC_CD_0_TCR_TG0 | + CTXDESC_CD_0_TCR_IRGN0 | CTXDESC_CD_0_TCR_ORGN0 | + CTXDESC_CD_0_TCR_SH0); + used_bits[1] &= ~cpu_to_le64(CTXDESC_CD_1_TTB0_MASK); + } +} + +struct arm_smmu_cd_writer { + struct arm_smmu_entry_writer_ops ops; + struct arm_smmu_master *master; + int ssid; +}; + +static void arm_smmu_cd_writer_sync_entry(const struct arm_smmu_entry_writer_ops *ops) +{ + struct arm_smmu_cd_writer *cd_writer = + container_of(ops, struct arm_smmu_cd_writer, ops); + + arm_smmu_sync_cd(cd_writer->master, cd_writer->ssid, true); +} + +static const struct arm_smmu_entry_writer_ops arm_smmu_cd_writer_ops = { + .sync = arm_smmu_cd_writer_sync_entry, + .get_used = arm_smmu_get_cd_used, + .v_bit = cpu_to_le64(CTXDESC_CD_0_V), + .num_entry_qwords = + sizeof_field(struct arm_smmu_cd, data) / sizeof(u64), +}; + +static void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, + struct arm_smmu_cd *cdptr, + const struct arm_smmu_cd *target) +{ + struct arm_smmu_cd_writer cd_writer = { + .ops = arm_smmu_cd_writer_ops, + .master = master, + .ssid = ssid, + }; + + arm_smmu_write_entry(&cd_writer.ops, cdptr->data, target->data); +} + int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, struct arm_smmu_ctx_desc *cd) { @@ -1203,16 +1261,19 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, */ u64 val; bool cd_live; - struct arm_smmu_cd *cdptr; + struct arm_smmu_cd target; + struct arm_smmu_cd *cdptr = ⌖ + struct arm_smmu_cd *cd_table_entry; struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; if (WARN_ON(ssid >= (1 << cd_table->s1cdmax))) return -E2BIG; - cdptr = arm_smmu_get_cd_ptr(master, ssid); - if (!cdptr) + cd_table_entry = arm_smmu_get_cd_ptr(master, ssid); + if (!cd_table_entry) return -ENOMEM; + target = *cd_table_entry; val = le64_to_cpu(cdptr->data[0]); cd_live = !!(val & CTXDESC_CD_0_V); @@ -1232,13 +1293,6 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, cdptr->data[2] = 0; cdptr->data[3] = cpu_to_le64(cd->mair); - /* - * STE may be live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=1. - */ - arm_smmu_sync_cd(master, ssid, true); - val = cd->tcr | #ifdef __BIG_ENDIAN CTXDESC_CD_0_ENDI | @@ -1252,18 +1306,8 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_master *master, int ssid, if (cd_table->stall_enabled) val |= CTXDESC_CD_0_S; } - - /* - * The SMMU accesses 64-bit values atomically. See IHI0070Ca 3.21.3 - * "Configuration structures and configuration invalidation completion" - * - * The size of single-copy atomic reads made by the SMMU is - * IMPLEMENTATION DEFINED but must be at least 64 bits. Any single - * field within an aligned 64-bit span of a structure can be altered - * without first making the structure invalid. - */ - WRITE_ONCE(cdptr->data[0], cpu_to_le64(val)); - arm_smmu_sync_cd(master, ssid, true); + cdptr->data[0] = cpu_to_le64(val); + arm_smmu_write_cd_entry(master, ssid, cd_table_entry, &target); return 0; }