From patchwork Mon Jul 31 07:48:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AAE30C001DE for ; Mon, 31 Jul 2023 07:40:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YGbM5VetIkSYuj2do+qnOFm9jdJrtwy1WGVlmKeeO6w=; b=2cfsIS/QsYNzM0 wg+M+bLeYBv2tD+WqTMlQuMAasxWSnAKwA+7ybEJBMyydSaJdVMC169nQjC2fHMNVdHiqwP5iyNmF i0e/VJx5yWUYwkzU5GpQCDLIHapVyuav6mfOk3bwz6D/eDPmy0oT4QpDnbhS14Lne7mr4ayikj6sQ qoIM2LwWmD0Cb+zoAcDrfTS2SdD68oLYI0MZleJB/2yegiwZLS9c925U3Hgk8zkIea3zbTgL6Yvt1 K7cgrnmgKR1WVh+Kx0t/0kIIDcFmNJPrwZS/IUqu5GElFgvLY/Xg+IEGAWMnggdod8pv/LW+1M3s6 /7Bp6JaFmZEUcLywsHDw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qQNVG-00ELVh-1m; Mon, 31 Jul 2023 07:39:42 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qQNV3-00ELMl-22 for linux-arm-kernel@lists.infradead.org; Mon, 31 Jul 2023 07:39:31 +0000 Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm627jmzNmP9; Mon, 31 Jul 2023 15:35:50 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:14 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 1/4] mm: hugetlb: use flush_hugetlb_tlb_range() in move_hugetlb_page_tables() Date: Mon, 31 Jul 2023 15:48:26 +0800 Message-ID: <20230731074829.79309-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230731_003930_317311_B009E140 X-CRM114-Status: GOOD ( 10.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Archs may need to do special things when flushing hugepage tlb, so use the more applicable flush_hugetlb_tlb_range() instead of flush_tlb_range(). Fixes: 550a7d60bd5e ("mm, hugepages: add mremap() support for hugepage backed vma") Signed-off-by: Kefeng Wang Reviewed-by: Mike Kravetz Acked-by: Muchun Song --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 64a3239b6407..ac876bfba340 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5281,9 +5281,9 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, } if (shared_pmd) - flush_tlb_range(vma, range.start, range.end); + flush_hugetlb_tlb_range(vma, range.start, range.end); else - flush_tlb_range(vma, old_end - len, old_end); + flush_hugetlb_tlb_range(vma, old_end - len, old_end); mmu_notifier_invalidate_range_end(&range); i_mmap_unlock_write(mapping); hugetlb_vma_unlock_write(vma); From patchwork Mon Jul 31 07:48:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96A4EC001DE for ; Mon, 31 Jul 2023 07:39:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p5K6w0nADC2XSZGiIrbEKeOhYYGhByMfmXfjcrIjXt4=; b=WtM0xG1s1cLAfB ZugmDaWDJQGm3C8SW0oUG7OgoQylTEJeA56f92zIpb2XG9a3NgVh5mSfGlBv09NtWg718KIuAs6uL 5+PRvCtxEPlj1AS8BPt63/Vzeo9FvGJo43z679Bz+RiLSUSaadUNvVT8btjTMizGofq4FmeYNdLtS IpfpxqIG6WMd9bSwx1hAgKkHkZuAexkTXRKFvSKJAdKsvPfReNKlkSOX/CVJBZt+guDskFYbxwd+E q7AEqbzHw465dEfCdaCtKSPd5kmVfQSwP3JROm1/tw0E2/T9TXbqTnPkkGncWTXyleDdy5A4emlO3 J+/ELmcsSGilSLaG9IkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qQNV5-00ELRI-0y; Mon, 31 Jul 2023 07:39:31 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qQNV1-00ELMd-28 for linux-arm-kernel@lists.infradead.org; Mon, 31 Jul 2023 07:39:29 +0000 Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RDqmD2YBwztQSD; Mon, 31 Jul 2023 15:35:56 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd() Date: Mon, 31 Jul 2023 15:48:27 +0800 Message-ID: <20230731074829.79309-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230731_003927_916539_0602B236 X-CRM114-Status: GOOD ( 11.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 11e06e4ab33b..1883205fa22b 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pmd_none(*new_pmd)); pmd_populate(mm, new_pmd, pmd_pgtable(pmd)); - flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE); + flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0090BC04A94 for ; Mon, 31 Jul 2023 07:40:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nXlCjM6JOmnZDYL/CyLzcmn8aRWmohBJ0d9U+fudvs4=; b=AuEmPXQO0/Mu8R Zqqrm9DMxojy0l03n5lYaHekC3GO+OD4/xMR1MIiM5RS+WhEp2FR0RkBweIxc7CaaZVmR+FhQ1Tku wnYuImXsAn/0v2hkV147LpelneUUq+QHwbhFwSz2V73LgzfRL2dkkhTlHo5UQ11xRvvwb2jUZ3XRH EaT/w85X8j0Vn7R7QDpHouapDFX3NVeTgxUdnTaMbU42uwxdUkq15xDbavRbpOxNk/GGy0Wxu3hAj 1DI8z2hGrzMCxQVMvAr9OuO4XMKC7gjPQrO+f40qg1M4fmc1D+tw15moYTmGcfuJGlcL2LAC9QLDC SV9z0kReNi65AiPDh8hA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qQNVG-00ELVP-09; Mon, 31 Jul 2023 07:39:42 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qQNV4-00ELMk-0P for linux-arm-kernel@lists.infradead.org; Mon, 31 Jul 2023 07:39:31 +0000 Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm74d7YzNmbC; Mon, 31 Jul 2023 15:35:51 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 3/4] mm: mremap: use flush_pud_tlb_range in move_normal_pud() Date: Mon, 31 Jul 2023 15:48:28 +0800 Message-ID: <20230731074829.79309-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230731_003930_368936_510D8192 X-CRM114-Status: GOOD ( 10.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: c49dd3401802 ("mm: speedup mremap on 1GB or larger regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 1883205fa22b..25114e56901f 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -333,7 +333,7 @@ static bool move_normal_pud(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pud_none(*new_pud)); pud_populate(mm, new_pud, pud_pgtable(pud)); - flush_tlb_range(vma, old_addr, old_addr + PUD_SIZE); + flush_pud_tlb_range(vma, old_addr, old_addr + PUD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9709BC001DF for ; Mon, 31 Jul 2023 07:40:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ja/r3KQhnBCYGHjyTF0ruSCXYOGlHpjGXfJbJzlv3hk=; b=B0afo7ebZduOau ewCztCWp/YLphyF+j4LQffGVeYtHFaGMO5jRqHQ3ScK+lkxdQI9NgiyXFEyPaUo477hdQateFQgi2 qXIkaUS7WAwrBQh/L0InxEOFkQKAedO/jW3Jtwl2bjhOwpdXvMwDxyptkNt95pStZw2oYMgv/zxrM SvwE3mj814PYfef02VgCM7sRQGg710Y1wcMfb39lLX9umsBASTvdziSU0avVCB1IJP/AReHumiSE8 8leUQ4JEWE9BylxbH/uiuMQAPn+1/VlEtltw26+mZRprgRlx2Msq9vMCRYr5zUN2o+/AL7zD99lgk LYn8N7zO5q6586M7ixCw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qQNVF-00ELUp-1s; Mon, 31 Jul 2023 07:39:41 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qQNV3-00ELMi-1q for linux-arm-kernel@lists.infradead.org; Mon, 31 Jul 2023 07:39:31 +0000 Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqp763FHzVjdw; Mon, 31 Jul 2023 15:37:35 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:16 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 4/4] arm64: tlb: set huge page size to stride for hugepage Date: Mon, 31 Jul 2023 15:48:29 +0800 Message-ID: <20230731074829.79309-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230731_003929_787158_A5E5D6F5 X-CRM114-Status: GOOD ( 12.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org It is better to use huge_page_size() for hugepage(HugeTLB) instead of PAGE_SIZE for stride, which has been done in flush_pmd/pud_tlb_range(), it could reduce the loop in __flush_tlb_range(). Signed-off-by: Kefeng Wang --- arch/arm64/include/asm/tlbflush.h | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 412a3b9a3c25..25e35e6f8093 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -360,16 +360,17 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ish); } -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - /* - * We cannot use leaf-only invalidation here, since we may be invalidating - * table entries as part of collapsing hugepages or moving page tables. - * Set the tlb_level to 0 because we can not get enough information here. - */ - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); -} +/* + * We cannot use leaf-only invalidation here, since we may be invalidating + * table entries as part of collapsing hugepages or moving page tables. + * Set the tlb_level to 0 because we can not get enough information here. + */ +#define flush_tlb_range(vma, start, end) \ + __flush_tlb_range(vma, start, end, \ + ((vma)->vm_flags & VM_HUGETLB) \ + ? huge_page_size(hstate_vma(vma)) \ + : PAGE_SIZE, false, 0) + static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) {