From patchwork Mon Sep 25 16:58:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Venkata Pyla X-Patchwork-Id: 13398144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E949CE7AA9 for ; Mon, 25 Sep 2023 16:58:44 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.132]) by mx.groups.io with SMTP id smtpd.web11.66607.1695661118645604481 for ; Mon, 25 Sep 2023 09:58:39 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.132, mailfrom: venkata.pyla@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 38PGwaOn4000755; Tue, 26 Sep 2023 01:58:36 +0900 X-Iguazu-Qid: 2rWgSWj5NamL9vn03U X-Iguazu-QSIG: v=2; s=0; t=1695661115; q=2rWgSWj5NamL9vn03U; m=/6Mjk93QZOTqBd12s/oOPEX6axzsU47u9h1lQqTs9qs= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1123) id 38PGwYTv3965549 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Sep 2023 01:58:35 +0900 From: venkata.pyla@toshiba-tsip.com To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: venkata pyla , dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core v2 1/2] dosfstools: Add recipe to build it locally Date: Mon, 25 Sep 2023 22:28:32 +0530 X-TSB-HOP2: ON Message-Id: <20230925165833.2696-2-venkata.pyla@toshiba-tsip.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <51a07da1-b0a1-4a3b-b463-739e3ea208f2@siemens.com> References: <51a07da1-b0a1-4a3b-b463-739e3ea208f2@siemens.com> MIME-Version: 1.0 X-OriginalArrivalTime: 25 Sep 2023 16:58:32.0529 (UTC) FILETIME=[83F62410:01D9EFD1] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 16:58:44 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/13220 From: venkata pyla mkdosfs doesn't generate fat file-systems reproducibly, due to the reason that the utility mkfs.fat is not honoring the variable SOURCE_DATE_EPOCH and including build time stamps in the file system headers, this issue has been fixed in upstream master branch but not release yet, so building it locally with the patch. This local build can be dropped when this patch[1] is available in Debian releases, issue[2] is already available to make release with this patch. [1] https://github.com/dosfstools/dosfstools/commit/8da7bc93315cb0c32ad868f17808468b81fa76ec [2] https://github.com/dosfstools/dosfstools/issues/179 Signed-off-by: venkata pyla --- kas/opt/reproducible.yml | 1 + .../dosfstools/dosfstools_latest.bb | 28 +++ ...Honor-the-SOURCE_DATE_EPOCH-variable.patch | 160 ++++++++++++++++++ 3 files changed, 189 insertions(+) create mode 100644 recipes-devtools/dosfstools/dosfstools_latest.bb create mode 100644 recipes-devtools/dosfstools/files/0001-Honor-the-SOURCE_DATE_EPOCH-variable.patch diff --git a/kas/opt/reproducible.yml b/kas/opt/reproducible.yml index 9b56b28..fada232 100644 --- a/kas/opt/reproducible.yml +++ b/kas/opt/reproducible.yml @@ -15,3 +15,4 @@ local_conf_header: reproducible-builds: | SOURCE_DATE_EPOCH := "${@bb.process.run("git -C ${LAYERDIR_cip-core} log -1 --pretty=%ct | tr -d '\n'")[0]}" WIC_DEPLOY_PARTITIONS = "1" + IMAGER_BUILD_DEPS += "dosfstools" diff --git a/recipes-devtools/dosfstools/dosfstools_latest.bb b/recipes-devtools/dosfstools/dosfstools_latest.bb new file mode 100644 index 0000000..ebdf983 --- /dev/null +++ b/recipes-devtools/dosfstools/dosfstools_latest.bb @@ -0,0 +1,28 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Toshiba Corporation 2023 +# +# Authors: +# Venkata Pyla +# +# SPDX-License-Identifier: MIT +# + +inherit dpkg + +CHANGELOG_V="+cip" + +SRC_URI = "apt://${PN}" +SRC_URI += "file://0001-Honor-the-SOURCE_DATE_EPOCH-variable.patch;apply=no" + +# this is a host tool +PACKAGE_ARCH = "${HOST_ARCH}" + +do_prepare_build() { + deb_add_changelog + + cd ${S} + quilt import ${WORKDIR}/*.patch + quilt push -a +} diff --git a/recipes-devtools/dosfstools/files/0001-Honor-the-SOURCE_DATE_EPOCH-variable.patch b/recipes-devtools/dosfstools/files/0001-Honor-the-SOURCE_DATE_EPOCH-variable.patch new file mode 100644 index 0000000..30c2132 --- /dev/null +++ b/recipes-devtools/dosfstools/files/0001-Honor-the-SOURCE_DATE_EPOCH-variable.patch @@ -0,0 +1,160 @@ +From 8da7bc93315cb0c32ad868f17808468b81fa76ec Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Bj=C3=B8rn=20Forsman?= +Date: Wed, 5 Dec 2018 19:52:51 +0100 +Subject: [PATCH] Honor the SOURCE_DATE_EPOCH variable +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Implement the SOURCE_DATE_EPOCH specification[1] for reproducible +builds. If SOURCE_DATE_EPOCH is set, use it as timestamp instead of the +current time. + +[1] https://reproducible-builds.org/specs/source-date-epoch/ + +Signed-off-by: Bjørn Forsman +--- + src/boot.c | 23 +++++++++++++++++++++-- + src/common.c | 18 ++++++++++++++++-- + src/mkfs.fat.c | 19 ++++++++++++++++--- + 3 files changed, 53 insertions(+), 7 deletions(-) + +diff --git a/src/boot.c b/src/boot.c +index 4de450d..8f78e1c 100644 +--- a/src/boot.c ++++ b/src/boot.c +@@ -33,6 +33,8 @@ + #include + #include + #include ++#include ++#include + + #include "common.h" + #include "fsck.fat.h" +@@ -672,6 +674,7 @@ void write_volume_label(DOS_FS * fs, char *label) + { + time_t now; + struct tm *mtime; ++ char *source_date_epoch = NULL; + off_t offset; + int created; + DIR_ENT de; +@@ -687,8 +690,24 @@ void write_volume_label(DOS_FS * fs, char *label) + if (de.name[0] == 0xe5) + de.name[0] = 0x05; + +- now = time(NULL); +- mtime = (now != (time_t)-1) ? localtime(&now) : NULL; ++ source_date_epoch = getenv("SOURCE_DATE_EPOCH"); ++ if (source_date_epoch) { ++ char *tmp = NULL; ++ long long conversion = 0; ++ errno = 0; ++ conversion = strtoll(source_date_epoch, &tmp, 10); ++ now = conversion; ++ if (!isdigit((unsigned char)*source_date_epoch) || *tmp != '\0' ++ || errno != 0 || (long long)now != conversion) { ++ die("SOURCE_DATE_EPOCH is too big or contains non-digits: \"%s\"", ++ source_date_epoch); ++ } ++ mtime = gmtime(&now); ++ } else { ++ now = time(NULL); ++ mtime = (now != (time_t)-1) ? localtime(&now) : NULL; ++ } ++ + if (mtime && mtime->tm_year >= 80 && mtime->tm_year <= 207) { + de.time = htole16((unsigned short)((mtime->tm_sec >> 1) + + (mtime->tm_min << 5) + +diff --git a/src/common.c b/src/common.c +index 6a2e396..4f1afcb 100644 +--- a/src/common.c ++++ b/src/common.c +@@ -30,6 +30,7 @@ + #include + #include + #include ++#include + #include + #include + #include +@@ -298,8 +299,21 @@ void check_atari(void) + uint32_t generate_volume_id(void) + { + struct timeval now; +- +- if (gettimeofday(&now, NULL) != 0 || now.tv_sec == (time_t)-1 || now.tv_sec < 0) { ++ char *source_date_epoch = NULL; ++ ++ source_date_epoch = getenv("SOURCE_DATE_EPOCH"); ++ if (source_date_epoch) { ++ char *tmp = NULL; ++ long long conversion = 0; ++ errno = 0; ++ conversion = strtoll(source_date_epoch, &tmp, 10); ++ if (!isdigit((unsigned char)*source_date_epoch) || *tmp != '\0' ++ || errno != 0) { ++ die("SOURCE_DATE_EPOCH is too big or contains non-digits: \"%s\"", ++ source_date_epoch); ++ } ++ return (uint32_t)conversion; ++ } else if (gettimeofday(&now, NULL) != 0 || now.tv_sec == (time_t)-1 || now.tv_sec < 0) { + srand(getpid()); + /* rand() returns int from [0,RAND_MAX], therefore only 31 bits */ + return (((uint32_t)(rand() & 0xFFFF)) << 16) | ((uint32_t)(rand() & 0xFFFF)); +diff --git a/src/mkfs.fat.c b/src/mkfs.fat.c +index 37fc8ff..1948635 100644 +--- a/src/mkfs.fat.c ++++ b/src/mkfs.fat.c +@@ -1074,7 +1074,7 @@ static void setup_tables(void) + } + + /* If is not available then generate random 32 bit disk signature */ +- if (invariant) ++ if (invariant || getenv("SOURCE_DATE_EPOCH")) + disk_sig = volume_id; + else if (!disk_sig) + disk_sig = generate_volume_id(); +@@ -1287,7 +1287,7 @@ static void setup_tables(void) + de->name[0] = 0x05; + de->attr = ATTR_VOLUME; + if (create_time != (time_t)-1) { +- if (!invariant) ++ if (!invariant && !getenv("SOURCE_DATE_EPOCH")) + ctime = localtime(&create_time); + else + ctime = gmtime(&create_time); +@@ -1477,6 +1477,7 @@ int main(int argc, char **argv) + int blocks_specified = 0; + struct timeval create_timeval; + long long conversion; ++ char *source_date_epoch = NULL; + + enum {OPT_HELP=1000, OPT_INVARIANT, OPT_MBR, OPT_VARIANT, OPT_CODEPAGE, OPT_OFFSET}; + const struct option long_options[] = { +@@ -1497,8 +1498,20 @@ int main(int argc, char **argv) + program_name = p + 1; + } + +- if (gettimeofday(&create_timeval, NULL) == 0 && create_timeval.tv_sec != (time_t)-1) ++ source_date_epoch = getenv("SOURCE_DATE_EPOCH"); ++ if (source_date_epoch) { ++ errno = 0; ++ conversion = strtoll(source_date_epoch, &tmp, 10); ++ create_time = conversion; ++ if (!isdigit((unsigned char)*source_date_epoch) || *tmp != '\0' ++ || errno != 0 || (long long)create_time != conversion) { ++ die("SOURCE_DATE_EPOCH is too big or contains non-digits: \"%s\"", ++ source_date_epoch); ++ } ++ } else if (gettimeofday(&create_timeval, NULL) == 0 && create_timeval.tv_sec != (time_t)-1) { + create_time = create_timeval.tv_sec; ++ } ++ + volume_id = generate_volume_id(); + check_atari(); + +-- +2.39.2 + From patchwork Mon Sep 25 16:58:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venkata Pyla X-Patchwork-Id: 13398145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D342CE7AA8 for ; Mon, 25 Sep 2023 16:58:44 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.158]) by mx.groups.io with SMTP id smtpd.web10.66332.1695661119691122037 for ; Mon, 25 Sep 2023 09:58:39 -0700 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.158, mailfrom: venkata.pyla@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 38PGwbxi4033810; Tue, 26 Sep 2023 01:58:37 +0900 X-Iguazu-Qid: 2rWgDRdQ1jgX6Hc16g X-Iguazu-QSIG: v=2; s=0; t=1695661116; q=2rWgDRdQ1jgX6Hc16g; m=tffQq5bZEEmNVRsZy4kkWEfx6tLb1YnSWM/mw3Dyk4s= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1121) id 38PGwZs73964790 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 26 Sep 2023 01:58:36 +0900 From: venkata.pyla@toshiba-tsip.com To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: venkata pyla , dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core v2 2/2] images: Add dosfstools as DEPENDENCY to all images Date: Mon, 25 Sep 2023 22:28:33 +0530 X-TSB-HOP2: ON Message-Id: <20230925165833.2696-3-venkata.pyla@toshiba-tsip.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <51a07da1-b0a1-4a3b-b463-739e3ea208f2@siemens.com> References: <51a07da1-b0a1-4a3b-b463-739e3ea208f2@siemens.com> MIME-Version: 1.0 X-OriginalArrivalTime: 25 Sep 2023 16:58:32.0607 (UTC) FILETIME=[84020AF0:01D9EFD1] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 16:58:44 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/13221 From: venkata pyla Signed-off-by: venkata pyla --- recipes-core/images/cip-core-image-kernelci.bb | 1 + recipes-core/images/cip-core-image-security.bb | 1 + recipes-core/images/cip-core-image.bb | 1 + recipes-core/images/common-image.inc | 13 +++++++++++++ 4 files changed, 16 insertions(+) create mode 100644 recipes-core/images/common-image.inc diff --git a/recipes-core/images/cip-core-image-kernelci.bb b/recipes-core/images/cip-core-image-kernelci.bb index 479c14c..8a58e7b 100644 --- a/recipes-core/images/cip-core-image-kernelci.bb +++ b/recipes-core/images/cip-core-image-kernelci.bb @@ -10,6 +10,7 @@ # inherit image +require common-image.inc DESCRIPTION = "CIP Core image for KernelCI" diff --git a/recipes-core/images/cip-core-image-security.bb b/recipes-core/images/cip-core-image-security.bb index 3421ce5..f114974 100644 --- a/recipes-core/images/cip-core-image-security.bb +++ b/recipes-core/images/cip-core-image-security.bb @@ -10,6 +10,7 @@ # inherit image +require common-image.inc DESCRIPTION = "CIP Core image including security packages" diff --git a/recipes-core/images/cip-core-image.bb b/recipes-core/images/cip-core-image.bb index 0ec7220..711d3d3 100644 --- a/recipes-core/images/cip-core-image.bb +++ b/recipes-core/images/cip-core-image.bb @@ -10,6 +10,7 @@ # inherit image +require common-image.inc ISAR_RELEASE_CMD = "git -C ${LAYERDIR_cip-core} describe --tags --dirty --always --match 'v[0-9].[0-9]*'" DESCRIPTION = "CIP Core image" diff --git a/recipes-core/images/common-image.inc b/recipes-core/images/common-image.inc new file mode 100644 index 0000000..863f817 --- /dev/null +++ b/recipes-core/images/common-image.inc @@ -0,0 +1,13 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Toshiba Corporation 2023 +# +# Authors: +# Venkata Pyla +# +# SPDX-License-Identifier: MIT +# + +# Install in build system for creating reproducible DOS filesystems +DEPENDS += "dosfstools"