From patchwork Sun Jan 29 14:00:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ritesh Raj Sarraf X-Patchwork-Id: 9544197 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E931D60415 for ; Sun, 29 Jan 2017 17:09:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB9CF26E54 for ; Sun, 29 Jan 2017 17:09:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFA2B28068; Sun, 29 Jan 2017 17:09:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, T_TVD_MIME_EPI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F284026E54 for ; Sun, 29 Jan 2017 17:09:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751876AbdA2RJm (ORCPT ); Sun, 29 Jan 2017 12:09:42 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:36187 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751833AbdA2RJh (ORCPT ); Sun, 29 Jan 2017 12:09:37 -0500 Received: by mail-pf0-f193.google.com with SMTP id 19so21750429pfo.3 for ; Sun, 29 Jan 2017 09:08:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=researchut.com; s=google; h=message-id:subject:from:reply-to:to:cc:date:in-reply-to:references :organization:mime-version; bh=YNxSTi6pCZCQzHGkCmr+36XFX3OeD2Ebq9A55s2CML0=; b=oqeQasBrf/sWzIsSgnasYl8gp/bKe7n3PYn4qC8ToR5uDpkb7dfJ+NaqnJ45rGf2aA xfvdfYtMc+zQEaqeZC0Rvip1DmTnEJAHYntqAYl49IxDbIZQdSfLC2+7+NdgsF4Epuol Ng//tNaM643iEsMT8gxHENF9QKXSJD05K4Muk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:reply-to:to:cc:date :in-reply-to:references:organization:mime-version; bh=YNxSTi6pCZCQzHGkCmr+36XFX3OeD2Ebq9A55s2CML0=; b=W44SouqVtrAHNboI20Hkv/qTsWCV7XnJlkiivtM58PbpxpqhfaksSF+eMb2yEIh0UB HupgeDqwWHBBy8TxdF+Sm7mIb6sjwykQ68CTDmuikqeTknig4PcMAGVFgnXB70xeaF22 Zggw8Mfd2a8Q9aoolZRMJYNzu55BsSL5spwQadU8gRgAqBE2HwnEaEjhQ/8NU9Bx7kCV mKxxrOcYv6whi0VYRauatzDCBa2KUyaf0TKuLTT2YNIP9AR8luqHhhMkSW9ONUpMujmK ebXw/E9n7v90urf7WRMrt6CclgVx8vZi53peoQdiqU2j+qERBmJIGe015sTJbmDSNihE CjkA== X-Gm-Message-State: AIkVDXKMrGswnAaCpXrZPig7GnC6YcVoMMgPHCwU2GYi/K68V1WFeKp2hMw6pgoQVsglkA== X-Received: by 10.99.232.5 with SMTP id s5mr19238193pgh.66.1485698428649; Sun, 29 Jan 2017 06:00:28 -0800 (PST) Received: from learner ([115.99.97.44]) by smtp.gmail.com with ESMTPSA id o1sm12118376pgf.25.2017.01.29.06.00.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 29 Jan 2017 06:00:27 -0800 (PST) Received: from localhost.localdomain ([127.0.0.1]) by learner with esmtp (Exim 4.88) (envelope-from ) id 1cXq1n-0003Ig-VD; Sun, 29 Jan 2017 19:30:24 +0530 Message-ID: <1485698423.24562.7.camel@researchut.com> Subject: squashfs-tools: Add -offset option to skip n bytes at the start of the file From: Ritesh Raj Sarraf Reply-To: rrs@researchut.com To: Phillip Lougher , linux-fsdevel@vger.kernel.org, "linux-kernel@vger.kernel.org" , probono@puredarwin.org Cc: Mention , probonopd/AppImageKit , probonopd/AppImageKit Date: Sun, 29 Jan 2017 19:30:23 +0530 In-Reply-To: References: Organization: RESEARCHUT X-Mailer: Evolution 3.22.4-1 Mime-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Sun, 2017-01-29 at 03:33 -0800, probonopd wrote: > Yes, I am not happy about that. > You could either push your change to the squashfs folks > Sent a PR half a year ago but no reaction: plougher/squashfs-tools#13 - is > something wrong with it? That repo seems inactive. So does the git repo at kernel.org. but I would rely on that one. Let's try fresh now. Given there's been not much development in the upstream repo, I've just taken your patch, as is, for submission. > Or you could just carry the delta and ask the user to fetch the squashfs-tools > source themselves > Do you mean "carry the delta" as in "provide a diff"? Doing now. Dear Squashfs Team, Can you please review/include the attached patch into squashfs-tools ? The mentioned changes are needed by AppImageKit. What is AppImage The AppImage format is a format for packaging applications in a way that allows them to run on a variety of different target systems (base operating systems, distributions) without further modification. https://en.wikipedia.org/wiki/AppImage AppImageKit is a concrete implementation of the AppImage format and provides tools such as appimagetool and appimaged for conveniently handling AppImages. https://github.com/probonopd/AppImageKit From f72f32b0fe7929e71edd9e6589fa1fcec1d86adc Mon Sep 17 00:00:00 2001 From: probonopd Date: Thu, 15 Sep 2016 18:24:10 +0200 Subject: [PATCH 1/2] Add -offset option to skip n bytes at the start of the input file --- squashfs-tools/unsquashfs.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/squashfs-tools/unsquashfs.c b/squashfs-tools/unsquashfs.c index 1323dd6..b6ba452 100644 --- a/squashfs-tools/unsquashfs.c +++ b/squashfs-tools/unsquashfs.c @@ -42,6 +42,7 @@ struct cache *fragment_cache, *data_cache; struct queue *to_reader, *to_inflate, *to_writer, *from_writer; pthread_t *thread, *inflator_thread; pthread_mutex_t fragment_mutex; +static off_t squashfs_start_offset = 0; /* user options that control parallelisation */ int processors = -1; @@ -631,7 +632,7 @@ int read_fs_bytes(int fd, long long byte, int bytes, void *buff) TRACE("read_bytes: reading from position 0x%llx, bytes %d\n", byte, bytes); - if(lseek(fd, off, SEEK_SET) == -1) { + if(lseek(fd, off + squashfs_start_offset, SEEK_SET) == -1) { ERROR("Lseek failed because %s\n", strerror(errno)); return FALSE; } @@ -2544,6 +2545,14 @@ int main(int argc, char *argv[]) exit(1); } dest = argv[i]; + } else if (strcmp(argv[i], "-offset") == 0 || + strcmp(argv[i], "-o") == 0) { + if(++i == argc) { + fprintf(stderr, "%s: -offset missing argument\n", + argv[0]); + exit(1); + } + squashfs_start_offset = (off_t)atol(argv[i]); } else if(strcmp(argv[i], "-processors") == 0 || strcmp(argv[i], "-p") == 0) { if((++i == argc) || @@ -2636,6 +2645,8 @@ int main(int argc, char *argv[]) "copyright information\n"); ERROR("\t-d[est] \tunsquash to , " "default \"squashfs-root\"\n"); + ERROR("\t-o[ffset] \tskip at start of input file, " + "default \"0\"\n"); ERROR("\t-n[o-progress]\t\tdon't display the progress " "bar\n"); ERROR("\t-no[-xattrs]\t\tdon't extract xattrs in file system" From 5a498ad24dcfeac9f3d747e894f22901f3ac10ef Mon Sep 17 00:00:00 2001 From: probonopd Date: Thu, 15 Sep 2016 21:09:52 +0200 Subject: [PATCH 2/2] Add -offset function to skip n bytes at the beginning of the squashfs file --- squashfs-tools/mksquashfs.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/squashfs-tools/mksquashfs.c b/squashfs-tools/mksquashfs.c index d221c35..92b6a31 100644 --- a/squashfs-tools/mksquashfs.c +++ b/squashfs-tools/mksquashfs.c @@ -98,6 +98,7 @@ int old_exclude = TRUE; int use_regex = FALSE; int nopad = FALSE; int exit_on_error = FALSE; +static off_t squashfs_start_offset = 0; long long global_uid = -1, global_gid = -1; @@ -516,9 +517,9 @@ int read_fs_bytes(int fd, long long byte, int bytes, void *buff) pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex); pthread_mutex_lock(&pos_mutex); - if(lseek(fd, off, SEEK_SET) == -1) { + if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) { ERROR("read_fs_bytes: Lseek on destination failed because %s, " - "offset=0x%llx\n", strerror(errno), off); + "offset=0x%llx\n", strerror(errno), off+squashfs_start_offset); res = 0; } else if(read_bytes(fd, buff, bytes) < bytes) { ERROR("Read on destination failed\n"); @@ -557,10 +558,10 @@ void write_destination(int fd, long long byte, int bytes, void *buff) pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex); pthread_mutex_lock(&pos_mutex); - if(lseek(fd, off, SEEK_SET) == -1) { + if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) { ERROR("write_destination: Lseek on destination " "failed because %s, offset=0x%llx\n", strerror(errno), - off); + off+squashfs_start_offset); BAD_ERROR("Probably out of space on output %s\n", block_device ? "block device" : "filesystem"); } @@ -2315,9 +2316,9 @@ void *writer(void *arg) pthread_cleanup_push((void *) pthread_mutex_unlock, &pos_mutex); pthread_mutex_lock(&pos_mutex); - if(lseek(fd, off, SEEK_SET) == -1) { + if(lseek(fd, off+squashfs_start_offset, SEEK_SET) == -1) { ERROR("writer: Lseek on destination failed because " - "%s, offset=0x%llx\n", strerror(errno), off); + "%s, offset=0x%llx\n", strerror(errno), off+squashfs_start_offset); BAD_ERROR("Probably out of space on output " "%s\n", block_device ? "block device" : "filesystem"); @@ -5341,6 +5342,15 @@ int main(int argc, char *argv[]) force_progress = TRUE; else if(strcmp(argv[i], "-no-exports") == 0) exportable = FALSE; + else if(strcmp(argv[i], "-offset") == 0 || + strcmp(argv[i], "-o") ==0) { + if(++i == argc) { + ERROR("%s: %s offset missing argument\n", argv[0], + argv[i - 1]); + exit(1); + } + squashfs_start_offset = (off_t)atol(argv[i]); + } else if(strcmp(argv[i], "-processors") == 0) { if((++i == argc) || !parse_num(argv[i], &processors)) { ERROR("%s: -processors missing or invalid " @@ -5641,6 +5651,9 @@ int main(int argc, char *argv[]) ERROR("\nMiscellaneous options:\n"); ERROR("-root-owned\t\talternative name for -all-root" "\n"); + ERROR("-o \t\tSkip bytes at the " + "beginning of the file.\n\t\t\t" + "Default 0 bytes\n"); ERROR("-noInodeCompression\talternative name for -noI" "\n"); ERROR("-noDataCompression\talternative name for -noD"