[v2,1/1] zlib.c: use size_t for size
diff mbox series

Message ID 20181012204229.11890-1-tboegi@web.de
State New
Headers show
Series
  • [v2,1/1] zlib.c: use size_t for size
Related show

Commit Message

Torsten Bögershausen Oct. 12, 2018, 8:42 p.m. UTC
From: Martin Koegler <martin.koegler@chello.at>

Signed-off-by: Martin Koegler <martin.koegler@chello.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---

After doing a review, I decided to send the result as a patch.
In general, the changes from off_t to size_t seem to be not really
motivated.
But if they are, they could and should go into an own patch.
For the moment, change only "unsigned long" into size_t, thats all

 builtin/pack-objects.c |  8 ++++----
 cache.h                | 10 +++++-----
 pack-check.c           |  4 ++--
 packfile.h             |  2 +-
 wrapper.c              |  8 ++++----
 zlib.c                 |  8 ++++----
 6 files changed, 20 insertions(+), 20 deletions(-)

Comments

SZEDER Gábor Oct. 12, 2018, 10:22 p.m. UTC | #1
On Fri, Oct 12, 2018 at 10:42:29PM +0200, tboegi@web.de wrote:
> From: Martin Koegler <martin.koegler@chello.at>
> 
> Signed-off-by: Martin Koegler <martin.koegler@chello.at>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> ---
> 
> After doing a review, I decided to send the result as a patch.
> In general, the changes from off_t to size_t seem to be not really
> motivated.
> But if they are, they could and should go into an own patch.
> For the moment, change only "unsigned long" into size_t, thats all

Neither v1 nor v2 of this patch compiles on 32 bit Linux; see

  https://travis-ci.org/git/git/jobs/440514375#L628

The fixup patch below makes it compile on 32 bit and the test suite
passes as well, but I didn't thoroughly review the changes; I only
wanted to get 'pu' build again.


  --  >8 --

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 23c4cd8c77..89fe1c5d46 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1966,7 +1966,8 @@ unsigned long oe_get_size_slow(struct packing_data *pack,
 	struct pack_window *w_curs;
 	unsigned char *buf;
 	enum object_type type;
-	unsigned long used, avail, size;
+	unsigned long used, size;
+	size_t avail;
 
 	if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
 		read_lock();
diff --git a/packfile.c b/packfile.c
index 841b36182f..9f50411ad3 100644
--- a/packfile.c
+++ b/packfile.c
@@ -579,7 +579,7 @@ static int in_window(struct pack_window *win, off_t offset)
 unsigned char *use_pack(struct packed_git *p,
 		struct pack_window **w_cursor,
 		off_t offset,
-		unsigned long *left)
+		size_t *left)
 {
 	struct pack_window *win = *w_cursor;
 
@@ -1098,7 +1098,7 @@ int unpack_object_header(struct packed_git *p,
 			 unsigned long *sizep)
 {
 	unsigned char *base;
-	unsigned long left;
+	size_t left;
 	unsigned long used;
 	enum object_type type;
Torsten Bögershausen Oct. 13, 2018, 5 a.m. UTC | #2
[]
> Neither v1 nor v2 of this patch compiles on 32 bit Linux; see
> 
>   https://travis-ci.org/git/git/jobs/440514375#L628
> 
> The fixup patch below makes it compile on 32 bit and the test suite
> passes as well, but I didn't thoroughly review the changes; I only
> wanted to get 'pu' build again.
> 

Oh, yes, I didn't test under Linux 32 bit (and neither Windows)
I will try to compose a proper v3 the next days.

[snip the good stuff]
Ramsay Jones Oct. 14, 2018, 2:16 a.m. UTC | #3
On 13/10/18 06:00, Torsten Bögershausen wrote:
> []
>> Neither v1 nor v2 of this patch compiles on 32 bit Linux; see
>>
>>   https://travis-ci.org/git/git/jobs/440514375#L628
>>
>> The fixup patch below makes it compile on 32 bit and the test suite
>> passes as well, but I didn't thoroughly review the changes; I only
>> wanted to get 'pu' build again.
>>
> 
> Oh, yes, I didn't test under Linux 32 bit (and neither Windows)
> I will try to compose a proper v3 the next days.

I had a look at this today, and the result is given below.

The patch is effectively your v2 patch with SZEDER's fix-up
patch on top! (I actually started with my own patch and 'pared
it back' by removing the off_t -> size_t changes, etc; so I was
somewhat amused to note that the end result was effectively
SZEDER's patch on top of your v2 ;-) ).

I have tested this on 32- and 64-bit Linux, along with 64-bit
cygwin (the test suite run hasn't finished yet, but I don't
expect any problem). I have an old Msys2 installation on Windows,
which is the closest thing I have to a windows dev system, so I
also built this on MINGW32 and MINGW64 along with some light
manual testing (the test suite has never passed on Msys2 for me).
This is not the same as testing on Gfw, of course.

ATB,
Ramsay Jones

-- >8 --
From: Martin Koegler <martin.koegler@chello.at>
Subject: [PATCH v3 1/1] zlib.c: use size_t for size

Signed-off-by: Martin Koegler <martin.koegler@chello.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---
 builtin/pack-objects.c | 11 ++++++-----
 cache.h                | 10 +++++-----
 pack-check.c           |  4 ++--
 packfile.c             |  4 ++--
 packfile.h             |  2 +-
 wrapper.c              |  8 ++++----
 zlib.c                 |  8 ++++----
 7 files changed, 24 insertions(+), 23 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index b059b86aee..3b5f2c38b3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -269,12 +269,12 @@ static void copy_pack_data(struct hashfile *f,
 		off_t len)
 {
 	unsigned char *in;
-	unsigned long avail;
+	size_t avail;
 
 	while (len) {
 		in = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
-			avail = (unsigned long)len;
+			avail = xsize_t(len);
 		hashwrite(f, in, avail);
 		offset += avail;
 		len -= avail;
@@ -1529,8 +1529,8 @@ static void check_object(struct object_entry *entry)
 		struct pack_window *w_curs = NULL;
 		const unsigned char *base_ref = NULL;
 		struct object_entry *base_entry;
-		unsigned long used, used_0;
-		unsigned long avail;
+		size_t used, used_0;
+		size_t avail;
 		off_t ofs;
 		unsigned char *buf, c;
 		enum object_type type;
@@ -2002,7 +2002,8 @@ unsigned long oe_get_size_slow(struct packing_data *pack,
 	struct pack_window *w_curs;
 	unsigned char *buf;
 	enum object_type type;
-	unsigned long used, avail, size;
+	unsigned long used, size;
+	size_t avail;
 
 	if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
 		read_lock();
diff --git a/cache.h b/cache.h
index 5d83022e3b..ba0ad73de1 100644
--- a/cache.h
+++ b/cache.h
@@ -20,10 +20,10 @@
 #include <zlib.h>
 typedef struct git_zstream {
 	z_stream z;
-	unsigned long avail_in;
-	unsigned long avail_out;
-	unsigned long total_in;
-	unsigned long total_out;
+	size_t avail_in;
+	size_t avail_out;
+	size_t total_in;
+	size_t total_out;
 	unsigned char *next_in;
 	unsigned char *next_out;
 } git_zstream;
@@ -40,7 +40,7 @@ void git_deflate_end(git_zstream *);
 int git_deflate_abort(git_zstream *);
 int git_deflate_end_gently(git_zstream *);
 int git_deflate(git_zstream *, int flush);
-unsigned long git_deflate_bound(git_zstream *, unsigned long);
+size_t git_deflate_bound(git_zstream *, size_t);
 
 /* The length in bytes and in hex digits of an object name (SHA-1 value). */
 #define GIT_SHA1_RAWSZ 20
diff --git a/pack-check.c b/pack-check.c
index fa5f0ff8fa..d1e7f554ae 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -33,7 +33,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
 	uint32_t data_crc = crc32(0, NULL, 0);
 
 	do {
-		unsigned long avail;
+		size_t avail;
 		void *data = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
 			avail = len;
@@ -68,7 +68,7 @@ static int verify_packfile(struct packed_git *p,
 
 	the_hash_algo->init_fn(&ctx);
 	do {
-		unsigned long remaining;
+		size_t remaining;
 		unsigned char *in = use_pack(p, w_curs, offset, &remaining);
 		offset += remaining;
 		if (!pack_sig_ofs)
diff --git a/packfile.c b/packfile.c
index f2850a00b5..013294aec7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -585,7 +585,7 @@ static int in_window(struct pack_window *win, off_t offset)
 unsigned char *use_pack(struct packed_git *p,
 		struct pack_window **w_cursor,
 		off_t offset,
-		unsigned long *left)
+		size_t *left)
 {
 	struct pack_window *win = *w_cursor;
 
@@ -1104,7 +1104,7 @@ int unpack_object_header(struct packed_git *p,
 			 unsigned long *sizep)
 {
 	unsigned char *base;
-	unsigned long left;
+	size_t left;
 	unsigned long used;
 	enum object_type type;
 
diff --git a/packfile.h b/packfile.h
index 6c4037605d..1fb482424b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -78,7 +78,7 @@ extern void close_pack_index(struct packed_git *);
 
 extern uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
 
-extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
+extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, size_t *);
 extern void close_pack_windows(struct packed_git *);
 extern void close_pack(struct packed_git *);
 extern void close_all_packs(struct raw_object_store *o);
diff --git a/wrapper.c b/wrapper.c
index e4fa9d84cd..1a510bd6fc 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -67,11 +67,11 @@ static void *do_xmalloc(size_t size, int gentle)
 			ret = malloc(1);
 		if (!ret) {
 			if (!gentle)
-				die("Out of memory, malloc failed (tried to allocate %lu bytes)",
-				    (unsigned long)size);
+				die("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
+				    (uintmax_t)size);
 			else {
-				error("Out of memory, malloc failed (tried to allocate %lu bytes)",
-				      (unsigned long)size);
+				error("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
+				      (uintmax_t)size);
 				return NULL;
 			}
 		}
diff --git a/zlib.c b/zlib.c
index d594cba3fc..197a1acc7b 100644
--- a/zlib.c
+++ b/zlib.c
@@ -29,7 +29,7 @@ static const char *zerr_to_string(int status)
  */
 /* #define ZLIB_BUF_MAX ((uInt)-1) */
 #define ZLIB_BUF_MAX ((uInt) 1024 * 1024 * 1024) /* 1GB */
-static inline uInt zlib_buf_cap(unsigned long len)
+static inline uInt zlib_buf_cap(size_t len)
 {
 	return (ZLIB_BUF_MAX < len) ? ZLIB_BUF_MAX : len;
 }
@@ -46,8 +46,8 @@ static void zlib_pre_call(git_zstream *s)
 
 static void zlib_post_call(git_zstream *s)
 {
-	unsigned long bytes_consumed;
-	unsigned long bytes_produced;
+	size_t bytes_consumed;
+	size_t bytes_produced;
 
 	bytes_consumed = s->z.next_in - s->next_in;
 	bytes_produced = s->z.next_out - s->next_out;
@@ -150,7 +150,7 @@ int git_inflate(git_zstream *strm, int flush)
 #define deflateBound(c,s)  ((s) + (((s) + 7) >> 3) + (((s) + 63) >> 6) + 11)
 #endif
 
-unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
+size_t git_deflate_bound(git_zstream *strm, size_t size)
 {
 	return deflateBound(&strm->z, size);
 }
Ramsay Jones Oct. 14, 2018, 2:31 a.m. UTC | #4
On 14/10/18 03:16, Ramsay Jones wrote:
> 
> 
> On 13/10/18 06:00, Torsten Bögershausen wrote:
>> []
>>> Neither v1 nor v2 of this patch compiles on 32 bit Linux; see
>>>
>>>   https://travis-ci.org/git/git/jobs/440514375#L628
>>>
>>> The fixup patch below makes it compile on 32 bit and the test suite
>>> passes as well, but I didn't thoroughly review the changes; I only
>>> wanted to get 'pu' build again.
>>>
>>
>> Oh, yes, I didn't test under Linux 32 bit (and neither Windows)
>> I will try to compose a proper v3 the next days.
> 
> I had a look at this today, and the result is given below.
> 
> The patch is effectively your v2 patch with SZEDER's fix-up
> patch on top! (I actually started with my own patch and 'pared
> it back' by removing the off_t -> size_t changes, etc; so I was
> somewhat amused to note that the end result was effectively
> SZEDER's patch on top of your v2 ;-) ).
> 
> I have tested this on 32- and 64-bit Linux, along with 64-bit
> cygwin (the test suite run hasn't finished yet, but I don't
> expect any problem). I have an old Msys2 installation on Windows,
> which is the closest thing I have to a windows dev system, so I
> also built this on MINGW32 and MINGW64 along with some light
> manual testing (the test suite has never passed on Msys2 for me).
> This is not the same as testing on Gfw, of course.

Ho, Hum. Of course, I have just noticed that, similar to
copy_pack_data(), check_pack_crc() should probably use a
call to xsize_t(len) when assigning to avail.

Sorry about that. (I need some sleep now!)

ATB,
Ramsay Jones


> -- >8 --
> From: Martin Koegler <martin.koegler@chello.at>
> Subject: [PATCH v3 1/1] zlib.c: use size_t for size
> 
> Signed-off-by: Martin Koegler <martin.koegler@chello.at>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
> ---
>  builtin/pack-objects.c | 11 ++++++-----
>  cache.h                | 10 +++++-----
>  pack-check.c           |  4 ++--
>  packfile.c             |  4 ++--
>  packfile.h             |  2 +-
>  wrapper.c              |  8 ++++----
>  zlib.c                 |  8 ++++----
>  7 files changed, 24 insertions(+), 23 deletions(-)
> 
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index b059b86aee..3b5f2c38b3 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -269,12 +269,12 @@ static void copy_pack_data(struct hashfile *f,
>  		off_t len)
>  {
>  	unsigned char *in;
> -	unsigned long avail;
> +	size_t avail;
>  
>  	while (len) {
>  		in = use_pack(p, w_curs, offset, &avail);
>  		if (avail > len)
> -			avail = (unsigned long)len;
> +			avail = xsize_t(len);
>  		hashwrite(f, in, avail);
>  		offset += avail;
>  		len -= avail;
> @@ -1529,8 +1529,8 @@ static void check_object(struct object_entry *entry)
>  		struct pack_window *w_curs = NULL;
>  		const unsigned char *base_ref = NULL;
>  		struct object_entry *base_entry;
> -		unsigned long used, used_0;
> -		unsigned long avail;
> +		size_t used, used_0;
> +		size_t avail;
>  		off_t ofs;
>  		unsigned char *buf, c;
>  		enum object_type type;
> @@ -2002,7 +2002,8 @@ unsigned long oe_get_size_slow(struct packing_data *pack,
>  	struct pack_window *w_curs;
>  	unsigned char *buf;
>  	enum object_type type;
> -	unsigned long used, avail, size;
> +	unsigned long used, size;
> +	size_t avail;
>  
>  	if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
>  		read_lock();
> diff --git a/cache.h b/cache.h
> index 5d83022e3b..ba0ad73de1 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -20,10 +20,10 @@
>  #include <zlib.h>
>  typedef struct git_zstream {
>  	z_stream z;
> -	unsigned long avail_in;
> -	unsigned long avail_out;
> -	unsigned long total_in;
> -	unsigned long total_out;
> +	size_t avail_in;
> +	size_t avail_out;
> +	size_t total_in;
> +	size_t total_out;
>  	unsigned char *next_in;
>  	unsigned char *next_out;
>  } git_zstream;
> @@ -40,7 +40,7 @@ void git_deflate_end(git_zstream *);
>  int git_deflate_abort(git_zstream *);
>  int git_deflate_end_gently(git_zstream *);
>  int git_deflate(git_zstream *, int flush);
> -unsigned long git_deflate_bound(git_zstream *, unsigned long);
> +size_t git_deflate_bound(git_zstream *, size_t);
>  
>  /* The length in bytes and in hex digits of an object name (SHA-1 value). */
>  #define GIT_SHA1_RAWSZ 20
> diff --git a/pack-check.c b/pack-check.c
> index fa5f0ff8fa..d1e7f554ae 100644
> --- a/pack-check.c
> +++ b/pack-check.c
> @@ -33,7 +33,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
>  	uint32_t data_crc = crc32(0, NULL, 0);
>  
>  	do {
> -		unsigned long avail;
> +		size_t avail;
>  		void *data = use_pack(p, w_curs, offset, &avail);
>  		if (avail > len)
>  			avail = len;
> @@ -68,7 +68,7 @@ static int verify_packfile(struct packed_git *p,
>  
>  	the_hash_algo->init_fn(&ctx);
>  	do {
> -		unsigned long remaining;
> +		size_t remaining;
>  		unsigned char *in = use_pack(p, w_curs, offset, &remaining);
>  		offset += remaining;
>  		if (!pack_sig_ofs)
> diff --git a/packfile.c b/packfile.c
> index f2850a00b5..013294aec7 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -585,7 +585,7 @@ static int in_window(struct pack_window *win, off_t offset)
>  unsigned char *use_pack(struct packed_git *p,
>  		struct pack_window **w_cursor,
>  		off_t offset,
> -		unsigned long *left)
> +		size_t *left)
>  {
>  	struct pack_window *win = *w_cursor;
>  
> @@ -1104,7 +1104,7 @@ int unpack_object_header(struct packed_git *p,
>  			 unsigned long *sizep)
>  {
>  	unsigned char *base;
> -	unsigned long left;
> +	size_t left;
>  	unsigned long used;
>  	enum object_type type;
>  
> diff --git a/packfile.h b/packfile.h
> index 6c4037605d..1fb482424b 100644
> --- a/packfile.h
> +++ b/packfile.h
> @@ -78,7 +78,7 @@ extern void close_pack_index(struct packed_git *);
>  
>  extern uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
>  
> -extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
> +extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, size_t *);
>  extern void close_pack_windows(struct packed_git *);
>  extern void close_pack(struct packed_git *);
>  extern void close_all_packs(struct raw_object_store *o);
> diff --git a/wrapper.c b/wrapper.c
> index e4fa9d84cd..1a510bd6fc 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -67,11 +67,11 @@ static void *do_xmalloc(size_t size, int gentle)
>  			ret = malloc(1);
>  		if (!ret) {
>  			if (!gentle)
> -				die("Out of memory, malloc failed (tried to allocate %lu bytes)",
> -				    (unsigned long)size);
> +				die("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
> +				    (uintmax_t)size);
>  			else {
> -				error("Out of memory, malloc failed (tried to allocate %lu bytes)",
> -				      (unsigned long)size);
> +				error("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
> +				      (uintmax_t)size);
>  				return NULL;
>  			}
>  		}
> diff --git a/zlib.c b/zlib.c
> index d594cba3fc..197a1acc7b 100644
> --- a/zlib.c
> +++ b/zlib.c
> @@ -29,7 +29,7 @@ static const char *zerr_to_string(int status)
>   */
>  /* #define ZLIB_BUF_MAX ((uInt)-1) */
>  #define ZLIB_BUF_MAX ((uInt) 1024 * 1024 * 1024) /* 1GB */
> -static inline uInt zlib_buf_cap(unsigned long len)
> +static inline uInt zlib_buf_cap(size_t len)
>  {
>  	return (ZLIB_BUF_MAX < len) ? ZLIB_BUF_MAX : len;
>  }
> @@ -46,8 +46,8 @@ static void zlib_pre_call(git_zstream *s)
>  
>  static void zlib_post_call(git_zstream *s)
>  {
> -	unsigned long bytes_consumed;
> -	unsigned long bytes_produced;
> +	size_t bytes_consumed;
> +	size_t bytes_produced;
>  
>  	bytes_consumed = s->z.next_in - s->next_in;
>  	bytes_produced = s->z.next_out - s->next_out;
> @@ -150,7 +150,7 @@ int git_inflate(git_zstream *strm, int flush)
>  #define deflateBound(c,s)  ((s) + (((s) + 7) >> 3) + (((s) + 63) >> 6) + 11)
>  #endif
>  
> -unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
> +size_t git_deflate_bound(git_zstream *strm, size_t size)
>  {
>  	return deflateBound(&strm->z, size);
>  }
>
Jeff King Oct. 14, 2018, 2:52 a.m. UTC | #5
On Sun, Oct 14, 2018 at 03:16:36AM +0100, Ramsay Jones wrote:

> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index b059b86aee..3b5f2c38b3 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -269,12 +269,12 @@ static void copy_pack_data(struct hashfile *f,
>  		off_t len)
>  {
>  	unsigned char *in;
> -	unsigned long avail;
> +	size_t avail;
>  
>  	while (len) {
>  		in = use_pack(p, w_curs, offset, &avail);
>  		if (avail > len)
> -			avail = (unsigned long)len;
> +			avail = xsize_t(len);

We don't actually care about truncation here. The idea is that we take a
bite-sized chunk via use_pack, and loop as necessary. So mod-2^32
truncation via a cast would be bad (we might not make forward progress),
but truncating to SIZE_MAX would be fine.

That said, we know that would not truncate here, because we must
strictly be shrinking "avail", which was already a size_t (unless "len"
is negative, but then we are really screwed ;) ).

So I kind of wonder if a comment would be better than xsize_t here.
Something like:

  if (avail > len) {
	/*
	 * This can never truncate because we know that len is smaller
	 * than avail, which is already a size_t.
	 */
	avail = (size_t)len;
  }

-Peff
Ramsay Jones Oct. 14, 2018, 3:03 p.m. UTC | #6
On 14/10/18 03:52, Jeff King wrote:
> On Sun, Oct 14, 2018 at 03:16:36AM +0100, Ramsay Jones wrote:
> 
>> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
>> index b059b86aee..3b5f2c38b3 100644
>> --- a/builtin/pack-objects.c
>> +++ b/builtin/pack-objects.c
>> @@ -269,12 +269,12 @@ static void copy_pack_data(struct hashfile *f,
>>  		off_t len)
>>  {
>>  	unsigned char *in;
>> -	unsigned long avail;
>> +	size_t avail;
>>  
>>  	while (len) {
>>  		in = use_pack(p, w_curs, offset, &avail);
>>  		if (avail > len)
>> -			avail = (unsigned long)len;
>> +			avail = xsize_t(len);
> 
> We don't actually care about truncation here. The idea is that we take a
> bite-sized chunk via use_pack, and loop as necessary. So mod-2^32
> truncation via a cast would be bad (we might not make forward progress),
> but truncating to SIZE_MAX would be fine.
> 
> That said, we know that would not truncate here, because we must
> strictly be shrinking "avail", which was already a size_t (unless "len"
> is negative, but then we are really screwed ;) ).
> 
> So I kind of wonder if a comment would be better than xsize_t here.
> Something like:
> 
>   if (avail > len) {
> 	/*
> 	 * This can never truncate because we know that len is smaller
> 	 * than avail, which is already a size_t.
> 	 */
> 	avail = (size_t)len;
>   }

Heh, you are, of course, correct! (that will learn me[1]). :-D

Hmm, let's see if I can muster the enthusiasm to do all that
testing again!

ATB,
Ramsay Jones

[1] Since I started with my patch, when I had finished 'paring
it back', the result didn't have this xsize_t() call. In order
to make the result 'v2 + SZEDER's patch' (which I thought was
quite neat) I added this call right at the end. :-P
Jeff King Oct. 15, 2018, 12:01 a.m. UTC | #7
On Sun, Oct 14, 2018 at 04:03:48PM +0100, Ramsay Jones wrote:

> > So I kind of wonder if a comment would be better than xsize_t here.
> > Something like:
> > 
> >   if (avail > len) {
> > 	/*
> > 	 * This can never truncate because we know that len is smaller
> > 	 * than avail, which is already a size_t.
> > 	 */
> > 	avail = (size_t)len;
> >   }
> 
> Heh, you are, of course, correct! (that will learn me[1]). :-D
> 
> Hmm, let's see if I can muster the enthusiasm to do all that
> testing again!

For the record, I am not opposed to including the comment _and_ using
xsize_t() to do the cast, giving us an assertion that the comment is
correct.

-Peff
Ramsay Jones Oct. 15, 2018, 12:41 a.m. UTC | #8
On 15/10/18 01:01, Jeff King wrote:
> On Sun, Oct 14, 2018 at 04:03:48PM +0100, Ramsay Jones wrote:
> 
>>> So I kind of wonder if a comment would be better than xsize_t here.
>>> Something like:
>>>
>>>   if (avail > len) {
>>> 	/*
>>> 	 * This can never truncate because we know that len is smaller
>>> 	 * than avail, which is already a size_t.
>>> 	 */
>>> 	avail = (size_t)len;
>>>   }
>>
>> Heh, you are, of course, correct! (that will learn me[1]). :-D
>>
>> Hmm, let's see if I can muster the enthusiasm to do all that
>> testing again!
> 
> For the record, I am not opposed to including the comment _and_ using
> xsize_t() to do the cast, giving us an assertion that the comment is
> correct.

Heh, I haven't found any enthusiasm tonight. Let's see if there
are any more comments/opinions.

Thanks.

ATB,
Ramsay Jones
Junio C Hamano Oct. 15, 2018, 4:22 a.m. UTC | #9
Ramsay Jones <ramsay@ramsayjones.plus.com> writes:

>> 
>> For the record, I am not opposed to including the comment _and_ using
>> xsize_t() to do the cast, giving us an assertion that the comment is
>> correct.
>
> Heh, I haven't found any enthusiasm tonight. Let's see if there
> are any more comments/opinions.

OK, in the meantime, I've replaced the thread-starter partch I had
in 'pu' with your v3.
Torsten Bögershausen Oct. 15, 2018, 5:54 a.m. UTC | #10
On 15.10.18 06:22, Junio C Hamano wrote:
> Ramsay Jones <ramsay@ramsayjones.plus.com> writes:
> 
>>>
>>> For the record, I am not opposed to including the comment _and_ using
>>> xsize_t() to do the cast, giving us an assertion that the comment is
>>> correct.
>>
>> Heh, I haven't found any enthusiasm tonight. Let's see if there
>> are any more comments/opinions.
> 
> OK, in the meantime, I've replaced the thread-starter partch I had
> in 'pu' with your v3.
> 
If that helps to move things forward:  I'm fully fine with v3.

Patch
diff mbox series

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index e6316d294d..23c4cd8c77 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -269,12 +269,12 @@  static void copy_pack_data(struct hashfile *f,
 		off_t len)
 {
 	unsigned char *in;
-	unsigned long avail;
+	size_t avail;
 
 	while (len) {
 		in = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
-			avail = (unsigned long)len;
+			avail = xsize_t(len);
 		hashwrite(f, in, avail);
 		offset += avail;
 		len -= avail;
@@ -1478,8 +1478,8 @@  static void check_object(struct object_entry *entry)
 		struct pack_window *w_curs = NULL;
 		const unsigned char *base_ref = NULL;
 		struct object_entry *base_entry;
-		unsigned long used, used_0;
-		unsigned long avail;
+		size_t used, used_0;
+		size_t avail;
 		off_t ofs;
 		unsigned char *buf, c;
 		enum object_type type;
diff --git a/cache.h b/cache.h
index d508f3d4f8..fce53fe620 100644
--- a/cache.h
+++ b/cache.h
@@ -20,10 +20,10 @@ 
 #include <zlib.h>
 typedef struct git_zstream {
 	z_stream z;
-	unsigned long avail_in;
-	unsigned long avail_out;
-	unsigned long total_in;
-	unsigned long total_out;
+	size_t avail_in;
+	size_t avail_out;
+	size_t total_in;
+	size_t total_out;
 	unsigned char *next_in;
 	unsigned char *next_out;
 } git_zstream;
@@ -40,7 +40,7 @@  void git_deflate_end(git_zstream *);
 int git_deflate_abort(git_zstream *);
 int git_deflate_end_gently(git_zstream *);
 int git_deflate(git_zstream *, int flush);
-unsigned long git_deflate_bound(git_zstream *, unsigned long);
+size_t git_deflate_bound(git_zstream *, size_t);
 
 /* The length in bytes and in hex digits of an object name (SHA-1 value). */
 #define GIT_SHA1_RAWSZ 20
diff --git a/pack-check.c b/pack-check.c
index fa5f0ff8fa..d1e7f554ae 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -33,7 +33,7 @@  int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
 	uint32_t data_crc = crc32(0, NULL, 0);
 
 	do {
-		unsigned long avail;
+		size_t avail;
 		void *data = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
 			avail = len;
@@ -68,7 +68,7 @@  static int verify_packfile(struct packed_git *p,
 
 	the_hash_algo->init_fn(&ctx);
 	do {
-		unsigned long remaining;
+		size_t remaining;
 		unsigned char *in = use_pack(p, w_curs, offset, &remaining);
 		offset += remaining;
 		if (!pack_sig_ofs)
diff --git a/packfile.h b/packfile.h
index 442625723d..e2daf63426 100644
--- a/packfile.h
+++ b/packfile.h
@@ -78,7 +78,7 @@  extern void close_pack_index(struct packed_git *);
 
 extern uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
 
-extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
+extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, size_t *);
 extern void close_pack_windows(struct packed_git *);
 extern void close_pack(struct packed_git *);
 extern void close_all_packs(struct raw_object_store *o);
diff --git a/wrapper.c b/wrapper.c
index e4fa9d84cd..1a510bd6fc 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -67,11 +67,11 @@  static void *do_xmalloc(size_t size, int gentle)
 			ret = malloc(1);
 		if (!ret) {
 			if (!gentle)
-				die("Out of memory, malloc failed (tried to allocate %lu bytes)",
-				    (unsigned long)size);
+				die("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
+				    (uintmax_t)size);
 			else {
-				error("Out of memory, malloc failed (tried to allocate %lu bytes)",
-				      (unsigned long)size);
+				error("Out of memory, malloc failed (tried to allocate %" PRIuMAX " bytes)",
+				      (uintmax_t)size);
 				return NULL;
 			}
 		}
diff --git a/zlib.c b/zlib.c
index d594cba3fc..197a1acc7b 100644
--- a/zlib.c
+++ b/zlib.c
@@ -29,7 +29,7 @@  static const char *zerr_to_string(int status)
  */
 /* #define ZLIB_BUF_MAX ((uInt)-1) */
 #define ZLIB_BUF_MAX ((uInt) 1024 * 1024 * 1024) /* 1GB */
-static inline uInt zlib_buf_cap(unsigned long len)
+static inline uInt zlib_buf_cap(size_t len)
 {
 	return (ZLIB_BUF_MAX < len) ? ZLIB_BUF_MAX : len;
 }
@@ -46,8 +46,8 @@  static void zlib_pre_call(git_zstream *s)
 
 static void zlib_post_call(git_zstream *s)
 {
-	unsigned long bytes_consumed;
-	unsigned long bytes_produced;
+	size_t bytes_consumed;
+	size_t bytes_produced;
 
 	bytes_consumed = s->z.next_in - s->next_in;
 	bytes_produced = s->z.next_out - s->next_out;
@@ -150,7 +150,7 @@  int git_inflate(git_zstream *strm, int flush)
 #define deflateBound(c,s)  ((s) + (((s) + 7) >> 3) + (((s) + 63) >> 6) + 11)
 #endif
 
-unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
+size_t git_deflate_bound(git_zstream *strm, size_t size)
 {
 	return deflateBound(&strm->z, size);
 }