[v4,07/11] git-p4: Add a helper class for stream writing

Commit Message

Linus Arver via GitGitGadget Dec. 4, 2019, 10:29 p.m. UTC

From: Ben Keene <[e-mail]>

This is a transtional commit that does not change current behvior.  It adds a new class Py23File.

Following the Python recommendation of keeping text as unicode internally and only converting to and from bytes on input and output, this class provides an interface for the methods used for reading and writing files and file like streams.

Create a class that wraps the input and output functions used by the git-p4.py code for reading and writing to standard file handles.

The methods of this class should take a Unicode string for writing and return unicode strings in reads.  This class should be a drop-in for existing file like streams

The following methods should be coded for supporting existing read/write calls:
* write - this should write a Unicode string to the underlying stream
* read - this should read from the underlying stream and cast the bytes as a unicode string
* readline - this should read one line of text from the underlying stream and cast it as a unicode string
* readline - this should read a number of lines, optionally hinted, and cast each line as a unicode string

The expression "cast as a unicode string" is used because the code should use the AS_BYTES() and AS_UNICODE() functions instead of cohercing the data to actual unicode strings or bytes.  This allows python 2 code to continue to use the internal "str" data type instead of converting the data back and forth to actual unicode strings. This retains current python2 support while python3 support may be incomplete.

Signed-off-by: Ben Keene <[e-mail]>
(cherry picked from commit 12919111fbaa3e4c0c4c2fdd4f79744cc683d860)
---
 git-p4.py | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

Comments

Junio C Hamano Dec. 5, 2019, 1:42 p.m. UTC | #1

"Ben Keene via GitGitGadget" <[e-mail]> writes:

> From: Ben Keene <[e-mail]>
>
> This is a transtional commit that does not change current behvior.  It adds a new class Py23File.

Perhaps s/transitional/preparatory/?  It does not change the
behaviour because nobody uses the class yet, if I understand
correctly.  Which is fine.

It is kind of surprising that each project needs to reinvent and
maintain a wrapper class like this one, as what the new class does
smells quite generic.

> Following the Python recommendation of keeping text as unicode internally and only converting to and from bytes on input and output, this class provides an interface for the methods used for reading and writing files and file like streams.
>
> Create a class that wraps the input and output functions used by the git-p4.py code for reading and writing to standard file handles.
>
> The methods of this class should take a Unicode string for writing and return unicode strings in reads.  This class should be a drop-in for existing file like streams
>
> The following methods should be coded for supporting existing read/write calls:
> * write - this should write a Unicode string to the underlying stream
> * read - this should read from the underlying stream and cast the bytes as a unicode string
> * readline - this should read one line of text from the underlying stream and cast it as a unicode string
> * readline - this should read a number of lines, optionally hinted, and cast each line as a unicode string
>
> The expression "cast as a unicode string" is used because the code should use the AS_BYTES() and AS_UNICODE() functions instead of cohercing the data to actual unicode strings or bytes.  This allows python 2 code to continue to use the internal "str" data type instead of converting the data back and forth to actual unicode strings. This retains current python2 support while python3 support may be incomplete.
>
> Signed-off-by: Ben Keene <[e-mail]>
> (cherry picked from commit 12919111fbaa3e4c0c4c2fdd4f79744cc683d860)
> ---
>  git-p4.py | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 66 insertions(+)
>
> diff --git a/git-p4.py b/git-p4.py
> index 7ac8cb42ef..0da640be93 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -4182,6 +4182,72 @@ def run(self, args):
>              print("%s <= %s (%s)" % (branch, ",".join(settings["depot-paths"]), settings["change"]))
>          return True
>  
> +class Py23File():
> +    """ Python2/3 Unicode File Wrapper 
> +    """
> +    
> +    stream_handle = None
> +    verbose       = False
> +    debug_handle  = None
> +   
> +    def __init__(self, stream_handle, verbose = False):
> +        """ Create a Python3 compliant Unicode to Byte String
> +            Windows compatible wrapper
> +
> +            stream_handle = the underlying file-like handle
> +            verbose       = Boolean if content should be echoed
> +        """
> +        self.stream_handle = stream_handle
> +        self.verbose       = verbose
> +
> +    def write(self, utf8string):
> +        """ Writes the utf8 encoded string to the underlying 
> +            file stream
> +        """
> +        self.stream_handle.write(as_bytes(utf8string))
> +        if self.verbose:
> +            sys.stderr.write("Stream Output: %s" % utf8string)
> +            sys.stderr.flush()
> +
> +    def read(self, size = None):
> +        """ Reads int charcters from the underlying stream 
> +            and converts it to utf8.
> +
> +            Be aware, the size value is for reading the underlying
> +            bytes so the value may be incorrect. Usage of the size
> +            value is discouraged.
> +        """
> +        if size == None:
> +            return as_string(self.stream_handle.read())
> +        else:
> +            return as_string(self.stream_handle.read(size))
> +
> +    def readline(self):
> +        """ Reads a line from the underlying byte stream 
> +            and converts it to utf8
> +        """
> +        return as_string(self.stream_handle.readline())
> +
> +    def readlines(self, sizeHint = None):
> +        """ Returns a list containing lines from the file converted to unicode.
> +
> +            sizehint - Optional. If the optional sizehint argument is 
> +            present, instead of reading up to EOF, whole lines totalling 
> +            approximately sizehint bytes are read.
> +        """
> +        lines = self.stream_handle.readlines(sizeHint)
> +        for i in range(0, len(lines)):
> +            lines[i] = as_string(lines[i])
> +        return lines
> +
> +    def close(self):
> +        """ Closes the underlying byte stream """
> +        self.stream_handle.close()
> +
> +    def flush(self):
> +        """ Flushes the underlying byte stream """
> +        self.stream_handle.flush()
> +
>  class HelpFormatter(optparse.IndentedHelpFormatter):
>      def __init__(self):
>          optparse.IndentedHelpFormatter.__init__(self)

Ben Keene Dec. 5, 2019, 7:52 p.m. UTC | #2

On 12/5/2019 8:42 AM, Junio C Hamano wrote:
> "Ben Keene via GitGitGadget" <[e-mail]> writes:
>
>> From: Ben Keene <[e-mail]>
>>
>> This is a transtional commit that does not change current behvior.  It adds a new class Py23File.
> Perhaps s/transitional/preparatory/?  It does not change the
> behaviour because nobody uses the class yet, if I understand
> correctly.  Which is fine.
>
> It is kind of surprising that each project needs to reinvent and
> maintain a wrapper class like this one, as what the new class does
> smells quite generic.

It is a rather generic class.  My intention was to avoid adding
any additional dependencies so a small class that only implements
the few methods we need seemed safest.

I cleaned up this commit message as well.

>> Following the Python recommendation of keeping text as unicode internally and only converting to and from bytes on input and output, this class provides an interface for the methods used for reading and writing files and file like streams.
>>
>> Create a class that wraps the input and output functions used by the git-p4.py code for reading and writing to standard file handles.
>>
>> The methods of this class should take a Unicode string for writing and return unicode strings in reads.  This class should be a drop-in for existing file like streams
>>
>> The following methods should be coded for supporting existing read/write calls:
>> * write - this should write a Unicode string to the underlying stream
>> * read - this should read from the underlying stream and cast the bytes as a unicode string
>> * readline - this should read one line of text from the underlying stream and cast it as a unicode string
>> * readline - this should read a number of lines, optionally hinted, and cast each line as a unicode string
>>
>> The expression "cast as a unicode string" is used because the code should use the AS_BYTES() and AS_UNICODE() functions instead of cohercing the data to actual unicode strings or bytes.  This allows python 2 code to continue to use the internal "str" data type instead of converting the data back and forth to actual unicode strings. This retains current python2 support while python3 support may be incomplete.
>>
>> Signed-off-by: Ben Keene <[e-mail]>
>> (cherry picked from commit 12919111fbaa3e4c0c4c2fdd4f79744cc683d860)
>> ---
>>   git-p4.py | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 66 insertions(+)
>>
>> diff --git a/git-p4.py b/git-p4.py
>> index 7ac8cb42ef..0da640be93 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -4182,6 +4182,72 @@ def run(self, args):
>>               print("%s <= %s (%s)" % (branch, ",".join(settings["depot-paths"]), settings["change"]))
>>           return True
>>   
>> +class Py23File():
>> +    """ Python2/3 Unicode File Wrapper
>> +    """
>> +
>> +    stream_handle = None
>> +    verbose       = False
>> +    debug_handle  = None
>> +
>> +    def __init__(self, stream_handle, verbose = False):
>> +        """ Create a Python3 compliant Unicode to Byte String
>> +            Windows compatible wrapper
>> +
>> +            stream_handle = the underlying file-like handle
>> +            verbose       = Boolean if content should be echoed
>> +        """
>> +        self.stream_handle = stream_handle
>> +        self.verbose       = verbose
>> +
>> +    def write(self, utf8string):
>> +        """ Writes the utf8 encoded string to the underlying
>> +            file stream
>> +        """
>> +        self.stream_handle.write(as_bytes(utf8string))
>> +        if self.verbose:
>> +            sys.stderr.write("Stream Output: %s" % utf8string)
>> +            sys.stderr.flush()
>> +
>> +    def read(self, size = None):
>> +        """ Reads int charcters from the underlying stream
>> +            and converts it to utf8.
>> +
>> +            Be aware, the size value is for reading the underlying
>> +            bytes so the value may be incorrect. Usage of the size
>> +            value is discouraged.
>> +        """
>> +        if size == None:
>> +            return as_string(self.stream_handle.read())
>> +        else:
>> +            return as_string(self.stream_handle.read(size))
>> +
>> +    def readline(self):
>> +        """ Reads a line from the underlying byte stream
>> +            and converts it to utf8
>> +        """
>> +        return as_string(self.stream_handle.readline())
>> +
>> +    def readlines(self, sizeHint = None):
>> +        """ Returns a list containing lines from the file converted to unicode.
>> +
>> +            sizehint - Optional. If the optional sizehint argument is
>> +            present, instead of reading up to EOF, whole lines totalling
>> +            approximately sizehint bytes are read.
>> +        """
>> +        lines = self.stream_handle.readlines(sizeHint)
>> +        for i in range(0, len(lines)):
>> +            lines[i] = as_string(lines[i])
>> +        return lines
>> +
>> +    def close(self):
>> +        """ Closes the underlying byte stream """
>> +        self.stream_handle.close()
>> +
>> +    def flush(self):
>> +        """ Flushes the underlying byte stream """
>> +        self.stream_handle.flush()
>> +
>>   class HelpFormatter(optparse.IndentedHelpFormatter):
>>       def __init__(self):
>>           optparse.IndentedHelpFormatter.__init__(self)

Patch

diff --git a/git-p4.py b/git-p4.py
index 7ac8cb42ef..0da640be93 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -4182,6 +4182,72 @@  def run(self, args):
             print("%s <= %s (%s)" % (branch, ",".join(settings["depot-paths"]), settings["change"]))
         return True
 
+class Py23File():
+    """ Python2/3 Unicode File Wrapper 
+    """
+    
+    stream_handle = None
+    verbose       = False
+    debug_handle  = None
+   
+    def __init__(self, stream_handle, verbose = False):
+        """ Create a Python3 compliant Unicode to Byte String
+            Windows compatible wrapper
+
+            stream_handle = the underlying file-like handle
+            verbose       = Boolean if content should be echoed
+        """
+        self.stream_handle = stream_handle
+        self.verbose       = verbose
+
+    def write(self, utf8string):
+        """ Writes the utf8 encoded string to the underlying 
+            file stream
+        """
+        self.stream_handle.write(as_bytes(utf8string))
+        if self.verbose:
+            sys.stderr.write("Stream Output: %s" % utf8string)
+            sys.stderr.flush()
+
+    def read(self, size = None):
+        """ Reads int charcters from the underlying stream 
+            and converts it to utf8.
+
+            Be aware, the size value is for reading the underlying
+            bytes so the value may be incorrect. Usage of the size
+            value is discouraged.
+        """
+        if size == None:
+            return as_string(self.stream_handle.read())
+        else:
+            return as_string(self.stream_handle.read(size))
+
+    def readline(self):
+        """ Reads a line from the underlying byte stream 
+            and converts it to utf8
+        """
+        return as_string(self.stream_handle.readline())
+
+    def readlines(self, sizeHint = None):
+        """ Returns a list containing lines from the file converted to unicode.
+
+            sizehint - Optional. If the optional sizehint argument is 
+            present, instead of reading up to EOF, whole lines totalling 
+            approximately sizehint bytes are read.
+        """
+        lines = self.stream_handle.readlines(sizeHint)
+        for i in range(0, len(lines)):
+            lines[i] = as_string(lines[i])
+        return lines
+
+    def close(self):
+        """ Closes the underlying byte stream """
+        self.stream_handle.close()
+
+    def flush(self):
+        """ Flushes the underlying byte stream """
+        self.stream_handle.flush()
+
 class HelpFormatter(optparse.IndentedHelpFormatter):
     def __init__(self):
         optparse.IndentedHelpFormatter.__init__(self)