Catch Universal Newlines But Preserve Original

So this is my problem, I'm trying to do a simple program that runs another process using Python's subprocess module, and I want to catch real-time output of the process. I know thi

Solution 1:

A possible way is to use the binary interface of Popen by specifying neither encoding nor error and of course not universal_newline. And then, we can use a TextIOWrapper around the binary stream, with newline=''. Because the documentation for TextIOWrapper says:

... if newline is None... If it is '', universal newlines mode is enabled, but line endings are returned to the caller untranslated

(which is conformant with PEP 3116)

You original code could be changed to:

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out = io.TextIOWrapper(proc.stdout, newline='')

for line in out:
    # line is delimited with the universal newline convention and actually contains
    #  the original end of line, be it a raw \r, \n of the pair \r\n
    ...