Pipe Output From Shell Command To A Python Script

I want to run a mysql command and set the output of that to be a variable in my python script. Here is the shell command I'm trying to run: $ mysql my_database --html -e 'select *

Solution 1:

You need to read from stdin to retrieve the data in the python script e.g.

#!/usr/bin/env pythonimport sys

defhello(variable):
    print variable

data = sys.stdin.read()
hello(data)

If all you want to do here is grab some data from a mysql database and then manipulate it with Python I would skip piping it into the script and just use the Python MySql module to do the SQL query.

Solution 2:

If you want your script to behave like many unix command line tools and accept a pipe or a filename as first argument, you can use the following:

#!/usr/bin/env pythonimport sys

# use stdin if it's full                                                        ifnot sys.stdin.isatty():
    input_stream = sys.stdin

# otherwise, read the given filename                                            else:
    try:
        input_filename = sys.argv[1]
    except IndexError:
        message = 'need filename as first argument if stdin is not full'raise IndexError(message)
    else:
        input_stream = open(input_filename, 'rU')

for line in input_stream:
    print(line) # do something useful with each line

Solution 3:

When you pipe the output of one command to a pytho script, it goes to sys.stdin. You can read from sys.stdin just like a file. Example:

import sys

print sys.stdin.read()

This program literally outputs its input.

Solution 4:

Since this answer pops up on Google at the top when searching for piping data to a python script, I'd like to add another method, which I have found in J. Beazley's Python Cookbook after searching for a less 'gritty' aproach than using sys. IMO, more pythonic and self-explanatory even to new users.

import fileinput
with fileinput.input() as f_input:
    for line in f_input:
        print(line, end='')

This approach also works for commands structured like this:

$ ls | ./filein.py          # Prints a directory listing to stdout.$ ./filein.py /etc/passwd   # Reads /etc/passwd to stdout.$ ./filein.py < /etc/passwd # Reads /etc/passwd to stdout.

If you require more complex solutions, you can compine argparse and fileinputas shown in this gist by martinth:

import argpase
import fileinput

if __name__ == '__main__':
    parser = ArgumentParser()
    parser.add_argument('--dummy', help='dummy argument')
    parser.add_argument('files', metavar='FILE', nargs='*', help='files to read, if empty, stdin is used')
    args = parser.parse_args()

    # If you would call fileinput.input() without files it would try to process all arguments.# We pass '-' as only file when argparse got no files which will cause fileinput to read from stdinfor line in fileinput.input(files=args.files iflen(args.files) > 0else ('-', )):
        print(line)

```

Solution 5:

You can use the command line tool xargs

echo 'arg1' | xargs python script.py

arg1 is now accessible from sys.argv[1] in script.py