Python subprocess Module
subprocessis a module in the Python standard library used to create and manage subprocesses.
subprocessIt allows you to execute external commands in Python programs and interact with those commands.
Throughsubprocessthe module, you can execute system commands, call other programs, and get their output or error messages.
Why use the subprocess module?
In Python, sometimes we need to execute system commands or call other programs to accomplish specific tasks. For example, you may need to run a shell command, launch an external application, or interact with a command-line tool.subprocessThe module provides a safe and flexible way to handle these needs.
Compared with the earlieros.system()oros.popen()approaches,subprocessthe module provides more powerful functionality and better control. It allows you to manage the input, output, and error streams of subprocesses more finely, and can handle more complex scenarios.
Core features of the subprocess module
1. Execute external commands
subprocess.run()YessubprocessOne of the most commonly used functions in the module. It can execute an external command and wait for the command to complete. Here is a simple example:
Example
# Execute a simple shell command
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)
# Print the output of the command
print(result.stdout)
In this example,subprocess.run()executed thels -lcommand and captured the output intoresult.stdoutthe result.
2. Handle input and output
subprocessThe module allows you to control the input, output, and error streams of subprocesses. You can pass data to the standard input of a subprocess, or read data from the standard output and standard error of a subprocess. Here is an example:
Example
# Execute a command and pass input to the subprocess
result = subprocess.run(['grep', 'python'], input='hello\npython\nworld', capture_output=True, text=True)
# Print the output of the command
print(result.stdout)
In this example,subprocess.run()executed thegrep pythoncommand and passed the string'hello\npython\nworld'as input to the subprocess.
3. Handle errors
subprocessThe module also allows you to handle errors of subprocesses. If a subprocess returns a non-zero exit status code,subprocess.run()it will raise aCalledProcessErrorexception.result.returncodeYou can check
Example
try:
result = subprocess.run(['ls', 'nonexistent_file'], capture_output=True, text=True, check=True)
except subprocess.CalledProcessError as e:
print(f"Command failed with return code {e.returncode}")
print(f"Error output: {e.stderr}")
Examplesubprocess.run()In this example,ls nonexistent_fileexecuted theCalledProcessErrorcommand. Because the file does not exist, the command failed and raised a
Advanced usage of the subprocess module
1. Using the Popen class
subprocess.Popen1. Using the Popen classPopenThe class provides a lower-level interface that allows you to control subprocesses more flexibly. You can use
Example
Example
process = subprocess.Popen(['ping', 'google.com'], stdout=subprocess.PIPE, text=True)
# Start a subprocess
while True:
output = process.stdout.readline()
if output == '' and process.poll() is not None:
break
if output:
print(output.strip())
# Read the output of the subprocess
return_code = process.poll()
print(f"Process finished with return code {return_code}")
# Get the exit status code of the subprocesssubprocess.PopenIn this example,ping google.comstarted a
2. Using pipes
subprocess2. Using pipes
Example
Example
p1 = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['grep', 'py'], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)
# Use a pipe to connect two commands
output = p2.communicate()[0]
print(output)
# Get the final outputls -lIn this example,grep pythe output of the command is passed topycommand, and the final output contains
Common methods, classes, and parameters of the subprocess module
The following is a description of common methods, classes, and parameters of the Python subprocess module, including functional descriptions and examples:
Core methods of the subprocess module
| Method | Description | Example |
|---|---|---|
subprocess.run() | Execute a command and wait for completion (recommended) | subprocess.run(["ls", "-l"], capture_output=True, text=True) |
subprocess.Popen() | Create a subprocess (low-level control) | proc = subprocess.Popen(["ping", "google.com"], stdout=subprocess.PIPE) |
subprocess.call() | Execute a command and return the exit code (legacy) | exit_code = subprocess.call(["python", "--version"]) |
subprocess.check_call() | Execute a command and raise an exception on failure | subprocess.check_call(["git", "commit"]) |
subprocess.check_output() | Execute a command and return the output (legacy) | output = subprocess.check_output(["date"], text=True) |
subprocess.CompletedProcess object attributes (return object of the run() method)
| Attribute | Description |
|---|---|
args | List of command arguments executed |
returncode | Process exit status code (0 indicates success) |
stdout | Standard output content (if setcapture_output） |
stderr | Standard error content (if setcapture_output） |
Common methods/attributes of the subprocess.Popen class
| Method/Attribute | Description | Example |
|---|---|---|
poll() | Check whether the process has terminated (returnsNoneTrue to indicate running) | if proc.poll() is None: print("Running") |
wait() | Block until the process ends | proc.wait() |
communicate() | Interactive input/output | stdout, stderr = proc.communicate(input="data") |
terminate() | Send termination signal (SIGTERM) | proc.terminate() |
kill() | Forcefully terminate the process (SIGKILL) | proc.kill() |
stdin | The standard input stream of the process | proc.stdin.write("input") |
stdout | The standard output stream of the process | print(proc.stdout.read()) |
stderr | The standard error stream of the process | errors = proc.stderr.read() |
Common parameter descriptions (applicable to run() and Popen())
| Parameter | Description | Example value |
|---|---|---|
args | Command (list or string) | ["ls", "-l"]or"ls -l" |
stdin | Standard input configuration | subprocess.PIPE(pipe),None(inherit) |
stdout | Standard output configuration | subprocess.PIPE、open('log.txt', 'w') |
stderr | Standard error configuration | subprocess.STDOUT(merged into stdout) |
shell | Whether to execute via Shell | True(supports string commands) |
cwd | Working directory path | "/tmp" |
env | Custom environment variables | {"PATH": "/usr/bin"} |
timeout | Timeout (seconds) | 30 |
text | Whether input/output is a string (not bytes) | True |
Example
Execute a command and capture output:
Example
print(result.stdout) # Output: "Hello\n"
Execute complex commands via Shell:
subprocess.run("grep 'error' log.txt | wc -l", shell=True, check=True)Get the output stream in real time:Example
while True:
line = proc.stdout.readline()
if not line: break
print(line.decode().strip())
Timeout control:
Example
subprocess.run(["sleep", "10"], timeout=5)
except subprocess.TimeoutExpired:
print("Command timed out!")