June 10, 2017: security problem when using "shell=True" in the Python "subprocess" module
The subprocess Python module allows to replace a series of obsolete Python functions (see
here):
os.system
os.spawn*
os.popen*
popen2.*
commands.*
But subprocess yields a security problem when using "shell=True" as argument of one of its functions. My goal here is to explain why, analysing the Python documentation content.
Security problem and its solution
One reason for using “shell=True” is to benefit from shell features (e.g. globbing). Except the security reason we are going to discuss, there are two reasons to avoid use of "shell=True". The first one is that Python offers replacement for many shell features (see
here:
If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory. However, note that Python itself offers implementations of many shell-like features (in particular, glob, fnmatch, os.walk(), os.path.expandvars(), os.path.expanduser(), and shutil).
The second reason is that
[...] in addition to that, the overhead of starting a shell to start the program you want to run is often unnecessary and definitely silly for situations where you don’t actually use any of the shell’s functionality.
But the main problem with “shell = True” is a security problem. The documentation gives the example:
>>> from subprocess import call
>>> filename = input("What file would you like to display?\n")
What file would you like to display?
non_existent; rm -rf /
>>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
The problem comes from ’;’ that allows to separate shell command. Any part of the input string following ’;’ will be executed as is. By the way, note that this would work identically if the user gives an existent file as first part of the string. Of course, the problem exists still more strongly if we invite the user to execute a command of his choice, since there would be even no need to use “;” in the input string:
>>> command_to_execute = input("What command would you like to execute?\n")
What command would you like to execute?
rm -rf /
>>> call( command_to_execute, shell=True) # Uh-oh. This will end badly...
But in this case this is not a surprise to get such a possible nasty behavior. In the former case (using “cat”) this is not expected at all, and this is the security problem.
To solve the problem, it is sufficient to use shell=False (the default):
>>> filename = input("What file would you like to display?\n")
What file would you like to display?
non_existent
>>> call( [ "cat", filename ], shell=False )
Indeed a “;” in filename would be understood by Python as part of a file name, not as a command separator since the string corresponding to the concatenation of the sequence [ "cat", filename ] is never passed to the shell. Instead, the string is passed as first argument to the "cat" program. Thus “rm -rf” has no chance to be executed (we get an error instead). This solves the security problem.
Note on syntax when shell = False
When using shell = False, the string must be the name (possibly the relative or the full path) of a program (see
here), without arguments:
If args is a string, the string specifies the command to execute through the shell.
If arguments have to be provided, a list has to be given:
If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.
So the string has to be correctly split for the command to be executed (cf
here):
>>> import shlex
>>> shlex.split("rm -f cou")
[’rm’, ’-f’, ’cou’]
>>> call( shlex.split("rm -f cou") )
0
For instance, the following does not work:
>>> call( [ ’rm’, ’-f cou’ ] )
rm: invalid option — ’ ’
Try ’rm --help’ for more information.
1
since it is equivalent to
$ rm "-f cou"
rm: invalid option — ’ ’
Try ’rm --help’ for more information.
The problem is that ’-f’ is not interpreted as a command line option specifier. The following line will not work as there is no program in $PATH called "rm -f cou":
>>> call( "rm -f cou" ) # does not work
[...]
OSError: [Errno 2] No such file or directory
When using “shell=False”, it will be necessary to use Python glob() function, since the shell is not used for that:
>>> call( shlex.split( "rm co*" ) )
rm: cannot remove ’co*’: No such file or directory
1
Conclusion
Executing shell commands that incorporate unsanitized input from an untrusted source makes a program vulnerable to shell injection, a serious security flaw which can result in arbitrary command execution. For this reason, the use of shell=True is strongly discouraged in cases where the command string is constructed from external input.
In fact, this should be avoided even for input constructed in a program, since it decreases the risk of unwanted data removal.
