Found by a Codex global repository scan of deepmodeling/dpdispatcher at commit 98a9e08.
Problem
Several HDFS operations build shell command strings from user-controlled paths and run them with shell=True. The SGE submit path also interpolates the remote directory and script filename without quoting.
Relevant code
|
cmd = f"hadoop fs -test -e {uri}" |
|
try: |
|
cmd = f"hadoop fs -rm -r {uri}" |
|
try: |
|
ret, out, err = run_cmd_with_all_output(cmd) |
|
cmd = f"hadoop fs -copyFromLocal -f {local_path} {to_uri}" |
|
try: |
|
ret, out, err = run_cmd_with_all_output(cmd) |
|
cmd = f"hadoop fs -copyToLocal {remote} {local_path}" |
|
|
|
try: |
|
ret, out, err = run_cmd_with_all_output(cmd) |
|
cmd = f"hadoop fs -text {uri}" |
|
try: |
|
ret, out, err = run_cmd_with_all_output(cmd) |
|
cmd = f"hadoop fs -mv {from_uri} {to_uri}" |
|
try: |
|
ret, out, err = run_cmd_with_all_output(cmd) |
|
def run_cmd_with_all_output(cmd, shell=True): |
|
with subprocess.Popen( |
|
cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.PIPE |
|
) as proc: |
|
out, err = proc.communicate() |
|
ret = proc.returncode |
|
return (ret, out, err) |
|
script_file_dir = self.context.remote_root |
|
stdin, stdout, stderr = self.context.block_checkcall( |
|
"cd {} && {} {}".format(script_file_dir, "qsub", script_file_name) |
Impact
Paths with spaces fail. Paths or URIs containing shell metacharacters can change the command that is executed.
Suggested fix
Prefer argv-list subprocess calls with shell=False. Where a remote shell string is unavoidable, apply shlex.quote() to every dynamic path, URI, and script filename.
Found by a Codex global repository scan of deepmodeling/dpdispatcher at commit 98a9e08.
Problem
Several HDFS operations build shell command strings from user-controlled paths and run them with
shell=True. The SGE submit path also interpolates the remote directory and script filename without quoting.Relevant code
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 17 to 18 in 98a9e08
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 40 to 42 in 98a9e08
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 82 to 84 in 98a9e08
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 104 to 107 in 98a9e08
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 122 to 124 in 98a9e08
dpdispatcher/dpdispatcher/utils/hdfs_cli.py
Lines 137 to 139 in 98a9e08
dpdispatcher/dpdispatcher/utils/utils.py
Lines 78 to 84 in 98a9e08
dpdispatcher/dpdispatcher/machines/pbs.py
Lines 233 to 235 in 98a9e08
Impact
Paths with spaces fail. Paths or URIs containing shell metacharacters can change the command that is executed.
Suggested fix
Prefer argv-list subprocess calls with
shell=False. Where a remote shell string is unavoidable, applyshlex.quote()to every dynamic path, URI, and script filename.