On spaces in the paths of programs and files on Windows

Today while debugging the build of Python 3.2 with Visual Studio, I ran into a caveat with invoking programs on Windows from directories with spaces in their name. The caveat applies equally to the standard C system call, to Windows batch files and cmd /c, and to any scripting/programming language with an interface to system, such as Python.

Suppose I have a program sitting in a directory with spaces in its name. For the sake of example let's take a trivial batch script called dumpfile.bat:

type %1

All it does is print the contents of the file passed to it as the first argument. I will place it in D:\temp\Spaces dir. How do I invoke this script with system? By quoting around the executable name:

import os

cmdline = r'"D:\temp\Spaces dir\dumpfile.bat" paths_with_spaces.py'
print ">> ", cmdline
os.system(cmdline)

This script is named paths_with_spaces.py, so it asks dumpfile.bat to print itself. And this works as expected.

Now, suppose I want to invoke dumpfile.bat on some other file, which also has spaces in its full path. For demonstration I will place a simple text file named file.txt also in D:\temp\Spaces dir.

Then, I write:

import os

cmdline = r'"D:\temp\Spaces dir\dumpfile.bat" "D:\temp\Spaces dir\file.txt"'
print ">> ", cmdline
os.system(cmdline)

Note that I placed the argument filename in quotes as well, since it also contains spaces.

Unfortunately, I get an error:

D:\temp\Spaces' is not recognized as an internal or external command,
operable program or batch file.

Using procmon, I can find out that the os.system call actually invokes:

C:\WINDOWS\system32\cmd.exe /c "D:\temp\Spaces dir\dumpfile.bat" "D:\temp\Spaces dir\file.txt"
>

under the hood. Running this manually from the command-line, I get the same error. So this is a problem with the Windows cmd.exe processor, not my os.system call.

Alas, cmd.exe is indeed exceptionally dumb. Here's a snippet from its documentation:

If /C or /K is specified, then the remainder of the command line after the switch is processed as a command line, where the following logic is used to process quote (") characters:

  1. If all of the following conditions are met, then quote characters on the command line are preserved:
  • no /S switch
  • exactly two quote characters
  • no special characters between the two quote characters, where special is one of: &<>()@^|
  • there are one or more whitespace characters between the the two quote characters
  • the string between the two quote characters is the name of an executable file.
  1. Otherwise, old behavior is to see if the first character is a quote character and if so, strip the leading character and remove the last quote character on the command line, preserving any text after the last quote character.

I emphasized the condition which my example fails to fulfill. The solution, it turns out, is to induce the behavior described in (2), by wrapping the whole command-line invocation in another pair of quotes:

import os

cmdline = r'""D:\temp\Spaces dir\dumpfile.bat" "D:\temp\Spaces dir\file.txt""'
print ">> ", cmdline
os.system(cmdline)

This works. Yet another little trick of Windows's wonderful command line.

Resources:

For comments, please send me an email.