Claus Brod: Don't quote me on this (18 Mar 2006)

topic end
<!--
   * Set TOPICTITLE = Claus Brod: Don't quote me on this (18 Mar 2006)
-->

<style type="text/css">
pre {background-color:#ffeecc;}
</style>

%STARTINCLUDE%
<a name="18"></a>
---+++ [[BlogOnSoftware20060318][Don't quote me on this]] (18 Mar 2006)

Let us assume that I'm a little backward and have a peculiar fondness for the
DOS command shell. Let us further assume that I also like blank characters in pathnames.
Let us conclude that therefore I'm hosed.

But maybe others out there are hosed, too. Blank characters in pathnames are not exactly my
exclusive fetish; others have joined in as well (=C:\Program Files=, 
=C:\Documents and Settings=). And when using software, you might be running
=cmd.exe= without even knowing it. Many applications can run external helper
programs upon user request, be it through the UI or through the application's
macro language.

<img src="%ATTACHURL%/directory.png" align="right" />
The test environment is a directory =c:\temp\foo bar= which contains
=write.exe= (copied from the Windows system directory) and two text files, one of
them with a blank in its filename.

Now we open a DOS shell:

<blockquote>
<verbatim>
C:\>dir c:\temp\foo bar
 Volume in drive C is IBM_PRELOAD
 Volume Serial Number is C081-0CE2

 Directory of c:\temp

File Not Found

 Directory of C:\

File Not Found

C:\>dir "c:\temp\foo bar"
 Volume in drive C is IBM_PRELOAD
 Volume Serial Number is C081-0CE2

 Directory of c:\temp\foo bar

03/18/2006  03:08 PM    <DIR>          .
03/18/2006  03:08 PM    <DIR>          ..
01/24/2006  11:19 PM             1,516 foo bar.txt
01/24/2006  11:19 PM             1,516 foo.txt
03/17/2006  09:44 AM             5,632 write.exe
               3 File(s)          8,664 bytes
               2 Dir(s)  17,448,394,752 bytes free
</verbatim>
</blockquote>

Note that we had to quote the pathname to make the =DIR= command work.
Nothing unusual here; quoting is a fact of life for anyone out there
who ever used a DOS or UNIX shell.

Trying to start =write.exe= by entering =c:\temp\foo bar\write.exe= in the
DOS shell fails; again, we need to quote:

<blockquote>
<verbatim>
C:\>"c:\temp\foo bar\write.exe"
</verbatim>
</blockquote>

And if we want to load =foo bar.txt= into the editor, we need to quote
the filename as well:

<blockquote>
<verbatim>
C:\>"c:\temp\foo bar\write.exe" "c:\temp\foo bar\foo bar.txt"
</verbatim>
</blockquote>

Still no surprises here.

But let's suppose we want to run an arbitrary command from our application
rather than from the command prompt. The C runtime library provides the
[[http://msdn2.microsoft.com/en-us/library/277bwbdz.aspx][system()]]
function for this purpose. It is well-known that under the hood
=system= actually runs =cmd.exe= to do its job.

<blockquote>
<verbatim>
#include <stdio.h>
#include <process.h>

int main(void)
{
  char *exe = "c:\\temp\\foo bar\\write.exe";
  char *path = "c:\\temp\\foo bar\\foo bar.txt";

  char cmdbuf[1024];
  _snprintf(cmdbuf, sizeof(cmdbuf), "\"%s\" \"%s\"", exe, path);

  int ret = system(cmdbuf);
  printf("system(\"%s\") returns %d\n", cmdbuf, ret);
  return 0;
}
</verbatim>
</blockquote>

When running this code, it reports that =system()= returned 0, and =write.exe=
never starts, even though we quoted both the name of the executable and
the text file name.

What's going on here? =system()= internally runs =cmd.exe= like this:

<blockquote>
<verbatim>
  cmd.exe /c "c:\temp\foo bar\write.exe" "c:\temp\foo bar\foo bar.txt"
</verbatim>
</blockquote>

Try entering the above in the command prompt: No editor to be seen anywhere!
So when we run =cmd.exe= programmatically, apparently it parses its input
differently than when we use it in an interactive fashion.

I remember this problem drove me the up the freakin' wall when I first encountered
it roughly two years ago. With a lot of experimentation, I found the right
magic incantation:

<blockquote>
<verbatim>
  _snprintf(cmdbuf, sizeof(cmdbuf), "\"\"%s\" \"%s\"\"", exe, path);
  // originally: _snprintf(cmdbuf, sizeof(cmdbuf), "\"%s\" \"%s\"", exe, path);
</verbatim>
</blockquote>

Note that I quoted the whole command string another time! Now the executable
actually starts. Let's verify this in the command prompt window: Yes, something
like =cmd.exe /c ""c:\temp\foo bar\write.exe" "c:\temp\foo bar\foo bar.txt""=
does what we want.

I was reminded of this weird behavior when John Scheffel, long-time user of our flagship
product <nop>OneSpace Designer Modeling and maintainer of the international
<nop>CoCreate user forum, [[http://cocreateusers.org/forum/showthread.php?p=15956#post15956][reported]] funny
quoting problems when trying to run executables from our app's built-in Lisp interpreter.
John also found the solution and documented it in a Lisp version.

Our Lisp implementation provides a function called =sd-sys-exec=, and you need to
invoke it [[http://www.fromconceptiontobirth.com/wotd/index.pperl?date=19981002][thusly]]:

<blockquote>
<verbatim>
(setf exe "c:/temp/foo bar/write.exe")
(setf path "c:/temp/foo bar/foo bar.txt")
(oli:sd-sys-exec (format nil "\"\"~A\" \"~A\"\"" exe path))
</verbatim>
</blockquote>

Kudos to John for figuring out the Lisp solution. Let's try to decipher all those quotes
and backslashes in the =format= statement.

Originally, I modified his solution slightly
by using =~S= instead of =~A= in the =format= call and thereby saving one level
of explicit quoting in the code:

<pre>
  (format nil "\"~S ~S\"" exe path))
</pre>

This is much easier on the eyes, yet I overlooked that the =~S= format specifier
not only produces enclosing quotes, but also escapes any backslash characters
in the argument that it processes. So if =path= contains a backslash (not quite
unlikely on a Windows machine), the backslash will be doubled. This works
surprisingly well for some time, until you hit a UNC path which already starts
with two backslashes. As an example, =\\backslash\lashes\back= turns into
=\\\\backslash\\lashes\\back=, which no DOS shell will be able to grok anymore.

John spotted this issue as well. Maybe he should be writing these blog entries,
don't you think? :-)

From those Lisp subtleties back to the original problem:
I never <i>quite</i> understood why the extra level of quoting is necessary for
=cmd.exe=, but apparently, others have been in the same mess before. For example,
check out
[[http://www.tug.org/ftp/texlive/Contents/live/xemtex/lisp/win32-native.el][this XEmacs code]]
to see how complex correct quoting can be. See also an online version of
the [[http://thesystemguard.com/TheGuardBook/CCS-Ext/helptext/Cmd-K3.txt.htm][help pages for <nop>CMD.EXE]]
for more information on the involved quoting heuristics applied by the shell.

PS: A very similar situation occurs in <nop>OneSpace Designer Drafting as well
(which is our 2D CAD application). To start an executable =write.exe= in a directory
=c:\temp\foo bar= and have it open the text file =c:\temp\foo bar\foo bar.txt=,
you'll need macro code like this:

<blockquote>
<pre>
LET Cmd '"C:\temp\foo bar\write.exe"'
LET File '"C:\temp\foo bar\foo bar.txt"'
LET Fullcmd (Cmd + " " + File)
LET Fullcmd ('"' + Fullcmd + '"')  { This is the important line }
RUN Fullcmd
</pre>
</blockquote>

Same procedure as above: If both the executable's path and the path of
the data file contain blank characters, the whole command string which
is passed down to =cmd.exe= needs to be enclosed in an additional
pair of quotes...

PS: See also http://blogs.msdn.com/b/twistylittlepassagesallalike/archive/2011/04/23/everyone-quotes-arguments-the-wrong-way.aspx
and http://daviddeley.com/autohotkey/parameters/parameters.htm

%COMMENT{type="below" nonotify="on"}%http://xkcd.com/1638/

-- Main.ClausBrod - 27 Mar 2016



---

%STOPINCLUDE%
to top
End of topic
Skip to action links | Back to top
You are here: Blog > BlogOnSoftware20060318

r1.9 - 12 Jun 2017 - 10:45 - ClausBrod to top
Blog