Making Executable Commands on Windows¶
Introduction¶
The user interface for any program starts with the command you use to
execute it. For example, to start Python, you run the command python
.
But it’s not as simple as that - the operating system needs to work out
what code to run, based on that command, and that’s where the complications
start.
Because things are complicated, I’ll give the punchline here. If you don’t want to read the gory details, feel free to just follow this advice. But if you feel yourself thinking “but what about...” or “surely...”, then please read the rest of this article. You’re probably wrong! There are a lot of solutions out there that work 99% of the time - but that 1% can make life really awkward for your users.
So, the advice:
Note
Always create an exe
file to run your main program.
How to create an exe¶
The biggest reason people don’t want to create an exe
file for their
program is that it’s hard. You really need a C compiler to build exes, and
most Windows users don’t have that. Also, editing exe
files means
editing the source and rebuilding. Not simple, especially compared to
a quick edit of your Python script (or whatever).
But typically, programming environments offer a means of building an
executable to “wrap” scripts. For Python, you can define setuptools
entry points in your setup.py
script, and these will be converted
into exe
files when you install your application.
For a more general solution, I’m in the process of developing Shimmy, an application to build executable “Shims” which can run any command you want. It’s a work in progress, but it is usable as it stands.
Why must I use an exe?¶
OK, so onto the meat of this article. Why do you have to use an executable?
The simple answer is that the Windows kernel treats exe
files specially.
No other format of file is handled as a first-class citizen. There are lots
of APIs that deal with other forms of “executable”, but they all end up needing
an exe
in the end.
The fundamental Windows API for creating a new process is CreateProcess
. It
takes the filename of an executable, a command line [1], and a few other
parameters that don’t really matter here. The filename can be omitted, if so
the command line is parsed for the first token, which is treated as the
filename.
The documentation for CreateProcess
explains the details, but in essence
Windows only looks for files with the extension .exe
when searching for a
command. You can specify the exact filename of the command, in which case you
can put any extension you like. But if you do, Windows will tell you the file
is not a valid Windows executable. So don’t do that.
There’s one exception - if you specify a Windows Batch file (.bat
or
.cmd
extension), Windows appears to run it successfully. But this isn’t
documented - indeed, the documentation explicitly says
To run a batch file, you must start the command interpreter; set lpApplicationName to cmd.exe and set lpCommandLine to the following arguments: /c plus the name of the batch file.
So not only is specifying a .bat
file undocumented, it’s actually
documented that you should use a different method of handling that case.
It works, but it’s probably going to bite you at some stage. And anyway,
the search process doesn’t try a .bat
extension, so your users have to
explicitly include the extension when running your command, which probably
isn’t what you want (and definitely isn’t a portable approach).
Are there any other issues?¶
If that wasn’t enough to make the point, here are some other issues commonly
seen when using alternatives to exe
wrappers:
- If you use a
.bat
file, you cannot call the command from another batch file. Calling one batch file from another transfers control (a “goto” rather than a subroutine call), which causes your original batch file to appear to terminate unexpectedly with no error. You have to use the “CALL” batch file command to call one batch file from another. And you (or one of your users) will probably forget... - If you interrupt a batch file with Ctrl-C you get the dreadful “Terminate batch job (Y/N)?” prompt. This doesn’t always happen, but do you really want your users to see that?
- If you use a
.py
file (or similar) and rely onPATHEXT
to make the shell know your file is executable, it will work fine in a command shell (cmd or powershell) but will fail when someone tries to call your command fromsubprocess.Popen
, or task scheduler, or any other means that usesCreateProcess
directly.
But I still don’t want to go to the trouble of using an exe!¶
Well, honestly, that’s fine. If it’s a script for your own use, or maybe for
your project, team, or company, then go for it. As long as you understand the
limitations, using PATHEXT
to make .py
files executable is a fine
solution. And some people even think .bat
files are a pretty neat thing ;-)
But be very careful before distributing your code to the wider world. Someone
out there will need to use it in a context where nothing but an exe
suffices. And they’ll blame you. You can point them to workarounds like Shimmy
(mentioned above) but why not just provide something robust in the first place?
Some final notes¶
This article has been pretty hard-line. Sorry for that, but I’ve spent a lot
of time debating this issue on various mailing lists, and I started as one
of the people arguing against needing exe
files. I was persuaded by many
people coming up with examples of issues with basically every solution I
proposed. So the advice here is pretty well battle-tested.
In the Python world, the issue of executable scripts was a huge topic of debate
in the early days of distutils. Even when setuptools introduced the idea of
console entry points, it wasn’t universally accepted at first. So plenty of
older projects still use other solutions (the scripts
argument in
distutils’ setup()
function). Don’t hate them for it - often such projects
were spending time maintaining Windows batch files when they didn’t use Windows
themselves, for which they should be commended - but feel free to suggest to
them that it’s probably time that they moved over to setuptools entry points.
Footnotes
[1] | Unix programmers take note, CreateProcess does not take a list of
arguments, and much of the fun in cross-platform handling of
subprocesses comes from hiding this fundamental difference between the
two operating systems. |