While this seems trivial, it is a very important question you'll have to answer if you want to pretend that a whitelisting solution will give you 100% protection. So lets take a shot at it:
An executable file is a file which contains machine code intended to be run on the CPU.
This looks right, doesn't it? Well, not entirely. What about batch files for example? They don't contain machine code per-se, however machine code is run when they are interpreted. The same thing can be said about Word documents, Excel spreadsheets, HTML files, etc. So lets extend the definition:
An executable file is a file which contains machine code intended to be run on the CPU or (during processing) results in instructions being executed on the CPU.
Congratulation! You just defined every file on the system. The processing of any file results in some instructions being executed. Even if we abstract away from software vulnerabilities, you are in a situation where you need to ask yourself the question: is it possible that the application will generate for some input file an instruction sequence which is undesirable?
To take this discussion to a more down-to-earth level, here is an example (taken from this Sophos blog post): imagine a shortcut (.lnk) pointing at the default command interpreter (cmd.exe) resulting in downloading an executable from an FTP site and running it. Fortunately whitelisting products have a chance to step in at the second stage and block it (also, some solutions, like Software Restriction Policies, treat shortcut files as executables).
However I can offer you even a more frightening file format: .msi (Microsoft Installer). This fileformat is intended by design to instruct the operating system to perform deep changes (copy/move files, change registry entries, run programs, etc). Even more problematic is the fact that Windows (Vista) assumes that setup programs need administrative privileges. The execution itself is performed by a service running with high privileges. Also, the MSI format allows for bundling and dynamically loading arbitrary DLLs (which may or may not be traced by the particular whitelisting product - many of them choose not to track it to avoid overwhelming the user with prompts).
Creating an MSI file is rather simple. For the following example I choose the WiX toolkit (both because it is easily installable and because it is open source - if you want to run it, you need the .NET framwork version 2.0). Save the following text as a file named "product.wxs":
<?xml version='1.0'?> <Wix xmlns='http://schemas.microsoft.com/wix/2003/01/wi'> <Product Id='12345678-1234-1234-1234-123456789012' Name='Test Package' Language='1033' Version='126.96.36.199' Manufacturer='Microsoft Corporation'> <Package Id='12345678-1234-1234-1234-123456789012' Description='My first Windows Installer package' Comments='This is my first attempt at creating a Windows Installer database' Manufacturer='Microsoft Corporation' InstallerVersion='200' Compressed='yes' /> <Directory Id='TARGETDIR' Name='SourceDir'> <Component Id='MyComponent' Guid='12345678-1234-1234-1234-123456789012' /> </Directory> <Feature Id='MyFeature' Title='My 1st Feature' Level='1'> <ComponentRef Id='MyComponent' /> </Feature> <Binary Id="wixca" src="wixca.dll"/> <Property Id="QtExecCmdLine" Value='"c:\windows\system32\cmd.exe" /c echo 1 > c:\out.txt' /> <CustomAction Id="QtExec" BinaryKey="wixca" DllEntry="CAQuietExec" Execute="immediate" Return="check"/> <InstallExecuteSequence> <Custom Action="QtExec" Sequence='1'/> </InstallExecuteSequence> </Product> </Wix>
Now run the following two commands:
candle product.wxs light product.wxsobj
And voila!, you have an .MSI file which - when executed - performs a command in the background silently.
Finally, I would like to outline a detailed attack scenario:
- A company/institution uses a whitelisting product to protect its PC's
- A user is tricked into running an MSI file. Now this user may or may not have administrative privileges (in which case the MSI file can be marked as not requiring elevated privileges)
- The instructions in the MSI file use the built-in Windows executables (which are on the whitelist) to perform any of the following actions:
- Add a user with a predetermined password (acting as a perimeter weakening malware)
- Lower the ActiveX security settings for IE.
- Opening a preprogrammed webpage. This can have two functions: signaling the existence of a new compromised host and/or containing the "stage 2" of the exploitation which can execute now, given that the security barriers have been lowered.
Conclusion: as many have said before me, there is no such thing as perfect security. What you can do, is to select a lockdown level appropriate for your activity and apply it. Using a lower than needed security level is irresponsible, however using a higher than needed level can prevent you from from doing your work. Be aware of the options and stay safe!