MSBuild Task, External programs and embedded quotes in the command-line

All right, here’s today’s topic…”How to waste time trying to use the MSBuild task to run an external program whose command-line contains embedded quotes”.

Need: I was trying to call to a custom console application that performs code-generation for the passed project. Due to the application needing fully-qualified file paths, I needed to place quotes around the parameters. Here are the relevant snippets from the TFS Team Build .proj file:

<CodeGenUtilityPath>“C:Program FilesCodeGenerationUtilityCodeGenerationUtilityCodeGenerationUtility.exe”CodeGenUtilityPath>

<Exec
Condition=‘@(SolutionToBuild->’%(HasCodeGeneration)’)’ == ‘true’
Command=$(CodeGenUtilityPath) /f=”%(SolutionToBuild.RootDir)%(SolutionToBuild.Directory)” /w=”$(WorkspaceName)” /l=”$(PathToLog)”
/>

Problem: When run, the following error was encountered:

Task “Exec”

Command:
“C:Program FilesCodeGenerationUtilityCodeGenerationUtility.exe”
/f=”C:ProjectsCommonContracts_2_0 ” /w=”VPC-XXXX_XXXX_Common SnapBuild” /cl=”Common SnapBuild_20060525.7″ /r /l=”
\Test4filedropCommon SnapBuild_20060525.7CodeGenLog.xml”

CodeGenerationUtility
=========================

Given a path to a folder, retrieves the list of files that match the pattern *Codegen?.*, checks them out of VSS, generates
code from the .xml files and checks them all back in.
– – – – – –

‘/f’ is not recognized as an internal or external command, operable program or batch file.

‘\Test4filedropCommon’ is not recognized as an internal or external command, operable program or batch file.

‘” ‘ is not recognized as an internal or external command,operable program or batch file.

Issue: As you can see, the command parser is correctly interpreting the quotes around the executable path (as shown by the console app’s help being displayed), but it is not passing the app’s command-line args to it. It is instead trying to run these as commands. It is also splitting the command line arguments without regard to the passed quotes around the paths. It sees the space within \Test4filedropCommon SnapBuild_20060525.7CodeGen.xml as an argument separator. This causes cmd.exe to believe that these are 2 separate arguments/commands. Ok, so how do we deal with this?

First off, let’s have a primer on the task (from MSDN):

This task is useful when a specific MSBuild task for the job that you want to perform is not available. One disadvantage of using the Exec task rather than a more specific task is that it cannot gather output from the tool or command that it runs.

The Exec task calls cmd.exe instead of directly invoking a process.

The last line is what concerns us. When you shell out to cmd.exe, you have a new set of rules to follow with regard to quotes around strings in your command-line.

Here is the relevant parts of the help text from running cmd.exe /?:

Starts a new instance of the Windows XP command interpreter

CMD [/A /U] [/Q] [/D] [/E:ON /E:OFF] [/F:ON /F:OFF] [/V:ON /V:OFF]
[[/S] [/C /K] string]

/C Carries out the command specified by string and then terminates
/K Carries out the command specified by string but remains

If /C or /K is specified, then the remainder of the command line after the switch is processed as a command line, where the following logic is used to process quote (“) characters:

1. If all of the following conditions are met, then quote characters on the command line are preserved:
– no /S switch
– exactly two quote characters
– no special characters between the two quote characters,
where special is one of: &<>()@^
– there are one or more whitespace characters between the
the two quote characters
– the string between the two quote characters is the name
of an executable file.

2. Otherwise, old behavior is to see if the first character is
a quote character and if so, strip the leading character and
remove the last quote character on the command line, preserving
any text after the last quote character.

So it looks to me like the task is calling cmd.exe with the /c option and thus we are seeing the behavior listed in #2 above. This gives us a totally messed up command line to process.

So how do we fix it????

Solution: We follow the old adage “If at first you don’t succeed, give up!” Actually we give up on forcing the MSBuild project to trigger the external app through the task and we build our own Custom Task! to fire this thing off. There a number of benefits to using a custom task for this, including:

  1. We have finer grain control over the running process and can send output to the build log. The task does not have this feature.
  2. We can be more descriptive in our project file as to what we are passing. So instead of seeing
    /f=”C:ProjectsContracts” /l=”\SomeServerfiledropCodeGenLog.xml”
    we can now see

    C:ProjectsContracts
    \SomeServerfiledropCodeGenLog.xml


  3. We can pass data back to the MSBuild engine upon task completion if we want to.

These are really great reasons to build your own custom task. Another reason is that it is really simple to do. There are a number of blogs and sites out there that describe the process. Instead of adding it to this post, I’m giving you a link to steps I used from Bart De Smet’s blog entry on Custom Build Tasks.

Conclusion: The task is great of you need to run a built-in command, such as dir, copy, or even cacls, but it isn’t robust enough to handle a large set of command-line arguments with embedded spaces. If you don’t have the embedded spaces, then you should be fine using . If you need a richer method of running an external process, offering logging, output parameters and descriptive metadata entries; then a custom task is the way to go.

TFSVC API – Workspace.Get() and GetOptions Enum

I’m writing a console app to do code generation during our Continuous Integration process. To do a Get from TFS Version Control, I’m hitting its managed API, specifically the Workspace object. The Workspace.Get() method has a number of overloads. The one I’m concerned with takes a local or server path and VersionSpec wrapped in a GetRequest object and a GetOptions parameter.

I want to force a get in all cases as well as ensure that I will overwrite any existing files on the local file system. Normally I would expect there to be a GetOptions entry for GetAll and Overwrite combined. There isn’t, but each of these values does exist on its own within the enumeration. What I did notice is that the values in the enumeration look a lot like bitmask values.

GetOptions Enumeration

NameValue
None0
Overwrite1
GetAll2
Preview4

So you can ue bitwise operations to combine these values for use in the Get() operation.

The code snippet below will perform the Get() which forces the TFS client to get the file every time and also to overwrite any writable version in the local folder:

Dim request as GetRequest
request=New GetRequest({local or server path}, VersionSpec.Latest)
wkspace.Get(request, _
CType(GetOptions.GetAll + GetOptions.Overwrite, GetOptions))


Since the GetOptions enum is a bitmask you can use the approach above or use a Bitwise-OR to combine these values.

wkspace.Get(request, CType(GetOptions.GetAll OR GetOptions.Overwrite, GetOptions))))

How to install TFS Build Server on build machine so it uses a Service Account

I wanted to create a remote build server to allow for Continuous Integration in my project. The TFS Build service needed to run under a “Service Account” (a non-interactive account with limited privileges). To install and configure the machine I followed the following process:

  1. Sign on to the machine with an account with Admin privileges
  2. Give “Service Account” membership in the local administrators group
  3. Log off and login into the server with the “Service Account”
  4. Install the TFS Build Service – which will provide the appropriate and limited permissions to the “Service Account”
  5. Log out
  6. Log back in with an account with Admin privileges and remove the “Service Account” from the administrators group.

This process has been working fine for me for the past 2 weeks

TFS Team Build can run forever

Problem: It is possible for TFS Server to lose track of a remote team build and believe that it is still running even if the build machine has not raised an event back to the TFS Server in a very long time (>36 hrs). It seems that the default timeout for a build is for the TFS server to wait forever.

Scenario: I was testing a team build that went awry. My build script deleted a large portion of the C: folder and hosed the machine (stop laughing). To stop the build (prior to learning that there is a TFSBUILD STOP command), I shutdown the build server. The machine was hosed and needed to be re-imaged. About 36 hrs later, I reviewed “All Builds” within Team Explorer and noticed that the TFS Server thought that the build was still running. So even after 36 hrs+, TFS hadn’t failed the build on a timeout.

Fix: I had to use the “TFSBuild.exe Stop” command to inform the TFS server that the build should be aborted. You should also run tf workspaces owner:* server:[MYSERVER] on the Build Server and the Client machine that initiated the build to update the workspace cache which will clean up any stray workspaces.

Comment: If you have a failure of a remote build machine during a build, you need to ensure that the build is cancelled on the TFS server or you may have a workspace collision when a new build is run when the machine is back up as the workspace’s local path is considered to be still “in use”. this “in use” status comes from the Build Machine or Client’s workspace cache being out of sync with the Team Foundation Server’s database. the tf workspaces command above will update the locaal cache from the server and clean this up.

Mike Ruminer has posted a step-by-step listing of the entire event on his blog.