User Story: Using the project build process in Visual Studio, compress project output quickly (using powershell)
Assembly Loading & Powershell from MSBuild (Example Application)
In my series on Assembly Loading, I used Powershell from MSBuild in order to compress assemblies during the build of the project. One of the handy tools I've come across is using Powershell from MSBuild to speed up development tasks or deployments. Using the same example application we'll cover how to run Powershell scripts from Visual Studio's MSBuild process.
Its unfortunate that MSBuild doesn't have a native feel to execute Powershell scripts from its pipeline. But even with a slight gap between the two technologies, its well worth using Powershell scripts inside Visual Studio / MSBuild.
Tools:
PowerGUI Script Editor. For those who prefer a stand-alone powershell editor.
– or –
PowerShell Tools for Visual Studio. For those who prefer the Visual Studio experience.
The project also makes use of 7zip (7za.exe, command line utility), which is included in the sample project.
Task 1: Create script to compress project output:
In the example project, Public.Dependency AfterBuild MSBuild event is setup to be called by including the Target file in the csproj file.
Edit the csproj to include the custom Targets file:
Scroll to the bottom of the csproj, and add a reference to a target file we're going to use to invoke our powershell script.
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="$(ProjectDir)..\Public.Process\Build\Public.Resource.AfterBuild.Target" />
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
</Project>
Note the use of relative paths, this is just so we can centralize the build scripts (and as a reminder to create them so they can be reusable).
Warning: Visual Studio has an issue using the Import Project=[targets file] where it only loads the referenced file when the solution is first loaded. So to updating the file requires closing and re-opening the solution (which is quick as long as you don't close the instance of Visual Studio).
Once you've edited the csproj, reload the project:
At this point, its worthwhile to stub out the Target file (which was already created, but empty) and named Public.Resource.AfterBuild.Target off inside the Build folder of Public.Process.
The stub for Public.Resource.AfterBuild.Target as follows:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<Target Name="AfterBuild">
<CallTarget Condition="'$(Configuration)'=='Compression'" Targets="CompressOutput"/>
</Target>
<!--- ... -->
<Target Name="CompressOutput">
<Message Text="Compressing Output Assembly ... $(AssemblyName).dll ==> $(AssemblyName).dll.zip" Importance="High" />
<!--<Exec Command="powershell..." />-->
</Target>
</Project>
By setting the Condition parameter for CallTarget, we can conditionally call the powershell script only when I have the build configuration set to Compression.
Task 2: Create a powershell script to (quickly) compress project output (avoid recompression if the assembly is newer)
We now have the call stubbed out inside our Target file, we need a powershell script that actually does the compression. We want to start with a one line call, with parameters to the powershell script. If you've worked with MSBuild before, you may have noticed that the <exec command="" /> (command line) is a bit clunky due to all the escape characters required. So as apart of this task, we need to make the powershell call as concise as possible (so we can maintain the script and not the command).
Warning: If you get the error 'File [XXX] cannot be loaded because the execution of scripts is disabled on this system. Pl ease see "get-help about_signing" for more details.' You need to likely change your execution policy. Run the command:
Get-ExecutionPolicy
Which will echo out the current execution policy (likely set to 'Restricted' if you or your workstation is new to powershell). To allow the script to run execute:
Set-ExecutionPolicy RemoteSigned
For starters, we need a script that takes the file to compress, the output path, and the path to the executable to zip the assembly. For sanity's sake, using a separate file to execute our test command makes the most sense:
Public.Process\Powershell\RunCompression.ps1 (calls .\Compress.ps1):
$scriptpath = $MyInvocation.MyCommand.Path $dir = Split-Path $scriptpath # Write-host "Working Directory: $dir" Push-Location $dir . "$dir\Compress.ps1" $inputFile = "Public.Dependency.dll"; $outputFolder = "$dir\..\..\Public.Dependency\bin\Compression" $exeZipLocation = "$dir\..\Utilities\7za.exe"; Compress -inputFile $inputFile -outputFolder $outputFolder -exeZipLocation $exeZipLocation; # revert to original directory Pop-Location
Next, we need to setup Compress.ps1 — . "$dir\Compress.ps1" includes the script inside RunCompression.ps1, once included, we can call the method Compress -inputFile $inputFile -outputFolder $outputFolder -exeZipLocation $exeZipLocation; erroring on the side of being explicit with most paths (these scripts need to be portable enough to work on a build server), our three variables are passed into Compress.
Public.Process\Powershell\Compress.ps1:
function Compress($inputFile, $outputFolder, $exeZipLocation)
{
Set-Location $outputFolder;
$resourceExists = Test-Path $inputFile;
$zipExeExists = Test-Path $exeZipLocation;
# if either file doesn't exist, echo it out and return
if(!$resourceExists)
{
Write-Host $inputFile "does not exist!";
return;
}
if(!$zipExeExists)
{
Write-Host $exeZipLocation "does not exist!";
return;
}
# since input parameters are valid, get the last modified datetime of the input file and the zip file (if it exists)
$resource = Get-Item $inputFile;
$resLastModDT = $resource.LastWriteTime;
$zipFileName = $inputFile + ".zip";
$source = $resource.FullName;
$destination = $resource.FullName + ".zip";
$zipExists = Test-Path $destination;
if($zipExists)
{
$zipfile = Get-Item ($destination);
if($resLastModDT -gt $zipfile.LastWriteTime)
{
Write-Host $inputFile "is newer than" $zipFileName "... Continuing with compression.";
& $exeZipLocation a -tzip -mmt -mx9 $destination $source;
}
else
{
Write-Host $zipFileName "is newer than" $inputFile "... Compression skipped.";
}
}
else
{
Write-Host $zipFileName "does not exists ... Continuing with compression.";
& $exeZipLocation a -tzip -mmt -mx9 $destination $source;
}
}
First, I want to validate that the input file and zip executable exists. If it does not exist, return.
Test that the input file paths are valid:
$resourceExists = Test-Path $inputFile;
$zipExeExists = Test-Path $exeZipLocation;
# if either file doesn't exist, echo it out and return
if(!$resourceExists)
{
Write-Host $inputFile "does not exist!";
return;
}
if(!$zipExeExists)
{
Write-Host $exeZipLocation "does not exist!";
return;
}
Now that we've established that the input files are valid, we want to get the last modified datetime of the files (assuming they exist) and construct our paths:
# since input parameters are valid, get the last modified datetime of the input file and the zip file (if it exists)
$resource = Get-Item $inputFile;
$resLastModDT = $resource.LastWriteTime;
$zipFileName = $inputFile + ".zip";
$source = $resource.FullName;
$destination = $resource.FullName + ".zip";
Most of these are being initialized for later use, because now we want to test whether or not the destination zip file already exists. Similar to testing the files for existence as we did prior:
$zipExists = Test-Path $destination;
if($zipExists)
{
# determine if the file is newer or older, if older, compress, if newer, then skip
}
else
{
# compress the file to the output folder
}
The first case, where the output file already exists, is slightly more complex because we want to skip compression if the file is newer (this is to enhance the build speed, assuming that this project, like most, will be built by your fellow developers — to them, its fast, to you, its dog food). Inside the conditional where the destination file already exists we test if its newer, if it is, skip, if older, overwrite:
if($zipExists)
{
$zipfile = Get-Item ($destination);
if($resLastModDT -gt $zipfile.LastWriteTime)
{
Write-Host $inputFile "is newer than" $zipFileName "... Continuing with compression.";
& $exeZipLocation a -tzip -mmt -mx9 $destination $source;
}
else
{
Write-Host $zipFileName "is newer than" $inputFile "... Compression skipped.";
}
}
else
{
...
}
If the output file does not exist, we just need to compress the file and output. Similar to the above step we compress as follows:
...
else
{
Write-Host $zipFileName "does not exists ... Continuing with compression.";
& $exeZipLocation a -tzip -mmt -mx9 $destination $source;
}
Task 3: Test out script against our simple test harness 'RunCompression.ps1':
On the first run, executing RunCompression.ps1 (we setup earlier) output:
Public.Dependency.dll.zip does not exists ... Continuing with compression. 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Scanning Creating archive C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.dll.zip Compressing Public.Dependency.dll Everything is Ok
On the second run (where the output zip file already exists, and the input file's last modified should be older than zip file), output:
Public.Dependency.dll.zip is newer than Public.Dependency.dll ... Compression skipped.
If we rebuild the application so that the input file is newer than the zip file, our output:
Public.Dependency.dll is newer than Public.Dependency.dll.zip ... Continuing with compression. 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Scanning Updating archive C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.dll.zip Compressing Public.Dependency.dll Everything is Ok
Success! We now have completed our script. Now all thats left is to wire it up to MSBuild and finish connecting the dots.
Task 4: Execute the Powershell script from MSBuild with the appropriate parameters.
Back to the Public.Process\Build\Public.Resource.AfterBuild.Target file
<Target Name="LibraryCompress">
<Message Text="Compressing Output Assembly ... $(AssemblyName).dll ==> $(AssemblyName).dll.zip" Importance="High" />
<Exec Command="powershell -Command "& { . \"$(ProjectDir)..\Public.Process\Powershell\Compress.ps1\"; Compress -inputFile \"$(AssemblyName).dll\" -outputFolder \"$(ProjectDir)bin\$(Configuration)\" -exeZipLocation \"$(ProjectDir)..\Public.Process\Utilities\7za.exe\"; }"" />
<Message Text="Compressing Output Symbols ... $(AssemblyName).pdb ==> $(AssemblyName).pdb.zip" Importance="High" />
<Exec Command="powershell -Command "& { . \"$(ProjectDir)..\Public.Process\Powershell\Compress.ps1\"; Compress -inputFile \"$(AssemblyName).pdb\" -outputFolder \"$(ProjectDir)bin\$(Configuration)\" -exeZipLocation \"$(ProjectDir)..\Public.Process\Utilities\7za.exe\"; }"" />
</Target>
The LibraryCompress target now calls the Compress.ps1 powershell script twice; first for the library (*.dll) and second for the symbols (*.pdb). Both work the same way, just with different arguments. In the case of the assembly (*.dll) compression, the inputFile argument is the $(AssemblyName).dll (i.e. Public.Dependency.dll), the outputFolder is the $(ProjectDir)bin\$(Configuration) (i.e. [Path]\bin\Compression\), and the zip executable location for 7za.exe (located un the Utilities folder inside the Public.Process project; 7za.exe, the 7zip command line executable, offers very decent compress vs. speed imho).
Now you understand why the call to the Powershell method needs to be concise. Powershell commands directly inside an MSBuild Target file is a maintenance nightmare due to the escape characters required for MSBuild to properly parse the powershell command. Any frustration you experience with " and & is well worth the effort though, as the mileage you gain with powershell is vast.
After editing the Target file (don't forget to reload the solution when marking Target include edits!), we can test our script by examining the output window in Visual Studio:
Build output for Public.Dependency project:
------ Build started: Project: Public.Dependency, Configuration: Compression Any CPU ------ Public.Dependency -> C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.dll Compressing Output Assembly ... Public.Dependency.dll ==> Public.Dependency.dll.zip Public.Dependency.dll is newer than Public.Dependency.dll.zip ... Continuing with compression. 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Scanning Updating archive C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.dll.zip Compressing Public.Dependency.dll Everything is Ok Compressing Output Symbols ... Public.Dependency.pdb ==> Public.Dependency.pdb.zip Public.Dependency.pdb is newer than Public.Dependency.pdb.zip ... Continuing with compression. 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Scanning Updating archive C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.pdb.zip Compressing Public.Dependency.pdb Everything is Ok ...
The subsequent build (where the zip file already exists and no project files have been modified) the build output looks like:
------ Build started: Project: Public.Dependency, Configuration: Compression Any CPU ------ Public.Dependency -> C:\Users\timwebb\Desktop\Public.Process\Public.Dependency\bin\Compression\Public.Dependency.dll Compressing Output Assembly ... Public.Dependency.dll ==> Public.Dependency.dll.zip Public.Dependency.dll.zip is newer than Public.Dependency.dll ... Compression skipped. Compressing Output Symbols ... Public.Dependency.pdb ==> Public.Dependency.pdb.zip Public.Dependency.pdb.zip is newer than Public.Dependency.pdb ... Compression skipped.
Now that we've successfully called Powershell from MSBuild using a Target include file, we repeat the process inside Public.SubDependency (since the script is parameterized correctly, it is readily reusable).
Calling Powershell from within MSBuild process is probably one of the best tools I've found for build and release management in general. There are many options available to you once inside a powershell script, everything from calling managed assemblies, restarting iis (or compressing output assemblies). Leave me a comment if you found this useful, let me know how you're using Powershell inside your MSBuild (aka PSBuild) process!
Enjoy!
This is great, just what I needed. However, I kept getting missing Curly Brace error in my .target file. I fixed by changing all the inner double quotes to single quotes: "e; to '
… "& { . '$(MSBuildProjectDirectory)\Compress.ps1'; … '$(exeZipLocation)' } "”
Works great….thanks