As a rule, whenever I find myself doing something that nobody else is doing, I have to be quite suspicious about what I'm doing. This is why I want to get code review to verify I'm not doing something insane with the build events for what it's not usually intended.
Contributing to Rubberduck VBA project, I have a PR that creates a new Visual Studio project, named Rubberduck.Deployment
. The main purpose of the project is to assist at the build time the extraction of the data that the installer will need to perform its thing. Therefore it will generate files at build time which can vary from build to build as the codebase is updated. Those generated files are then consumed by Inno Setup. However, I must support both building locally and remotely via the AppVeyor so that the contributors have easy time building and can do so transparently.
The key code to review is this entry in the Post Build.
<PreBuildEvent>
%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe
-ExecutionPolicy Bypass
-command "& '$(ProjectDir)PreInnoSetupConfiguration.ps1'
-WorkingDir '$(ProjectDir)'"
</PreBuildEvent>
<PostBuildEvent>
%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe
-ExecutionPolicy Bypass
-command "& '$(ProjectDir)BuildRegistryScript.ps1'
-config '$(ConfigurationName)'
-builderAssemblyPath '$(TargetPath)'
-netToolsDir '$(FrameworkSDKDir)bin\NETFX 4.6.1 Tools\'
-wixToolsDir '$(ProjectDir)WixToolset\'
-sourceDir '$(TargetDir)'
-targetDir '$(TargetDir)'
-projectDir '$(ProjectDir)'
-includeDir '$(ProjectDir)InnoSetup\Includes\'
-filesToExtract 'Rubberduck.dll|Rubberduck.API.dll'"
</PostBuildEvent>
As indicated, we invoke PowerShell scripts which is included in the Visual Studio project (and being a script, it doesn't directly participate in the building of the Visual Studio project). Here's complete code for the post build script.
# The parameters should be supplied by the Build event of the project
# in order to take macros from Visual Studio to avoid hard-coding
# the paths. To simplify the process, the project should have a
# reference to the projects that needs to be registered, so that
# their DLL files will be present in the $(TargetDir) macro.
#
# Possible syntax for Post Build event of the project to invoke this:
# C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
# -command "$(ProjectDir)BuildRegistryScript.ps1
# -config '$(ConfigurationName)'
# -builderAssemblyPath '$(TargetPath)'
# -netToolsDir '$(FrameworkSDKDir)bin\NETFX 4.6.1 Tools\'
# -wixToolsDir '$(SolutionDir)packages\WiX.Toolset.3.9.1208.0\tools\wix\'
# -sourceDir '$(TargetDir)'
# -targetDir '$(TargetDir)'
# -projectDir '$(ProjectDir)'
# -includeDir '$(ProjectDir)InnoSetup\Includes\'
# -filesToExtract 'Rubberduck.dll'"
param (
[Parameter(Mandatory=$true)][string]$config,
[Parameter(Mandatory=$true)][string]$builderAssemblyPath,
[Parameter(Mandatory=$true)][string]$netToolsDir,
[Parameter(Mandatory=$true)][string]$wixToolsDir,
[Parameter(Mandatory=$true)][string]$sourceDir,
[Parameter(Mandatory=$true)][string]$targetDir,
[Parameter(Mandatory=$true)][string]$projectDir,
[Parameter(Mandatory=$true)][string]$includeDir,
[Parameter(Mandatory=$true)][string]$filesToExtract
)
function Get-ScriptDirectory
{
$Invocation = (Get-Variable MyInvocation -Scope 1).Value;
Split-Path $Invocation.MyCommand.Path;
}
# Invokes a Cmd.exe shell script and updates the environment.
function Invoke-CmdScript {
param(
[String] $scriptName
)
$cmdLine = """$scriptName"" $args & set"
& $Env:SystemRoot\system32\cmd.exe /c $cmdLine |
select-string '^([^=]*)=(.*)$' | foreach-object {
$varName = $_.Matches[0].Groups[1].Value
$varValue = $_.Matches[0].Groups[2].Value
set-item Env:$varName $varValue
}
}
# Returns the current environment.
function Get-Environment {
get-childitem Env:
}
# Restores the environment to a previous state.
function Restore-Environment {
param(
[parameter(Mandatory=$TRUE)]
[System.Collections.DictionaryEntry[]] $oldEnv
)
# Remove any added variables.
compare-object $oldEnv $(Get-Environment) -property Key -passthru |
where-object { $_.SideIndicator -eq "=>" } |
foreach-object { remove-item Env:$($_.Name) }
# Revert any changed variables to original values.
compare-object $oldEnv $(Get-Environment) -property Value -passthru |
where-object { $_.SideIndicator -eq "<=" } |
foreach-object { set-item Env:$($_.Name) $_.Value }
}
# Remove older imported registry scripts for debug builds.
function Clean-OldImports
{
param(
[String] $dir
)
$i = 0;
Get-ChildItem $dir -Filter DebugRegistryEntries.reg.imported_*.txt |
Sort-Object Name -Descending |
Foreach-Object {
if($i -ge 10) {
$_.Delete();
}
$i++;
}
}
Set-StrictMode -Version latest;
$ErrorActionPreference = "Stop";
$DebugUnregisterRun = $false;
try
{
# Clean imports older than 10 builds
Clean-OldImports ((Get-ScriptDirectory) + "\LocalRegistryEntries");;
# Allow multiple DLL files to be registered if necessary
$separator = "|";
$option = [System.StringSplitOptions]::RemoveEmptyEntries;
$files = $filesToExtract.Split($separator, $option);
# Load the Deployment DLL
[System.Reflection.Assembly]::LoadFrom($builderAssemblyPath);
# Determine if MIDL is available for building
$devPath = $Env:ProgramFiles + "*\Microsoft Visual Studio\*\*\Common*\Tools\VsDevCmd.bat";
$devPath = Resolve-Path -Path $devPath;
if($devPath)
{
# Additional verifications as some versions of VsDevCmd.bat might not initialize the environment for C++ build tools
$result = Get-Module -ListAvailable -Name "VSSetup" -ErrorAction SilentlyContinue;
if(!$result)
{
Write-Warning "VSSetup not installed; extracting...";
Expand-Archive "$projectDir\OleWoo\VSSetup.zip" "$([Environment]::GetFolderPath("MyDocuments"))\WindowsPowerShell\Modules\VSSetup" -Force
}
try {
Import-Module VSSetup -Force:$true;
$result = Get-VSSetupInstance | Select-VSSetupInstance -Latest -Require Microsoft.VisualStudio.Component.VC.Tools.x86.x64;
} catch {
$result = $null;
Write-Warning "Error occurred with using VSSetup module";
Write-Error ($_);
}
if(!$result)
{
$devPath = $null;
Write-Warning "Cannot locate the VS Setup instance capable of building with C++ build tools";
}
}
if(!$devPath)
{
Write-Warning "Cannot locate the VsDevCmd.bat to initialize C++ build tools; falling back to tlbexp.exe....";
}
Write-Host "";
foreach($file in $files)
{
Write-Host "Processing '$file'";
Write-Host "";
$dllFile = [System.String]$file;
$idlFile = [System.String]($file -replace ".dll", ".idl");
$tlb32File = [System.String]($file -replace ".dll", ".x32.tlb");
$tlb64File = [System.String]($file -replace ".dll", ".x64.tlb");
$sourceDll = $sourceDir + $file;
$targetDll = $targetDir + $file;
$sourceTlb32 = $sourceDir + $tlb32File;
$targetTlb32 = $targetDir + $tlb32File;
$sourceTlb64 = $sourceDir + $tlb64File;
$targetTlb64 = $targetDir + $tlb64File;
$dllXml = $targetDll + ".xml";
$tlbXml = $targetTlb32 + ".xml";
# Write-Host "Variable printout:"
# Write-Host "dllFile = $dllFile";
# Write-Host "idlFile = $idlFile";
# Write-Host "tlb32File = $tlb32File";
# Write-Host "tlb64File = $tlb64File";
# Write-Host "sourceDll = $sourceDll";
# Write-Host "targetDll = $targetDll";
# Write-Host "sourceTlb32 = $sourceTlb32";
# Write-Host "targetTlb32 = $targetTlb32";
# Write-Host "sourceTlb64 = $sourceTlb64";
# Write-Host "targetTlb64 = $targetTlb64";
# Write-Host "dllXml = $dllXml";
# Write-Host "tlbXml = $tlbXml";
# Write-Host "targetDir = $targetDir";
# Write-Host "";
# Use for debugging issues with passing parameters to the external programs
# Note that it is not legal to have syntax like `& $cmdIncludingArguments` or `& $cmd $args`
# For simplicity, the arguments are pass in literally.
# & "C:\GitHub\Rubberduck\Rubberduck\Rubberduck.Deployment\echoargs.exe" ""$sourceDll"" /win32 /out:""$sourceTlb"";
# Compile TLB files using MIDL
if($devPath)
{
$idlGenerator = New-Object Rubberduck.Deployment.IdlGeneration.IdlGenerator;
$idl = $idlGenerator.GenerateIdl($sourceDll);
$encoding = New-Object System.Text.UTF8Encoding $true;
[System.IO.File]::WriteAllLines($idlFile, $idl, $encoding);
$origEnv = Get-Environment;
try {
Invoke-CmdScript "$devPath";
if($targetDir.EndsWith("\"))
{
$targetDirWithoutSlash = $targetDir.Substring(0,$targetDir.Length-1);
}
else
{
$targetDirWithoutSlash = $targetDir;
}
& midl.exe /win32 /tlb ""$tlb32File"" ""$idlFile"" /out ""$targetDirWithoutSlash"";
& midl.exe /amd64 /tlb ""$tlb64File"" ""$idlFile"" /out ""$targetDirWithoutSlash"";
} catch {
throw;
} finally {
Restore-Environment $origEnv;
}
}
# Compile TLB files using tlbexp.exe
if(!$devPath)
{
$cmd = "{0}tlbexp.exe" -f $netToolsDir;
& $cmd ""$sourceDll"" /win32 /out:""$sourceTlb32"";
& $cmd ""$sourceDll"" /win64 /out:""$sourceTlb64"";
}
# Harvest both DLL and TLB files using WiX's heat.exe, generating XML files
$cmd = "{0}heat.exe" -f $wixToolsDir;
& $cmd file ""$sourceDll"" -out ""$dllXml"";
& $cmd file ""$sourceTlb32"" -out ""$tlbXml"";
# Initialize the registry builder with the provided XML files
$builder = New-Object Rubberduck.Deployment.Builders.RegistryEntryBuilder;
$entries = $builder.Parse($tlbXml, $dllXml);
# For debugging
# $entries | Format-Table | Out-String |% {Write-Host $_};
$writer = New-Object Rubberduck.Deployment.Writers.InnoSetupRegistryWriter;
$content = $writer.Write($entries, $dllFile, $tlb32File, $tlb64File);
# The file must be encoded in UTF-8 BOM
$regFile = ($includeDir + ($file -replace ".dll", ".reg.iss"));
$encoding = New-Object System.Text.UTF8Encoding $true;
[System.IO.File]::WriteAllLines($regFile, $content, $encoding);
$content = $null;
# Register the debug build on the local machine
if($config -eq "Debug")
{
if(!$DebugUnregisterRun)
{
# First see if there are registry script from the previous build
# If so, execute them to delete previous build's keys (which may
# no longer exist for the current build and thus won't be overwritten)
$dir = ((Get-ScriptDirectory) + "\LocalRegistryEntries");
$regFileDebug = $dir + "\DebugRegistryEntries.reg";
if (Test-Path -Path $dir -PathType Container)
{
if (Test-Path -Path $regFileDebug -PathType Leaf)
{
$datetime = Get-Date;
if ([Environment]::Is64BitOperatingSystem)
{
& reg.exe import $regFileDebug /reg:32;
& reg.exe import $regFileDebug /reg:64;
}
else
{
& reg.exe import $regFileDebug;
}
& reg.exe import ($dir + "\RubberduckAddinRegistry.reg");
Move-Item -Path $regFileDebug -Destination ($regFileDebug + ".imported_" + $datetime.ToUniversalTime().ToString("yyyyMMddHHmmss") + ".txt" );
}
}
else
{
New-Item $dir -ItemType Directory;
}
$DebugUnregisterRun = $true;
}
# NOTE: The local writer will perform the actual registry changes; the return
# is a registry script with deletion instructions for the keys to be deleted
# in the next build.
$writer = New-Object Rubberduck.Deployment.Writers.LocalDebugRegistryWriter;
$content = $writer.Write($entries, $dllFile, $tlb32File, $tlb64File);
$encoding = New-Object System.Text.ASCIIEncoding;
[System.IO.File]::AppendAllText($regFileDebug, $content, $encoding);
}
Write-Host "Finished processing '$file'";
Write-Host "";
}
Write-Host "Finished processing all files";
}
catch
{
Write-Error ($_);
# Cause the build to fail
throw;
}
In start of the script, we do this:
[System.Reflection.Assembly]::LoadFrom($builderAssemblyPath);
Which means we load the DLL that was just built by the project (it's literally the output of the Rubberduck.Deployment
project) which the powershell script goes on to invoke methods and then eventually write out a file:
$builder = New-Object Rubberduck.Deployment.Builders.RegistryEntryBuilder;
$entries = $builder.Parse($tlbXml, $dllXml);
....
$writer = New-Object Rubberduck.Deployment.Writers.InnoSetupRegistryWriter;
$content = $writer.Write($entries, $dllFile, $tlb32File, $tlb64File);
$regFile = ($includeDir + ($file -replace ".dll", ".reg.iss"));
$encoding = New-Object System.Text.UTF8Encoding $true;
[System.IO.File]::WriteAllLines($regFile, $content, $encoding);
The autogenerated file is then used as a input to the Inno Setup compiler which is used during the AppVeyor build to build a complete installer for the Rubberduck addin. But for a local debug build where we don't use installer, we run this section:
# Register the debug build on the local machine
if($config -eq "Debug")
{
if(!$DebugUnregisterRun)
{
# First see if there are registry script from the previous build
# If so, execute them to delete previous build's keys (which may
# no longer exist for the current build and thus won't be overwritten)
$dir = ((Get-ScriptDirectory) + "\LocalRegistryEntries");
$regFileDebug = $dir + "\DebugRegistryEntries.reg";
if (Test-Path -Path $dir -PathType Container)
{
if (Test-Path -Path $regFileDebug -PathType Leaf)
{
$datetime = Get-Date;
if ([Environment]::Is64BitOperatingSystem)
{
& reg.exe import $regFileDebug /reg:32;
& reg.exe import $regFileDebug /reg:64;
}
else
{
& reg.exe import $regFileDebug;
}
& reg.exe import ($dir + "\RubberduckAddinRegistry.reg");
Move-Item -Path $regFileDebug -Destination ($regFileDebug + ".imported_" + $datetime.ToUniversalTime().ToString("yyyyMMddHHmmss") + ".txt" );
}
}
else
{
New-Item $dir -ItemType Directory;
}
$DebugUnregisterRun = $true;
}
# NOTE: The local writer will perform the actual registry changes; the return
# is a registry script with deletion instructions for the keys to be deleted
# in the next build.
$writer = New-Object Rubberduck.Deployment.Writers.LocalDebugRegistryWriter;
$content = $writer.Write($entries, $dllFile, $tlb32File, $tlb64File);
$encoding = New-Object System.Text.ASCIIEncoding;
[System.IO.File]::AppendAllText($regFileDebug, $content, $encoding);
}
The writer will actually create the registry in the developer's HKCU registry to register the debug build to COM among other things. As it does that, it also generates a registry script which is then saved to the disk that enables deletions of all keys it created. That file is then used in subsequent build to delete the previous build's old keys, ensuring that developer don't end up with 1000s of stale keys as they make changes to the objects that may or may not change the registration.
This all works all nicely but as I said at start, this is quite unusual use of Post Build event and I'm wondering if that is a code smell in itself. I'm also interested in hearing whether we can do this better to make the process more seamless and less rube-goldsberg-esque.
Note that while I could have just invoked the powershell script and the DLL inside the AppVeyor, this would not achieve the goal of being able to build and generate the files when building locally which can't be then used for unit tests, debugging, or just simply inspecting. More importantly, I don't want to have one process for building locally and other process for building on AppVeyor, as that has potential to create bugs that we can't see due to missed differences in the build.
Questions to answer:
Is this a reasonable method to customize a build process? Are there better ways?
I don't like that the macro
$(FrameworkSDKDir)
is basically undocumented to a C# project. AFAICT, it's only used on a C++ project. Am I going to run into trouble for using that macro? In tests, it seems to work but...? What about a new version? All this is just to be able to runtlbexp.exe
which AFAICT isn't in thePATH
variable.I currently have no exact error handling strategy; any error will cause the build as whole to fail. That might be acceptable since no output means build can't be completed anyway but it might violate the principle of least astonishment. I'm also annoyed that currently any errors will just give out a generic
the command exited with exit code -1
in theError List
without giving any actual information about the errors. Currently, the contributors must go to theOutput
for theBuild
stream to see the actual error from the powershell script. Can we improve on that?Can we improve the environment check? Because we use MIDL compiler which is technically a C++ build tool and thus not a part of a normal C# build process, we must configure the environment. However, I've found it quite hard to reliably detect whether the installed Visual Studio supports C++ build tools or not. I originally checked for existence of
VsDevCmd.bat
but this has had false positives which causes build process to attempt to compile using MIDL and then fail.
Note on #4: Since the original path, a contributor found a hole in the logic and the script was since tightened on its check, with a section dedicated to analyzing the presence of C++ build tools, starting on line 106
C#
if there isn't any C# involved ? \$\endgroup\$$builder = New-Object Rubberduck.Deployment.Builders.RegistryEntryBuilder;
\$\endgroup\$