Fuzzy's Blogic

Deploying SSRS Reports from Powershell

2008-11-25T22:17:00Z

My current client has a BI group that writes SSRS reports. In the "legacy" ALM tool they built, they used VSS, nAnt,Cruise Control and a product called "Visual Intercept" to "build" (actually just extract RDL files) and a nAnt script with a custom task to Deploy the SSRS report (RDL File) to Reporting Services.

We migrated to TFS and use a lot of Powrshell for the deploy scripts. I searched for some solutions from Powershell to deploy RDL's that I felt was a good fit. This solution: http://blogs.devhorizon.com/reza/?p=751 is for a Sharepoint Reporting Service Integration. These are stand-alone SSRS reports - so I'm not even sure if I have the .DLL it relies on "ReportService2006.dl" so it was no go.

The other solution that looks quite promising, from Paul Stovell at Readify http://www.paulstovell.com/blog/reporting-services-automation is an excellent approach, but relies on a utility "rs.exe". But that wasn't installed on the server they run the deploys from (actually the TFS Build server for the time being) and I didn't want any dependencies I didn't have complete control over.

Being a C# guy - I figured the easiest thing was just to write copy the code from the original nAnt custom Task that called the SSRS Web Service to Deploy the report. I wrote a very simple Wrapper to the webservice as a Static Method to a Static Class. I then just copy the generated DLL's to the Build/Deploy server - and call that wrapper from the Deploy script. Simple enough.

I cleaned the original code a bit - adding an integer return that shows 0 for success, 4 for warnings and either 12 or 16 for a fatal error. The C# Code is here. I just added a .NET 2.0 style WebService Reference (as Derick Whitaker explains here: http://devlicio.us/blogs/derik_whittaker/archive/2008/07/21/referencing-2-0-web-services-asmx-in-visual-studio-2008.aspx in case you don't know how) to http://localhost/reportserver/reportservice.asmx?wsdl (don't use ReportService2005.asmx - that's a different web service!)

using System;
using System.Collections.Generic;
using System.IO;
using System.Xml;
using System.Linq;
using System.Text;
using SSRSDeployer.ReportServer;
using System.Web.Services.Protocols;

namespace SSRSDeployer
{
    public static class SSRSDeployer
    {
        public static int Deploy(string webServiceURL, string rdlPath, string parentFolder, string reportName, out string warnings)
        {
            //create an instance of the WebService Proxy and set URL and credentials
            ReportingService rs = new ReportingService();
            rs.Url = webServiceURL;
            rs.Credentials = System.Net.CredentialCache.DefaultCredentials;

            Byte[] definition = null;
            Warning[] warns = null;
            StringBuilder sb = new StringBuilder();
            int ret = 0;

            try
            {

                //read the file as a filestream into a byte array
                FileStream stream = File.OpenRead(rdlPath);
                definition = new Byte[stream.Length];
                stream.Read(definition, 0, (int)stream.Length);

                //    Call the web service to create the report
                rs.Timeout = 1000000;
                warns = rs.CreateReport(reportName, parentFolder, true, definition, null);

                if (warns != null)
                {
                    foreach (Warning warning in warns)
                    {
                        //ignore the warning about shared data source
                        if (!(warning.Message.StartsWith("The data set ‘") && warning.Message.Contains("’ refers to the shared data source ‘") && warning.Message.EndsWith("which is not published on the report server.")))
                        {
                            // otherwise return it to output
                            sb.Append(string.Format("Warning returned from WebService: {0}", warning.Message));
                            ret = 4;
                        }
                    }
                }
            }
            catch (IOException)
            {
                //report any IO error reading the RDL file
                sb.Append(string.Format("Error: IOExcption reading file: {0}", rdlPath));
                ret = 12;
            }

            catch (SoapException e)
            {
                //report any Errors from the WebService call
                sb.Append(e.Message);
                ret = 16;
            }

            warnings = sb.ToString();
            return ret;
        }
    }
}

To Call it from Powershell, simply load the assembly and call the Method: This Powershell script will deploy all RDL's found in the specified folder with a report name matching the file name. Make sure you put all the generated DLL's from the compile to the folder referenced by the $lib= statement

[string] $webServiceURL = "http://SSRSServer/ReportServer/ReportService.asmx"
[string] $rdlPath = "C:\File13"
[string] $parentFolder = "/SSRS.Folder";

$file = Get-Item $rdlPath
$lib="C:\File13\SSRSDeployer\SSRSDeployer.dll"
$x=[Reflection.Assembly]::LoadFrom($lib)

$files = Get-ChildItem $rdlPath *.rdl
ForEach ($file in $files) {
[string] $reportName = ([string] $file.Name.ToUpper()).Replace(".RDL","")
[string] $warnings = ""
"Deploying Report: $reportName"
[Int32] $result = [SSRSDeployer.SSRSDeployer]::Deploy($webServiceURL, $file.FullName, $parentFolder, $reportName, [ref] $warnings)
if ($result -eq 0) {"Deployed without Errors"}
elseif ($result -eq 4) {"Warnings from Deploy: $warnings"}
elseif ($result -eq 12) {"IO Error reading RDL File"}
elseif ($result -eq 16) {"Error Deploying Report: $warnings"}
}

Feeling Like Homer and exclaiming "D'Oh!"

2008-11-24T17:49:00Z

I'm a big Simpsons fan - and one of the appeals is watching Homer do something really stupid that he should have known better and exclaim "D'oh" when the inevitable happens. So there's nothing more painful when you end up "pulling a Homer"...

I had one today. I was trying to solve a problem, and made it worse by doing something that I should have known better. At my client I'm doing some pretty complex TFS build automation. Since some of what I'm doing is pushing the limits of what I personally think should be done using MSBuild, I wrote a C# custom MSBuild task that calls PowerShell. My complex scripting therefor is done in what I believe is by far the best scripting language on the Windows platform, instead of a tool that is excellent for determining dependencies between source code...

While trying to debug a problem of getting my PowerShell script in an MSBuild script, all of sudden things that were previously working started breaking. After annoying "cubical neighbors with exclamations of frustration" - I realized what I had just done to make things worse.

I was deploying changes to the build server using a "C$" type file share (I have admin access to the "box") and notepad. I had the "Format-Word wrap" option in notepad checked - and it was inserting line breaks "randomly" in the scripts - wreaking havoc in both PowerShell and MSBuild...

D'oh!

Removing VSS Bindings to Migrate SSRS 2005 Solutions to TFS Using Powershell

2008-11-18T22:35:00Z

The last task I have for a VSS to TFS Migration is to convert a large number of SSRS (SQL Server Reporting Services) Projects. While Microsoft has a tool to migrate from VSS to TFS - I've avoided using it. Most "experts" have advised against it for a number of reasons. Particularly, a desire to do some refactoring on the organization of the project folders, and just to give a "clean start" for TFS.

For this clients other applications, generally there are a reasonable number of solutions, so manual efforts, like un-binding from VSS and re-binding to TFS in Visual Studio has not been that big of a deal. Unfortunately my client has close to 100 SSRS projects and about half as many solutions. One of the Power Tools "tfpt.exe" supposedly automated re-binding to TFS - but it didn't work for me. I think it may only be intended to use in conjunction with the VSSConverter tool.

Another promising tool was suggested to me by fellow Magenicon-TFS-Expert Donn Felker. This tool claims to remove the binding, but it nuked the entire .sln file for me - and I don't know if it fixed the .rptproj (I got too frustrated to pursue it further) Donn mentioned having similar problems, but somehow got a workaround. I never figured it out and punted.

I decided to write my own PowerShell script to do it. My approach is to do a "Get-Latest" from VSS to it's working directory, then copy the files to the Workspace mapped Directory for TFS. Then run the script, then add the files to TFS. Here's what the script does:

Clear the Read-Only flag on all the files
Delete the files I don't want in TFS (*.suo, *.user, *.*scc)
Remove the binding section from the sln files
Remove the "State" from the .rptproj files

The last one was the most interesting discovery. SSRS Project Files are not MSBuild projects (like most other Visual Studio projects) but they are XML documents. They have a "State" element that has SOMETHING encoded. I've seen many references for SSRS issues that offer solution to "manually edit the .RPTProj and remove the "State" element. So I knew it was okay to delete them. I also discovered that through trial-and-error that this was necessary to remove the binding to VSS for the individual reports in each solution.

After the end of the post I have my PowerShell script that accomplishes it. It's not very fancy - without error detection etc - since this is a "One Shot" type thing - but it does what is needed. The only thing I'm unhappy about - I've been able to find ANYTHING as far as an automated "binding" of solutions to TFS. So my users will need to manually do this as they edit reports. Many haven't been touched since they were written in 2006 - so this is hopefully not a big deal...

Here's the script in all it's glory

$Folder = "C:\File13\StARS.Reports"
#first clear the read-only flag from all files
get-childitem "$folder" -Recurse | % {
        # Test for ReadOnly flag and remove if present
     if ($_.attributes -band [system.IO.FileAttributes]::ReadOnly) {
            $_.attributes = $_.attributes -bxor [system.IO.FileAttributes]::ReadOnly
   }
}

#next delete all files that are *.suo, *.user, and *.*scc - we don't want thim in TFS
Get-ChildItem $folder *.suo -Recurse -Force | Remove-Item -Force
Get-ChildItem $folder *.*scc -Recurse -Force | Remove-Item -Force
Get-ChildItem $folder *.user -Recurse -Force | Remove-Item -Force

#next get all the .sln file - and remove the VSS binding information
$files = Get-ChildItem $folder *.sln -Recurse
foreach ($file in $files) {
$fileout = $file.FullName + ".new"
Set-Content $fileout $null
$switch=0
Get-Content $file.FullName | % {
  if ($switch -eq 0) {
   if ($_ -eq " GlobalSection(SourceCodeControl) = preSolution") {
    #we found the section to skip - so set the flag and don't copy the content
    $switch=1}
   else {
    #we haven't found it yet - so copy the content
    Add-Content $fileout $_
   }
   }
  elseif ($switch -eq 1) {
   if ($_ -eq " EndGlobalSection") {
    #last line to skip - after it we start writing the content again
    $switch=2}
   }
  else
   { #write remaining lines
    Add-Content $fileout $_}
}
#remove the original .sln and rename the new one
$newname = $file.Name
Remove-Item $file.FullName
Rename-Item $fileout -NewName $newname
}
##next we need to remove the <State> Node from the .rptproj files - it also contains binding information
$files = Get-ChildItem $folder *.rptproj -Recurse
foreach ($file in $files) {
#read the project file as XML
$proj = [xml] [string]::Join("`n",
  (get-content $file.FullName))
$stateNode = $proj.SelectNodes("/Project/State")
$root = $proj.Project
[Void]$root.RemoveChild($stateNode.item(0))
#save file todo
$proj.Save($file.FullName)
}

Error loading SSRS Report Project in VS2005: "Could not load file or assembly 'ReportingServicesLibrary, Version=9.0.242.0"

2008-11-13T16:51:00Z

I had this problem and "Google'ing" it turned up nothing relevant. So obviously this is prime material for blogging about it...

The background is that I returned to a client I had previously consulted for after a couple months. They had deleted and added an account for me on their domain - so I when I logged into the Virtual PC I use for their domain - Windows considered it a new account (I think this may have had something to do with the problem but who knows). I had used that VPC for over two years - and had installed various different editions of Visual Studio 2005 over that period.

I'm working on migrating their applications from VSS (and a custom-baked build system) to Team Foundation Server and my last project is for their SSRS projects. But every single report I tried loading gave me a couple of errors. This happened whether I opened the solution or the Report Project (*.rptproj) First I got the error: "Project item '#item°#' does not represent a file. " That workaround was to manually edit the rprtproj and remove the "<State"> element. I also so entries complaining that the *.user files should not be checked into version control - (which they were) - so I deleted those as well...

But when I cleared those errors - I started getting the ""Could not load file or assembly 'ReportingServicesLibrary, Version=9.0.242.0" error. I was stumped - and tried to look around for solutions. I contacted the Release Manager (who I report to) to discuss it - in a previous position - she had developed quite a few of those reports- who suggested if when I do a "New Project" wizard - if the templates for "Business Intelligence Projects" was an option - and it wasn't! Somehow that package got corrupt in my install of VS 2005.

I have a good idea that it happened when "mucking around" in the "C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies" directory - but that's another story...

I found some google posts on some fixes - but nothing worked - until I went to the extreme measure of COMPLETELY uninstalling ALL of Visual Studio 2005. I then had to completely uninstall ALL of SQL Server 2005. Then I had to reinstall everything. This was no small task - but it did finally work in the end.

Cool Powershell Trick - Stopping and Starting a service on a remote server without having to explicitly load the .NET assemblies

2008-10-29T17:56:00Z

I was chatting with my colleague at Magenic Donn Felker about two of our favorite topics TFS and Powershell. He found a cool way to load scripts into a profile. It reminded me of a cool Powershell Trick i found on the web before I started blogging...

I wrote deployment scripts in Powershell, which the client wanted to run on a single server regardless of where they were deploying to - so I needed to stop and start services on a remote machine. However, the current version of Powershell doesn't include direct support for managing services on a remote server. The solution (as is so often the case in Powershell is to use .NET Framework classes). To do so - you generally need to load the Assemblies explicitly with code that looks like this

#load assemblies
$key = Get-ItemProperty HKLM:\SOFTWARE\Microsoft\VisualStudio\9.0
$dir = [string] (Get-ItemProperty $key.InstallDir)
$dir += "PrivateAssemblies\"
#set to $x so it won't emit the "success" to the output stream
$lib = $dir + "Microsoft.TeamFoundation.Client.dll"
$x=[Reflection.Assembly]::LoadFrom($lib)
$lib = $dir + "Microsoft.TeamFoundation.VersionControl.Client.dll"
$x=[Reflection.Assembly]::LoadFrom($lib)

So with the assemblies loaded you can call methods on the classes they implement like this:

#Set up connection to TFS Server

$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($tfsServer)

#get version control$versionControlType = [Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer]

$versionControlService = $tfs.GetService($versionControlType)

(This sample above is to load assemblies to use the TFS Version Control API)

So you pretty much need to know where the Assembly is loaded from, and use the Reflection.Assembly class to load it.

Here’s the trick. While the Powershell “Cmdlets” to manipulate services only work on the machine the script is running on, they use the .NET framework classes “behind the scenes” so if you execute the “Get-Services” cmdlet – all the dependent assemblies are automatically loaded in your process space.

So the code below works without having to explicitly load any assemblies!I

# dummy run of Get-Service CMDLet to force .NET Assemblies to be loaded

$dmy = Get-Service

#Get the services on the Remote Machine

$allServices = [System.ServiceProcess.ServiceController]::GetServices($NSServer)

#Get the NS Service

$nsService = $allServices | where {$_.Name -like 'NS$steriworksinstance'}

#Call Stop on the Service if it is Running - if so stop it

if ($nsService.Status -eq "Running")

{

$nsService.Stop()

}

While this might “break” in a future release of Powershell, they are adding support for remote servers, so switching to the cmdlets for that would be preferred anyway…

Oh and in case you are interested - the code to start the service

$nsService.Start()

Extending TFS Presentation

2008-10-07T14:24:00Z

I've given the presentation "Extending Team Foundation Server 2008-SP1 " twice in the past couple of months: at "Codeapalooza" on September 6, 2008 and at the Chicago VSTS User's group last week on October 2, 2008. I've had at least one attendee ask for a copy of the Slides and the Proof-of-Concept code. Both the code and the slides can be downloaded here. In this presentation I discussed some different ways TFS can be extended with detailed examples. For the TFS API, I walk-through code for a Proof-of-Concept I wrote which will create "Bug" Work Items for unhandled exceptions.

If you are involved with a users' group or similar organization in the Midwest and would be interested in this presentation I'd be happy to do so! Please contact me using the email link to the right.

Implementing VB6 on TFS

2008-08-27T20:13:00Z

I just finished implementing VB6 build for my client. It was pretty easy, but I learned a few things.

To do it, I simply added the source code to Version Control (naturally). Then created a build type like any but with one exception. Obviously I couldn't use the "Create Project File" Wizard, since there will be no SLN Solution file to find, so I created the TFSBuild.proj entirely manually before creating the Build Type. In that project I added the following:

PropertyGroup>
  <VB6>$(ProgramFiles)\Microsoft Visual Studio\VB98\VB6.exe</VB6>
  <VB6Output>$(BinariesRoot)\VB6\</VB6Output>
  <VB6Timeout>150000</VB6Timeout>
</PropertyGroup>

</Target>

With that, I got a good VB6 Build!

The next thing I did was put the standard customization the client uses for it's Build Number/Version Stamping. Once the Build Number was calculated, we edit files appropriately to get that into the EXE's and DLL's - the heavy lifting done using the Tigris MSBuild Community Tasks. This always worked fine but when I added it to my VB6 Solution, VB6 wouldn't compile with the message:

The project file 'c:\TFSBuild\23\Sources\Source\[ApplicationDirectory]\[application].vbp' contains invalid key 'Type'.

With a "garbage" character in front of the capital "T" of Type.

Well once again, the same issue I blogged about a few months ago: UTF-8 is not ASCII! Evidently VB6 can't handle UTF-8 encoded files, which is the default in .NET. Fortunately, the <FileUpdate> tasks exposes the "Encoding" Property to the tasks, so specifying "Encoding=ascii" solved the problem!

The other issue I had was with the installer for the TFS MSSCCI Provider. This is a plug-in for "legacy" applications (like Visual Studio 6 and Visual Studio 2003 for example) to use TFS. I had installed an earlier version (and had successfully checked-in and out VB6 modules from VS6 and TFS Version Control). I saw that a new version of the Provider was released in early August (to support SP1 of TFS2008 I would guess in addition to some new features) - so I installed it. It had an MSI - so I assumed that it could handle dealing with earlier versions.

I then tried to check-out files and got a cryptic error:

"method not found: Void Microsoft.TeamFoundation.Msscci.Client.Context.EditItems(System.String[], Microsoft.TeamFoundation.VersionControl.Client.RecursionType, Microsoft.TeamFoundation.VersionControl.Client.LockLevel,"

Using Reflector I found that, in fact, such an overload did not exist. So after some digging I noticed two entries in "Add/Remove Programs" for the MSSCCI Provider. Evidently installing the new version on top of the old one breaks it. Here is what I had to do to fix it:

Uninstall one of the Entries in Add/Remove Programs
If you try to uninstall the second (now only remaining one) you get an error - so -
Double-click the installer (MSI) for the second one, and choose "Repair"
Now you can uninstall the remaining entry
Install the newest version

Voilà, the MSSCCI provider works find in VB6 (or anywhere else for that matter). So the moral of the story, if you use TFS with the provider, whenever a new release comes out, make sure to manually unintall it before installing the new one.

Fun with VS2008SP1 and VPC

2008-08-24T17:06:00Z

So I'm preparing for my presentation at the Upcoming Codeapalooza event: which will be Saturday September 6 at 2:30 PM in Wheaton (like how I got a little plug in on this blog post?). This will be a "hands-on" presentation so I have a separate Virtual PC representing the Client workstation from my TFS Virtual Server. Naturally I want to demo with the latest and greatest, so I upgraded my TFS Virtual Server (installed on Windows Server 2003) and the Client (Windows Vista Business) both with their corresponding Service Pack installs.

I had zero issues with the TFS server, both the Visual Studio SP (since I have Team System Web Access installed I have to have Team Explorer installed). But I'm chugging along on the client and decide to add some stuff to source code. Since I'm used to SP1 features (man, it doesn't take very long does it?) I try to drag a folder from Windows Explorer to Source Code Explorer in Team Explorer (I think I just lost a few IQ points typing that..) But nothing happened!

Hmm, something clearly isn't right. I discover, somehow, the SP1 installer, which reported no errors, didn't upgrade the Team Explorer component. Well at first I figure I can (unhappily) live without those features, until I get a Work-Item customized. Then, every time I tried to open the Work Item forms from Team Explorer I get a "Could not load type 'Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemTypeDeniedOrNotExistException' from assembly 'Microsoft.TeamFoundation.WorkItemTracking.Client" exception. When I first saw that I thought I might have made a mistake with the customization, so I open the item up in TSWA and no error. Same with Team Explorer run directly on the TFS server.

The funny thing, everything works fine on the client, I just constantly got that error (and I mean constantly, every time I opened a Work Item, ran a query, or whatever). The "annoyance factor" overrode the "nothing is broke but the error message so I can live with it" factor (especially given that I am planning on demoing with the VPC and would look pretty shabby with a "Yeah, I get this error all the time, just ignore it" in my presentation.

So I decide I had to fix it. Not much info out there on google beyond what I already knew (you get this when SP1 isn't installed). So next I tried to re-install SP1, but no dice. For some reason the installer for SP1 just didn't recognize that I had Team Explorer installed, so it didn't update it.

So my only recourse was to uninstall Visual Studio 2008 and reinstall the application and the Service pack. The interesting thing, when I did reinstall VSTS 2008, (from the Team Suite Edition I got from a MSDN subscription, which I have a copy of thanks to my former colleague and Team System MVP Steve St. Jean). Before I installed SP1, I decided to run Visual Studio 2008. And the curious thing, this time Team Explorer didn't get included automatically. I had to do a separate install. I'm not sure why - but it likely had something to do with the previous issues.

I then proceeded to install SP1 and this time, Team Explorer got updated properly as expected. The downside, on a Virtual Machine, even on a host with a decent processor (Intel Q6600) and 4GB of memory, it took a significant amount of time.

Perhaps there were other work-arounds that would have been accomplished faster, but I do think a straight uninstall/reinstall is "cleanest".

Output Issue with Transcript in Powershell

2008-07-14T14:30:00Z

As I mentioned in my previous blog post, there is a "bug" in the current version of Powershell with the Transcript. For "external" commands, (that is programs that aren't Powershell commands) the transcript will log the input but not output. I first encountered this when I started calling "Robocopy" instead of "Copy-Item" because of the speed and features Robocopy provides over the native "copy-item" command.

My work around is very simple. I need to capture the output anyway to determine if there is an error. So I assign the output stream from Robocopy to a variable and then simply use "Write-Output" to display it in the stream. The drawback is that the interactive user is going to have to wait until the command has completed to see the output - but again, I need to parse the output for errors - so not much I can do about that.

Here's my code snippet:

using Robocopy for speed and more power with exlusion

"copying files with robocopy with the following command: $exec"

$exec="robocopy $sourcePath $destPath /XD _config* Deploy /s"

$out = Invoke-Expression $exec

#write the log to stream

Write-Output $out

#check if there was an error

$fail = $out -match "ERROR :"

if ($fail)

{

throw("Failure in deploying ProcessLauncher Files")

}

Another solution was provided by a reader to the blog Seb from New Zealand (thanks for reading Zeb!) which he got from the Microsoft Powershell Newsgroup: which is to use:

Function Fix-Transcript ()
{
Process{
Write-Host $_
}
}

ipconfig | Fix-Transcript

Stop-Transcript

While it’s a little “noisier” I think I prefer the Write-Output as I think it is a little clearer what’s happening.

"Logging" Script Output in Powershell

2008-07-12T13:41:00Z

I've been using Powershell for deploy scripts from TFS builds at my current gig. I COULD have used MSBuild, but I chose Powershell for a number of reasons (which I'll detail in a future article on the subject). One important feature (particularly for something like deploying "bits" to production) is to save a "log" of all output from the script. In case of failure, you'll want to review the errors. For troubleshooting in the future, it's especially important to know what files were copied etc. And auditors love stuff like that...

A long time ago, in a technology far far away, I wrote database scripts for a Sybase database on UNIX using Kornshell - so I have some (albeit decade old) experience with scripting. Normally you would create the "log" by re-directing the output from the script to a file. The problem is that you have to wait for the script to finish and type a command to see the results. So I would use the "tee" command that would split the output to the console and a file.

When writing my Powershell scripts I found they have the "tee-object" so all was good. I called my script with a line like this:

.{.\_DeployHostApplication.ps1 -Env QA -WebTargetName "\\corpgp01\ARCS-Services-bin"} | tee-object $file

The problem, tee-object only copies the "success" output-stream and not errors. In Kornshell there was stdout and stderr and you could redirect with 2>& (or some such completely obscure and unreadable syntax, it's been a while) that redirects both output streams.

So doing research with the handy tool known as "Google" I looked at various approaches and found out that Powershell has a feature built in (ah those Powershell authors are so smart!) with commands "start-transcript" and "stop-transcript" that writes all output automagically to a file! This was EXACTLY what I needed!

The only issue, without error control, if a severe error occurs that causes the script to fail, the "stop-transcript" won't execute. That "transcripting" continues and everything you type at the Powershell command line continues to be written to the file. (I was getting "recursive like behavior as I was typing the log file and it would append the output, doubling the size of the log as it went on!).

But this is no problem in Powershell as it has Error Trapping built in. It's sort of like the old classic VB "On Error" (as opposed to Try-Catch) where you have a script block that executes if an error occurs. So I wrapped the stop-transcript command in an error trap and all was fine. Here is the new code:

$dt = Get-Date -format "yyyyMMdd_hhmm"

$log = ($MyInvocation.MyCommand.Name).Replace(".ps1","_$dt.log")

$file = "..\..\" + $log

start-transcript $file

trap { stop-transcript; break}

.\_DeployHostApplication.ps1 -Env QA -WebTargetName \\corpgp01\ARCS-Services-bin

stop-transcript

So here is the final script. The first three lines determine the filename I want to log to. The name will match the Powershell script name with a date and time stamp followed by “.log” instead of “.ps1” and two directory tree levels above where the script was located.

Lines 4-5 start the transcript, and uses the error trapping code to stop the transcript and then break out of the script (alternatively I could use “continue” instead of “break” to continue the DeployHostApplication script instead of breaking at the line that caused the error)

Line 6 executes my script, and line 7 will do the stop-transcript when there are no errors.

The only other issue I want to mention, “stop-transcript” only writes POWESHELL output. A subsequent blog post will show how I solved that issue…

I'm a published author!

2008-06-18T18:09:00Z

My article "An Introduction to Continuous Integration with Team Foundation Server 2008" was published in the June 2008 Edition of .NET Developer's Journal! The PDF can be viewed The URL is: http://pdf.sys-con.com/Dotnet/Walk_Spread.pdf (I think that may only be valid while it's the "current issue" and back issues require a paid subscription...)

UTF-8 is NOT ASCII!

2008-06-18T14:42:00Z

At my current client we had an interesting (and probably very common issue). The project we're on is to integrate my client's internal accounting applications (a mixture of Microsoft Dynamics 6.5 and custom applications) with "GetPaid" from Sungard. The vendor asked for a file, we asked what encoding they wanted and they stated "UTF-8". We produced the software to extract the data and run the vendor's utility code to load it - and it fails because of "bad data" in the comments field.

The comments field is a free form field for a customer. My client's intrepid QA Tester showed screen shots of the data in the Accounting System that had no issues, but did include a backwards quote (apostrophe, accent grave or whatever you want to call it). My colleague from Magenic Scott Janssens had written a general-purpose C# utility that took an XML document as input which would specify a stored procedure name and specifications to convert columns from the query to text. As instructed, Scott specified UTF-8 encoding.

Scott did some research (a cautious developer always takes into consideration THEY did something wrong). A further bit of confusion being the database in question was SQL 2000. SQL 2000 doesn't use UTF-8 but the now obsolete UCS-2. But ADO.NET makes that conversion without any issue.

Scott researched the issue and explains it as well (or better) than I can as follows:

Here's the skinny on encoding:

Firstly, UTF-8 is not ASCII. I had this wrong and I suspect SunGard is mistaken in the same way I was.

ASCII (aka ANSI, aka Plain Text) stores a character using a single byte.

UTF-8 is a Unicode encoding which stores a character using 1, 2, 3, or 4 bytes.

In UTF-8 all the ASCII characters are represented with one byte. This makes UTF-8 backwards compatible with ASCII.

The problem we're seeing is when the UTF-8 text in the database contains non-ASCII characters. In the case below, the character is a Unicode apostrophe. This is different than the ASCII apostrophe. The Unicode apostrophe is represented by three bytes. When read with the proper encoding these three bytes display as an apostrophe. When read with the ASCII encoding, each of the three bytes is displayed (because the encoding assumes each byte is a character) with whatever ASCII characters the byte values correspond to. The result is gibberish.

These values in the database are probably the result of the user cutting and pasting text from Word or Outlook.

The file we're sending Sungard is currently encoded in UTF-8. I suspect they want it in ASCII.

Depending on the needs of the project sending the data in ASCII may or may not be an issue. When converting to ASCII, any character that doesn't exist in ASCII is replaced with a question mark. So in our example below, "doesn't" becomes "doesn?t". If we need to keep the strings as they appear in our database, Sungard will need to properly read UTF-8. If we don't care about non-ASCII characters appearing as question marks, then we can send the data to Sungard in ASCII format.

The change to make the converter output ASCII is a one line change, changing the encoding object from UTF8Encoding to ASCIIEncoding.

Interesting stuff - good work Scott!

Found a Bug in Team Build

2008-04-24T16:21:00Z

I reported a problem on the MSDN TFS Build Automation Forum and Jason Pricket from Microsoft verified that it is, in fact, a bug in TFS 2008. My client previously hasn't used CI - so I setup the builds for the Application in TFS as "Daily Builds". Or specifically the build's "Trigger" is "Build Every Week on the following Days" with Monday through Friday checked. There's also a Check box for "Build even if nothing has changed" - which I have UNchecked.

The issue is that I have build automation using a task from MS Build Community Tasks that takes a version number from a Text File to ultimately update the Assembly Version, which I of course then check-back in to Version Control. Obviously, I don't want that check-in to be considered when evaluating whether "nothing has changed"

For CI, this would cause an infinite number of builds, so they added a feature where if you include "***NO_CI***" as a comment on your check-in, that check-in isn't considered when determining whether to trigger a build.

While having builds happen every night regardless of whether anything changed obviously isn't a serious of an issue as infinitely triggering builds, but this problem makes the "Build even if nothing has changed" useless - cluttering up your universe of "Builds" with unneeded ones. This could especially be an issue for Shops with lots of small applications that don't have a lot of development activity against them.

Of course - this is an argument for just turning on CI....

I reported this bug to Microsoft here. Forum post where Microsoft acknowledged it IS a bug is here

Another Build Glitch

2008-02-15T19:15:00Z

So here I am cruising along on my TFS Implementation. Got the code into version control redoing the File References in a way that will end the misery we're living with on the old system and ready to start building the Application. Instead of one big solution, I'm breaking the Application into it's "Deployable Components". For this app, they have two Web Services Projects, a Web application, a DLL for Notification Services, a Console Application and a Click-Once Deployed Windows Form Application. So I created six solutions instead of the one. I create my build types and build.

I start with one of the Web Services and the web application. I get my DLL's built quite nicely, but there's no _PublishedWebSites folder in my binaries tree (let alone on the drop site). I try to dig in and see what's wrong with my build type definition. Can't find anything. I first try Googleing on "How to build Web Applications in TFS" and just about EVERYTHING refers me to a page on MSDN that no longer exists (gotta love that). I keep digging and I'm SURE I'm doing everything correctly.

So I get frustrated and post the problem to the MSDN Forum. That helps me focus - and I decide the next step is to build a "Hello World" application. I build a Web Application with a single page that says "Hello World" and an <ASP:Label> where I pass in the date. Add it to Source Control, setup a build type, build it, and yup - I get the proverbial _PublishedWebsites folder nicely organized to deploy to a server.

Then I did some more Googleing and ran across This blog post from Donn Felker, which I had scanned past and initially didn't realize was my problem. So I open up the .csproj for Hello World - and yup there is a line in there for the Microsoft.WebApplication.targets. I then opened up the csproj for the application I'm migrating from VSS and unlike Don's where the line was commented out, in my projects, it was missing entirely.

So I edited the file, inserted the lines - and presto - I get a _publilshedWebsites folder!

So thanks Donn for blogging about your solution - saved me some serious time!

Now to move on to ClickOnce publishing....

Keep those Versions Straight!

2008-02-07T21:52:00Z

I had been pulled away from the TFS project for a week to help out on another and finally was able to get back to TFS. My client has a Framework they wrote that is a mix of CSLA, Microsoft Enterprise Library and custom code. I have that building successfully so I had moved on to the Application itself. I made some extensive changes to how the referenced DLL's work (including the aforementioned Framework) - so I did that all manually after removing the bindings to source safe.

So I go ahead and add the solution to TFS Source Control and create a build type and BOOM I get a very bizarre error:

Target "CoreCompileSolution" in file "C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets" from project "c:\TFSBuild\5\BuildType\TFSBuild.proj":
Using "WriteLinesToFile" task from assembly "Microsoft.Build.Tasks.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "WriteLinesToFile"
C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(978,5): error MSB3491: Could not write lines to file "c:\TFSBuild\5\Sources\Steriworks\Main\Source\SteriWorks.sln.Debug.vsprops". Could not find a part of the path 'c:\TFSBuild\5\Sources\Steriworks\Main\Source\SteriWorks.sln.Debug.vsprops'.
Done executing task "WriteLinesToFile" -- FAILED.

Googleing the problem brought me to this blog post by our friendly neighborhood Magenic TFS guru Steve St Jean: which explained the error - but a symptom caused by different reason.

What's going on here is that the Build was trying to create a vsprops file in the solution directory - but this directory is read-only (needless to say - coming straight from the Source Control) so it "blowed up real good". But the question was why?

Digging into the "C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(978,5): " was even stranger. Here it is:

<WriteLinesToFile
          File="$(VsPropsFile)"
          Lines="$(VCOverridesOpen) OutputDirectory=%22$(OutDir)%22%3E%0D%0A$(VCOverridesCodeAnalysis)$(AdditionalVCOverrides)$(VCOverridesClose)"
          Overwrite="true" />

Well, my project isn't a C++ project and furthermore the variable there refers to Code Analysis which I'm not using (at the moment at least).

So I decide to branch the code (Ah such TFS goodness - that's something always to be avoided in VSS), start whacking parts of the Solution to find out what's causing it. I start to do that and then go to create a new build type when I realize:

I had created the build type in Visual Studio 2005!

My client is still using VSTS2005 for their development. Several of the developers are still on Windows 2000 (they skipped XP and are migrating from Win2K to Vista) and haven't decided when they want to take the plunge to 2008. But since we started the TFS project after Team System 2008 was RTM we decided to use the 2008 Version of TFS.

So when I am working on the source code in Visual Studio - I need to use VS2005 - but when I do some customization for TFS - I need to use 2008. However the Icons are the same - (I changed on my desktop - but on the Task bar - they are the same) - so it can be tricky keeping which one is straight. I knew that you couldn't "queue" a TFS2008 build from VS2005 - and I never intended to create the build type in VS2005 - but I did Obviously, this is another incomparability between 2005 and 2008 as far as creating Build Types are concerned.

So I deleted the Build Type, recreated in Visual Studio 2008 and presto! The error went away!

Now to get the build to work! (It's broken - but at least it's errors I expect to get at this point in the process)

Digging into the "C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(978,5): " was even stranger. Here it is:

Well, my project isn't a C++ project and furthermore the variable there refers to Code Analysis which I'm not using (at the moment at least).

I had created the build type in Visual Studio 2005!

So I deleted the Build Type, recreated in Visual Studio 2008 and presto! The error went away!

Now to get the build to work! (It's broken - but at least it's errors I expect to get at this point in the process)