Just A Programmer We're just programmers

22Nov/121

Announcing poshrunner.exe so MyScript.ps1 can use MyScript.ps1.config instead of powershell.exe.config

I have a tendency to do odd things with technology so that things don’t just work. When I point out the obscure edge cases I find, most people tell me, “well don’t do that.” I usually ignore them and dream of tilting windmills. Well today a windmill has been tilted, and this is the epic tale.

I’m a developer that fell in love with PowerShell. As such I often call .NET API functions from powershell scripts. This usually just works. However, it kind of falls apart when you have to use settings in the app.config file. This means its basically impossible call functions from a DLL that use NHibernate, Entity Framework or WCF Service references. (However, WCF Services can be called direcctly from PowerShell quite easily)

The solution is to run the PowerShell script in a new PowerShell Runspace in a second AppDomain that uses its own app.config. However, things quickly fall apart because you need to write three classes that inherit from PSHostRawUserInterface, PSHostUserInterface and PSHost respectively or else Write-Host will throw an exception.

Now all this is a lot scarier than it sounds. However, it stops two important groups of people from ever using PowerShell to call DLLs that absolutely require you to manipulate your app.config:

  • People scared off by the word AppDomain
  • People that realize they have better things to do than everything I described above

Lucky for these two groups of people, I wasted my time so they didn’t have to! The project is currently called AppDomainPoshRunner, and I ILMerge it (via IL-Repack) into poshrunner.exe. Right now poshrunner takes one command line argument, the path to a script. If the script exists it will run it in an AppDomain whose config file is scriptname.config. Log4net configuration is read from a file called ADPR.log4net.config in the same directory as poshrunner.config.

The full background is to long and convoluted for this post. This was all born out of a problem with calling New-WebServiceProxy twice in the same PowerShell console. I use log4net to write the console messages so this has the potential to be quite extendable. Then Stan needed to run PowerShell scripts from msbuild and was complaining to me about it over twitter. He didn’t like the hacky solution I had then. Eventually I realized this was the way to simplify my previous solution.

So download the zip file. Try it out. Complain to me when you find bugs!

TLDR; Unlike powershell.exe -file foo.ps1, which uses the shared powershell.exe.config, poshrunner.exe foo.ps1 uses foo.ps1.config, for great justice. Download it now!

8Nov/120

Windows Internals Study Group – First meeting

Last night myself and two others had out first planning meeting via google huddle for our Windows Internals Study group. We will be meeting next Wednesday 2012-11-17 at 20:30 EST to discuss the first two chapters of Windows Internals 6th edition. If you still want to participate its not too late, just let me know.

One thing we decided was to make all our notes public. Right now they are being stored in a google drive shared folder that is publicly accessible. The information there will grow with time.

22Oct/120

Excel Link Dump

I was recently told I had to train one of my companies clients to access our API’s using Excel. This ended up being a miscommunication, and the client wanted to use C#. However, I spent a day re-learning Excel VBA before I got the correct information. Relearning involved writing a simple app with out API and doing a lot of googling to fill in the large gaps in my knowledge.

I decided to post the list of useful links I found here, mostly for my own reference. If I ever have to touch Excel again, I’ll add to this list, and perhaps curate it a little better.

Perhaps I might be inspired to rewrite the Excel Reverse DNS macro I wrote many years ago.

18Oct/120

Continuous Integration with Windows Azure SDK 1.7, Powershell and TFS Build too!

Environments change and the solutions to support them have to keep up. I was very entertained with my old deployment solution for a good while. However, we eventually moved to Azure, and I needed to scramble to find something new. Tom Hollander’s Automated Build and Deployment with Windows Azure SDK 1.6 filled that void until I upgraded my project type to the 1.7 SDK. At that point, I realized I had to roll up my sleeves and cobble something new together.

From an automated deployment standpoint, the crippling change between the 1.6 to the 1.7 SDK, is the lack of an “ImportAfter” folder, which allowed us to include legacy msbuild files to attach to the build process. This is what Tom used to attach a Powershell deployment script to the SDK’s Publish build target. However, with the Azure 1.7 SDK, I had to figure out how to execute that PowerShell script myself.

Microsoft: Downloads for Managing Services in Windows Azure

GitHub: StanleyGoldman/AzurePublishHelpers

Creating a Management Certificate & Publish Settings file

Visual Studio has a link which allows you to download a publish settings file, without completely explaining what the side effects are. I myself didn’t understand the problem when I encountered the first symptom, you have reached the maximum number of management certificates. I was forced to understand the situation when I tried to get publish settings files for the 2nd and 3rd Azure Subscriptions my account was associated with. The link creates a management certificate, uploads it to your azure account and provides you with a .publishSettings file to install onto your machine. Life is actually easier when we start taking control of our management certificates.

It is easy enough to create a management certificate. Gaurav Mantri’s blog post really helped on this.

makecert -sky exchange -r -n "CN=JustAProgrammer" -pe -a sha1 -len 2048 -ss My "JustAProgrammer.cer"

We can take this certificate and upload it to the Management Certificates console in Azure. Take note of your subscription id and the thumbprint of the certificate. As you will need it to create your publish Settings file.

Using the PublishSettingsCreator utility, we can create a publish settings file to carry our management information.

PublishSettingsCreator.exe "AzureExample" "ecd7cc1d-12ec-8cf6-a60b-0cf14db32020" "99DFFB9D05D0B5B92893FBBA35988DE281E01E9E"

In order to use the Azure Powershell Cmdlets we have to import the Azure Modules into our Powershell session.

PS C:UsersAdministratorDesktop> Import-Module 'C:Program Files (x86)Microsoft SDKsWindows AzurePowerShellAzureAzure.psd1'

And now we can import the publish settings file.

PS C:UsersAdministratorDesktop> Import-AzurePublishSettingsFile .ecd7cc1d-12ec-8cf6-a60b-0cf14db32020.publishsettings
Setting: AzureExample as the default and current subscription. To view other subscriptions use Get-AzureSubscription

Importing the settings file sets the subscription as default. We can get the default subscription as follows.

PS C:UsersAdministratorDesktop> Get-AzureSubscription
 
SubscriptionName        : AzureExample
SubscriptionId          : ecd7cc1d-12ec-8cf6-a60b-0cf14db32020
Certificate             : [Subject]
                            CN=JustAProgrammer
 
                          [Issuer]
                            CN=JustAProgrammer
 
                          [Serial Number]
                            9157E76A714509B54F8B853A1B80555B
 
                          [Not Before]
                            10/18/2012 8:21:06 AM
 
                          [Not After]
                            12/31/2039 6:59:59 PM
 
                          [Thumbprint]
                            99DFFB9D05D0B5B92893FBBA35988DE281E01E9E
 
ServiceEndpoint         : https://management.core.windows.net/
SqlAzureServiceEndpoint :
CurrentStorageAccount   :
IsDefault               : True

Before we can use this subscription to deploy, we have to set the storage account for the subscription. You can CurrentStorageAccount is not set in the subscription definition above.

PS C:UsersAdministratorDesktop> Set-AzureSubscription -SubscriptionName "AzureExample" -CurrentStorageAccount "azureexample"

With the publish settings file and Import-AzurePublishSettingsFile, Set-AzureSubscription commands we can allow any machine to deploy using said Management Certificate. The certificate and publish settings file should be guarded well, either of these files allow access to your azure subscription.

Customizing the Build

Create a new build definition and configure it to build the solution. Be sure to add the “Publish” target to your build. The argument will cause the Azure 1.7 SDK to create the deployment package during build.

If you are using TFSBuild, you can do this while configuring your build, look for the field ‘MSBuild Arguments’, add set the value ‘/t:Publish’.

Finish configuring the build definition and queue the build. We are going to use the output from the build to test the powershell script.

Testing the Powershell script

Using the output from the build, we should be able to execute the powershell script. Copy the script over to your build server. Execute it with the path to the publish file, location where the output Package and Configuration file can be found and the name of the package file in that location. A tag can optionally be specified to help identify the build. I usually use the version number of the binary here, but the build label works just as fine.

PS C:UsersAdministratorDesktop> C:BuildsAzureDeploy.ps1 
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzureProfilesAzureExampleProduction.azurePubxml'
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzurebinDebugapp.publish' 
    'AzureExample.Azure.cspkg' 
    'Test LabelTag'

Note: I’ve had some problems with the Azure Powershell Cmdlets and relative paths.

If your machine is configured correctly, this should deploy without any problems. If you are using TFS the next section is useful for you to wrap this all together. If you are using a build system other than TFS, you already have what you need to continue. Enjoy!

TFS Build Process Template

The Build Process Template included in the package takes a few arguments and handles the execution of the Powrshell script rather nicely. After choosing it as the template for your build, you just have to specify a few arguments.

Deployment Configuration: The build configuration of the Azure project, “Debug” or “Release”
Deployment Profile: The name of the profile file to be used
Deployment Project Name: The name of the project Azure project
Deployment Script: (Optional) In case you dont keep your deployment script in the same spot as mine

The build process template uses the Build Label as a tag for the build. It might be a bit more useful to use something like github: martinbuberl/VersionTasks to tag the builds.

Enjoy Continuous Integration

It never seems worth the effort until after you are done with it.

15Oct/120

SQLCLR wrapper for RAISERROR()

Recently I was writing a SQLCLR stored procedure. I made several calls to RAISERROR() in the procedure. While this meant that my stored procedure would be easier to debug in the future because of useful error messages, there was a lot of ceremony involved in these RAISERROR() calls. Therefore I decided to encapsulate all this ceremony into a simple static method.

RAISERROR() presents a small challenge in wrapping in a C# function. That challenge is the variable parameter length due to the printf() like parameter substitution. The first step to overcoming this is to use the params keyword for the substitution parameters. The second step is avoiding the possibility of SQL Injection.

Many built in CLR functions use the params keyword. For example, if you use the String.Format(), Console.WriteLine() or Debug.Print(), with substitution parameters you have used overloads with the params keyword. For example, lets look at these two calls to String.Format();

String.Format("Name: {0}", "Justin");
String.Format("Name: {0} Occupation: {1}", "Justin", "Developer");

It may look like I’m using two overloads, but I am not. One might also conclude that String.Format() takes one string and eleventy billion object parameters all of which have a default of null. However, parameter defaults are new in .NET 3.5, and even the long forgotten .NET Framework 1.0 had a fully functional String.Format(). In actuality, the signature for String.Format() is as follows:

public static string Format(string format, params Object[] args)

Basically, the params keyword lets the user pass a variable size list of parameters, and populates them in a singe argument as an array. So now that we know about params, it seems like it shouldn’t be too hard to write our function lets give it a go.

private static void RaisError(SqlConnection cn, string message, short severity, short state, params object[] args) { . . . }

Now of course you want to protect against SQL injection. The best way to do that is to use the SqlCommand.Parameters collection to scrub the parameters. While you may be very clever, the ADO.NET code is very well battle tested. The only problem is if you don’t know how many parameters you are passing, how do you know how to make the string. The answer, is to generate SqlCommand.CommandText dynamically.

This may seem like you are opening yourself up for SQL injection. However, you aren’t. You simple have to generate a string that contains @arg1, @arg2, @arg3 . . . based on the length of the object array that gets passed to params. You then populate the SqlCommand.Parameters collection with the values of that array. If you don’t feel comfortable with this, then perhaps a stackoverflow answer with 268 votes at the time I wrote this article advocating this practice would convince you.

So putting it all together here is our function:

The above function is copyright 2012 myself (Justin Dearing). However, I release it under the MIT license, so feel free to use it in any of your code.

24Sep/122

Looking to improve my software craftsmanship in Cambridge, England on Saturday 2012-10-20

I’ll get right to the point. My employer needs me to be in London on Friday 2012-10-19, and I decided to stay the weekend. I settled a nice youth hostel in Cambridge. At first I was thinking of touring the museums and university. Then I figured, why not see if I can become a better programmer while I am there?

So first the basics. These days I’m fluent in C#, PowerShell, T-SQL and MongoDB. I work for a large company that produces propeitary closed source software that I cannot talk about or share. However, I do contribute to some open source software, and theres nothing stopping me from working on your projects for the weekend. My full resume is here. Also, I’ve never done this sort of thing before, or even tried pair programming so this is entirely new to me.

I’m looking for someone to propose an interesting project for me to pair with them for a day on. If it’s some closed source project you are working on, then I’d expect some form of compensation. It could be, dinner, single malt scotch, a used iPad, etc. If its open source, then no compensation is necessary (but I’ll never turn down a well earned free lunch). I’d like to see tangiable results from our work in a day. If those tangiable results are simply turning unit tests from red to green, or increasing code coverage, that’s ok . I don’t care if the situation works out that one of us is a better programmer or we are both equal programmers. There is something to be learned on both sides of a teacher/student relationship.
Are you interested? You can reach out to me in the comments, or send me a mention on twitter. My twitter handle is @zippy1981.

6Sep/122

Why should developers learn and use PowerShell?

Its no secret that I’m a PowerShell true believer. I’m often frustrated why many of my fellow developers don’t fall in love with PowerShell the way I do. However, today, I found this interview by Doug Finke in which he explains why developers should use PowerShell.

BTW, Doug wrote a great book called PowerShell for Developers. Grab a copy today!

10Aug/120

Saying Thank You and the Stack Exchange Summer of Love

Recently Joel Spolsky kicked off the Stack Exchange summer of love as a way to be proactive against the rudeness that occurs in all mature online forums. Then Josh Heyer clarified the role of niceness in a Q&A forum. I applaud this effort in general. However, I think they forgot one thing. They should remind people to say thank you.

As a participant on several of the Stack Exchange sites, mostly the original trilogy, I am no Jon Skeet, but I do try to answer questions when possible. When I answer questions, two things really annoy me:

  • Those who never upvote or mark an answer as correct
  • Those who never ever use the comments for saying thank you, or It worked.

See, I’m a fairly social person, and while I’m not always good at it, I’ve made an effort in recent years to remember to say thank you in emails. On top of that, I like to help people. My reward for answering your question is you saying “that worked for me.” A thank you simply does that better than a green check mark. Don’t get me wrong, I’m all about the upvotes, green check marks, and badges. The gamification is a driver, but please make me feel like I’ve help out a fellow human being to solve a problem.

29Jul/120

MSIExec and Far Manager Part 1: Command line install and uninstall of FarManager 3

This is part one in a series of blog articles in which I shed light on the internals of MSIs using the example of the MSI for Far Manager 3. It was inspired, amongst other thing, by lessons learned while creating the Far-2 and Far-3 packages for chocolatey. While the idea of writing those packages was so others would not have to learn the dark arts of command line manipulation of MSIs, I though I would write this series for those interested in how the sausage gets made.

I’ve written about my love of the File and Archive Manager before. I’m a command line guy so this utility naturally appeals to me. In addition to being a command line guy, I’m also very much a MSI guy. I always prefer installers to unzipping files somewhere, and I prefer MSI installers to exe based installers such as those made with the Nullsoft Scriptable Installer System. On the surface it might seem like these two loves would be in conflict. After all, when you run an MSI you get a GUI. However, there is a command line executable for installing MSIs, called MSIExec.

So using the example of  a recent 64 bit nightly build of Far Manager 3.0, C:userszippyDownloadsFar30b2746.x64.20120624.msi, lets explore how we can install and uninstall an MSI from the command like.

Getting some help with msiexec /?

MSIExec comes with built-in help. To see it, type msiexec /? from the command line.  Strangely, it will display that help in a window instead of the console. This is similar to the behavior of ntbackup.

Let’s install Far Manager!

Looking through the command line install options, /i is the switch to install an MSI. To have no GUI feedback I can use /quiet. If I want just a status bar I can use /passive. So  to install far automatically, I can use the following command:

msiexec /i "C:userszippyDownloadsFar30b2746.x64.20120624.msi" /passive

This will give you a default install of Far Manager 3.0. However, if you use Far, you are by definition a power user, and you probably don’t pick the default install options. It is possible to customize what features to install during a command line install. I’ll explain your options for doing that in part 2.

Time to uninstall it

I can only see one reason to want to uninstall Far Manager 3.0. That would be of course when Far Manager 4.o comes out! Since Far 3.0 is still under development, that event is probably at least 2 years away. However, we want to be prepared for that day. Also, Far is just the example we are using. There are other programs that are installed with an MSI that deserve to be uninstalled.

If you examine the msiexec /? documentation, then you will see that the uninstall syntax is msiexec </uninstall | /x> <Product.msi | ProductCode>. The /uninstall and /x switches are identical, but I prefer /x because its terser. ProductCode is a GUID that I’ll explain how to get later. For now, lets use the path of the original MSI. So in our example the command is.

msiexec /x "C:userszippyDownloadsFar30b2746.x64.20120624.msi" /passive

That’s fairly simple. However, you don’t always have the MSI available for something you want to uninstall.

Finding the Product Code

The ProductCode is a GUID. The method for finding it is not obvious. I’ve resorted to using WMI, but there are probably other methods. Specifically, I use the Win32_Product class. The best way to do this from the command line is  with the PowerShell cmdlet  Get-WmiObject. The command to search for all instances of Far Manager installed via MSI on your system is:

Get-WmiObject Win32_Product -Filter ‘Name LIKE "Far Manager %"’

This returns the following on my machine:

IdentifyingNumber : {143F0C11-D9F3-4F1E-9037-67BBFDD379AD}&lt;br /&gt;<br />
Name              : Far Manager 2 x64&lt;br /&gt;<br />
Vendor            : Eugene Roshal &amp;amp; Far Group&lt;br /&gt;<br />
Version           : 2.0.1807&lt;br /&gt;<br />
Caption           : Far Manager 2 x64&lt;/p&gt;<br />
&lt;p&gt;IdentifyingNumber : {B0911B7C-968A-4448-BAF2-0FF2DA34B805}&lt;br /&gt;<br />
Name              : Far Manager 3 x64&lt;br /&gt;<br />
Vendor            : Eugene Roshal &amp;amp; Far Group&lt;br /&gt;<br />
Version           : 3.0.2779&lt;br /&gt;<br />
Caption           : Far Manager 3 x64

As you can see, I have versions 2.0 and 3.0 of Far installed. I could uninstall Far 2.0 with the command msiexec /x {143F0C11-D9F3-4F1E-9037-67BBFDD379AD} /passive. If you want to get fancy and uninstall both at the same time, you can with the help of the Start-Process cmdlet.

Get-WmiObject Win32_Product -Filter ‘Name LIKE "Far Manager %"’ | %{ start msiexec ‘/x’,$($_.IdentifyingNumber),’/passive’  -Wait }

Special thanks to Trevor Sullivan (blog|twitter), for helping me out on a Sunday with the Start-Process syntax.

I’ve barely scratched the surface of what you can do with the MSIExec. However, its a good start. In Part 2 I’ll discuss how to customize the install.

23Jun/122

Firing with dignity: In defense of the immediate escort

Earlier this week, Chris Dixon (blog|twitter) wrote about firing from a startup. I’ve never worked in a start-up. I’ve also never ran a company or fired anyone. However, I have been fired from an established mature company. In the comments I shared my experiences on being fired and how I felt that being escorted out the door and not being allowed to say goodbye to my coworkers actually added dignity to the process. Another commenter, Phillip Rhodes disagreed. I wish to expand upon my thoughts on the matter here.

This is the first of a two part series. In this part I will talk about my experiences at an organization where coworkers of mine were fired. In the second I will talk about the time I was fired.

One of my first jobs was at a medium sized local ISP. It was not a perfect company by any means, but I learned a lot there, and my time there shaped my opinions about many things including my belief that it is in the fired person’s best interest to be immediately escorted out the door.

Several people were let go during my tenure at this company. There was one period of layoffs due to a lost client. However, most were firings due to performance and there was only one I’d argue as being an outright mistake. I believe that some level of escorting was involved in all of them, but I only witnessed the immediate aftermath of one.

The one I witnessed was my boss. I had at this point risen to the rank of Unix/Midrange administrator. Two senior unix admins gave notice recently, and two more were brought on. One was to be boss of myself and the other new hire. It quickly became apparent that he was not a good fit. At some point I was commanded (he actually told the help desk operator that relayed this order, “This is my command!”) to document the entire unix infrastructure, and declared I’d be the only one on call until this was completed. I did not bother to object to this because I knew his end was near. The inhouse recruiter’s whiteboard had UNIX ADMIN written real big on it.

One day shortly thereafter, the CTO called me and the other junior unix admin into his office. I put on a solem looking face as I correctly guessed we were about to be told to change all the passwords. We divided up the systems between the three of us. Then I walked back with the other junior to his cube. I needed to break the solemnity at this point so I said something to the effect of, “well thats the biggest non-suprise of the year!” At that point, my now ex-boss walked up to us with a VP escort and proclaimed, “guys I am fired.” He then walked to his desk, and gathered his things with much ado.

To this day, I’ve never had a more awkward moment in my professional career then having to interact with my ex boss who was at that point a dead man walking. I would never describe that particular person as ever conducting himself with dignity, but this was a new low for him.

In this particular case, an attempt was made to immediately escort the fired person out. Perhaps more could be done. He could be fired over lunch. Honestly, this particular person was prone to emotional outbursts. I don’t think a public setting would encourage the man to restrain himself. The only way to give the person his dignity would be to deliver the news in an abandoned building. Finally, regardless of your general policy on the removal of fired persons, this particular case warranted immediate removal for the safety of everyone in the buildings.

That had been the only firing I had witnessed at that company. However, I was involved in the password reset for another one. This is incidentally the firing I disagreed with. This is a person who’s hand I would have shook had I encountered him immediately upon being given the news, when he was a “dead man walking.”

Here’s the thing, I did get to shake his hand. We had lunch several weeks later after he started his new job. He contacted a few coworkers, and we all spread the word. He got to interact with us on his terms, with the pride and dignity of someone that recently started a new job.

Now these experiences are all vicarious to me, and I know not what lies in another mans heart, so I don’t know how these people really took  the news. However, these were formative vicarious experiences to me. As such, they shaped how I dealt with my own termination, which is the subject of the next installment in this series.

Tagged as: 2 Comments