Dealing with rejection in git, resetting your master to theirs and starting over
Posted by Justin on July 3rd, 2010 filed in Just A Programmer6 Comments »
You haven’t really started using git until you’ve gotten your first patch rejected. Due to the nature of git, merging is easier, but rejection is harder. Well if I was using branches it wouldn’t be that big of a deal, but thats besides the point.
You see I developed a love/hate relationship with a wonderful NoSQL database called MongoDB, that led me to contribute patches to it. I loved it because its awesome for a few specific tasks I need to do, but found its windows support a little lacking. It ran and performed great in windows, and even ran as a proper NT Service. However, it needed a little spit and polish. It was also a great excuse to do some hardcore windows system programming, and logging calls. Ask any sysadmin that ever had to support a program I wrote, I love verbose debug logs.
So every once in a while one of my pull requests got rejected. The first few times it took me a while to deal with it. I don’t mean it took me a while emotionally to deal with the fact that my code is anything but perfect. I mean that it took me a while to reset my git repo with the main 10gen repo. Sure I could have deleted it and started over, but I wanted to learn how to use git properly.
So in the end I asked on the Long Island Linux User Group’s mailing list and got a helpful reply from Mark Drago. I actually linked to my reply to his reply, since I make a correction. So without further ado, here what I did.
Ok a little more ado. My repo is refereed to as origin in the config and 10gen’s is referred to as 10gen. I wanted to save my master branch’s current state to a newly named branch delete my master and pull from 10gen’s master. In the end my problem was I never did a git fetch. The final sequence of commands I did type was:</ado>
- git checkout master
- git checkout -b oldmaster
- git branch -d master
- git fetch 10gen #Needed to get the list of branches in master.
- git checkout remotes/10gen/master
- git checkout -b master
- git push -f origin master
And that was it. I was free to continue to write code.
A really simple guide to creating SQL Server Table with a FILESTREAM backed column
Posted by Justin on July 3rd, 2010 filed in Just A ProgrammerComment now »
I recently helped to develop a small website to share images. I decided to use the new FILESTREAM feature of SQL Server 2008 to store the images. Along the way I encountered some gotchas, so I wrote this article to document them here.
Usually, I am the last person to want to store files in a relational database. Windows comes with a datastore highly optimized for storage of arbitrary blobs of binary data called the NTFS filesystem. However, SQL Server 2008 has a really nice feature where you can store the content of varbinary fields directly on the filesystem. This is a best of both worlds approach. Each blob in the varbinary column of your table is stored in a separate file on your filesystem. However, you can access the contents of this file through a SQL query.
Example downloads
The complete example can be found here. The zip archive contains both the SQL script to create an example database, and a sql backup file you can restore on your system.
Enabling FILESTREAM on your SQL Server instance.
There are two steps to do this. The first is enabling it either at install time via the installer, or via a vbscript after the fact. The second is turning it on via a system stored procedure.
Enabling After the Fact
You probably did not enable filestream support when you installed SQL Server 2008, unless you went out of your way to enable every feature. If you did already enable it, running this script won’t hurt anything.
The vbscript is available via the Microsoft SQL Server Engine Code Sample Project on Codeplex. That page does a good job of documenting all the command line arguments. The main concern most developers using this will have is the named instance of the SQL server you wish to enable filestream support on. Most developers trying this out are doing it from their machine. They are probably using the express edition of SQL Server 2008 which has a default instance name of SQLEXPRESS. The script defaults to attempting to enable filestream support on the named instance MSSQLSERVER. Therefore you will most likely have to specify the option “/Instance:SQLEXPRESS”.
So once you verify the named instance you want to enable filestream support on, you run the following command:
cscript filestream_enable.vbs [/Instance:InstanceName]
That was simple. Now we move on to turning on filestream support.
Turning on FILESTREAM support
Ok now fire up sql management studio or sqlcmd and execute the following:
RECONFIGURE
GO
Once again, you can run this as many times as you want without negative side effects.
Enabling FILESTREAM on Your Database
Now filestream is completely enabled on your sql server instance. However, you have to enable it on your database. We do this by adding a filestream FileGroup. In a SQL Server database, all the data and log files, .mdf and .ldf files respectively are part of one or more filegroups. With filestreams, you have a special filegroup that is a collection of directories that contain the filestream files.
Many application developers are ignorant of the specifics of sql server files and filegroups. We take these for granted because we can just type “CREATE DATABASE foo” and we get a data file and log file with sane enough defaults created automatically. We then let the DBA figure out how to setup the filegroups on staging, UAT, and production. However, we need to get down and dirty when we want to use a filestream. Luckily, with a little dynamic SQL, we can create a filestream filegroup and file without thinking too much.
When making a data file in sql server, even if that file is really a directory containing filestream data, you have to specify an absolute path. I prefer the convention of storing my filestreams alongside my mdf and ldf files. On the SQL Server 2008 R2 Express instance on my laptop, that is “C:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS2K8R2\MSSQL\DATA.” However, the exact data folder is different per each instance of SQL server. After some research, I took Walden’s slightly hackish suggestion of using the sysfiles table. This suggestion assumes you did not customize your files and filegroups in your CREATE DATABASE statement.
DECLARE @FileStreamPath VARCHAR(MAX)
SELECT
@FileStreamPath=REPLACE(fileName, DB_NAME() + '.mdf', 'FileStreamPhotos')
FROM sysfiles
WHERE fileid=1
DECLARE @sqlString varchar(max)
SET @sqlString =
'ALTER DATABASE [dbWithFileStream] ADD FILE
(
NAME = PhotoStore,
FILENAME = ''' + @FileStreamPath + '''
) TO FILEGROUP FileStreamGroup_Photos'
EXEC sp_sqlexec @sqlString
GO
Its dirty but it works.
Creating Your FILESTREAM stored VARBINARY column
There’s no point to a filestream if your not going to store VARBINARY columns in it. To store a varbinary column in a filestream, All we have to do is add the keyword FILESTREAM to our column definition, as illustrated below.
CREATE TABLE photos (
id UNIQUEIDENTIFIER ROWGUIDCOL NOT NULL
CONSTRAINT PK_photos PRIMARY KEY NONCLUSTERED
CONSTRAINT DF_photos_id DEFAULT NEWID(),
photo VARBINARY(MAX) FILESTREAM NOT NULL
GO
Getting data into and out of this column is another topic for another article. For now know that all methods for getting data in and out of normal VARBINARY columns works, plus new better ones.
Conclusions
FILESTREAM is certainly “ready for prime time,” in that it is safe and performs well. However, the feature feels very 1.0-ish. My adventures in getting it enabled were mostly due to not knowing there were two distinct steps to enabling it. If I needed to store a large amount of binary data for a project in the future, I’d certainly consider using SQL server filestreams again. However, I’d probably lean towards writing the files to the filesystem myself, and store the path to the file in a varchar column.
Appcmd.exe and his amazing superfriends in %systemroot%\system32\inetsrv
Posted by Justin on June 29th, 2010 filed in Just A ProgrammerComment now »
Preface: Regardless of the size of your organization and segregation of job roles, every programmer must sometimes assume sys-admin related duties. Sometimes the programmer is also the sys-admin in a one man shop, and even in a large shop with many programmers and many system administrators, a programmer must develop the administrative procedures for the software he develops, and usually administers the working of his own box. Therefore, even though this is a programming blog, system administration tasks will be mentioned here.
Every once in a while I ask a question knowing I will be embarrassed to learn the answer. However, if google cannot lead me to the answer, there has to be someone exactly as stupid as me with he same problem. I write these words for thats persons enlightenment, and to allow everyone else to mock me.
So I was looking for methods for generating reports about IIS virtual hosts, when google pointed me to appcmd.exe. This is a swiss army knife executable for command line administration of several aspects of windows. Much of its IIS related functionality previously existed in vbsripts. The only problem was I could not find it. It was not available on my path and explorer search did not find it. I tried to find an appcmd feature or role to windows and did not find one. Finally, I violated the unwritten code of men and asked for directions. I opened a ticket with out co-location provider.
The response I got was short and professional. The executables full path is “c:\windows\system32\inetsrv\appcmd.exe”. I realized the following steps were necessary at this point:
- Add “c:\windows\system32\inetsrv\” to the system path.
- Write this article
- Figure out why my usage of explorer search did not turn up this executable.
- Write a follow-up article on how to modify ones path and other envirormental variables in Windows Server 2008.
- Write a followup article on how to search for files in explorer in Windows 2008
So stay tuned for more ar
My new favorite tool, the Far File manager
Posted by Justin on June 21st, 2010 filed in Just A ProgrammerComment now »
Strange things excite me, things even other programmers would consider strange to be excited about. Every once in a while, something comes along that excites me in multiple ways. One of those things is the orthodox file manager, Far.
The far manager was developed by Eugene Roshal, who created WinRar. It was originally shareware, but has recently been made open source.
I’ve known about Far for a while. I first discovered it looking for a file manager that could handle a directory with thousands of files at a job where I was doing ETL operations. It was installed on my machine by a developer of ReSharper who was troubleshooting a very strange bug on my system remotely. Also I worked in a company where several Russian’s used it daily.
However, while I toyed with it several times, I never took the time to really get to know it until a few weeks ago. By really getting to know it I meant installing several plugins, and experiencing the “theres a plugin for that” joy several times over.
Overview
Far is a command line based file manager with two columns and a command prompt. The command prompt behaves similar to cmd.exe, but not exactly. For example, in a standard command prompt typing “cd e:\foo” whike you are on the c:\ drive will change the current directory on the E: drive but you still have to type e: to get to that folder. In Far typing cd e:\foo does both. One other difference, that bothered my unix sensibilities, is “cd ~” changes to the folder that far is installed to. In unix this changes to the users home directory so I was expecting similar behavior. There are probably many other useful command line enhancements that I’ve yet to discover yet as well.
Far File Manager
Installing Far and plugins.
Far is available on http://farmanager.com. There is a 1.7 and 2.0 version. The 2.0 version supports unicode asnd the 1.7 version us the legacy ascii version. You can get 64 bit binaries for both versions. You can install far via an MSI, or a 7-zip archive.
After you install Far, you will want to install several plugins. I will highlight my favorite ones here. ote that while binaries compiled against the far 1.7 SDK will work with Far 2.0, 32 bit plugins will not work with 64 bit far. For this reason you probably want to install the 32 bit version of Far, unless you are like me and like pain.
Except where mentioned, these plugins can either be found at the plugring site, or for 64 bit binaries, the evil programmers google code project. I will go through some of the plugins I like below.
7-Zip
As far as I know, there is no 64 bit version of this available yet. However, I probably just haven’t found it yet. If you install far without this plugin, you can browse the contents of most archives in Far. However, you will not be able to copy files out of them. I’ve yet to try getting the built-in archive support full working. However, with all the archives supported by 7-zip, I’m in no hurry to.
Event Viewer
This works like a text mode only version of eventvwr.exe. I’ve yet to find a truly compelling case to use it over the standar gui version. However, its nice to have an alternative tool for any job.
Service Manager
This is really convenient. It lists drivers and services temperately. It also allows you to edit things you can’t in the mmc snap-in, such as the path to the binary the service executes. Finally, it lets you create a new service. You rarely need to do this, but when you do its hard to find a good tool for the job.
User Manager
This one is really useful, especially on XP Home edition. Functionality is similar to the “Local Users and Groups” section of the Computer Management MMC snap-in on XP Pro. The thing I really love about it is you can set the “User must change password at next logon” flag on a user in XP Home Edition. I spent the good part of a train ride from Penn Station to Islip on Friday failing to achieve this in other ways. I’m not saying its the only way this task can be done. I’m just saying that this plugin will let me accomplish this task easily.
User Must Change Password At Next Logon
WinSCP
The arbitrariness of alphabetical order has put what is perhaps the most useful plugin last. There is a GUI scp/sftp client for windows called WinSCP. The author also made a Far plug-in based on the same code.
This plug-in, along with the 7-zip one, also take advantage of one of the most powerful intrinsic features of Far. With Far, you can copy any file from one panel to another, regardless of whether the panels contain a local folder, a unc path, the inside of an archive, or a sftp folder. Because of this, Far is a great tool for moving files to and from remote servers.
Conclusion
Far is a great file manager, and I will spend more time getting to know it. I think all programmers and sys-admins that work with Windows should get familiar with it as well.
CoApp: Open Source Package Manager for Windows
Posted by Justin on April 11th, 2010 filed in Just A ProgrammerComment now »
As I mentioned before, I am a big fan of msi installers on windows. So I was really excited to see Garrett Serack announce an OSS package management system called CoApp that he has been authorized to work full time on.
In my point of view, Garrett really gets it. I’m hoping as a side effect of this project, WiX will get all sorts of dependency handling improvements.
I’ll be watching how this develops very closely.
My strange ternary operator indenting
Posted by Justin on April 7th, 2010 filed in Just A Programmer1 Comment »
I’ve always indented ternary operator (?:) statements like so in languages that support them:
int fooCount = (isSeven)
? 7
: int.MaxValue;
I’ve never seen anyone else do it this way , but no one has complained about my style to my knowledge. If anyone has a strong opinion on the matter, for or against it, I’d like to know.
Cassini is now on github.
Posted by Justin on April 5th, 2010 filed in Just A Programmer1 Comment »
This weekend I created a github repository for Cassini, the small, single executable, Ms-PL web server which Microsoft released for developers. This was motivated by my SVG experiments, because Cassini does not serve svg files with the proper content type.
I coded a simple workaround to set the mime type and wanted to give it back to the community. While there are two codeplex projects with Cassini enhancements, Cassini++ and CassiniDev, I felt github was the place to host Cassini changes. There are a multitude of small enhancements that have been made to Cassini, and Cassini is not the sort of project that works well with strong central leadership.
Currently the version of Cassini available in my github repo sports the following features:
- Everything in standard Cassini v3.5.0.2, The lastest version from Dmitry Robsman’s blog
- SVGs are sent with the proper mime type. This is the itch that inspired this fork. I have a feeling that there is a better way to it, but no one on stackoverflow provided an answer to my question.
- The ability to optionally serve remote urls. Thanks to vzelenko for making his changes available.
I’m hoping that hosting my changes on github will foster the sharing of Cassini enhancements. I will reach out to both Cassini projects on codeplex and let them know of my changes so that can incorporate them into their projects.
Adventures with SVG (a lessons learned post)
Posted by Justin on April 4th, 2010 filed in Just A Programmer3 Comments »
Go into the open dessert. Draw a circle around you. All that is inside the circle represents your knowledge. All that is outside the circle represents that which you do not know. The line represents that which you know you do not know. What happens as you make the circle bigger?
– Unknown
I don’t remember where I first read a version of the above story. However, it rings true for my recent experiences. You see I wrote a simple jQuery plugin, and decided to use inkscape to generate the graphics used by it. I then discovered it was possible to animate the SVG files with javascript. Before I knew it I found myself wandering through a dessert awaiting the promised land of SVG. I’ve yet to reach the promised land, but I’ve learned a lot in the experience.
Lesson Zero: SVG is poorly documented on the web.
I need to qualify the section header a bit. The actual standard is pretty well defined. There are examples on the web, and w3schools has a section on SVG. However, SVG is not popular enough for you to be able to steal all your code from google. You will have to experiment to do a lot of simple tasks.
Lesson One: SVG has very html like DOM.
This one was a pleasant surprise actually. SVG elements have event attributes like onload and onclick. Animating is as simple as recursively calling a function via window.setTimeout(). You could call console.log() to trace your javascript’s execution via firebug or the chrome developer console. In general I was able to leverage a lot of the javascript experience I’ve had with html.
Lesson Two: I should be animating with SMIL instead of javascript.
I’ll call this a lesson I’ve not really learned. If I truely learned my lesson I would have learned SMIL and done my animations with it instead of javascript. However, I was more concerned in exploring what could be done by combining javascript and SVG than animating SVGs.
Lesson Three: SVG is inconsistently implemented in practice.
I’m not just talking about amongsT the browsers, but that too is a problem.
First of all, you need to use one of several plugins to render SVGs in Internet Explorer. IE9 will finally rectify that. The no longer maintained Adobe SVG plugin was the most popular IE plugin. However, the still alpha flash and javascript based svgweb by google is gaining popularity. That one has the advantage of not requiring any special software besides flash, which is probably already installed. However, it seems it does not support javascript embedded in the svg file.
Second, SVG support is inconsistent amongst opera, firefox, and chrome. Chrome has a nasty habit of not rendering svg embedded in html via an <object/> tag if the mime type is not properly set. I discovered this while serving html and svg through Cassini.
(As an aside, I have made available a modified version of Cassini on github so no one will have to suffer as I have.)
Third, inkscape, the editor I use to draw my SVGs, is not a complete SVG editor, and it implents several extensions to the SVG standard. The extended elements have not prevented SVGs I drew in Inkscape from being rendered in browsers since they have different XML namespaces. While I’ve discovered how to reference external javascript files in a svg through inkscape, I have to use a separate text editor on the SVG files to embed javascript in the document.
Conclusions
My first conclusion is SVG is not quite ready for prime time as a deploy directly to the web technology. However, there is enough buzz happening that early adopting might be prudent for some scenarios. Browser support is improving, and the tools are improving.
My second conclusion is that SVG is ready as a “mastering format,” like PSD, XCF (gimp native) or Illustrator. If you want to draw with a rector tool, and then render to a bitmap, SVG is a solid format. Inkscape is a good editor.
My third and final conclusion, is that the desert of SVG is very big, and my circle in the sand is quite small.
jquery.collapsiblePanel: A collapsable panel plugin for jQuery
Posted by Justin on March 31st, 2010 filed in Just A ProgrammerComment now »
I’m pleased to announce my first open source jquery plugin, collapsiblePanel.
Basically, this plugin surrounds a given html element with a box and places a title bar on top. Clicking on the title bar will toggle between expanding and collapsing the element.
My main motivation for creating this plugin is to for the tangential lessons I will learn. Some of the tangential lessons I am learning as a result of this:
- How do you use the google closure compiler?
- How do you use git, and github specifically.
- How do you automate the release build process for a static html website. This means automatically renaming the javascript file with the version number, compressing it, and creating a git branch.
- Whats is a good javascript editor? Right now I’m liking komodo edit.
- Can I use javascript and svg to create animated graphics?
The plan is to blog as I learn, sharing my experiences with you all.
Using the registry to resolve Visual Studio reference paths.
Posted by Justin on January 28th, 2010 filed in Just A ProgrammerComment now »
Note: To skip the long journey of what lead me to figuring this out, click here to go to the howto.
Recently I was asked to look at a fiddler plugin Stan, the founder of this blog, was developing. He gave me a SVN path and asked me to build it and test it.
So I checked out the source code and hit F5. I got a bunch of compiler errors relating to the fact that I didn’t have fiddler installed. I rectified that matter and still got errors. The problem was that the hintpath of fiddler.exe was wrong. On my machine, Fiddler is installed in ‘C:\Program Files\Fiddler2\’, while on Stan’s machine it is installed to ‘C:\Program Files (x86)\Fiddler2′. I consulted the mighty google, which led me to a stackoverflow question. The question pointed out that you can have multiple hintpaths to an assembly. However, I wanted a better solution. What if someone installed Fiddler to a custom location?
I got the idea of using the registry. Fiddler has an installer. Surely the installer records its install location to the registry. It does in ‘HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Fiddler2′ (Apparently fiddler is written by a Microsoft Employee). So the question is how to get MSBuild, the tool that visual studio uses to parse project files, to read a value from the registry.
The answer was found in a post on the MSBuild team blog.
How To
Unfortunately, Visual Studio does not allow you to edit hintpaths to referenced assemblies. So you’re going to have to edit your vcproj or vbproj file in notepad or some other text editor. Here are the steps:
- Open the project file in your text editor.
- Look for the <Reference/> element for fiddler.exe. It should look similar to this:
<Reference Include=”Fiddler, Version=2.2.7.5, Culture=neutral, processorArchitecture=MSIL”>
<SpecificVersion>False</SpecificVersion>
<HintPath>C:\Program Files\Fiddler.exe</HintPath>
<Private>False</Private>
</Reference>
- Change the hintpath as follows:
<Reference Include=”Fiddler, Version=2.2.7.5, Culture=neutral, processorArchitecture=MSIL”>
<SpecificVersion>False</SpecificVersion>
<HintPath>$(Registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Fiddler2@InstallPath)\Fiddler.exe</HintPath>
<Private>False</Private>
</Reference> - Save the file
- Visual studio will detect the file change and ask you to reload the file. If you are using SharpDevelop as you’re IDE, you will have to close and reopen the solution.
Thats all there is to it. Happy coding!