I have had the good fortune of a ring-side seat to the creation and shipping of several widely used software products. I have also been a voracious consumer of software products. To a lot of software users who have not been on the other side – the experience is very similar to going to a restaurant for a bite. The ambience, the friendliness of the staff, the presentation of the food, the cost of the visit are some of the key factors that determine your experience. Very rarely do you get a glimpse at the kitchen and almost never do you get to see the inventory sheet, the recipe list etc etc.
I bring this up because I happened upon what I consider a very englightening and admirable post from Chris Sullivan, a Program Manager on the Home Server team at Microsoft. In said post, Chris goes into some detail on how many bugs/feature-requests were recieved against the product since it’s gone into beta and how they are being addressed.
I am heartened to see a team connect with their beta testers (and the rest of the world) in such a fashion. For someone who spends hours patiently installing a whole new operating system and configuring it and dealing with hardware and driver idiosyncracies, to know that the team is looking closely at what you and the other have reported and that they take the time to talk back to you is great encouragement to continue helping to make the product better.
Then there are the downsides. The innards of software are not always beautiful. The process of getting to a beautiful end product is not always beautiful. Connecting to the world outside as an exception to established norm might cause trouble. Would Chris have thought that his blog post would get picked up by media? In a world where every move that your company makes is scruitinized under the microscope, there is always that chance that the information that has been presented would undergo the same treatment.
Not everyone who has looked into the restaurant kitchen react calmly.
While the number of bugs reported (and also the impressive number of them fixed) seems large, one needs to understand that the product in question is a whole operating system – undoubtedly one of the most complex pieces of consumer software in existence. For comparison I did a quick query on one of my old hunting grounds: the Firefox bugzilla. Searching for bugs filed against FF2 after it’s beta release and before it’s final release was a similarly large number. I imagine if you looked at the bug counts on other beta software, you might see a number that to the average “kitchen-blind” consumer might seem like a lot.
I do hope that there will be more engagement of this sort from other product teams with their consumer communities. Home Server team and Chris – good job!