The Busy Web Developer and the Bad Dogfood 

Saturday, June 25, 2005 10:40:13 AM
Previous Post << >> Next Post
Rate this Content 0 Votes

This post is intended to be a playful counterpoint to Gonzalo's post about the Bad Bug Reporter and the Chocolate Factory. Though I also am serious in pointing out some major problems with the latest mono release I hope this will be taken with the good humor intended.

Once upon a time there was a web developer who worked very hard to become the best web developer he could and was always learning and improving.? His web software ran on the leading proprietary operating system of the day but one day he found out about a thing called mono that would allow his software to run on linux and other unix like operating systems. He was overjoyed at this discovery and began using mono and linux to run his software.? He found a few things he had to do differently to make sure his software always worked on mono and linux but by following a few key guidelines he could be usually get his new features working. By and by he discovered bugs in mono and he would report them to the mailing list and the answer was usually "Its fixed in svn" which meant that if you had the same version of mono as the mono developers the bug was not there. So he would wait for the next release and yes usually the problem was fixed which was good.? Then he realized that if he was to be taken seriously as a bug reporter he must setup a machine to use the same code from svn that the mono developers use so that the answer could not be "Its fixed in svn".? This was not an easy task for the web developer, he struggled long and hard and had to read a lot of documentation and learn a lot of things new to him in order to get setup like a mono developer. But he was persistent and eventually he got there and was able to report bugs that were not fixed in svn.

From time to time he would find a bug that he could work around and he would not bother reporting it because he was busy developing new features for his software and since he had found a workaround that solved the problem for him there was no pressure on him to take the time to create a test case. For he had learned that a bug report without a test case was frowned upon by the mono developers. They like test cases so they can run the tests when they make changes and try not to introduce new bugs or bring old ones back to life and he understood that and could relate to their position.?

Then one day a release called 1.1.8 came along that had a lot of problems. The developer filed a bug report for one of them but really wanted to be working on his new features instead of creating test cases for a whole bunch of bugs.? He felt that there would be not as much need for test cases if the mono developers were eating their own dogfood and running their sites on mono instead of php and other technologies.? He theorized that if they did this they could test the new releases of mono on their own sites before making a release and this would surely improve the quality of the releases because problems would be very obvious to the mono developers if they were happening on their own sites.? Since this developer was working hard to make really good web site software that runs on mono he even wished that the mono developers would use his software for their own sites. If there were features they needed he would build them.? All the mono developers would have to do is ask.

The web developer decided that he could not take the time to create test cases for all the problems he found but he would blog about them. Here is what he blogged:

Here are steps to produce bugs in mono 1.1.8

Create a web user control with a private property foo. Add the control to a web page and put an attribute in the markup Foo="bar" This will raise an error on mono 1.1.8 but not on 1.1.7 or on Windows

Create an asp Hyperlink like with a navigate url this in a repeater and it works in previous versions of mono and Windows


but in mono 1.1.8 the server tags end up as part of the url

There were even more bugs reported by users of the web developers software that the web developer could not reproduce himself but maybe that just means "Its fixed in svn"

The web developer continues working away on his software and maintains his hope that one day the mono developers will start to eat their own dogfood on a regular basis and maybe even consider using his software to run their sites.? We will have to stay tuned to see if he is a dreamer or just another bad bug reporter.

Share This Using Popular Bookmarking Services
Copyright 2003-2010 Joe Audette
Previous Post << >> Next Post

re: The Busy Web Developer and the Bad Dogfood

Saturday, June 25, 2005 1:10:40 PM DougHolton
Yeah I think before they release a new version, they would do well to test some of the common complex software apps used by Mono users, such as your Mojoportal, nant (which broke again with 1.1.8), etc.


re: The Busy Web Developer and the Bad Dogfood

Wednesday, July 20, 2005 12:52:07 PM David
It seems to me that the mono releases aren't done how most of us software types do them. This is how a typical release cycle runs for us:

- Add features and fix bugs as we go. Still open to input from users.
- Midway through, decide which features will be defered til next release. New minor features can still be added.
- Finish off all outstanding feature requests, and begin feature freeze.
- Thorough testing, bugfixes. No new features.
- Package up and release.
- Repeat for next version

Mono's release cycle seems to be:

- Add features and fix bugs as we go, Still open to input from users.
- Wake up one morning and decide it is about time for a release.
- Take a snapshot from svn, package it up and call it a release.
- Continue development as if nothing had happened.

My point is, is that the crucial difference between a stable and development version is a feature freeze period that allows bugs to be fixed in a stable environment. As a long time follower of the mono lists (users, devel and a stack of others), I haven't seen a feature freeze since 1.0.0.

re: The Busy Web Developer and the Bad Dogfood

Wednesday, July 20, 2005 3:20:38 PM Joe
I don't really have any problem with the way the project is managed or the release cycle. In talking about bad dogfood I'm really just referring to a bad batch of dogfood while most of the time I think it is pretty good. 

I don't think the mono release cycle is that haphazard and also they are implementing an ecma specification its not a matter of deciding what features to add based on input from users they know what needs to be implemented and they include whatever they think is stable in the releases.

To me the only problem is that unit testing doesn't catch everything, you've got to have some real world application tests to make sure there aren't major problems that the unit testing doesn't cover.  I'm going to help with some testing before the next release which should help make sure its stable at least for ASP.NET since that is the area I'm working in.

I'm not criticizing your opinion, just saying the release cycle doesn't seem a problem to me. I know in my own work I lean toward short iterations and I make releases based on convenient points in the progress chain. Of course my projects are much smaller in scope than the mono project so its easier to manage.



re: The Busy Web Developer and the Bad Dogfood

Wednesday, July 20, 2005 4:10:52 PM David
I hear what you're saying Joe, but I want to take you up on a couple of points.

they are implementing an ecma specification its not a matter of deciding what features to add based on input from users

True, but they still implement features between releases. There are parts of the specification that they haven't implemented yet and this is what makes up part of each release. This sort of change can still destabilised a project.

they include whatever they think is stable in the releases

But the problem as I see it is that they don't just include what is stable. Everything goes into the release. Every patch that is committed to svn comes with the next release, whether it causes a bug or not (and these bugs are the undetected ones you are talking about).

To me the only problem is that unit testing doesn't catch everything, you've got to have some real world application tests

I totally agree with you Joe. This is what I'm talking about. They need to have a period where they simply stop developing and start testing. Nothing but testing. And when they find a bug they fix it then they do all their tests again (not just unit tests, but acceptance tests too, as you say). I'm not seeing this though. All I'm seeing is that they collect up the release notes, run all the regression tests and ship it.

I don't think their release cycle needs to change that much. Short iterations and releases based on convenient points in the progress chain are great, but it isn't what I'm seeing from mono.

Finally, and this is kind of just a rant, we've been trying to our ASP.net application running on mono since 0.28. By the time we were up to 1.0.6, we were sick of trying since one bug would be fixed, only to reveal another, and each bug took so much of our time to track down to a simple enough test case that it simply wasn't economical. We switched our project back to running on IIS, Windows and MS.Net. Having seen the recent advances with the io-layer in 1.1.7 I'm certain we'd have more luck with the most recent versions of mono. However, I also fear that we would run into other problems.


re: The Busy Web Developer and the Bad Dogfood

Wednesday, July 20, 2005 4:46:46 PM Joe
You make some good points, I really can't argue against them. I've only been working with mono since about 1.0, I can only imagine how it was back at 0.28.

But I still consider mono a young project  that has accomplished an amazing amount of work in a relatively short time and I am very optimistic for the future.

I think it is just now getting to a point where people should consider starting to test and tweak their ASP.NET apps for running on mono and make their own judgment on if and when its ready for use in production with their app.  I'm an early adopter and you are even more of an early adopter :)  For some projects it might make sense to wait a bit and let mono mature but for others it might be a good time to get started.

I don't think they just use whatever is in svn the morning of the release :) I currently build mono from svn but for testing the next release they are going to send me a link to a tarball a week ahead of the release for testing, so at some point they tag it and they continue developing the main branch but they test the tag and use it for the release.

In any case I am optimistic that the next release will be better because I will make sure it doesn't break anything in mojoportal before the release. That may not catch everything but it should catch anything really glaring.



Comments are closed on this post.
Donate Money to support the mojoPortal Project. View Joe Audette's profile on LinkedIn View Joe Audette's profile on The Guild of Accessible Web Designers site