Lines are starting to get drawn for the Great Quality Wars of '09.  On one side you have camp led by Joel Spolsky and the now infamous Stack Overflow Podcast #38, and the other camp led by Uncle Bob.  I'm sitting this one out, from what I can see so far, both sides are going to get bloodied up where the discussion turns from a reasonable and positive debate to personal attacks.  We have a lot of problems in our industry and dividing it into factions won't help anyone.  Especially when I see this religious war based on how do I get there, not where do we want to go.

This, being on a much greater scale, reminds me of the Great Coding Convention Battles I participated in within a company in the early 90's.  One of the stupidest things we discussed over the course of a month of meetings was whether to use the following syntax for one line after an if statement.

if(true)
   DoSomething();

or

if(true)
{
   DoSomething();
}

Looking back this was a tremendous waste of time, albeit a relatively fun discussion in our meetings where both sides decided the other side was a bunch of hacks. 

I was in the

if(true)
   DoSomething()

camp unless I wasn't. 

For the case of:

if(true)
    while(someCondition)
        DoSomethingElse();

I would use the convention:

if(true)
{
    while(someCondition)
         DoSomethingElse();
}

where I would be in both camps.  What does this mean?  Does this mean I'm a moderate?  Well no, I do have very strong views on both coding (and politics but I'll spare you the pain on discussing my political views here).  I think the main reason I had strong feelings on my approach was that the other camp wasn't using C/C++ in their day to day work and kept bringing up examples and what *experts* said about the topic and under no circumstance should we have a block of code not enclosed in curly brackets.  My camp lived in a C/C++ editor for most of our work.

Do I agree with the statement that Quality doesn't matter?  No.  Do I agree with the statement that Quality is the most important thing?  No.  For any system there is really only one true measurement of success:

In the long term is your user community satisfied and will they continue to use your software to enhance their business or personal lives. 

Anything else is pure rubbish.  You may have a very high quality system that is just a joy to maintain, but unless people use your software, you probably won't (and shouldn't) get past V1.0.  You also could have a system that you deploy within a month or two that people just love, but it has tons of bugs in the core architecture and can't be maintained, you probably won't get beyond V1.0 either.

Quality is an important attribute of your system, however quality is not free.  Building a quality systems is not ensured by a set of tools, techniques or processes.  Quality is a result of an individual's commitment and a culture within your organization.  

There is a danger on both ends of the spectrum.  On one end of the spectrum there is "we don't need these new fangled tools and techniques, our Cobol and Assembly Language application runs just fine", yeah right, there's a real competitive edge.  I'm probably not going to buy or use your software, and if I do, I'm probably not going to apply any updates unless absolutely necessary.  On the other end of the spectrum, we try to rigorously follow the latest fads something like not writing a single line of code without a test, attempting to achieve 100% code coverage, or program exclusively against interfaces.  While this does sound good in reality are you putting too much trust in your metrics?  Code coverage of 100% (or even 90%) means your code was run, that's it.   I may be different, but I know most of my bugs are edge conditions I don't account for even if I do have a test suite covering that chunk of code.  The code coverage metric does little to protect me against these problems. 

What is the middle ground here?  It's actually pretty simple, think for yourself, make sure you write lots code that makes it into production, it doesn't do much good to get your code to a certain point and gloss over all the little details.  Also read as much as possible. 

Although I think the SOLID Principles offer a number of good concepts,  I personally don't agree with all those concepts.  You may agree with all those concepts.  That's OK.  One area I disagree with is the Single Responsibility Principle or SRP.  I've seen implementations of this in code a code base I needed to extend, and it's just not my cup-of-tea, too much noise, classes are not free.  The developer swore by this technique.  That's OK. 

I wish I could find the transcript, but let me paraphrase a discussion of SRP from Uncle Bob's Hanselminutes interview (right around 3 inutes into the podcast)

Uncle Bob: This is somewhat out of the norm for object oriented design.  Early Object Oriented Design Principals had us grouping together functions of that operated on the same data structures so that the methods of the class manipulate the same variables. 

Scott: That definitely flips things on its head.

Early on I learned one of the core principals of Object Oriented Analysis and Design is it's all about the data and methods acting upon that data via encapsulation.  Over time your business rules will probably change, however the structure of your data will remain fairly static.  The way I design LOB type apps is ruthlessly focus on my physical data model.  If you don't get that right, you will be fighting with an impedance mismatch all through your development cycle.  Although I could certainly see the value of a different approach, I keep my business objects 1:1 in sync with the database tables.  The primary reason for this is simplicity, once you break that relationship you introduce a mapping layer that is sometime necessary but comes at a cost of additional code.  The only code you know has zero bugs is code you don't write.

Maybe I'm "old-school" but my process has allowed me to pay the bills for the past 20 years now.  In some cases the developers picking up my code said, this is so incredibly easy to maintain and the hand off went smoothly with very few support calls.  In other cases, people just said it was a mess.  It seemed like people that liked my code have been in the industry 10+ years and have handed over legacy code themselves.  The people that thought I was a hack usually had less than 5 years and already knew everything there was about writing software.  They were just waiting for their first chance to start from scratch, build a product and show us old-timers how it's done.  I suspect, at one time I thought I knew everything about writing code.  Maybe this is a right-of-passage?

Using classes to group together functions just doesn't smell right to me, it may to you. That is ok.  I have the right to express my opinion just as Joel and Uncle Bob did, neither of these two people are hacks and I don't think I am either.  We as an industry need to do a better job of leaving out the religion and personal attacks, software is hard.  Period.  What works for one person on one project would cause a different person to fail.  

Another thing to think about in adopting and evaluating new techniques and technologies is one of the concepts of the Software Capability Maturity Model or CMM.  If you are a Level 1 organization, you cannot skip directly to a Level 3 or 4 organization.  You must go through Level 2.  If you attempt to implement processes found at Level 3 or 4 you will fail. 

At any point in your career your toolbox contains a finite number of tools, over time you should be continually adding tools to your tool box or you won't advance in your career.  If you attempt to implement something like the SOLID principals before having in place effective requirements gathering, defect tracking, or a continuous integration process your time would be much better spent focusing on blocking and tackling exercises rather than more advanced techniques.  It's surprising how many organizations I've worked with that try to adopt the latest advanced technique, but their bug tracking is emails, source control is sorely lacking and requirements gathering consists of meetings where the main goal is for people to hear themselves talk and sound important! I've seen many more projects fail because of the lack of effective leadership, poor requirements or project management than I have because the programmers used the wrong technique or pattern.  Before we put stealth on our airplane, let's make sure our airplanes fly.

Over time there have a been considerable advances in our field of software engineering, some stick around some don't.  Are we still using those old Rational Rose Puffy Clouds.  I liked my Puffy Clouds!  I want my Puffy Clouds back!  None of these should be considered 100% bad and ignored, learn something from these, but don't just jump in and blindly implement without thinking.  Put what works for you in your tool box, have a full tool box and select the exact right tool to solve your problem and more importantly, know why you picked that tool.

I'm looking forward to the "smack-down" between Joel and Uncle Bob in an upcoming Stack Overflow episode!  Would be very interesting to see them on UFC as well.  In the mean time, I'm going to keep writing software, reading blogs and keep adding more tools to my tool box.

Development or Discovery

The biggest cost in any software development effort is always a factor of time.  It became pretty obvious to me while working on a new reporting project.  I've worked on a "ton" of systems over the past few years and with very few exceptions I can sit down and start making tangible progress very quickly (this may not be evident to my clients while I'm working on the plumbing, but that's a topic for another post) but with this project I've got about 4-6 hours into it and although I'm making headway on a couple of fronts I'm starting to sense the "churn" where it's more of a discovery process than a development effort.  How much of your time is spent in discovery and how much in pure development?  I think the ratio of discovery to development is major factor in developer efficiency and thus the time/cost in a software development effort.  Although some external factors such as training, pair programming, documentation and mature processes may help with this ratio, the biggest impact on the ratio will be the person doing the development.

Discovery does not provide any real business value.  Discovery is a critical investment that needs to be made.  Just as you invest in your financial future, you should invest in your professional one as well.  Once you’ve made this investment you start building capital, the capital can then be used to increase your Development time, this is where your investments should pay off.  As Frederick Brooks states in his timeless classic “The difference between an exceptional developer and a Mediocre one is generally an order of magnitude”.  Why is this the case?  One reason is Developer Efficiency  but the other is the capital the developer brings to the effort.  Effective program manager realize this.  When we talk about capital/experience/knowledge this can really be in three categories, general technology, domain specific, implementation specific.

Technology Capital

You, my dear reader are 100% responsible for this.  If you work for a corporation that’s nice in that they may pay you to learn this knowledge, but in the end, this is how to program an Http Handler in ASP.NET or write a SQL Query, create a clustered index and so on.  These skills are not implementation specific.  The wider background of skill sets you have the more capital you will have and the more value you bring.  As a consultant, if I am stuck doing “discovery” on a new technology I very rarely bill the client for this time.  My clients pay me to “develop” they assume I know they technology.

Domain Specific

This is where you can start developing additional capital.  Unless you pick on specific vertical, there will always be a portion of the time working on development efforts where you will be doing discovery within that domain.  An example of this is my experience working in a Sales Force Automation company for 3 years on a number of different efforts.  Although there was an initial learning curve in this vertical, over time the value I brought to the table during the system design meetings increased, the more knowledge I brought to the table, the quicker we could talk about what made their problems unique and in most cases these so-called “unique” problems were actually solved in other efforts and that experience could be capitalized on.  My goal going forward is to pick a new domain every other year and start gaining that knowledge.  Have you worked in one industry all your life?  How many jobs are there in that industry?  How well are you compensated in your industry with respect to others?

Client/Implementation Specific

Now is where it get’s interesting, each client does indeed have certain technologies, systems, database schemas that are unique to their organization.  I would suspect that a considerable number of developers are “experts” in this area.  As a consultant I worked at one client for about 3 or 4 years.  They made a considerable investment in me in their specific data, not only just the structure, tables but also the subject matter.  I made a significant investment in time in their data.  At the point I started scaling back my work on their efforts, I had a much better understanding of the data within their systems then anyone did internally.  This was a great engagement for both parties (Client & Software Logistics) however when I look back over those 3 or 4 years, what can I take away from that client specific data knowledge?  Not very much as it turns out.  From a billing perspective, I don’t have any problem doing “discovery” of a clients systems on their “nickel” especially if in the long run that knowledge won’t be useful anywhere but their systems.

The bottom line is in software development, the more capital you bring to the table, the less time you are doing discovery and the more time you do performing development.  You are paid by your organization as an employee or a consultant to do development and not discovery.

-ec

Putting your Trust in Metrics

I'm a big fan of automating as much as you can, but blindly automating processes controlled by metrics may cause more problems than they solve.  The idea behind capturing metrics or in the case I'm currently working on, writing a suite of unit tests is to automate the testing so I can make changes at-will, run my unit test and know everything is still working good.  If I come to rely on these unit tests (or really any sort of metrics) to help me make business decisions as to when I am code complete and ready to ship;unless I'm 100% sure that the unit tests take into account 99.99% of all the edge cases; I'm doing the recipients of the software a disservice.  As an efficient coder, I want to take what the customer tells me they need and turn it into the compiled bits that successfully solve their problem in a straight of a line as possible.  Unit testing and other software related automated processes go a long ways to helping ensure an efficient path, but as these automated processes are developed we need to pay special attention to make sure these are complete.  In a large sense, if you are capturing KPI or Key Performance Indicators about your organizations performance, are you sure that the core measures you capture are indeed complete and accurate?  Remember the purpose of capturing these metrics is to influence business decisions.  If your metrics are not accurate or your measures just can't be trusted, there are three core problems that this causes.  First it clouds the judgment of the people directing the efforts, they may think they need to do A, but the metrics tell them B really needs to happen, A is clearly the right thing to do, but since B is out there as a potential solution it may dilute the affect of implementing A.  Second it puts in place a false sense that you are indeed doing something to solve your problems and all you are really doing is delaying the true solution costing the most valuable asset in software development, time.  Finally it takes time to capture metrics and if done properly it is a great tool.  If your metrics are worthless you waste the peoples time building the infrastructure to capture the metrics, you waste the peoples time to capture the metrics and you may even unintentionally sabotage future metrics capturing initiatives.

In a previous post I discussed the race car analogy, where the drivers have a significant amount of safety gear in their cars.  They need to be 99.99% sure that if they get in an accident the gear will do it's job and protect them.  If they can rely on their gear, they can take more risks to give themselves a competitive edge.  Are your sure the test cases, scenarios and metrics your are employing would help you prevent or survive a crash and give you a competitive edge?

-ec

Developer Efficiency

At the very essence of what we do as technical developer folk is take the needs of our business stake holders and turn those into 1's and 0's (and maybe in our lifetime we might figure out how to get 2 ) that business stake holder can run as software on our computers.  Any thing else is pure rubbish! 

So what are the attributes of an efficient coder?

Technical Aptitude

This is a baseline of becoming an efficient coder, when it's all said and done you must have a certain level of Technical Aptitude, that's not to say you need to be a "Rocket Scientist" however you should have certain skill such as problem solving, a general nature to ask "hmmm...I wonder how that actually works?" or "boy that sure is some cool technology, I wonder if I could build that?".  I think this is something you either have or you don't.  Not having it is not a bad thing it just means this type of work may not be a good fit.  I know any time I try to do something creative with any sort of graphics problem, I can spend 3-4 hours on trying new things and in the end it probably looks worse that when I started.  This isn't a bad thing, but the sooner you can recognize your weaknesses, the sooner you can find other people around that complement your strengths that make up for those weaknesses.

Memory

As a developer there are just too many different technologies that could be implemented to solve a problem, too many Languages and API's to learn, too many object models, too many command prompt arguments and keywords, you get the idea.  The more of these you can stash away and bring to the fore-front when necessary the quicker you can continue developing and not doing discovery.

Perceptiveness

Being able to read between the lines and think to a long term strategy is important when developing your solutions.  I see Software Development as chess match.  At any given time you are presented with a certain set of facts or knowledge, each time you sit down to design/code the moves you make have a major impact in how the pieces get moved and sets you up for your next designing/coding session.  I've just seen too many software efforts take the fundamentally wrong approach early on, and those projects never really recover.  Although an extreme example, let's say you wanted to write a simple contact manager, do you think you would start writing this in Assembler?  How about some sort of auditing function where a copy of all items was made into a database every 15 minutes, then a process was run to detect and report on changes, this was a real world scenario where a developer worked on getting this process working for about four weeks, and never did...the efficient coder made the correct chess moves and had implemented in an afternoon.  I see this is a very common problem in software development.  Do you start coding before you have a true understanding of the problem you are trying to solve?

Attention to Detail

This is a really hard one for me, coming up with creative solutions and looking at the big picture sometimes blurs some of the details.  What's interesting is that when I'm doing low-level programming such as embedded systems development, my eye for detail is very sharp...go figure...to be a efficient coder means developing system with little or no bugs.  The challenge here is that if you pay too much attention to the details, you may loose site of the big picture.  Coding something that implements the fundamentally wrong solution even if perfect and bug free is worth little to no value.  I think this is one of the bigger areas where we can use technologies, tool and processes to improve our efficiency.  After all computers aren't very good at creativity however they are extremely good at the details.

Experience Level

I think by far the biggest impact on this ratio is the experience of the person doing the development.  This experience is not just in the actual programming languages such as Java, C, C++, or the Software Development Life Cycle/methodologies, but to an even greater extent programming a wide variety of business processes.  Just as with software, certain things can be "abstracted" the same is true with business processes.  This experience also helps facilitate the conversations with the business stake holders, the more you understand about business in general, the more time you can more quickly get to the specifics of what makes this problem unique. 

Experience only comes with time.  You can do certain things that may not seem that important, but really are.  There is no substitute for spending time with the business stake holders even is you just sit in the room and not contribute anything.  That is not to say they should have a direct line or to influence your day-to-day scheduling.  But as a manager early on, I used any possible opportunity to get the developers in a room with the stake holder.  When doing embedded systems work it was always a good reminder going to the field and realizing that in the real world when my software is working it's not just a little LED that comes on at the right time but it's actually a 20 ton rock crusher that would be very bad if it came on at the wrong time.  It puts everything you do in a different perspective.  How often do you have actual conversations with your business stake holders?

Passion

What really differentiates you as an "efficient coder"?  Do you really just plain and simple love what you do?  Or are you just coding to pick up a paycheck?  Life is just too short to spend the majority of it working in a job just to pickup a paycheck.  If at a certain extent you have qualities mentioned above and your are passionate about taking the needs of the business stake holders and turning them into the 1's and 0's in exactly the right order as efficiently as possible (what ever that means on your particular effort) the chances of you being an "efficient coder" are pretty good!

Are you an "efficient coder"?

- ec

How can you tell how efficient your software development/migration process is?

It might be interesting to develop a set of metrics that capture the amount of time and steps involved with doing certain common activitives on a development effort.  A simple but I think effective measurement might be what is involved with adding a data-driven, context sensitive drop down list to a WinForm or WebForm.  This would involve steps similar to the following:

• Adding the data source to populate the drop down
• Adding the field to be controlled to the data base
• Getting the source from your source control repository
• Modifying the data transfer objects to get the selected value between the database and the control
• Modify the user interface to add the control
• Checking in the changes
• Add some sort of help documentation
• Develop any required tests
• Migration to the three typical environments (Dev, Stage, Prod) for both the application code and database objects.

Come to think of it if this process was captured (even without any metrics) it would go a long ways for developer documentation.

I recently came across an interesting tool for looking at source code metrics SourceMonitor from Campwood Software.  This tool looks at source within a set of sub-directories and gives you basic measures about your software.

The following are a sample of measures captured through this application:

• Total Source Lines
• Total Statements
• Percentage of documentation
• Complexity
• Block Depth

A very interesting chart is the summary one below; this gives you an idea of how well your source fits within a predefined set of measures.