Test Driven Development In the Real World

Well its been a couple years now and I've got enough experience doing TDD to feel that I can speak intelligent about what works (at least for me) using this development methodology.  I've seen some people say that they should never write a line of code before writing the tests, well that might be all-and-good for if your primary goal is to get an A on your test this semester, but in the real world, I'm not sure this is the best use of time.  Again, this is speaking from my past experience and the way I develop systems so your mileage may vary.

In my experience, as software developers we have to assume a certain level of stability in the building blocks we create.  Should we create test cases around our business objects that ensure when we execute a SQL Command against the database the SQL Connection object opens the connections and successfully executes the command? 

Let's look at the Red-Green refactor cycle.  At any one time when you are developing software you are attempting to solve a specific problem.  If you aren't sure of the problem you are trying to solve you might not be ready to start coding, it might be time to drop back to the white-board and draw some boxes/puffy clouds, or you may need to revisit your data model.  Either way just getting that little green light to come on within your screen may prove your current API is valid, but I'm guessing that, the API may change a little bit once you continue on in your efforts.

So if you made it this far, you have an open mind and we can continue our discussion...so let me say this, is there a place for TDD?  Abso-fricky-lutely I just did a little effort that was really built as more of a test concurrent type of development effort, but it's almost the same thing.  This was a very complex process where we had a disconnected database on a remote device that synchronized with a server.  We had to create some new "stuff' on the device that all needed to work locally before we could talk to the server.  No problem you say right?  Well where it gets interesting is the fact that the server uses integers to establish the relationships between the tables, that means if I create a new record on the device, I can't use an integer as the primary key, because that will only be available once we look at the identity column on the server database.  We need to build up all our referential integrity on the device database using Guido's then replicate those changes back to the server, create the "real records" send those "real records" back down to the device and make sure all our relationships are still happy.  For about the past year, I've been very happy using NUnit, however the client I'm building this for uses VSTFS so built the testing using this technology...really about the same thing, just slightly different syntax.  My new favorite little utility TestDriven.NET works like a charm with both.  So anyway, I was able to create some really nice scenarios that create the item on the device, add this to it, add that to it, change this, change that, then synchronize with the server and make sure the data I had on the device was the same as after I synchronize.

Another awesome experience I had with TDD was the development of a binary serialization algorithm for DataSets.  This I did as a pure TDD experience, I built my tests for the different data types I needed to serialize, then I created the methods and work on this until each of the unit tests passed.

My other experience wasn't so much with TDD but more of a formalized Unit Testing via NUnit.  I created a VB.NET ASP.NET 2.0 application, with about 150 unit tests, I'm not sure I gained a ton in development efficiency, however it did same my rear-end a couple times with my build cycle.  As part of the continuous integration environment that was in place for this project, we ran the unit tests, if they didn't succeed, we knew we shouldn't deploy.  Worked out slick and once the test were built it worked for "free" so that was kind of nice.

So what's my final opinion on TDD?   Well just as in anything moderation...and finding the right tool to solve the right problem.  I'm not convinced you need to write a test case to make sure the set and get properties work on your business objects, however if you have a complicated workflow, it is certainly worth the investment in time.

-ec

The Deadly Cycle or Keeping up with the Latest Technologies

As we progress through another year as Software Engineers, I'm starting to sense an alarming trend.  Please indulge me as I explain, if you are a software architect you are responsible for picking the right technologies to construct your system, to do this effectively you really need a fairly in-depth knowledge about the existing technologies or building blocks that are not only available today, but what is also available as either a CTP or beta release.  Now you pick the right technology and start constructing your system, let's say it's not a huge system, but it isn't trivial either, let's say you have a cycle time of 6 months.  As you build your system (and you picked the right technology) you will gain the experience necessary to master the technology, and if you planned it right and with a little luck, by the time you are ready to ship, they beta release you were working with go golden so you are set!

Now you are off to your next effort and design and build a project using your newly acquired expertise.  Another 6 month effort, great solution and everyone is happy.  What's happened in the past year?  Well, another new cycle of technology came out requiring even more detailed understanding.  This new technology may be so fundamentally different that your previous experience may now be obsolete.

Let's look at this another way, you know that about 1 year from now a fundamentally new technology is coming out, you need to build something, but you don't want build it off of a technology that will be obsolete when that technology so you hold off for the new technology.  Well your not shipping or even working on your product until you can get a stable CTP, even with that you run the risk of the API's changing by the time your technology goes live.  So you find the right time start constructing your system get a few months into it and -whammo- you see another technology that would be just perfect for your implementation that is really mutually exclusive to the technology you selected.  What do you do?  Just wait another 6 months for that technology to ship?  Probably not...

What are some things we can do to minimize the impact of new technologies?

Keep up with the absolute bleeding edge of technology.  There are many ways to do this.  In the past I only attended one professional conference a year.  Starting last year I attended two conferences www.medc2006.com and www.devconnections.com this year I'll be attending Dev Connections both in the spring and the fall, also MEDC 2007 and MIXX so this year it will be four conferences, with the incredible number of new technologies being release it really only seems like the best way.  With the new technologies coming out there are usually a number of important subjects within that technology.  These conferences aren't really a good way to get a mastery of the technology however it does give a good overview that will let you know the key parts.  These are also good for determining what is real and what isn't.  As to this point, there really isn't a substitute for experience.  How many times have you been to some sort of session or demo where they say "Build an enterprise site within 10 minutes with no lines of code"?  Well I've got news, as Frederick Brooks says in the Mythical Man Month, there's no silver bullet.

Another way that I think we can attempt to isolate ourselves from the rapid change in technology is to focus on the data.  Even if you choose to implement an SOA architecture (what ever you define that to be) when it comes down to it, at some point the rubber is going to meet the road, your going to need some objects with properties and method that actually does the work.  The way you factor these objects may be the biggest factor as to how well you can adopt the latest technology and put the latest and greatest user interface or offer the functionality provided in your objects as services.

Something that may be an option, but I haven't been able to practice is move away from coding.  Even though some people make it look easy building non-trivial software is hard and complex.  Things that look good at 50,000ft start to fall apart if you can't also understand how they work at the 1ft level.  How many times have you worked with the architects in the ivory towers that haven't coded in years and establish the all encompassing architecture that looks good on paper, but when you have to start building those objects with the properties and methods just falls apart or you create way too many lines of code.

On a final note, we need to start thinking about the software develop as business assets, not just lines of codes and assemblies.  What we develop should try to be at the right level of granularity for the task at hand.  If we build our projects where the majority of the business logic is in the UI, well we may have something we can sell and ship (which is by no means a bad thing) but we really don't have something that we can glue together using the latest technologies.

I'm sorry I just can offer any "silver-bullets" here but sometimes recognizing the problem is half the battle.  I think that we as software engineers need do the best we can to keep up with the latest technologies, invest a significant amount of time reading and researching new technologies and last an probably most important keep our customers and end users in mind.  If we do this things will just fall into place, our products will ship and we will start developing the goal of not just developing software assets, but business assets.

-ec

Another Strategy/Tactics Post

As I look over the last 6 years as a consultant, one thing becomes very clear, technical solutions by themselves will only get you so far.  As a great leader once said “Good tactics can save even the worst strategy. Bad tactics will destroy even the best strategy” (George S. Patton) this is very true for software development, but with a slight twist.  As long as you continue an effective tactical operation, the problems with the strategy may remain hidden.

From 2001-2005 I developed a mission critical system for a very large company, over the course of these years, I did what it took (including searching out Internet café’s in Key West) to keep this system operating as efficiently as possible.  As an outside technical development asset, my primary mission was make sure I kept the business constituents happy, the scope of the change I could affect on the organization was limited, for long term, this change would be required to embrace and support system.  As I left this engagement in early 2006, it was clear that this system was not going to be properly supported.  In March of 2007 for better or worse, Software Logistics was re-engaged on this effort and will make a significant positive impact.

So how does this come back to my original quote from GSP?  In this effort I worked on the “Product” and very little on the “Process”.  The product succeeded at the tactical level with assistance of Internet cafés in Key West, however at the strategic level the proper systems and protocols where not being put in place.  It was clear the technical/tactical part of the solution was sound however the strategic part needed some help.   What was required for long term success on this project was a complementary skill set.  Enter my business partner David E. McCracken, I’m extremely excited to work with Dave on this project going forward.   There is a considerable amount of power and flexibility within the system that was developed however this power and flexibility came at a cost of an investment in understanding two domains, the first being a non-trivial understanding of the subject matter data and the second being an understanding of the architecture.  The adoption beyond the immediate team required considerable effort that in some way is mutually exclusive to the day-to-day activities.    This effort is not in creating the code, stored procedures or even the documentation around these software artifacts.  This effort is in making sure that our tactical efforts that we perform as software engineers today not only satisfy the immediate and pressing needs of the business, but also are done is such a way that they can be repeated on a consistent and predictable fashion in to support strategic goals.  In the short term tactics can make up for poor strategics, however in the long run this is not a sustainable model.  Some great gems of knowledge can be found on David's blog on some ideas on how these efforts can take place.

Bottom line is how much are you working to satisfy your business constituent's tactical and immediate needs and how many fail-safes do you have in place to ensure the long term strategic organizational goals?  Do you have the right infrastructure in place for both these equally important journeys?

-ec

DevConnection Orlando 2007 - Day 1

I was up in the air as to going to DevConnections earlier this March, but after making the decision and attending Day 1, I'm extremely glad I showed up.

NiVek J.D.'s Maiden Voyage

Everything came together today for NiVek's J.D.'s official first journey.  NiVek J.D. is a remote control tractor purchased from Target that was converted over to be controlled via a small on-board computer programmed in Java.  The same basic board I used for NiVek I was used for this "robot".

This robot has two cool features not on NiVek I.  The first is it has a GPS module purchased from Parallax.  The second and which I think is really exciting is it using a Windows Mobile device as a "Repeater" that allows for communications from the NiVek embedded computer to a a PC.  This design is based upon software components that are part of WiMo or Windows Mobile Robot, you really need to check out that site!  This consists of some kewl software components that are made up of a Compact Framework 2.0 application that runs on Windows Mobile and a collection of services that run under Microsoft's Robotics Studio.

NiVek J.D.'s Hardware

Basically the radio was just ripped out of the existing remote control tractor and and the motors were connected to the NiVek embedded computer.  As with the "original" WiMo, the Windows Mobile device was mounted on a servo with a CD-ROM.  The original WiMo used a SmartPhone not a Pocket PC so that probably worked out a little better, it's nice to be able to pan the camera however with my driving skills (smacking into walls) one of these days, I'm pretty sure the CD-ROM is going to end up in pieces .  I need to re-think that part of the design.

NiVeK J.D.'s Maiden Voyage (well at least one of the first few)

50,000ft System Overview

• The actual robot itself is controlled by an embedded computer based upon a small PIC processor with some additional components that allow it to be programmed in Java with 32K RAM & ROM this was purchased from Parallax and is called a Javelin Stamp.  (See this post for more information)
• The embedded computer has a BlueTooth transceiver module that allows it to communicate with a Windows Mobile Device.
• The Windows Mobile Device has a Compact Frameworks 2.0 application running code available from the WiMo Bot web site.
• The Windows Mobile WiMo application communicates with the robot via BlueTooth.  It also has the ability to listen on a socket for connections from a remote application.  Since this is a Windows Mobile device (in my case a phone), it will not only work while it's connected via a local LAN via WiFi, but it can also communicate via the GPRS radio and be a sort of "repeater" that will allow it to communicate to a host controller program anywhere it has cellular reception, just think about this...this is very kewl!  A nice feature is on the opening screen shot of WiMo it tells you the IP address of the device.
  
• On the PC side you have a set of Microsoft Robotics Studio services.  When these services first start you will be greeted by a dialog that will allow you to enter the IP address of the remote Windows Mobile device.
  
• Once you press "Connect" (and the software gods are shining on you) you should establish a connection from your PC to the WiMo application.
• At this point a couple of additional forms will show up from the MSRS services.  The one in the upper left is displaying console messages from MSRS (Microsoft Robotics Studio).  This is a great way to see what's actually going on with your services. 
• The one in the upper right is from a service that came with WiMo (with the addition of buttons to control the motors).  Another really cool built in feature with WiMo is the ability to use the camera on your Windows Mobile device to send pictures back from your robot.  This from also sends messages to the core WiMo communications MSRS service to pass those to the WiMo device application.  These messages allow for control of the robot from the PC.
• The dialog in the bottom is an additional MSRS service that was built that subscribes the the TextMessageReceived event from the core WiMo service. 
  • The NiVek embedded computer spits out GPS readings every second (probably need to change this so it only sends when the location changes). 
  • This gets sent from NiVek to the the WiMo software on the device with a simple <stx><etx> binary protocol.  The WiMo software turns it into a simple text message. 
  • For our GPS WiMo constructs a simple text message "GPS: #Sats=4 Lat=28.4.042 Long=82.42.5522". 
  • This text message is sent over the wire from the Windows Mobile application to the WimoComm MSRS service. 
  • The WiMo MSRS service picks up the text message and finds any services that subscribe to this type of incoming event.
  • The GPS Point plotter MSRS service subscribes to these messages so it takes those readings and plots the on the crude dialog you see below.  The challenge here is that for a robot this size of NiVek J.D. if it moves 50ft that's a long distance, and the resolution on the GPS module I purchased just doesn't seem to be all that accurate.

Finally one last picture of you host at the controls!

Looks like my day job is going to be busy over the next few weeks with DevConnections and MEDC but I hope to sneak in a few hours every once in a while to push this effort forward!

Can anyone figure out what NiVek stands for?

- ec

Client Project - ConnectWise Mobile

A couple of years ago, I was contacted through a current client that they had a "Fun Project" to work on, normally when I here this I go running for the hills!  This one was different, ever since my early days as an embedded systems programmer, I had a passion for mobile devices.  I've always played around with these technologies since early Windows CE devices but as in this post until the last couple of years the hardware and development platforms just weren't out there to realistcally do anything but non-trivial apps without a very significant development effort.  The application was for a company that has a professional services automation tool and is called:

Once I completed the initial requirements gathering and design, this application started out life as a Compact Frameworks 1.0 application on Windows PPC 2003 using SQL Server CE.  It currently runs on the Compact Frameworks 2.0 on Windows Mobile 5.0 and I will be attending MEDC 2007 to learn about any changes required or new capabilities offered in new framework to be offered later this year.  Along the way I converted CFNunit to work with the Compact Frameworks 2.0 and add a little more functionality on the PC for running these tests.  I also learned a ton about what it takes to make a PPC CF application work without a stylus, this doesn't really come that way out-of-the box.  Finally we also have a desktop version of this app using the same Business Objects and Control Library.

This site discusses the features of this application much better than I could do here so check it out!

-ec

It's not a destination, it's the journey

While trying to implement a Project Delivery Optimization initiative, it's not the destination it's the journey.  Where you want to get to has a lot to do with where you are.  As you set off on the road to "make-things-better" when delivering software you need to look for the right steps to make increment improvements today without limiting your options in the future.  The processes and tool you need to take you to the next level will serve as a set of building blocks for the level after that.  There is nothing wrong with incremental gains, however as you decide on a process, tool or technology, you should have an understanding of what their limits are what can be done when you hit those limits.  For any non-trivial development organization as you achieve your goals, one of the biggest changes you will make will be to the culture of the organization.  As you make these changes, you will establish momentum.  If a certain process you start has limits that will only take you so far, you don't want to lose all that momentum.

Let's say you establish a project specific initiative, you see progress beyond what you expect.  The tools and processes you come up with  work like clockwork on that initiative.  You want to repeat that on another project so you implement that initiative on the next project.  Now you have a couple/few projects working like a well oiled machine.  You decide to implement that for the other 12 projects in your area.  You do this, however you see that things are starting to slip and you aren't seeing the improvements in efficiencies you expect.  What happened?  Maybe the initiative you implemented works well at the tactical project level but not at the strategic organizational level.  The way this initiative/tools work doesn't take into account any additional commitments beyond the project level and there is no way to change that.  You've made gains but now you are at a dead end and changing course could mean starting from scratch.  If you start out with the strategic end in mind and keep that in mind while making tactical maneuvers your journey toward the utopia of project delivery execution will be much smoother!

Have you ever made any decisions that where absolutely the right decision at the time, however as time went on you find that the investment you made in that decision limited any possibility for future growth without significant pain?

-ec

Microsoft Robotics Studio Communications Protocol

Ah – the weekend, time to do a little coding for fun!  One of my on-going projects it to build out the functionality of my latest robot NiVek I.  Although the robot itself has a little computer it doesn’t have much of a user interface, that is provided via a PC.  Therefore for the PC to configure and display the telemetry from the robot a communications channel needs to be established.

Exploring SharePoint 2007 with PowerShell

The more I work with SP2007, the more I get pumped about the technology.  Although there is a lot you can do right out-of-the-box, the programming model behind this really allows it to be a great framework for any development platform.

To get started programming, you probably want to get an idea of how the Object Model works, a real simple way to do this is with PowerShell.  I have an SP2007 dev server I setup that I installed PowerShell & PowerShell IDE on.  Once you do that, just get a hold of the SP2007 SDK documentation, this will give you an idea of the structure of the Object Model then start playing!  The following screen shot should be enough to get you going:

Obviously this same concept would work to exercise any object model!

Enjoy!

- ec