Blog Home  Home RSS 2.0 Atom 1.0 CDF  
The Efficient Coder - Monday, March 16, 2009
There has got to be a better way of communicating with our computers!
 
 Monday, March 16, 2009

With the introduction of Windows Azure I was looking for something I could write where I could kick the tires and see what’s involved with this new technology.  For a quick overview what I came up with, please see the video video I created for my Mix09 “Show Off” entry hosted on SoapBox.  My first (of hopefully many) Azure applications I created is online at http://www.tweetmyrun.net.  The user experience needs a little tweaking but it seems to be working great!

This application uses the following technologies:

  • Windows Live Authentication
  • Windows Azure Hosting Services
  • Windows Azure Table Storage
  • Silverlight 2.0 “talking” to Azure via WCF

Here’s are some high-level bullet points you need to get started:

  1. Download the Windows Azure SDK
  2. Download the Windows Visual Studio Tools for Azure
  3. Apply to participate in the beta.
  4. Create a library project to contain your DAL, I found this post as a great starting point.
  5. Start your development storage from the start menu after configuring the your table storage settings with the following settings:
    1. AccountName = devstoreaccount1
    2. Shared Key = Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
    3. Storage Endpoint = 127.0.0.1:<StoragePort>
  6. Create BAT file to extract the settings from your DAL assembly to create tables to simulate you Azure Table Storage with something similar to:
    "Program Files\Windows Azure SDK\v1.0\bin\DevtableGen.exe" /forceCreate "/server:localhost\SQLExpress" "/database:MyDatabase" obj\Debug\MyDAL.dll
    pause
  7. For more information on configuring IIS 7.0 click here.
  8. Although you can certainly test your application on the local development fabric, as long as you create your site as a web role, you can test most of the features as just a standard IIS web site.
  9. When working with Live Authentication redirection back to your server, I found creating an entry in your HOSTS file works good.

If you have any questions, please free to message me on Twitter via @bytemaster

-ec

3/16/2009 1:23:16 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]    |  Trackback
 Friday, February 27, 2009
I'm building a multi-tier application in JavaScript.  This is implementing the Model View ViewModel pattern to keep as much of the code generic as possible, while allowing for customizations in both the UI and data layers for specific browsers and data access methods such as a local SQLLite database or calls to the server.  I'm using code-generation to create my models, but need a good way to create the Controllers, Views, and ViewModels.  I had been using a simple template, but cutting, pasting and replacing text seemed a little inefficient, so I decided to create some Visual Studio Templates to accomplish this goal.

It's fairly simple to create a trivial template, but I needed to add just bit more complexity, I need to build up a custom namespace for my classes, so just a simple ZIP file containing my template didn't cut it.  The answer was to implement IWizard in the Microsoft.VisualStudio.TemplateWizard namespace

We need to create two components for our new templates, a ZipFile containing, *.vstemplate and a .NET Assembly containing the IWizard implementation.

Create your IWizard Implementation
1) Create a Windows Class Library
2) Add the following References
  • EnvDTE
  • Microsoft.VisualStudio.TemplateWizardInterface
3) Create an empty class, and implement the interface Microsoft.VisualStudio.TemplateWizard.IWizard
4) The method you care most about is:

      public void RunStarted(object automationObject, Dictionary<string, string> replacementsDictionary, Microsoft.VisualStudio.TemplateWizard.WizardRunKind runKind, object[] customParams)

5) In this method you can add additional tokens to be used in your template by adding name value pairs to the replacementDictionary
          replacementsDictionary.Add("$modulename$", "MyName");

6) In addition you can add a standard windows form to your wizard to collect information
      _inputForm = new ModuleInputForm();
      _inputForm.ShowDialog();

      replacementsDictionary.Add("$modulename$", _inputForm.ModuleName);

7) Within the Project Properties and the Signing Tab, you need sign your assembly.  You will be installing within the GAC, so it needs a strong name.
8) Install your Assembly in the GAC.  The easiest way to do this is to just copy the file from your build output directory to %WINDIR%\assemblies between two explorer windows.
9) After you install your Assembly note the Assembly Name and Public Key Token, you will need these to create your vstemplate.



10) If you make updates to your wizard, you have to make sure you close all instances of Visual Studio .NET so your wizard is reloaded.

Create your vstemplate:
We will need to create a ZIP file that contains at least three components (more if you want to generate multiple files):
1) SomeTemplate.ext - This will contain the template used to create the Visual Studio Project Item
2) Specification.vstemplate - An XML file that contains the configuration for your template
3) Image.ico - An image that will be displayed in the new project item dialog for your custom project template.

What I did was just create a temp directory where I created these files.  Once complete I'll zip them up and show you where to put them so Visual Studio will recognize them.

SomeTemplate.ext, or in my case Controller.js
This is your custom template.  It can contain anything, but what makes this work is the ability to replace tokens within the template.  I wanted to use a custom module name and class name within my JavaScript files, this is only a fragment from the file, but you get the idea.

MyCompany.Controllers.$modulename$.$safeitemname$.registerClass('MyCompany.Controllers.$modulename$.$safeitemname$', null, Sys.IDisposable, MyCompany.Controllers.IController);

Specification.vstemplate or in my case Controller.vstemplate
Next we need to create our .vstemplate file.  This is a small XML file that tells Visual Studio how to build your populated instance of the template, it's format is as follows:

<VSTemplate Version="2.0.0" xmlns="http://schemas.microsoft.com/developer/vstemplate/2005" Type="Item">
  <TemplateData>
    <DefaultName>Controller.js</DefaultName>
    <Name>M-V-VM Controller</Name>
    <Description>Model View ViewModel Controller</Description>
    <ProjectType>CSharp</ProjectType>
    <SortOrder>10</SortOrder>
    <Icon>__TemplateIcon.ico</Icon>
  </TemplateData>
  <TemplateContent>
    <References />
    <ProjectItem SubType="" TargetFileName="$fileinputname$.js" ReplaceParameters="true">Controller.js</ProjectItem>
  </TemplateContent>
  <WizardExtension>
    <Assembly>MyCompany.
Templates.WizardTemplateInstance, Version=1.0.0.0, Culture=Neutral, PublicKeyToken=a2c453bf57a7f5d7</Assembly>
    <FullClassName>
MyCompany.Templates.WizardTemplateInstance</FullClassName>
  </WizardExtension>
</VSTemplate>


Pretty straight forward. but here's a little more information about the file format, one important note, make sure in your ProjectItem node, you have ReplaceParameters="true" to update your tokens.  In addition to the custom tokens we added in the wizard we created above, here is a list of built in tokens.

You need to find yourself an icon for your template.  Make sure it's in the same directory and is specified by the <Icon> node.

Once you have this completed, zip all three files and place them in the directory on your machine similar to:
     [USERNAME]\Documents\Visual Studio 2008\Templates\ItemTemplates

Generate your Template
Start Visual Studio, within your solution tree, click on Add New Item and in the bottom section on MyTemplates you should see the following:


Change the name and if the Software God's are shining on you you should see:



where you can enter the name of your module.

Once you hit Save, your new file should get generated any tokens you specified be replaced.  In my case here is a portion of the generated file with the module name of "Sync" and file name of "History" is as follows:

MyCompany.Controllers.$modulename$.$safeitemname$.registerClass('MyCompany.Controllers.$modulename$.$safeitemname$', null, Sys.IDisposable, MyCompany.Controllers.IController);
MyCompany.Controllers.Sync.History.registerClass('MyCompany.Controllers.Sync.History', MyCompany.Controllers.ControllerBase, Sys.IDisposable, MyCompany.Controllers.IController);

Yes that is Javascript, if you are doing client side programming, why aren't you taking advantage of the Microsoft Ajax Client Library?


-ec
2/27/2009 12:06:05 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   Software Engineering | Code Generation  |  Trackback
 Tuesday, February 17, 2009
I've always had a drive to take apart anything electronic and figure out how it worked.  Even better if the thing had a microprocessor since then I would try to start up a conversation with it.  I think my first experience with this was hooking up an old teletype machine given to me by my junior high school to a VIC-20 computer to use a printer.  Then came the first XBox.  I enjoyed playing games on it, but I wasn't really a gamer.  Then I found out about modding the darn thing...I think the main reason for this was to be able to make *back-ups* of your games.  I really didn't so much care about that, I wanted to find a version of the XBox SDK and get my own Hello World programming running.



That brings us to current day, Microsoft did something really cool and made game development main-stream for the XBox 360 in the form of the XNA Game Studio.  Last summer I downloaded Version 2.0 of the SDK and started my journey on learning this new technology.  Other than playing around with the Hydra Game Development Kit, I knew next to nothing about game programming, this was a chance to learn a brand new technology. 

I'm still in the process of learning but I thought it was interesting on how I'm building my first game.  Most of the time when I write an application, I really want to have an in depth knowledge of what's happening with the code.  I don't like calling methods, even if they are in a library or frameworks unless I at least have a general knowledge of how that software will be ran by the processor.  In January I started a project that I will be publishing in March as a Community Game on the XBox 360.  To make this work, I needed to do a bunch of stuff I really had no clue how to program.  These were things like explosions, smoke from a missle trail etc...  I found some great tutorials and started cutting and pasting my way to an application that pretty much did what I needed to, although the program was a mess.  I did feel dirty doing this, but just reading about code isn't enough, implementing it into something that actually runs is a much better way to learn things. 

So I figured out how to create 3D models with the XSI Mod Tool, that was no small chore, but kind of fun and something I did on the couch while watching TV.  Next I found some code to render the model in my game, I cut-and-pasted that into my application and be-hold the model was spinning in 3D, although I didn't know how it worked it was rewarding.  Then I added a few more models and decided that the code just didn't "smell right" anymore.  It was a good time to create a class that knew how to render models in my game, this was important since my game had some specific needs and I couldn't just cut-and-paste anymore I needed to know how the code worked.  This is when the lights started coming on.  Next was implementing the flight of a missile.  I found some algorithms for acceleration and gravity as well as a smoke plume and cut-and-pasted the code.  This got me about 75% of the way there and was a mess, but at least it sort-of worked and was rewarding.  I refactored the code, learned how it worked and the light bulbs came on again.  I repeated this process for other things such as rendering water and implementing a generic controller that works with either a keyboard or game controller.  So the basic pattern here was add the features, you may need a dependency that you don't know how to implement, that's OK, instead of just banging your head against the wall for 3 days, find some code the sort-of does what you want, get it working.  Then the real critical part, refactor the hell out of it and make that code your own!



This was a project I wrote for myself, most of the time people pay me to write software so I can't recommend this process for any client work.  However now if a client contracted with me build an XNA game (fat chance, but you never know), I can point to some real world experience and feel good that I'll be delivering a solution and not learning on the job.



I think to a certain extent, this is probably how we learn to implement technologies most of the time, but in smaller steps.  This was just very obvious since I had to start from scratch with many core concepts and the process of turning the cut-and-paste code into my own code was very clear and effective.  So don't feel bad about cutting and pasting code, just make sure you don't leave it like that once you get it working.  Apply your style, refactor, rinse and repeat and make the code your own!

-ec



2/17/2009 8:25:31 AM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   Software Engineering | XNA  |  Trackback
 Thursday, February 12, 2009

Lines are starting to get drawn for the Great Quality Wars of '09.  On one side you have camp led by Joel Spolsky and the now infamous Stack Overflow Podcast #38, and the other camp led by Uncle Bob.  I'm sitting this one out, from what I can see so far, both sides are going to get bloodied up where the discussion turns from a reasonable and positive debate to personal attacks.  We have a lot of problems in our industry and dividing it into factions won't help anyone.  Especially when I see this religious war based on how do I get there, not where do we want to go.

This, being on a much greater scale, reminds me of the Great Coding Convention Battles I participated in within a company in the early 90's.  One of the stupidest things we discussed over the course of a month of meetings was whether to use the following syntax for one line after an if statement.

if(true)
   DoSomething();


or

if(true)
{
   DoSomething();
}


Looking back this was a tremendous waste of time, albeit a relatively fun discussion in our meetings where both sides decided the other side was a bunch of hacks. 

I was in the

if(true)
   DoSomething()


camp unless I wasn't. 

For the case of:

if(true)
    while(someCondition)
        DoSomethingElse();


I would use the convention:

if(true)
{
    while(someCondition)
         DoSomethingElse();
}


where I would be in both camps.  What does this mean?  Does this mean I'm a moderate?  Well no, I do have very strong views on both coding (and politics but I'll spare you the pain on discussing my political views here).  I think the main reason I had strong feelings on my approach was that the other camp wasn't using C/C++ in their day to day work and kept bringing up examples and what *experts* said about the topic and under no circumstance should we have a block of code not enclosed in curly brackets.  My camp lived in a C/C++ editor for most of our work.

 

Do I agree with the statement that Quality doesn't matter?  No.  Do I agree with the statement that Quality is the most important thing?  No.  For any system there is really only one true measurement of success:

In the long term is your user community satisfied and will they continue to use your software to enhance their business or personal lives


Anything else is pure rubbish.  You may have a very high quality system that is just a joy to maintain, but unless people use your software, you probably won't (and shouldn't) get past V1.0.  You also could have a system that you deploy within a month or two that people just love, but it has tons of bugs in the core architecture and can't be maintained, you probably won't get beyond V1.0 either.


Quality is an important attribute of your system, however quality is not free.  Building a quality systems is not ensured by a set of tools, techniques or processes.  Quality is a result of an individual's commitment and a culture within your organization.  

 

There is a danger on both ends of the spectrum.  On one end of the spectrum there is "we don't need these new fangled tools and techniques, our Cobol and Assembly Language application runs just fine", yeah right, there's a real competitive edge.  I'm probably not going to buy or use your software, and if I do, I'm probably not going to apply any updates unless absolutely necessary.  On the other end of the spectrum, we try to rigorously follow the latest fads something like not writing a single line of code without a test, attempting to achieve 100% code coverage, or program exclusively against interfaces.  While this does sound good in reality are you putting too much trust in your metrics?  Code coverage of 100% (or even 90%) means your code was run, that's it.   I may be different, but I know most of my bugs are edge conditions I don't account for even if I do have a test suite covering that chunk of code.  The code coverage metric does little to protect me against these problems. 


What is the middle ground here?  It's actually pretty simple, think for yourself, make sure you write lots code that makes it into production, it doesn't do much good to get your code to a certain point and gloss over all the little details.  Also read as much as possible


Although I think the SOLID Principles offer a number of good concepts,  I personally don't agree with all those concepts.  You may agree with all those concepts.  That's OK.  One area I disagree with is the Single Responsibility Principle or SRP.  I've seen implementations of this in code a code base I needed to extend, and it's just not my cup-of-tea, too much noise, classes are not free.  The developer swore by this technique.  That's OK. 

 

I wish I could find the transcript, but let me paraphrase a discussion of SRP from Uncle Bob's Hanselminutes interview (right around 3 inutes into the podcast)

 

Uncle Bob: This is somewhat out of the norm for object oriented design.  Early Object Oriented Design Principals had us grouping together functions of that operated on the same data structures so that the methods of the class manipulate the same variables. 

Scott: That definitely flips things on its head
.


Early on I learned one of the core principals of Object Oriented Analysis and Design is it's all about the data and methods acting upon that data via encapsulation.  Over time your business rules will probably change, however the structure of your data will remain fairly static.  The way I design LOB type apps is ruthlessly focus on my physical data model.  If you don't get that right, you will be fighting with an impedance mismatch all through your development cycle.  Although I could certainly see the value of a different approach, I keep my business objects 1:1 in sync with the database tables.  The primary reason for this is simplicity, once you break that relationship you introduce a mapping layer that is sometime necessary but comes at a cost of additional code.  The only code you know has zero bugs is code you don't write.

 

Maybe I'm "old-school" but my process has allowed me to pay the bills for the past 20 years now.  In some cases the developers picking up my code said, this is so incredibly easy to maintain and the hand off went smoothly with very few support calls.  In other cases, people just said it was a mess.  It seemed like people that liked my code have been in the industry 10+ years and have handed over legacy code themselves.  The people that thought I was a hack usually had less than 5 years and already knew everything there was about writing software.  They were just waiting for their first chance to start from scratch, build a product and show us old-timers how it's done.  I suspect, at one time I thought I knew everything about writing code.  Maybe this is a right-of-passage?

 

Using classes to group together functions just doesn't smell right to me, it may to you. That is ok.  I have the right to express my opinion just as Joel and Uncle Bob did, neither of these two people are hacks and I don't think I am either.  We as an industry need to do a better job of leaving out the religion and personal attacks, software is hard.  Period.  What works for one person on one project would cause a different person to fail.  


Another thing to think about in adopting and evaluating new techniques and technologies is one of the concepts of the Software Capability Maturity Model or CMM.  If you are a Level 1 organization, you cannot skip directly to a Level 3 or 4 organization.  You must go through Level 2.  If you attempt to implement processes found at Level 3 or 4 you will fail. 

 

At any point in your career your toolbox contains a finite number of tools, over time you should be continually adding tools to your tool box or you won't advance in your career.  If you attempt to implement something like the SOLID principals before having in place effective requirements gathering, defect tracking, or a continuous integration process your time would be much better spent focusing on blocking and tackling exercises rather than more advanced techniques.  It's surprising how many organizations I've worked with that try to adopt the latest advanced technique, but their bug tracking is emails, source control is sorely lacking and requirements gathering consists of meetings where the main goal is for people to hear themselves talk and sound important! I've seen many more projects fail because of the lack of effective leadership, poor requirements or project management than I have because the programmers used the wrong technique or pattern.  Before we put stealth on our airplane, let's make sure our airplanes fly.

 

Over time there have a been considerable advances in our field of software engineering, some stick around some don't.  Are we still using those old Rational Rose Puffy Clouds.  I liked my Puffy Clouds!  I want my Puffy Clouds back!  None of these should be considered 100% bad and ignored, learn something from these, but don't just jump in and blindly implement without thinking.  Put what works for you in your tool box, have a full tool box and select the exact right tool to solve your problem and more importantly, know why you picked that tool.

 

I'm looking forward to the "smack-down" between Joel and Uncle Bob in an upcoming Stack Overflow episode!  Would be very interesting to see them on UFC as well.  In the mean time, I'm going to keep writing software, reading blogs and keep adding more tools to my tool box.

 

-ec
2/12/2009 12:44:54 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   Business of Software | Project Delivery | Software Engineering | Software Metrics  |  Trackback
 Friday, February 06, 2009
With all the grim economic news, there has never been a better time to be a developer (well maybe during the internet bubble, but that wasn't sustainable).  As an independent consultant, I'm always just a little worried about what the next six months will bring.  I've been doing this for about 8 years now and so far there has been no shortage of work, but you never know what the future will hold.  I think 2009 may provide a different way of thinking about how I perform my craft to pay the bills, let me explain.  When doing client work there is really no long term guarantee.  It's a process of identifying a need for their organization, doing an excellent job filling that need, get a check, rinse and repeat.  You are really establishing a "dependency" on your client, their needs and cash flow have a big influence on your future.  As a developer we know that in most cases dependencies are not a good thing.

That's where I think there are some great opportunities these days for us developers to break these dependencies and really take control of our future.  As much as I enjoying developing systems to make my clients succeed, I would rather develop systems to make myself succeed!  How is this possible? 
Well I'm pursuing two opportunities right now that you can too!  The costs and barriers for entry are negligible.



The first one is really sort of fun, I'm building a game for XBox Community Games.  The idea is I build software, publish it, and it becomes available to millions of XBox users to download and play for a very small fee where a large portion of that goes into my account.  This means if I can make the game interesting enough to tap into an extremely small percentage of the millions of users, the income may not be all that trivial.  I'm not buying that 54 foot yacht yet, but who knows.  The point here is that I have a high level of control over the success and outcome of this effort, the advertising and sales are taken care of via the Community Games site.  I can focus on cuttin' code and not have to build a business with dozens of employees and all the headaches that come with that.



I think the next opportunity is even more exciting and beginning the second half of this year will merit a considerable investment in my time.  I'm already getting my feet wet with this technology and I'm excited for the RTM date.  This approach and techonology changes things, period.  Over the past eight years, I've been working on a product I call The Chaos Filter.  I've spun off a few little products that generate some revenue, but really haven't cranked up the marketing engine yet.  My thought is once I do this, my primary role will shift from building and extending the product to building and extending the business to support that product.  The challenge has always been, how can I focus on what I enjoy and outsource 95% of everything else yet remain in enough control to be successful.  My product is really a set of highly configurable services that work together well.  My goal is to allow subject matter experts; non programmers but technical folks; to very quickly customize and assemble those services into things I'm calling MicroApps.  These will be built for highly specialized niche markets.  My thought is not to sell these for hundreds or thousands of dollars, but a monthly subscription fee that will usually be less than $20.  Basically a high value, low cost solution that will be very easy for people to sign up for and keep coming back.  To make any money at this effort, I'm going to need a ton of people signing up.  That's where I see the Azure Services Platform come in.  Without building a large company, I can partner with subject matter experts to build these MicroApps.  Then make these products available to the Windows Live user base.  This also offers a highly scalable platform that as my product grows and I need more capacity, I'll just need to adjust the number of server instances in a configuration file and wha-la I'm more scalable.  Again, as with the XBox game, I can focus on my core competency, cuttin' code.



Another awesome opportunity would be to start writing applications for the iPhone and iPod Touch.  Although being a .NET developer, I'll probably focus on the two opportunities I've outlined above, however this once can't be ignored.  Apple did a great job of making it super simple for people to trade you their money for your apps.  I purchased a MacBook and have started to learn Objective C.  Even though this seems very appealing, I just don't see spending my valuable time on this anytime soon.  I guess the XBox and Azure Live Services just turn out to be a fad *cough* there are other opportunities out there.

I'm spending about two thirds of my time doing client work and the other third trying to figure out what type of killer app I'm going to develop that will allow me to spend all my time programming for fun, not necessarily to pay the bills.  This approach isn't for everyone, if you enjoy the security of working for a company and thrill of being part of a high performance team you may have found your niche, if not, maybe it's time to start looking at what you can do on your own.

-ec
2/6/2009 10:39:35 AM (Eastern Standard Time, UTC-05:00)  #    Comments [1]   Business of Software | Project Delivery | Software Engineering  |  Trackback
 Wednesday, February 04, 2009

Yesterday I had a dog-and-pony show to discuss a product I've been contracted to build for a client of mine.  The demo was in three parts, a technical design review by their lead developer, then a status review with their CTO and finally a recap with the CEO.  The first two parts went just fine, then when the CEO came into the room my Gateway Vista 64 bit laptop froze up.  My mouse would move, but wouldn't respond to any clicks or keyboard input.  To be fair, I had a VM running and I think this may contributed to the failure.  Since this product needed to run on all sorts of OS's & devices, I had my MacBook, iPod Touch and Google phone so I was able to give a few quick demos, just nothing on the projector...I hate it when those types of things happen.

Overall my experience with Vista hasn't been that bad, but also can't say it was all that great, this was the final straw.  I upgraded my main custom built Quad Core machine last December to Windows Server 2008 64 Bit Standard Edition and that machine has just been rock solid.  I decided until Windows 7 goes RTM, I'm done with Vista and will be putting Server 2008 on all my dev boxes.  In a small way I miss the nice Aero Glass interface in Vista and I know I can configure it by turning on "Desktop Experience" in the Add Features configuration section, however right now my system is running so good, I just don't want to make any changes.

On my way home from the client meeting yesterday afternoon, I stopped off at Best Buy and picked up a 2.5" 7200 RPM drive for my laptop and have decided to upgrade.  I picked up the drive for about $109 plus tax.  With the price of hard drives these days, going forward when I need to repave or upgrade a machine, my plan is going to be to just go ahead and pickup a new hard drive then keep the old one around for a bit.  For this upgrade, my computer has two drive bays, so now my laptop will have 640GB of storage (well less than that since a 320GB drive really isn't a 320GB drive, but that's off topic).  That should be plenty of room for a bunch of VM's so I don't have to bring my external HD with me when I travel.  One of the many nice things about this laptop is that that it has a eSATA port that makes copying the 20-30GB VMs quick work.  Once I get my laptop upgraded, I'll report back.  If the drivers just aren't there, I'll just use the second SATA drive as extra storage and put my original one back in and be ready for business!  The moral of the story is with drive prices these days it's just as easy and considering time involved just as a cheap to pickup a second drive and keep the old one around.

-ec

2/4/2009 9:32:57 AM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   Hardware  |  Trackback
 Sunday, February 01, 2009

As I reach the 1/2 point in my career developing software, I started thinking about where we are at in the industry of Software Development and how much progress (or lack thereof) as well as some observations and things I’ve learned.  If you would have asked me 10 years ago if I would be at the half way point in 2009, I would say, heck no, I’ll be retired and doing this for fun…so much for the internet bubble and stock options.

 The following are some observations and thoughts on some things I've learned at the different positions and work I’ve done with the last 19 years.  If you make it to the end of my "ramblings" drop me a note and let me know what you think.

After graduating from Minnesota State University in 1990 with a degree in Computer Science and Electrical Engineering Technology, my first job was as a Software Engineer responsible for designing and programming embedded systems for weight and force measurement devices.  We had about 5-6 Software Engineers on staff, each week we had meetings to discuss ways we could improve the process of developing software.  This was a long and painful process where our primary objective was to build consensus on development standards.  We started with coding conventions for FORTH, C & C++.  What made this extremely painful is that as we all had opinions (although only a couple of us actually programmed in C/C++ at the time) and at one point we spent weeks discussing putting curly brackets around single lines after an if statement.  Anyway, my point is that although it was important to build consensus this approach wasted way too much time, and although I think we did make some modest gains, these meetings were probably better held in an academic setting.  I had an excellent mentor and the company probably still doesn’t want to know the number of hours the two of us spent shooting the sh*t discussing software engineering and software architectures instead of programming devices.  Luckily the two of use seemed to be the only ones to ship on time so I guess they didn’t mind too much.  I considered those years as almost a “Masters” level education.  Some of the observations/lessons from this position:

  • Designing the hardware, writing the low-level drivers and then the software that made the microcontroller do something meaningful is truly an enlightening experience.  It still influences how I program in high level languages such as C# today.
  • Another invaluable lesson learned at this job was the difference between sales and marketing.  And the good, the bad and the ugly of working with other areas within a company.
  • Get your developers in front of the customer and in the real world as much as reasonably possible.  It puts a different perspective on things when you see your software run a huge rock smasher or rail-road train scale instead of turning on $0.05 LEDs in the right sequence that simulates the actual hardware.  This is also true if your developers experience is limited to only seeing a 1 with a bunch of zero’s that represents a decimal data type on a sales order instead of a real order for a gazillion dollars that represents real money.  It's an eye opener to see how real users actually use your software.
  • Keep your developers coding, and then have them code some more and when they are done doing that, have them code a little more.  As with any craft you get better with experience.
  • If you don't know where you are going a map is no help, developers shouldn't be the ones making up the requirements, if they are get them in front of the customer as much as possible (note the conflict with the previous item)
  • Software estimating even on small projects is very difficult.  Estimates from developers should be doubled.
  • When marketing drives certification efforts such as ISO9000, the results are very shallow.

Next I went to work for a Sales Force Automation company, starting out as a software engineer and ending up with the title of “Director of Tools and Technology”.  I had two primary roles as a director, first was to dispatch fire fighting teams get projects out of the ditch and the second was to put in a set of processes and tools to help keep those projects out of the ditch in the first place.  I had the luxury of hiring my own senior staff.  I’m a firm believer that right people are single most influential factor in the success for failure of a software project.  I actually found that people with engineering degrees in some respect were better suited for this type of work than ones with a pure comp-sci degree.  In my previous job I was introduced to the classic Managing the Software Process by Watts Humphrey this is a classic must read book.  A number of concepts seemed to make considerable sense when working with large projects with a duration of 1+ years and teams of 6-12 developers.  Much more so than the single person 3-4 month projects at my previous position.  One of the lessons learned here is that different types of software require different set of processes and talent.  Some other observations from this position:

  • Developing large systems is just plain hard, there is no silver bullet, but with the right people it is possible and can even be fun.
  • Team building is important to the success of your project.  Finding the right chemistry is worth the effort, bad apples or “negative producers” must be swiftly removed from the team.
  • It’s really all about data, for the type of systems we were building (and even the ones I build today) getting the data model right will make or break your project.
  • While the business analysts are figuring out what to build, it's a good time to have your senior and lead developers plan out your architecture and figure out what common functionality needs to be organized into the application frameworks.
  • It's worth the investment to build scaffolding to test your application.  At the time we weren’t doing any formal unit testing, but had all sorts of little tools and utilities to help make our life easier.  Really a necessity when the MFC compiler took 45 minutes to do a full build.
  • I was formally introduced to “Design Patterns” and the GoF.  Coming from an engineering background, I had always thought Software Development needed to mature and “Design Patterns” were going to be our building blocks.
  • Platform Independence sounds good when on the marketing cut-sheet’s but in reality…well…you know the story here.
  • Senior, top level management buy-in and participation is absolutely critical to the success of any software process improvement effort.  The primary reason for this is there are more departments in your organization other than development and everyone needs to play together nicely to ensure successful projects.
  • When spec’ing and selling multi-million dollar projects, make sure you have representatives from the dev staff involved with pre-sales.  Without that I’ve found there is usually an expectation mismatch, this is true in smaller projects but critical in larger ones.
  • Our company was organized into Product and Professional Services divisions, using real world requirements from Professional Services goes a long way to help product development planning.
  • There should be an inverse proportion of the amount of code you write with respect to your level within the management food chain.  When I started as a software engineer, I spent 90% of my time coding.  As a director, I spent about 5% coding if I was lucky.  The challenge here is to keep your skill sharp, you need to code, period.  To be a good manager you need to manage period, if you are coding you are doing your employees a disservice.
  • In our industry, a manager works for their employees.  That is to say a managers first and foremost responsibility is to equip her coders with everything they need to succeed and aggressively remove road blocks.
  • Although difficult, setting and managing expectations is critical to the success of delivering software.
  • Although I’m sure the agile folk’s won’t agree, I’ve found the longer I can stay out of the “problem domain” and keep working on my application frameworks (factoring out common/abstract functionality) the sooner I’ll ship my software.  This deserves another post to explain in more detail.
  • Technical solutions generally don’t work to fix process problems
  • Towards the end of my time there I had a keen understanding of why executives “don’t like surprises”.
  • If you get a stock option grant (especially one big enough to retire with) it's a good idea to get a lawyer involved early on so you don't end up with Toilet Paper.
  • It’s all really about People, Process and Technology

 

PPT Solutions, LLC

Three of us left the Sales Force Automation company and started our own venture it was called PPT Solutions (People Process & Technology, in that order as we liked to say).  We put together our business plans, cash flow projections and since this was in the internet bubble we were convinced there is no way we could fail.  Although I wouldn’t consider this business a success, we did manage to ship a product, and I did learn a lot about going into business for yourself.

  • We were told going into a business partnership with friends was a bad idea, we thought we were different.  We weren’t.  The reality is that someone needs to be in charge and equity needs to be based upon contribution to the company.  Friendships complicate this, especially in our industry where we think we are all “rock-stars”.  Eight years later the wounds from this are finally healed.
  • Sales is critical, sales is not easy, sales requires a different skill set than developers, we sold our product into Imation (3M Spin off and electronic media company) I think we landed the sale because I was sleeping with the global sales training manager (who I've been married to for the last nine years and still madly in love with).
  • Although it sounds good to get a sale and then use that money to develop a product and then market it that way, in reality the product will contain all sorts of very specialized features for the company paying for the majority of the effort.  Care should be taken so that specialized requirements don't contaminate the core product.
  • Make your estimate than double it, both time and dollars
  • It’s a double edged sword working out of your house.  It's nice, but it requires discipline to stay focused.  Having weekly goals is a good idea.  You will probably also be working on weekends to a certain extent.  Do whatever possible to separate your work week from your weekend, this helps burn-out.
  • If you partner with other people to start a business, clearly define expectations and responsibility up front.

 

Next up was a small startup, let's just say I don't have a lot of good memories from that place.  This wasn't really a technology learning experience, however I did learn how to successfully get what I needed in a difficult environment to ship software.  The company came in at the tail end of the internet bubble and had visions of making a pile of cash with very little effort and a lot of 0's and 1's in the right order.  Enough said…moving On….

  

That takes me up to what I'm doing today, Software Logistics, LLC.  My company.  I don't have any employees, but use contractors as necessary.  I seemed to have found a nice niche.  I spend about 90% of my week coding.  Although I do miss the thrill of leading a team into battle, I guess the pure joy of carving out excellent software makes up for it.  I've worked with very small companies and very large companies during the past eight years.  Each has their strengths and weaknesses and enjoy working with both.  This has been a great learning experience and I’ve shipped a number of projects in the process.

  • To get started, networking and word of mouth is very important.  Even more so is networking with the right people.  There is time for friends and time for business.  As with my previous venture, these don't often mix very well.  That doesn't mean you can't have fun with the people you work with, however since this is going to be your primary source of income it's important to stay objective.
  • Even though you are out on your own, that doesn't give you an excuse to cut corners or skip basic SCM principles such as source control, issue tracking, build environments etc…  What's nice about being on your own is you don't have to reach consensus and you can experiment with different processes and technologies.  Although still in evolution, my CI and deployment environment rivals if not exceeds most of companies I work with.
  • Invest in yourself.  I attend at a minimum two conferences a year.  Even those these are on my own nickel this is a must.  Going to conferences allows you the time to focus on what's new and get a quick overview of a wide variety of technologies.  Don't expect to become a master at any of these, but you should get an idea of what's real and what's worth investing more time in.  Even more important is getting a sense of that latest flavor of the month so you don't waste your time or sabotage your project.
  • If done right, it's generally important to come off strong in the first 15% of the project and finish strong in the last 25%.  This doesn't mean you go "dark" but during the development phase it's a balance between keeping your client comfortable with the progress and staying out of the details enough to keep on your schedule.
  • Communications is critical, if I had to say there is one common denominator across all the clients I've worked with is that they all need to improve their communications.  There needs to be a balance between using tools such as a defect tracking and verbal communications.  SCM work tracking tools are good for capturing and managing a large amount of details, however even with a "priority" ranking, they just don't convey the sense of urgency and importance that verbal communications can provide.
  • When in a meeting (even in sunny Tampa) no-one likes to see your arms, wear a nice button down dress shirt and khaki's.  Remember you are a professional, impressions matter, especially when negotiating the yearly rate increase.  I don't think it's important to drive a BMW, but you should project a feeling of success.
  • You can't generally rely on the QA of your clients.  If you can't do it yourself (which is very difficult if not impossible) hire someone and get a good issue tracking system.  I really like Visual Studio Team System, if you have a premium subscription, you should be able to leverage the workgroup license.
  • Hold frequent status meetings, you should run these meetings, always have an agenda.  Set the expectations as to what you will cover in your agenda and ask for feedback if they have any concerns.
  • Respect your clients time, most people have more work than time.  At the start of the meeting ask for how much time you have and adjust your schedule accordingly.
  • You are viewed to be the expert, and to be successful, you should be a jack of all trades and a masters of a lot of them.  However it’s impossible to know everything.  If you don't know something don't BS your client.  Most of the time the people you are working with can tell, you need to keep your credibility and trust.  If you don't know something, find out and make sure you follow up promptly.
  • Your demos will almost never go as planned.  I remember one demo that I practiced a number of times and it worked flawlessly, I got to the client site and they had custom DNS settings on their internal network.  It broke my demo.  Think quick, think on your feet, and adapt, in my case a simple tweak of the HOSTS file was all that was necessary.
  • At all costs do not get involved in company politics.  All companies have them.
  • Don't expect the same level of support/services from other people at your client as you did when you worked as an employee within a company.  People are generally overworked and requests from the consultant come after requests from their internal managers.
  • If you are billing hourly, make sure you have a good system for capturing and tracking your time, even if your client doesn't ask for it.  Never over bill your client, if your client isn't paying you enough to live on, ask for more money find an additional or a different client.  Remember it's your name on the line.
  • If a client pays you to build software, they own the source code (unless you work out a different arrangement).  Don't use that source code in a different project or for a different client without their written approval.  That isn't however saying that you can't re-implement the same algorithm, they own the source, not the concept with certain obvious exceptions such as industry or trade secrets.
  • I've found that there is no right answer to the right amount of documentation you should provide.  I had one client that complained about the level of documentation I produced.  When it came down to detail and specific documents with senior management, I could always produce the documents in their repository and the response was "I didn't know we had that".  I don't think the documents were even looked at.  I guess my point is for a developer, documentation is no fun.  If no one is going to read it it's your call on how much to deliver.  It's good for CYA.  Another client I have uses a WIKI for documentation, nice and simple, yet effective.  If your client doesn't have one, set one up for them on your server.  If it gets traction install on their server.
  • Don't skimp on your computer setup and hardware.  Time is money, would you rather spend your time installing and configure software, or would you rather cut code and get paid.
  • I've found that using a tool like VMware Workstation works great for doing client work.  I have a base setup that has all generic development tools.  I can than copy that base VM and configure it with the specific tools and source to work on client projects.  An added benefit here is that at the end of the project, you can always just give them the VM and they can pick up where you left off.  My main dev boxes generally are multi-core and very fast, when I'm working in a VM, I just can't tell a difference.
  • If the environment is right, offer more than you ask for, I really enjoy mentoring young developers, and showing them new tools, technologies and techniques, be careful here though not to step on other internal developers toes.

If I haven't bored you too much and you made it this far (I'm a coder not a writer), I can offer some general thoughts of where we are as an industry:

  • Every project is different, yet there are many similarities amongst different projects.  If anyone says they have the one true way to develop software, you can tell them, ok, whatever, let me buy you beer and let’s poke holes in the process for what I’m working on.  There are also differences in techniques required when working on the same project.  An example is a non-trivial LOB application, there is the application frameworks, data layer, business objects, UI library, each of these is different and requires a different approach and the investment in time and design.  My point here is that there seems to be a lot of great work done developing tools, technologies and processes, we just need to start doing a better job of discussing where they should be used.  And general sweeping statements such as any code with out tests is considered “legacy” (read crap) don’t do much for helping us bring the right tool to the right problem.
  • I'm not sure there are any metrics out there, but I would suspect the majority of developers out there work on what I would consider Line of Business (LOB) type of applications.  Generally the most difficult part of this development is the ability to track and manage changes to the business rules and requirements.  I've seem a few implementations, that probably were much more complicated than need be.  Sometimes it's fun to pick up the latest technology or pattern when all you really needs is some simple OO principles and a nice Forms over Data application.  Some of us need to build Jaguar's and some of us need to build F-150's.
  • I mentioned it above, but one of the core problems I see in our industry today is the lack of effective communications, this was true back in 1990 and is still true today.  This may not be true of all companies, but I would suspect it is for the majorities of the companies out there.  I think Software Development can be described as managing thousands of little details, understanding how they act together and how to effectively communicate those details with others.  We do this by a series of translations; users translate their needs to a user advocate, the user advocate translates those needs to a business analyst, the business analysts turns those into requirements and prioritizes, the developer picks up those requirements, the develop translates the requirements to a design, then the design into code and finally the compiler turns the code into something that can run on a computer.  Even if you effectively implement agile techniques (which I suspect most companies that say they are agile really aren't) this is just plain difficult.  Take into account Fredrick's Brooks law of adding man power to a late software project makes it later and you get an idea of the scope of the challenge.  So, you say, what's the answer?  I'm not sure there really is one.  What I do know is some people are better at this than others, I think this is probably an important attribute that is overlooked when hiring.  Sure you may be a rock star coder, but if you don't code the right thing, oops…what's the point?
  • Everyone in our industry thinks they are a "Rock Star" in reality very few people are.  Another theory from the class Mythical Man Month is that there is an order of magnitude between an average developer and ones that are truly "Rock Stars".  I’ve found this to be the case.  On some big projects I’ve worked on I’ve found that 20% of the team completes about 80% of the work.  The trick that I've learned is accepting that I'm probably not one of the “Rock Stars” and as the Verizon quote goes try to "Make Progress Every Day" into becoming a better developer.  This includes reading all the classics, writing tons of code that gets into production.  Coding for fun generally glazes over the details, details are what makes software hard.  I've also found it to be beneficial to work with really smart people, they push you, you push them.
  • I still keep in contact with the ninja's that worked for me to setup what we had thought was a decent development organization.  Since then, we went down two different roads, they went to work at a huge enterprise, where the basic idea is to manage change and make sure the gazillion lines of code compile and bugs don't cost too many millions of dollars per year.  I'm cuttin' code pretty much every day.  I don't know which path is better, I'm not sure there is a better path, I know I love what I'm doing now.  When we get together a couple times a year, it sounds like progress is being made in managing change, however the systems are getting more and more complex.  The net result is that things are just as screwed up as ever and the same fires we put out back then are still flaring up however they are handled much better.  Kudos to them, I have a lot of respect for what they do.
  • It appears that more and more companies tend to rely on libraries and third party and open source technologies.  I think this is both good and bad, if you are working on a small 1-2 month project go ahead and use the 3rd party building blocks.  If you are working on a large system, it may be better to re-invent the wheel.  I think a good example of this is your Data Access Layer, if you are doing a small one off type of project absolutely use something like Linq 2 SQL or NHibernate.  If you are working on a fairly large project with a number of developers, it may be worth it to carve out your own DAL.  This is a non-trivial process, but 6-8 months into the project, you should recoup the investment by having exactly the feature set you need and understanding the inner workings of the critical piece of technology.  Spend the time up front, think through your architecture and requirements.  Realize that those 1/2 hour demos you saw where someone created a CRM site with just 5 lines of code, probably leaves out a considerable number of details.  Sometimes just dropping in a library introduces problems as noted by Joel Spolsky's Law of leaky abstractions

Well I guess I should probably close as I could ramble on for another few pages, I think in summary I would have to say that our industry is maturing with new processes, technologies and techniques but at the same time the systems are developing are getting much more complex so I would say our overall progress as an industry is probably flat.  At one time I had thought we would mature to a more engineering approach to delivering software but I think with the dynamic nature of the problem space and technologies, I'm not convinced that will happen anytime soon.  As to being at the 1/2 way point in my career, I’m still trying to build the “Killer App” who knows….

-ec  

2/1/2009 5:30:45 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   Software Engineering  |  Trackback
 Wednesday, November 26, 2008

In my ChaosFilter project, I'm factoring all the text and strings into a resource class.  The intent is to allow for customization by the end user for different installations and possibly translation at some point.  My solution is to place all my text into a class with properties that figure out how to get the text based based upon a unique key and context (which company/application they are running in).  Currently these are stored in the database and then loaded into a cached dictionary object.  This appears to work very fast, however with my architecture and a bit of code gen this could easily be replaced by more traditional resources. 

In the past it's been a pain the a** since I needed to create the label, put the marker into my class file then insert the field into the database.  A better solution certainly exists.  My solution is to build a DXCore custom refactoring.  The idea would be if I'm in a C# class within a string literal or an ASP.NET page within some static HTML, I could hit the refacting key and and take that text and convert it into a message or a simple label.  The difference being a label is only a few words where the message could be sentance or two.  For larger blocks of text I have a different approach.

My Solution

For a ASP.NET page:

For a C# code file (in a library assembly or code behind):

Then to enter the text:

 

So how was this done?  Actually it was fairly trivial, I built part of it early this week in a couple hours, the rest this morning.  Probably < 4 hours for the whole thing and I haven't built a plugin for a couple years.

 

Getting Started

This link does a much better job than I could to show the basics of creating a DXCore plugin, but unless I missed it, it didn't cover too much about creating custom refactorings.

http://community.devexpress.com/forums/t/68994.aspx

 

So assuming that you have the frameworks setup for your plugin (I'm going to skip a few steps not relevant and found elsewhere), here's what you need to do:

1) Create a new DXCore Plug for your refactoring:

 

2) By default the Refactoring Provider that you need to create a custom Refactoring is not in the toolbox, it can be added by "Choose Items" and selecting the RefactoringProvider

3) Once you add that, drop that onto your custom design surface for your Refactoring plugin, it should look something boring like:

4) The click on Properties for your component and add appropriate values:

5) Now go in and click on the highlighed events to build up your stubs:

6) Finally go in and add your code (painting code from: http://tinyurl.com/5a3pe2):

private bool _handlingEditorForegroundPaint = false;
private SourceRange
_previewRange;
private DevExpress.CodeRush.UserControls.CodePreviewWindow _previewWindow = null
;

// DXCore-generated code...

#region InitializePlugIn
#region FinalizePlugIn

private void ConvertToCustomCaption_Apply(object sender, ApplyContentEventArgs ea)
{
   var customLabel = new ui.CustomCaption
();
   ea.Element.SelectFullBlock();
   customLabel.ShowWithString(CodeRush.Selection.Text, CodeRush
.Caret.ScreenPosition);
   if
(!customLabel.Cancelled)
      ea.Selection.Text = string.Format("Customization.Captions.{0}"
, customLabel.LabelKey);
}

private void ConvertToCustomCaption_CheckAvailability(object sender, CheckContentAvailabilityEventArgs ea)
{
   ea.Available = CodeRush
.Caret.InsideString && ea.Element.InsideClass;
}

private void ConvertToCustomCaption_HidePreview(object sender, HideContentPreviewEventArgs ea)
{
   if (_previewWindow != null
)
   {
      _previewWindow.HidePreview();
      _previewWindow = null
;
   }

   _previewRange = SourceRange
.Empty;
   if
(_handlingEditorForegroundPaint)
   {
      _handlingEditorForegroundPaint = false
;
      EventNexus.EditorPaintForeground += new EditorPaintEventHandler
(EventNexus_EditorPaintForeground);
   }
}

private void ConvertToCustomCaption_PreparePreview(object sender, PrepareContentPreviewEventArgs ea)
{
   PrimitiveExpression ele = ea.Element as PrimitiveExpression
;
   if (ele != null
)
   {
      _previewRange = ele.Range.Clone();
      EventNexus.EditorPaintForeground += new EditorPaintEventHandler
(EventNexus_EditorPaintForeground);
      CreatePreviewWindow(ea, "Customization.Captions.[New]"
);
   }
}

private void CreatePreviewWindow(PrepareRefactoringPreviewEventArgs ea, string codeToPreview)
{
   _previewWindow = new CodePreviewWindow
(ea.TextView, _previewRange.Top);
   _previewWindow.AddCode(codeToPreview);
   _previewWindow.ShowPreview();
}

private void InvalidatePreviews(RefactoringPreviewEventArgs ea)
{
   if
(_previewRange.IsEmpty)
      return
;

   int doubleSpaceWidth = ea.TextView.SpaceWidth * 2;
   int
textViewLineHeight = ea.TextView.LineHeight;
   Rectangle
previewRect = ea.TextView.GetRectangleFromRange(_previewRange);
   previewRect.Inflate(doubleSpaceWidth, textViewLineHeight);
   
ea.TextView.Invalidate(previewRect);
}

void EventNexus_EditorPaintForeground(EditorPaintEventArgs ea)
{
   if
(_previewRange.IsEmpty)
      return
;

   using (StrikeThrough strikeThrough = new StrikeThrough())
   {
      strikeThrough.TextView = ea.TextView;
      strikeThrough.FillColor = Color
.Red;
      strikeThrough.Range = _previewRange;
      strikeThrough.Paint(ea.Graphics);
   }
}

The "thingy" that let's me enter the text is just a simple WinForm represented by ui.CustomCaption, I won't bore you with that implemetentation since it is fairly tightly coupled intergrated into my architecture.

Enjoy!

-ec

11/26/2008 3:36:25 PM (Eastern Standard Time, UTC-05:00)  #    Comments [0]   ASP.NET | DXCore  |  Trackback
Copyright © 2012 Kevin D. Wolf. All rights reserved.
DasBlog 'Portal' theme by Johnny Hughes.
Pick a theme: