DevConn 2007

I had a hard time convincing my self to go to Fall DevConnections 2007 in Las Vegas.  I was really glad I went, I think I'm finally feeling that I'm on the tail end of the learning curve with the new technology releases from Microsoft.  I can't by any stretch of my imagination an expert in any of the new technologies but at least with some recent development experiences and attending numerous sessions over the past few days at think I've got a good lay of the land, at least to a point where I have a good sense of the capabilities of the new technologies and should be able to match up a problem to the right solution. 

Here are some of my findings from this conference:
1) I'm anxious to start using VS.NET 2008 (to be released later November), specifically I'm interested in the better support for developing AJAX sites with intellisense and script debugging.  I'm also interested to see the performance enhancements in the new product.  I'm starting to get a little frustrated with some performance issues in VS.NET 2005.

2) I'm starting to "feel-the-love" with Silverlight 1.0.  Although it's extremely exciting to think that Silverlight 1.1 can be programmed with any .NET languages instead of just JavaScript as in 1.0, I'm just not there yet for 1.1.  I like it that there is a nice and small 1-2MB file that can be loaded and installed with little to no friction is great for 1.0, but although I appreciate the ambition of moving .NET into all the different worlds with Silverlight 1.1 I think there might be too many moving parts.  I think the XAML Browsers applications are probably a better alternative...we will see, I sincerely hope to be proved wrong :).

3) I think I'm starting to feel a little better about LINQ as well.  In the past LINQ seems to have been presented as a completely different way of working with data, well yes it is, but in demos I've seen this was going to start leaking into the UI layer.  It seems like there are two really different uses for LINQ, the first working with pure objects via collections of different sorts, and the second is LINQ to SQL.  As with my previous post, it seems like for the most part we should let the Database handle the getting/filtering of data and that is what LINQ to SQL does (saw some awesome demos of seeing the SQL that LINQ generates and the execute it).  I'm still not sold on the other approach (at least in the 90% apps that sit on top of a database) where we use LINQ to work with collections of objects.  I can see some edge cases where this might be kind of nice, but I just can't see this in the vast majority.  I need to get my hands dirty with LINQ to SQL, it might be a nice alternative to CodeGen which I'm still a big fan of...stay tuned.

4) I attended a session on the Microsoft Sync Platform, very cool...kind of packages up a lot of nice features I built into a client application.  It was good verification that I got the architecture right!

5) There are some new additions to WF in 2008, specifically the ability to create WCF end points for sending and receiving messages.  I can't say I completely understand this yet, but there most certainly appears to be some interesting capabilities here.  There is also a project to embed a WF designer into client applications.  This could get very interesting.

6) It seemed like in a number of sessions I went to the presenters had one problem or another with their technologies, I guess this is probably a case of being on the tail end of the betas for Microsoft's latest releases.  I also give a few presentations on some of the latest cutting edge stuff and it makes me feel just a bit better that I'm not the only one that runs into some "technical challenges" doing demos.

In summary...another good conference, the next one will be in April in Orlando 

-ec

LINQ Thoughts

We'll it's been a long time since my last post, I've been extremely busy over the past few months, both working on The Chaos Filter as well as client work.  I have spent a bit of time on some new technologies that hopefully I'll get a little time to do some additional posts.  These include Silverlight, Microsoft Office Communications Server Speech Server, and learned a lot from Marshall Harrison of GotSpeech.NET fame.  I've also built a nifty little WCF service that does call backs to .NET clients, this could be very interesting.

Any way, I should probably get back on topic...it seems like everyone is going ga-ga for LINQ, yes, I agree this is a very cool technology, but I'm just not feeling the love from the perspective of providing a new set of capabilities we just didn't have before.  Maybe it's the type of systems I build that is prompting my skepticism.  Most of the systems I would consider are "transactional" or maybe stateless would be a better description.  This includes ASP.NET web page processing and other services oriented programming, basically take a bit of information do something with it (talk to the database, store a file transform it, cook it up in a pan) and send something back to the user.  That's not to say however that I'm not implementing an object oriented approach.  It's just there is a distinct start point, interaction with a whole bunch of objects to do "stuff" and a distinct end point and other than some very simple state and context information (usually stored in some sort of persistence storage such as a DB) I just don't keep state in memory between transactions.

Saying that, let's discuss a different class of application.  I'm currently re-inventing an IM server (OK, I got a good reason to do that, but that's not the point).  With this IM server, I'm using the WCF service as I mentioned above to make call backs to clients out in the cloud that have initiated a session (still need to understand the plumbing and underlying mechanism here, but again that's not the point).  So basically I have a service running in the background, clients connect and open a session, interact with that session and I can send call backs when invitations come in or send out a message to a conversation.  What this means is I'm creating almost an In Memory Data Store that contains relevant information to manage these sessions.  Basically right now, I've got everything working off of System.Collections.Generics.Dictionary objects where it's fairly simple to do a lookup to get basic state information for what I need.  Alright we are three paragraphs into this, I guess I should finally get back to LINQ and how it relates.  My in-memory database of state information is starting to get a bit more complicated and I would love a better way to do lookups against this database, I can see the value of LINQ in a BIG-WAY for this type of application, hopefully if you made it this far, you do to.  Unfortunately for this application it's a .NET 3.0 app, and I'm not ready to jump to .NET 3.5 to leverage this new technology.  So setting aside the using LINQ for relational DB access and the political debate that it brings, is this a valid use for LINQ in things that we just couldn't do before?  If so how many of us are writing these types of stateful applications of those of us that are writing these how many of these should actually be stateless and thus minimizing the impact that the new LINQ .NET 3.5 features provide?  I'm not proposing I have the answer here, but I'm posing the question...

-ec

Hi, My name is Kevin and I'm a VS.NET Junky

Now to the rant, well for the most part I have two main problems that are really starting to drive me nuts, first is performance and the second is VS.NET crashing.  Another side bar please, I've been in this industry since the early ninety’s and developing programs since the early eighties we've come a very long ways.  I remember a couple of multiyear long projects I worked on in the mid to late nineties.  These were VC++ apps that literally to 30 to 45 minutes to compile each code-build-link-test cycle, some of us built some test harnesses to cut this down, but still it took 3 to 5 minutes.  As I continue my rant looking back at those projects really puts things in perspective. 

So what do I mean by performance issues, I have what I think is a fairy high-end developer machine, not like this monster machine that Scott Hanselman put together (oh yes - I will have a quad core box sitting under my desk by the end of the summer) but a decent machine none the less, I have a Core 2 Duo running at 2.13GHz with 4GB ram and a decent SATA hard drive, I've also got a USB drive I'm using to leverage ReadyBoost on my Vista machine.  Here is a link to a better description of my setup.  I'm currently working on a fairly large C# solution with about 25 projects and 4 web applications.  The application was written so that the user interface is a collection of Custom Controls or more specifically WebParts so each time I make updates I need to compile my control libraries or business objects libraries.  If my system has recently been rebooted I can usually do the edit-compile-test cycle in about 15-20 seconds, that includes the ASP.NET page recompiling with the new control library.  I have to admit, when you think about all the work it's doing, that is really incredible.  With this type of edit-compile-test cycle you can really achieve the state of flow as describe in the wonderful book PeopleWare productive projects and teams, a classic must read.  I have a terrible habit of working on too many things at once, I'm lucky, in that I can very quickly context switch however this generally results in a number of applications being open at one time on my computer.  After about 3-5 days of working on my Vista machine (for some crazy reason I installed Vista Ultimate, might that be part of the problem?) my compile times slowly start creeping up to where they are starting to approach 45-60 seconds.  In reality this just doesn't seem that long however it's just long enough to interrupt the "flow".  I think the thing that really makes my blood boil is when I'm watching the output pane in VS.NET and it says the compile completes, its 100% complete with zero errors, then, as I usually run my computer with task manager open, that little green bar sits at about 50%-75% VS.NET is non-responsive and I usually say something to my computer like..."That's OK, I'll just sit here and wait, no problem I don't have anything better to do...are you finished?...are you finished?"  (I get some strange looks from my better half, but I think she is getting used to it) then about 15-20 seconds later I can see if what I did actually worked.  Alright again only about 15-20 seconds but it certainly blows-the-flow.  That closes out the first part of my rant...in all reality, what VS.NET does is incredible, but the extra 30 seconds is just enough to let my mind wonder, check my emails, IM etc... (ding...maybe that's the real problem I have AADD and should look to fix my compile time issue with medications, trade one addition for another).  One thing that I try to do to "keep-the-flow" going is with an excellent tool called TestDriven.NET this get's my edit-compile-test time down to a matter of a few seconds when I'm working on my business logic, if you aren't using this why not?  In full disclaimer, I have CodeRush and TestDriven.NET plug-ins installed, I don’t think they are the problem.  My prior machine was a 3.0GHz Pentium D and 3GB RAM, I installed Vista Business and the VS2008 Beta 2 (not in a Virtual PC).  I don't plan on installing any other software on that machine and will see how it performs.

Rant number two...this one won't be nearly as long, about every 30-50th compile, as soon as I see that "Build Successful" message in my status bar, within a blink of eye Visual Studio disappears from my screen.  No error message no log entry...nothing, it's just gone.  I start it up and load my project (which takes between 2-3 minutes) and have never lost any work, but my solution settings (such as which files are open etc...) get lost.  Really a little annoying.  Whenever I get a free 4-5 hours I'm going to repave the machine and only install VS.NET and the key tools necessary for doing development, this time I mean it for sure, really only my development apps.

There, I got that off my chest after looking at the good the bad and the ugly, the reality of it for the size of our solutions and all the competing junk I have on my machine, I should be happy these compile in 5 minutes and I'm probably being greedy, but as with most developers I like to rant and complain.

Bottom line, thank you Microsoft for feeding my habit and providing me with a great (but not perfect) set of tools to write software to keep my clients happy and thus pay my bills, let’s see if we can do it a tad better in Orcas.

-ec

WF/Everywhere

(Work From Everywhere)

We'll it's summer time again and time to head out to the beach.  We have a travel trailer my wife and I take down to Fort Myers Beach in Florida to a place call the Red Coconut nice place right on the beach.  As an on line addict I need to have a decent Internet connection or I go through withdrawls...pretty sad.

What I'm currently using is a Sprint wireless card along with Linksys router that it plugs into.

This combination rocks!  I'm getting excellent download rates

It also has a built in WiFi radio, so I can sit outside my travel trailer and get on this connection.  It also works great as a switch to distribute connectivity.  I have my MCE 2005 box hooked into it and we watched a streaming version of Lost from the ABC site with a perfect picture!  This also works great if you have a Sling box player (highly recommended).

So if you haven't looked at EVDO lately I highly recommend it, for about $60 a month it's an awesome solution.

-ec

MEDC 2007 - Day 1

Actually the first day was yesterday with MIX, I went over there with high-hopes of watching another great key-note by Scott Guthrie, I made the trip from the Monte Carlo to the Venetian early that morning, got registered and headed up to the meeting room.  I was greeted by a not-so-friendly security guard saying the MIX sticker on my badge wouldn't let me onto the fourth floor...guess there was something in the fine print, oh well.  I came back that afternoon and went to a couple great sessions, one on identity, it really seems like OpenID is really going to merit a series investment in time over the coming months.  The other conference I went to was on security with AJAX enabled applications.  This was a great session as a refresher/reminder that building a secure application really needs to be started from day one and built into the architecture, at least its much easier that way.  Overall the $395 I paid to add MIX to the MEDC conference was well worth it

The conference started out with a bang with the Key Note by Robbie Bach the president of Microsoft’s Entertainment & Devices Division.  He made a great case for developing software as a service and one of the biggest clients for those services is going to be mobile devices.  I also so some great demos on another one of the things I love to work on in my spare time (what's that?) Home Automation.  I really need to look at some of the latest developments in this area starting with Microsoft's Home Server.

Key points on what I took away from today's sessions

• Microsoft/Verisign finally came up with a "sane" way of signing mobile CAB files for WM 5.0.  Before you had to sign all your files, send them to Verisign, let them do their thing, get those files back.  Package them into a CAB and send it back up to them to sign the completed package.  What a pain in the ass, no real way to automate.  Just introduced this week, you can just start sending up the CAB file and it only costs one signing event!  Now it seems reasonable to sign the files...
• There is now an applications out there called fakeGPS.  I have a project coming up in May where I need to report on GPS data from a device, I'm looking forward to digging into this technology.
• According to Microsoft, WM6.0 and the Compact Frameworks are 100% back-wards compatible.  I have a fairly large WM5.0 application I've developed so I'll put this through the test, I really hope this is the case!
• On WM6.0 devices the Compact Frameworks 2.0 SP2 and SQL Server Compact edition are baked into the ROM...nice!
• On the large WM5.0 application I developed there is a good chunk of the code that deals with one handed operation on a Pocket PC type device (read not smart phone) I always thought this was a real hack...it was nice to see the Microsoft presenters give a demo of how to do this (and their implementation was really even more of a hack ;) ). 
• In Windows Mobile there is an event that I can subscribe to (something like Connection Status Changed) this will be fired when the connection via cell phone changes.  As part of my mobile application based upon a schedule I check the server for changes...kind of messy and probably wasteful the easy solution here would be to use an SMS message to kick of replication, yet this isn't that great either.  What I'm thinking about is that I'll build some sort of registration type service where the device can register with the server it's IP address as the connection status changes, then the server could make a TCP/IP call to the device to initiate replication (or maybe just send changes)
• New terminology for WM6.0 (Don't shoot the messenger)
  • Smartphone (no touch pad) - Now called "Windows Mobile Standard"
  • Pocket PC Phone - Now called "Windows Mobile Professional"
  • PDA's (no phone) - Now called "Windows Mobile Classic"

Overall this was a great day...picked up lot's of tips & tricks and validated all my design considerations over the past year.

-ec

64 Bit SSIS & Importing Access Data

Day 1 in Las Vegas for MIX & MEDC 2007...conference starts on Monday, I arrived on Sunday and had planned for a nice relaxing day of enjoying what Las Vegas has to offer.  Only a small bit of work (or at least that's what I thought) I had been manually kicking off a process that imported Shoretel Phone records into a SQL Server 2005 database to do some reporting on phone usage.  I knew this week would be busy so I figured I'd get this setup as a job within SQL Server so I didn't have to worry about it.  I moved my package to the production box created a job with one step that would start the package and send me a status email.  As a good developer I tested it before I headed on my adventure...oh-no it didn't work!

To make a long story short, after screwing around with this for about four hours (which I really couldn't bill my client for) I finally figured out that the OLE DB Jet drivers don't come in a 64 bit flavor...so executing it with the job scheduler didn't turn out to be an option, I finally ended up just creating a simple windows "Task Scheduler" job that kicks off a the 32 bit version of dtexec and life was good...

Here's hoping somebody finds this post and doesn't waste as much time as me!

-ec

The Learning Curve

What is the acceptable learning curve on a new technology?  How much time will you invest in something before you realize you are spinning your wheels and you need to get some actual work done?  I recently had a reporting project as that as I like to say just "Screamed" for an OLAP type solution.  I had considerable experience with MS Reporting Analysis Server 2000 building numerous cubes that just worked great.  My past couple of years (at least since the release of SQL Server 2005) just didn't present any needs for this.  This new project was a perfect excuse to pick up the 2005 Analysis server technology, however this project just wasn't that big and didn't have a lot of hours allocated for it.  I figured, eh this new technology can't be that much different so let's give it a try.  I had bought the books a while ago and took them out over the weekend to the back porch lit up a cigar and read what I thought were the important chapters.  Then I went back in and fired up VS.NET found the right project templates and got started.  This is certainly not a post on how to use MSAS (Microsoft Analysis Server) 2005, but I created my data sources, created my data source views and was off to the races.  Then I created my cubes via the Cube Wizard and I just couldn't get them to compile.  I put the errors I was getting into Google and started the research process, I made incremental gains but it really was a struggle, a lot of new concepts and I didn't have the luxury of any type of formal training.  After a few hours of struggling I was starting to think; well, I'm not going to bill my client for my learning on this technology so do I just need to create some simple stored procs with some aggregate queries and get this project done with the old tried and true big frickin' hammer approach.  I thought just a little big longer, then all the pieces started to fall in to place, than all those pieces started to make a TON of sense.  This solution that MS came up with in their 2005 Analysis Server product release just really kicks ass!  Bottom line is that with the 2000 release of the product it was kind-of like building (or maybe should I say hacking) as solution in VB 6.0 or Access, with MSAS 2005 we have a formal development environment to build very tight well engineered cubes.

So looking back on this at the point I was thinking about abandoning this learning process I was probably 80% of the way there.  I'm very glad I kept with this, overcome the learning curve and have a new tool in my toolbox.

What did I take away from this exercise? First, even if something seems a little difficult and it may be easier to fall back on your existing tool set to solve the problem don't give up on the new technology too soon.  What you are doing is making an investment in your future, making your self more valuable and effective as a developer.  Bringing the right tool to the right problem is critical to solving technical problems today.  Yes, I could have probably built these reports using simple SQL, however with the extra effort and tenacity of not letting this new technology built me resulted in a solution that not only solved the problem at hand but with MSAS solved it quicker (even with the investment in the learning curve).  It also opened a new door of presenting these cubes to the end users so they could use MS Excel to run their own ad-hoc queries against the data and thus present a new set of options to the business to solve future problems. 

Second thing we can learn is when developing our new technologies and frameworks what can we do to reduce (although I don't believe every eliminate) the learning curve associated with other people adopting these technologies.  What can we do to help this?  I'm not sure that building pages upon pages of word documents or web pages is really the right answer here.  I did the perquisite reading and got some of the basic concepts with MSAS but I'm not sure that provided a major benefit.  What we can do is build very simple sample applications, not much more than the typical "Hello World" applications.  Don't get too fancy with these, focus on building small tutorials that are very complete for the topic or concept you want to present but don't go to far to confuse the "learner" with details that don't apply.  Create these tutorials that are somewhat interesting that represent real world problems a bite size chunk at a time.

If you are doing any real productive development chances are you have about 12 hours of work to do but only eight hours of time to get that work done.  What can you do?  Say this the right solutions is just to hard and fall back on your current skill set and spend the 12 hours, work until 10PM, get up the next morning at six and continue the cycle.  Or do you make the investment of your time that builds your tool box to a point where the next time a similar problem comes up, you can bring up the exact right solution and that 12 hour problem gets solved in 4 hours and your solution is exactly the right fit that may also be a building block to solve other business problems.  I'm glad I invested the time in this effort, the day or so I spent getting up to speed on MSAS is already paying back dividends.  Don't give up to soon, and don't always fall back on your tried and true technologies.  Saying all of that, this is where experience comes in, recognizing problems and identifying existing and "real" solutions is just something that good developers have a knack for.  Our customers (either internal or external) don't pay us to play with all the newest and latest technologies they pay us for solutions and before you invest your time you should have a good sense of what you expect from the technology.

Hang in there, there are a ton of new technologies out there, it's going to be a fun ride until us as software engineers have the same basic building blocks as more evolved engineering disciplines such as electrical or mechanical engineering.

Good luck and enjoy your journey!

-ec

Test Driven Development In the Real World

Well its been a couple years now and I've got enough experience doing TDD to feel that I can speak intelligent about what works (at least for me) using this development methodology.  I've seen some people say that they should never write a line of code before writing the tests, well that might be all-and-good for if your primary goal is to get an A on your test this semester, but in the real world, I'm not sure this is the best use of time.  Again, this is speaking from my past experience and the way I develop systems so your mileage may vary.

In my experience, as software developers we have to assume a certain level of stability in the building blocks we create.  Should we create test cases around our business objects that ensure when we execute a SQL Command against the database the SQL Connection object opens the connections and successfully executes the command? 

Let's look at the Red-Green refactor cycle.  At any one time when you are developing software you are attempting to solve a specific problem.  If you aren't sure of the problem you are trying to solve you might not be ready to start coding, it might be time to drop back to the white-board and draw some boxes/puffy clouds, or you may need to revisit your data model.  Either way just getting that little green light to come on within your screen may prove your current API is valid, but I'm guessing that, the API may change a little bit once you continue on in your efforts.

So if you made it this far, you have an open mind and we can continue our discussion...so let me say this, is there a place for TDD?  Abso-fricky-lutely I just did a little effort that was really built as more of a test concurrent type of development effort, but it's almost the same thing.  This was a very complex process where we had a disconnected database on a remote device that synchronized with a server.  We had to create some new "stuff' on the device that all needed to work locally before we could talk to the server.  No problem you say right?  Well where it gets interesting is the fact that the server uses integers to establish the relationships between the tables, that means if I create a new record on the device, I can't use an integer as the primary key, because that will only be available once we look at the identity column on the server database.  We need to build up all our referential integrity on the device database using Guido's then replicate those changes back to the server, create the "real records" send those "real records" back down to the device and make sure all our relationships are still happy.  For about the past year, I've been very happy using NUnit, however the client I'm building this for uses VSTFS so built the testing using this technology...really about the same thing, just slightly different syntax.  My new favorite little utility TestDriven.NET works like a charm with both.  So anyway, I was able to create some really nice scenarios that create the item on the device, add this to it, add that to it, change this, change that, then synchronize with the server and make sure the data I had on the device was the same as after I synchronize.

Another awesome experience I had with TDD was the development of a binary serialization algorithm for DataSets.  This I did as a pure TDD experience, I built my tests for the different data types I needed to serialize, then I created the methods and work on this until each of the unit tests passed.

My other experience wasn't so much with TDD but more of a formalized Unit Testing via NUnit.  I created a VB.NET ASP.NET 2.0 application, with about 150 unit tests, I'm not sure I gained a ton in development efficiency, however it did same my rear-end a couple times with my build cycle.  As part of the continuous integration environment that was in place for this project, we ran the unit tests, if they didn't succeed, we knew we shouldn't deploy.  Worked out slick and once the test were built it worked for "free" so that was kind of nice.

So what's my final opinion on TDD?   Well just as in anything moderation...and finding the right tool to solve the right problem.  I'm not convinced you need to write a test case to make sure the set and get properties work on your business objects, however if you have a complicated workflow, it is certainly worth the investment in time.

-ec