Microsoft Cutting Edge - Too Much Of A Good Thing?

I confess that lately I'm feeling a bit overwhelmed.  Microsoft is churning out an enormous amount of new web-application development stuff these days.  And they're doing so at a pace that makes it hard to keep up with the announcements; forget about actually understanding it all.

For ASP.NET developers, we now have a number of new technologies to consider when working on a project :

While we're mulling over the ASP.NET specific stuff from above, we also need to consider the following new approaches for creating our data tier (and possibly maintaining it - I'm not yet convinced that ongoing DAL maintenance is being addressed):

And don't forget Microsoft's Silverlight project, in which we're all really interested, but not entirely sure how it will fit into our whole web application development picture.  It seems like Silverlight is being positioned as a Flash/Flex competitor for the rich web application development space.  Certainly the idea of using the same old C# or VB we know and love instead of mastering Flex sounds good.

Much of this stuff strikes me as really cool, compelling, useful and productivity-enhancing.  Yeah, I know, I'm sounding like a shill for Microsoft right about now.  But to my credit, I did say "seems" and not "is".   

Up until October of this year (2007), we had an equally cool and compelling third-party option for some of this stuff in Rob Conery's awesome SubSonic product (and it's free, no less).  You still do have SubSonic, for now at least.  But in October, Microsoft assimilated Rob (all your SubSonic are belong to Microsoft - resistance is futile, don'tcha know).  Personally, I'm kind of happy about this development because I think it shows that Microsoft is showing good judgement in assilimateees (did I spell that correctly?).  And Rob appears to have retained the right to develop SubSonic as he sees fit, free of overt influence from the Bor...uh, Microsoft.  Rob blogs some about choosing a data access layer here: Rob on choosing a data-access method.   

So I've been on the "I'll sleep when I'm dead" track, trying to stay abreast of all these latest developments in development.  Much of this stuff is built upon on new language features in the latest .NET framework, some of which are not entirely trivial to immediately grasp at an elemental level (LINQ expression trees, for instance).

With all this new and cool stuff streaming out of Redmond, one would think that developers are in a state of bleeding-edge bliss.  And to some degree, I think they are.  But I also think that developers are in a state of frustration and confusion over the vast number of choices of tools and techniques to use on new projects.  I know I am.

The thought process goes something like this:  "I'm starting a new development project that will take man-years to complete.  I want to avoid the mistakes of the past while simultaneously taking advantage of the progress that's been made in development tools and techniques thereby improving my (and my development group's) productivity."  Sound familiar?

Back in the day, if you were writing an application for Microsoft Windows, such decisions were considerably easier.  Mostly, it was a choice between writing directly against the Win32 API, or using MFC (Microsoft Foundation Classes), or using VB.  If you were rooted in the Borland camp, the choice was even easier: you just used OWL (Object Windows Library) or, later on, Delphi.

There were some other, moderately viable, choices that came along in the meantime, mostly centered around the 4GL (fourth generation language) movement (or hysteria, as I like to think of it).  Companies were coming out with products that were supposed to remove the grunt-work of coding an infrastructure freeing us to just build features for our users.  And it would eventually bring programming to the masses.  Uh... yeah, right.  Things didn't quite work out the way they predicted.

I mention the 4GL phase because I think it's instructive to consider why it failed.  I think that at least part of the reason 4GL's failed to take the world by storm has to do with the typical mind-set of seemingly most developers.  I believe, although I may just be projecting, that people who did serious application development for a living were put off by the lack of control inherent in using the so-called 4GL products.  And in the world of 4GL development, developers, removed from all worries of coding details, were implicitly promoted to application designers and/or architects.  Yet, as we know, not all developers make good, happy, designers or architects.

A lot, but certainly not all, of the new stuff coming out of Microsoft is aimed at increasing our productivity through a combination of designer-based and database-schema-based code generation.  It has a slightly familiar feel to me; not entirely unlike the 4GL craze, albeit with some important differences to keep developers happy.  Developers use wizards and designers and drag-and-drop operations to create nifty applications while writing very little code.  Well, very little code by hand, to be more precise.  There is a very high coolness factor to this approach.  It's really kind of fun and empowering and fast to be able to create a web application through the designer that way.  And the generated code is nicely written - as well as or better than most people would do by hand, I suspect.

The problem, as I see it, is that so much code is being generated.  Tons of it.  Throw together a database of 15 tables with relations and constraints, mix in 30 views (web pages) for viewing, editing and reporting on that data.   Use, for instance, ListViews to display and manage the data and LINQ-to-SQL to render the data access layer.  When you're done, you wind up with more generated code than you can shake a stick at.  Who's going to maintain that code?  How easy will it be to update all that code to take advantage of the latest, greatest tools and techniques?  Will we have to regenerate the code when our database model changes? 

At least at the moment, it seems more accurate to say that the stuff coming out of Microsoft is aimed at increasing our startup or initial productivity.  It doesn't, however, seem to do much to help us maintain or extend our application. 

This only serves to increase my confusion and frustration over which tools and techniques to use on my new projects.  I want to get these projects done quickly (well I don't want that so much; it's usually others that do).  But I also want to ensure I don't create some huge, unmaintainable library of code files. 

So now I find myself wending my way through MSDN documentation and various blogs, diving down into the details of the recently released stuff, trying to find how I can leverage it all in a non-designer, non-code-generated fashion.  I've got some ideas.  And I'm ready to start coding them up.  But yet I'm frozen.  I'm wondering if, any day now, Scott Guthrie's going to announce that MS has made available a CTP release of a product that sits on top of all this LINQ-AJAX-ENTITY-MVC stuff, that no longer generates code to physical files, but instead generates it dynamically, if at all.  Sort of like SubSonic's build provider approach, except that it doesn't require a build step.  Instead, the DAL is refreshed, programmatically or automatically, after deployment (which allows for client-specific modifications "in the field").

I've run low on iced-tea (caffeine) so I'll cut short my stream-of-consciousness rant, make a refrigerator-round-trip to reload my cup and get back to work.  If, by some freak combination of flawed web search, combined with an extra-normal display of patience, you find yourself having actually read all this, I would love to hear what you think.