BLOBTo BLOB or not to Blob – Part One

So it’s time to consider another face of the new 2010 SharePoint product as well as the conceptual content management issues of “Storage” the evil face of every IT department. Where do we put the stuff one Southern SQL DBA once told me when we were talking about content LOL. So where to start this little journey? Traditionally the concept of file storage has been shifting from the base roll of file server in the early 1990’s to the content systems file storage systems like SharePoint. Microsoft has pronounced the death and doom of file servers over the new base technology of SharePoint or the writing has been on the wall for quite some time that it would be dyeing . Sun setting file server technology has long been the call to SharePoint’s content battle cry so it’s perfectly normal that it coming to an end with the 2010 product line.  But recently I began examining the issue of storage with a client who was upgrading a number of MOSS 2007 sites into a single new 2010 system. And yes you guessed it…. Storage became the central issues and only logically I might add as I look back now over 10 years of SharePoint and the growing data stores of information from the previous versions that must now be accumulating in the hallowed server rooms and data centers across the world. Yep in our effort to take over the content management space it seems like we the designers and architect have failed to look at the cumulative effect of storing data in SharePoint and so the question I consider today is whether it is nobler in the eyes of men to Externalize BLOB’s ………… or TO BLOB or NOT TO BLOB in an effort of control cost and capacity of memory in a SharePoint farm.

Fortunately Microsoft has been thinking the same thing for a few years and the issue becomes one of basically understanding the path and method they have chosen to direct the management large data farms and the effect of such farms storage requirements on a SharePoint resource. So I took the afternoon to revisit the newly released Remote Blob Storage in SharePoint 2010 articles and video from the 2010 conference in Vegas last year. And there it was. Overlooked by all the other presentation about features and capabilities, set up and structure …………. THE BLOB Store.

At the core the BLOB storage issue is the cuprite of SharePoint evil…. The Externalization of BLOB’s. It turns out to be about 70% of the actual storage size of document and information stored from content so it then is 70% of the real storage issue In SharePoint.

So in a natural server farm that is made up of say 2 terabytes of content data 1.7 terabytes is the surface space of large blobs of data, thus the term evil BLOB the fat of content. The remaining 30% makes up the really usable Meta data that is joined and manipulated in list and search indexes while the BLOB holds the real content of the artifact once the user has found or wishes to enable information. So the theory is that this meta data is the core or working elements of SharePoint information and the BLOB’s just hang around in large vats of content associated with the meta data and although valuable, are more or less data overhead. So what to do with overhead the story goes. After all overhead is not a good thing in any real system either human or machine.

TO BLOB then becomes the next evolution of managing information in SharePoint data and there are really a number of creative ways of looking at BLOB data. It can be considered a newly evolved requirement of the process that has moved us here. Much like the pack rats of the world (and most companies with no enforced document polices are really like this) we hold and hold our data in repositories and back-up systems for the eternity to come. A lot of the holding is simply the lack of policy management or administrative support to management the life cycle of the documents. A lot of the issues is just corporate laziness.  So for the next few weeks I wanted to touch on some of the basic issues around BLOB and the new technology of BLOB management and SharePoint 2010 and look at creative ways to approach BLOB storage and content in a migration event as I work with my customer to find a solid long-term solution to the BLOB’s in SharePoint 2010.

Stay tune!