The Data Highway

   Too many data warehousing projects are failing in corporate America today.  Given the wealth of experience in the 1990’s and 2000’s, we are overlooking some fundamentals. 

  Many factors play into why business intelligence initiatives fail in today’s corporate environment.  In the 1990’s, data warehousing methodology was still in it’s infancy.  The objectives were different.  Data integration and availability were the prime goals.  Selling the project was a matter of convincing management that data integration provides a more complete reporting picture for management.  ROI was measured on how much visibility KPIs and global statistics brought to the executives making decisions.  Project teams were allowed more time to develop reporting products.  Some projects that succeeded evolved into areas such as EAI.   

   In the 2000’s, corporate America became more sophisticated in the business intelligence arena.  Executives understood and often have seen the benefits of the data integration age.  KPIs are well developed.  Middle management began to depend on operational metrics measuring performance, profitability and the cost of managing their business areas.  

Which brings us to the top five reasons data warehouse initiatives fail:

1. Biting off too big a bite.  Projects that scope work in the big bang approach are doomed for failure.  Project teams usually find success by scoping the project into many smaller increments, giving three to five months to deliver.
2. Competing projects.  Warehouse projects are expensive to begin and require a unified effort to provide true integration.  Multiple projects with data integration as the goal will guarantee your project (if it is allowed to continue) will be just another data silo consuming corporate resources.
3. Lack of corporate vision.  Most companies are very focused on how to deliver products and services.  Global reporting requires the same diligence in developing an data integration strategy.  Key performance indicators (KPIs) used to drive business decision requires coordination across the business.  A clear, concise vision of what the corporation will look like will help drive clear goals and objectives.
4. Dirty data.  Data integration brought out the ugly side of what’s been going on at the lowest levels.  Reports that are produce incomplete or inaccurate results will hurt the initiative.  The need for data governance and business stewardship is the answer.
5. Insufficient technical design.  Reporting initiatives are more often victims of cost-cutting, econo-designs.  Look carefully at server capacity and dedicate services whenever possible.  Separate the Stage environment from the ODS and Fact area.  This will expand bandwidth and allow modularization of key processes.  Allow plenty of resources for persistent areas, even Staging.  Separate reporting and ETL load functions into different application server environments.  Look at this also for the database schemas as well.  The extra cost and planning will allow for faster processing and happier customers.

   More is expected from IT today.  And with less.  It is critical to get some wins for the business and executives early in the initiative.  Everyone understands the promise of data warehousing, it is our job to execute and capitalize on that promise.