“For too long we as an industry have been ignoring the fact that we take too long to build things, and this is because we assume that we have to get them right — but we don’t ever get them right! That’s why you see people still doing satellite data marts, still doing their own Excel spreadsheets, still doing [operational data stores]: it’s because IT can’t get it right the first time.”
Agile BI Proponent at TDWI World Conference, San Diego 2010
There is a lot of buzz in BI circles these days about “agile BI.” And it’s no secret that there have been many failed data warehousing and BI projects over the years. That being said, we are not as pessimistic as the agile proponent cited above. In fact, the successes of dozens of winners (and hundreds of contestants) in the annual TDWI Best Practices in Business Intelligence and Data Warehousing competition attest to IT’s ability to get BI right.
Why BI Initiatives Fail
Based on BI/DW industry experience and our own work with major companies in a wide range of industries, our perspective is that the root causes of many failed BI/DW initiatives include:
- failure to effectively align the BI/DW initiative with business strategies and core business processes and manage BI as a program;
- failure to define and enforce an overarching data architecture that promotes information reuse, scalability, cost-effective data management, and rapid iterative BI development;
- failure to leverage proven business process improvement and change management methods to drive the use of BI applications into core business processes that impact business results;
- failing to adapt IT policies and shared services management practices to the needs of best practices BI development;
- spending millions of dollars to bring a bunch of data into a “warehouse” in hopes that business users will figure out – after the fact – how to use that data to improve business results; and
- failure to use one of the proven best practices lifecycle methodologies taught at TDWI conferences and other venues.
Viewed in relation to the above failure points, the agile BI approaches advocated by agile proponents do little to address some of key problem areas associated with BI/DW failures. While the high degree of business engagement associated with agile BI certainly is a positive step, if we’re not careful, agile BI could amount to doing the wrong things – and still failing – more rapidly.
Building a Solid BI Foundation
To avoid this risk and address the common causes of BI/DW failures, here are five things you can do to ensure your BI/DW initiative is built on a solid foundation. Collectively, these things can be done in 8-12 weeks, and sometimes less time, to set the stage for successful BI/DW agile development efforts:
- Develop a pragmatic, comprehensive BI Strategy and Roadmap aligned with both short-term and long-term business goals and strategies to ensure both a balanced business focus and a more holistic perspective on where BI opportunities exist to achieve a rapid return on investment
- Define a data architecture, leveraging a proven architecture approach that can be used during agile development to facilitate sound data management practices
- Investigate the limitations of current data sources in relation to BI information requirements, understanding where the quick-win agile projects do and do not exist
- Use a proven BI business requirements approach to ensure that there is explicit traceability between agile projects and business information requirements that align with and explicitly support key business goals and strategies
- Determine roles and responsibilities for making the necessary business process changes needed to ensure that that new BI capabilities are incorporated into the business and the potential business value of new BI capabilities is captured
- Leverage proven agile development approaches to achieve rapid high-impact BI capabilities
These proven practices address the root causes of BI/DW failures, which are the problems we need to solve to be successful in the BI world. We’ll explore these problems and approaches in more detail in future posts.