A major focus at the TDWI Conference in August 2010 was Agile BI, and a core idea is that using Agile BI development methods results in BI releases in “weeks instead of months” – with six weeks being a commonly cited cycle. In a previous blog post – Agile BI: What Problem are you Trying to Solve? – we shared our view that Agile BI does not address the common risks and failure points associated with large-scale BI initiatives. Essentially, Agile BI is not intended as a large-scale, program approach to business intelligence and analytics. That being said, at the BI project level, agile BI may well be very useful as long as certain pre-conditions are met. This post takes a look at when and how agile BI can be used to achieve rapid delivery of business intelligence and business analytics.
A Business Intelligence Strategy and Data Architecture Sets the Stage for Agile BI
The CIO and the BI Director are charged with key tasks such as:
- explaining the business benefits of business intelligence to the business units;
- defining and enforcing a data architecture to enable rapid, low-cost development of BI and business analytics and high data reuse for new BI applications; and
- building and articulating a business case for BI based on a sound business intelligence strategy and a pragmatic BI program plan.
Our experience across industries, and the experience of judging TDWI Best Practices winners, shows that BI can be delivered rapidly, which is the central premise of agile as a development approach. To accomplish these tasks, it is essential to use a robust BI lifecycle methodology, such as those taught at TDWI Conferences. Essentially, a solid BI methodology sets the stage for BI development by:
- establishing an over-arching BI strategy and prioritized BI project portfolio;
- establishing a business-driven BI data architecture;
- establishing any needed plans for the technical infrastructure, including a data migration strategy for any existing data warehouses and/or operational data stores, and a platform migration strategy if need be; and
- establishing a BI program charter, program management plan, and change management plans.
With a solid foundation for business intelligence success in place, companies can then turn to developing a release strategy to guide the BI development team. This is where agile BI may come into play.
Proven BI Methods are supposed to enable Agile BI
Many of the proven BI lifecycle methodologies advocate iterative, incremental delivery of BI applications via 90-day projects that build out fully functional slices of the data architecture. Best practices require that each release must put a BI or business analytics capability into the business users’ hands, such that the new capability can be used as designed to create incremental revenues or reduce existing costs. In this context, agile BI proposes to deliver BI in slightly less time – 45 days or so versus 90 days.
How Should Agile BI be Used?
In our view, delivering micro-slices of BI to achieve “delivery in days or weeks rather than months” probably boils down to a BI release scoping decision. For example, are two smaller monthly releases more useful to business users than one larger 90-day release? The answer to that question depends on the specific business context. The business requirements for business intelligence are expressed as business questions, and they are grouped into analytical subject areas. For example, food manufacturers need to manage the shelf life of their finished goods inventory in order to avoid the costs of inventory obsolescence. This means that they need to be able to answer questions such as these:
- how many cases of product 123 have less than 60 days of shelf life remaining?
- which customers buy product 123?
- which of those customers have product 123 on their shelves as of today’s date?
- which of those customers have product 123 in their warehouses as of today’s date?
- what is the average velocity of product 123 for those customers?
- for which of those customers is it likely that their inventory of product 123 will lapse into obsolescence?
- what is the spot market price today for product 123?
Business intelligence best practices suggest that a company would need to be able to answer most/all of the above questions in order to effectively reduce the cost of obsolete inventory, Accordingly, an agile BI approach that answers some of the questions in 45 days would not leave the operations function in a position to create business value. The unit would need to wait until the remainder of the business questions were answerable, e.g. it would have to wait for the agile BI approach to deliver one or two subsequent releases. Viewed in this light, traditional incremental BI development would be a sound BI strategy. On the other hand, if analytical subject area is such that answering a couple business question in, say, 6 weeks or less is useful, then an agile BI approach would make sense.
Key Takeaways for Using Agile BI
Our perspective is that proven BI methods enable rapid delivery of useful BI applications while also addressing the root causes of BI failures. So by all means let’s be agile, but let’s also be fundamentally sound. Toward that end, we recommend that companies:
- develop a BI strategy, roadmap, and program plan to set the stage for BI development
- understand that at the project level, agile BI and iterative, incremental BI development methods are identical in their intent to rapidly deliver useful BI and analytics
- define and enforce a data architecture that allows for use of agile BI when it makes sense – without sacrificing sound data architecture principles
- focus on delivering complete analytical capabilities as rapidly as possible.
We always welcome your thoughts on agile or any other of the topics we discuss here.