Feeling Good: The what’s and why’s of Agile BI
Just mentioning Agile BI feels good. Everybody wants to be agile and Business Intelligence is the holy grail of business management. Truth to be told, Agile BI is understood by some, practiced by few, and desired by many.
Agile BI is simply the skills, development practices, and supporting technologies that enable companies to accelerate the time required to deliver value from BI projects. While there is “religious debate” raging in the BI community between the proponents of the application of Agile and Waterfall software development methodologies to BI projects, we at CTI have a different focus: the customer, not the methodology, drives us. However, from a pragmatic perspective, the application of the Agile disciplines best addresses the discovery required to define evolving BI needs, as well as the need for speed and flexibility in BI projects.
Business Intelligence is no longer about traditional reporting and a dashboard for executives. Rather, it is increasingly providing the ‘intelligence” for real-time decisions by front-line employees, executives striving to quickly react to market conditions, and customers and suppliers who are deeply embedded in the business processes of companies who need BI to effectively deliver their goods and services. With these needs there has never been a better time for the adoption of Agile BI.
Most corporations have blindly adopted the traditional waterfall software development methodology of Design-Change-Integrate-Unit Test-Validate-and Deploy for its BI projects. However, this approach has numerous challenges that lead to excessive time to deliver BI projects that do not quite meet business requirements.
First, it takes too long identify and explain requirements. Second, it can takes month to access new data sources —especially from sources outside the control of the corporation— and change the data warehouse to add critical data. Third, it takes many iterations to get right data and reports. To compound problems, any changes break integration points and impact applications. Finally, directly accessing operational systems is not possible or ideal for that matter.
A smart application of proven Agile software development techniques have the following characteristics that deliver BI faster and with more flexibility to change:
- On-going scoping to meet time-boxed deliverables
- Rapid iterations that deliver working components
- Evolving requirements through discovery and application
- Use of scrum sessions that employ real-time decision-making processes based on actual events and information.
- Frequent and thorough testing
- Continual and close business and development team communication throughout the development process
Like most formal methodologies, the application of Agile BI techniques must be done pragmatically with the supporting tools. In our experience, the following are the key principles to successfully apply Agile BI:
- Continually set management expectations (Agile BI is not a silver bullet).
- Select the most applicable use case (involved management, small components delivered iteratively address important business need).
- Establish the right technical environment with tools that support Agile BI (key technologies and software tools to be examined and adopted Data Virtualization, Data Integration, Data Quality, and Data Profiling).
- Find the right balance (Current Sprint activities vs. Finish Line, composition of functional team members, scope and Time-what can be delivered in a tight cycle)
The next Agile BI blog entry will focus on implementation details illustrated by “how to” lists and a case study. Stay tuned…