BI Best Practices Benchmark Report, Part 5: Performance Metrics

Date: 11-08-2011
Author: Bit Software

One of the main goals of Business Intelligence is to organize and distribute metrics on the performance of the business. But one of the trickiest things to measure is the performance of the BI platform itself. A few statistics about BI performance are relatively easy to obtain and quantify, such as usage data drawn from the logs of specific tools or portals. One trend that jumped out from Gleanster survey is that almost all companies seem to be more focused on metrics tied to business outcomes and not at usage metrics, which are easy to measure but not as valuable to the business.

Below is a list of „performance metrics”, according to the participants responses (in descending order of importance):

1. Time-to-decision / time-to-response for information requests. The majority see BI as a business accelerator, a tool for driving faster, better informed decision making. But this may be a somewhat subjective measure for high-level strategic decisions.

2. Information comprehensiveness, accuracy and consistency. Audit for desired quality measures, analyzing extracts from the data warehouse to identify missing or incomplete data. By scoring for information quality, set goals for improving it and make specific plans to correct the inaccuracies.

3. Volume / quality of actionable insights. Track the number of decisions that can be traced to data or patterns exposed by BI tools. Work toward continuous improvement of both quality and quantity.

4. Financials (revenue, operating margins, etc.) Look for evidence that the BI strategy is making the company more valuable. If the evidence is not there, reconsider both the data and the models used by the BI platform. Finally, track if decision makers are, in fact, using the BI, cause a system that is not being used will never achieve its intended results.

5. Number of active users. How many users have accounts on the BI platform, compared with the penetration you would ideally like to see across the organization? How frequently do they access the system? Those are the questions raised by survey participants.

6. System usage (number of queries, dashboard views). Almost three out of four Top Performers cited number of users and technical measures of system usage as important metrics, a higher percentage of those in the Everyone Else category focused on these things over speed of decision making or quality of insights. According to survey participants, is important to find the answers to the following questions: How many times per day are BI tools employed, overall and by targeted classes of business users? Is that number rising? What parts of the BI platform are getting the most use?

7. Employee performance / productivity. Employees ought to be more productive when they do not have to waste time compiling reports or searching for information that is now readily available through the BI platform.

8. Employee satisfaction. By making measures of individual and departmental performance clearly understood and readily available through dashboards and reports, the organization provides employees with more control and eliminates uncertainly.