Layout

Summary Results from
The OLAP Survey 3
Executive Summary . 4
Key Findings - General Conclusions .
Key Findings - Business Intelligence Products .5
Selected Results .6
Conclusion . 14
Executive Summary
The OLAP Surveys (http://www.survey.com/olap) are the leading independent survey of the online analytical pro- cessing (OLAP) or Business Intelligence (BI) market. As the industry's most thorough benchmark of BI products, The OLAP Survey 3 is the direct result of a comprehensive survey of real-world experiences of 1,047 organizations implementing BI. As the name implies, this is the third year in which this survey has been conducted, with the methodology and survey base improving each year. The Survey provides insight into customers' actual BI imple- mentation experiences as well as their experience using various BI products. This year, ten products were in use in a large enough number of sites to be analyzed in depth. Other BI product surveys and analyst product reviews exist but they are largely based on the perceptions of the reviewer - and do not have the sample breadth and statistical rigor of The OLAP Survey 3. The survey was pur- posely conducted across all geographies, with the following geographic split: North America 59.9%, Europe 34.0%, and rest of world 6.2%. This largely mirrors the geographic distribution of the BI market itself. The survey respondents were from organizations in over 20 different vertical categories with Healthcare, Insurance, Retail, Finance, Government, and Telecommunications having the greatest representation with approximately 45% of respondents. Respondents came from a variety of corporate sizes, with the median size corporation having It is important to note that this survey is 100% vendor independent. In contrast to other surveys, The OLAP Survey was not commissioned, suggested, sponsored or influenced by vendors in any way. No vendors were involved with the survey at any point in time, including the drafting of survey questions, editing of survey findings or analy- sis. Vendors were not even informed The OLAP Survey 3 was underway.
This fierce independence might seem almost like vendor-hostility but it is not. Vendors purchasing this study can be reassured that, unlike many other published surveys, it was not influenced or inspired by their competitors.
This strict vendor independence makes The OLAP Surveys a more reliable source for all readers.
Key Findings - General Conclusions
Overall, 48% of OLAP buyers said they had conducted a multi-product evaluation before buying. Organizations which had performed a formal multi-product evaluation were more likely to achieve their business goals and much more likely to realize business benefits than those who did not.
The Survey indicates that BI product selection impacts project success. Choice of vendor and product appears to have significant impact on whether business goals are attained and desired benefits are Organizations plan to increase their web-based BI deployments and the rate at which they deploy via an extranet in the coming year. Organizations which had deployed web-based BI to at least 30% of their users reported the highest rates of business success.
Microsoft Windows server is the most commonly used operating system platform for OLAP Servers at a 74.1% market share. UNIX-based servers had a 22.5% market share; Linux-based servers had 1% share.
'Soft' benefits, such as faster reporting and better business decisions are much more likely to be achieved than 'hard' bottom line benefits like increased revenues and reduced costs. The least likely benefit to be achieved was saving IS headcount. Business benefits were most likely to be achieved if projects went live within three months of product selection. More than 80% of the organizations surveyed would like to deploy OLAP solutions more widely.
Key Findings - Business Intelligence Products
The OLAP Survey 3 measured project success by the Satisfying Business Goals metric, or the extent to which orig-
inal business goals were met. Business benefit was measured, in aggregate, by the Combined Benefit Achieve-
ment metric. Each of the BI products was assessed for its ability to deliver eight discrete business benefits.
The survey found that products differ widely in their ability to deliver project success and provide business bene- fit. MicroStrategy-based projects were far more successful than the average of those conducted with other products as measured by Combined Benefit Achievement and were among the most likely to report success as
measured by Satisfying Business Goals.
MicroStrategy's product support was rated "Excellent - Accurate and Timely" by 27.5 % of MicroStrategy's cus- tomers, more than any other vendor. For the second consecutive year, MicroStrategy was the highest overall rated BI specialist and had a Product Support Quality composite rating 15 % higher than the average of the other
Data scalability is measured by the Average Data Volume metric. In the survey, respondents reported wide varia-
tions in the average amount of data supported by each BI product.
For the third consecutive year, MicroStrategy customers reported the largest Average Data Volume, at 313 GB.
The next closest ranked product had only one-fifteenth the Average Data Volume, at 20 GB.
Web user scalability is measured by the Web Deployment Rate metric and is defined by the percentage of Web-
deployed users within an organization. BI web deployment rates ranged from a high of 83% to a low of 6%, with a For the third consecutive year, MicroStrategy had the highest Web Deployment Rate at 83%; this was one-third
higher than the next closest BI product.
The degree of extranet deployment, or extension of a BI solution outside the organization to customer, suppliers and partners, is measured by the Extranet Deployment Rate metric. For the third consecutive year, MicroStrategy
had the highest Extranet Deployment Rate with 25.7% of all MicroStrategy customers having already deployed
OLAP data to external users, over 2 times the average rate of the other products.
The relative extent of a product's reach, or how widely a product is used within an organization, is measured by the Prevalence Rate metric.
With a Prevalence Rate of 64.8 %, MicroStrategy was the highest rated BI specialist product and was 33% more
prevalent in organizations than the average of all other products.
The degree to which a product is purchased at the enterprise level is measured by the Average Number of
Named User Seats metric.
With an Average Number of Named Users Seats of 1110, MicroStrategy is the highest ranked BI product, greater
than the average product by a factor of 2.
The products vary in their market share when used as an OLAP product front-end to the various relational databases. MicroStrategy has the highest market share (29.7%) when used as a front-end against Teradata and is in the top 3 (10.7%) market share against IBM databases. MicroStrategy was in the top 5 in accessing all other relational SELECTED RESULTS
Project Success and Business Benefit
The OLAP Survey 3 measured project success by the Satisfying Business Goals metric, or the extent to which original business goals are met. Business benefit was measured in aggregate by the Combined Benefit Achieve- ment metric. Each of the BI products was assessed for its ability to deliver eight discrete business benefits.
As Figure 1 illustrates, MicroStrategy customers are among the most likely to satisfy business goals and achieve busi- ness benefit. Furthermore, MicroStrategy ranked in the top 1 or 2 in each of the following business benefit criteria: Both Business Objects and Cognos customers reported below-average scores for Satisfying Business Goals and
Combined Benefit Achievement. MicroStrategy scored better than both Cognos and Business Objects in each of
the eight business benefit criteria. SAP BW's performance dropped in both absolute and relative terms, resulting in the gap between SAP BW and the other BI products growing even wider.
Source: Table compiled by MicroStrategy using data from The OLAP Survey 3. Key: Higher number means greater business benefit/success.
Figure 1: Business Benefit Scores by Product
"MicroStrategy continues to be a clear leader in helping organizations achieve business goals and in attain- ing business benefits as measured across a spectrum of important revenue-generating and cost-saving crite- ria," according to Nigel Pendse, author of The OLAP Survey 3. He also noted, "The fact that MicroStrategy customers have the largest and some of the more successful BI applications, as measured by the business benefit criteria we evaluated, confirms MicroStrategy's high functionality and suitability for highly scalable Web deployments - key drivers in delivering tangible business value." Below average ratings for Satisfying Business Goals and Combined Benefit Achievement by both Business
Objects and Cognos likely stems from their low web deployment rates and their dis-integrated product lines which increases end-user training and deployment times. Product Support Quality
The quality of a vendor's product support is becoming increasingly critical as BI becomes more integral to organi- zations' operations. Better product support often results in higher application success rates and helps to ensure customers get full value from their BI investments.
The OLAP Survey 3 found that major differences exist in the quality and timeliness of a vendor's product support.
As Figures 2A shows, MicroStrategy's product support was rated "Excellent" by 27.5% of MicroStrategy's cus- tomers, more than any other vendor. As Figure 2B shows, MicroStrategy was the highest-scoring BI specialist as ranked by the Product Support Quality composite metric.
In contrast, Business Objects had poor Product Support Quality ratings in 2003, showing a large decrease from
their 2002 rating. SAP BW also showed a large decline in Product Support Quality ratings; ranking last among all
vendors in 2003. This poor support is unfortunate for BW users, given the high number of technical problems SAP users reported they encountered with the product.
% of Respondents Who Received Excellent Product Support
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3.
Figure 2A: Product Support Quality by Product
"MicroStrategy was the top performing specialist BI vendor when it comes to quality and timeliness of prod- uct support, just as it was in 2002". According to Nigel Pendse, author of The OLAP Survey 3: "MicroStrategy had more top ratings for support and the fewest customers complaining of poor support. This must be at least in part due the stability and consistency of MicroStrategy's products, which were all developed by one com- pany, using a single architecture, rather than being assembled from multiple origins. MicroStrategy is also fully focused on this one product line, unlike the large, general-purpose vendors." Low overall Product Support Quality ratings by Business Objects likely stems from their strategy to acquire rather
than to build their ETL and reporting products. Product acquisitions present significant business and technical integration issues and often strain product support resources. To further illustrate this point, Cognos' planning product Adaytum (not shown in the chart above) now has the lowest Product Support Quality of any product.
Given the historic poor support of acquired technologies, it is likely the Business Objects acquisition of Crystal will create on-going support challenges for Crystal customers.
Product Support Quality
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Higher number means better Product Support ratings.
Figure 2B: Product Support Quality by Product
Data Volume
Increasing information intensity, and in particular, the desire to electronically capture and store every business transaction, has made the terabyte-size data warehouse commonplace. While analysis of summary data is often a launching point for understanding business trends, organizations need to view transaction-level detail to dis- cover anomalies, exceptions and trend drivers that pre-defined aggregations often obscure. Median Input Data Volumes (GB)
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Gigabytes of source data volumes.
Figure 3: Median input data volumes (Gigabytes) by Product
Despite the ever increasing need to analyze larger amounts of data, products differ considerably in their data vol- ume support, as can be seen in Figure 3. For the third consecutive year, MicroStrategy customers indicated they analyzed the largest amounts of data - a median of 312.5 Gigabytes (GB). This is larger, by a factor of 15 than Brio, the second-place product, over 23 times larger than Business Objects' and over 120 times larger than "As usual, MicroStrategy sites reported by far the largest data volumes of all, but the gap was wider than ever this year, said Nigel Pendse, author of The OLAP Survey 3. "Despite most vendors claiming to be able to handle large, detailed applications, it is clear that only MicroStrategy's customers are actually implementing such transaction-level BI databases. This puts it ahead in a whole class of applications." Cube-based OLAP products, like those employed by Business Objects, Cognos, and Hyperion, inherently limit the data that can be analyzed. They are optimized for analysis of summary level or small volumes of data but are not suitable for many of the more valuable, transaction-level BI applications. In contrast, MicroStrategy's Relational OLAP (ROLAP) approach doesn't require a- physical intermediate cube and provides good query response from large databases by generating enhanced SQL that for instance, is sparse aggregate-aware and can decompose logically complex queries into multiple, smaller 'bite-size' queries that can be executed far more efficiently.
Web Deployment Rate
The OLAP Survey 3 indicates that web-deployed BI applications deliver greater business value and benefits than those deployed via client/server technology. The web continues to be the most cost-effective delivery mecha- nism for deploying BI to the whole enterprise. Information reach is improved and setup and maintenance costs Median Web-Deployment Rates (%)
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Gigabytes of source data volumes.
Figure 4: Median Web-Deployment Rates (%) by product
The Survey also indicates that Web deployment success varies considerably by BI product. Major differences in the average web deployment rates among products exist, as can be seen in the cumulative Web-deployment rates plotted in Figure 4. For the third consecutive year, MicroStrategy had the highest median Web-deployment rate at 83%, meaning half of MicroStrategy customers reported deployment to at least 83% of their users. This is one- third higher than the next highest ranked vendor, Brio at 60%, and higher than the median of 20 % by a factor of 4. "Once again, MicroStrategy sites had by far the highest rates for Web-deployment - the sheer consistency of this finding proves that it is no fluke," said Nigel Pendse, author of The OLAP Survey 3. "Uniquely among major BI products, the modern MicroStrategy product line was designed from the ground up for large scale Web deployments, rather than being converted from older desktop or client/server architectures. This seems to pay off for its customers, who report on far more data, to more users, who are much more likely to be Web Surprising in the above chart is the low level of web deployment by Business Objects customers. It's likely that the Business Objects thick-client architecture and limited user scalability are limiting web adoption. Business Objects has announced plans to improve their web product in the future.
Extranet Deployment Rate
Extranets enable high return-on-investment (ROI) BI applications as they extend BI out to customers, suppliers and partners. Given the high returns extranets offer, The OLAP Survey 3 reports that over 60% of sites plan to have a BI extranet eventually. Extranets demand a higher level of BI architectural integrity since more stringent product security, greater ease-of-use, and higher user scalability are often prerequisites for extranet deploy- ments. Unfortunately, these elevated architectural requirements are not met by all vendors, which explains the wide variations in Extranet Deployment Rates found by BI product in The OLAP Survey.
Extranet Deployment Rates (%)
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Higher rates mean greater web extranet deployments. Rate indicates the percentage of customers where a specific product has already been used in the deployment of OLAP data to external users.
Figure 5: Extranet Deployment Rates (%) by Product
As Figure 5 shows, MicroStrategy has the highest Extranet Deployment Rate at 25.7% - twice the average for all
products. This means that over one-fourth of surveyed MicroStrategy customers have already deployed BI applica- tions to external users. In addition, MicroStrategy customers forecast the highest future extranet deployment rates, perhaps implying the MicroStrategy product line is well-suited to the particular demands of extranet applications. "Most vendors boast of their product's extranet usage, but as with Web usage generally, MicroStrategy sites were the most likely to have an extranet in production," reported Nigel Pendse, author of The OLAP Survey 3.
"Curiously, not only did MicroStrategy sites have the highest levels of extranet usage, but they were also the only group whose extranet usage grew faster than had been predicted by the previous year's sample. All the users of other products had lower growths than they had expected, which suggests that the MicroStrategy extranet deployments are more problem-free than those based on other products." MicroStrategy's investments in its security architecture, user scalability, and ease-of-use have likely helped its cus- tomers more successfully deploy BI extranets. Surprisingly, Business Objects, which has long claimed to have an extranet-ready architecture, fared worse than average in successful extranet deployments.
Prevalence Rate
The Survey shows that organizations buy multiple BI products, and that actual usage of these products vary within an organization. Some products are used sparingly by a few users, others remain deployed within small departmental pockets, while others are deployed widely across the enterprise. The Survey uses the metric, 'Prevalence Rate' as
an indicator of which products are actually used, as opposed to merely purchased. Prevalence rates are also a composite proxy for number of BI applications, number of users and overall BI product penetration in an organization. Product Prevalence (Usage) Rates (%)
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Higher rates mean more widespread usage of a specific BI product at an organization. Measures the amount or penetration of usage of a BI product at an organization.
Figure 6: Prevalence Rates (%) by Product
As shown in Figure 6, Prevalence Rates vary from a high of 64.8% for MicroStrategy to a low of 41.8% for
"The fact that MicroStrategy is one of the most prevalent BI products is an indication of its suitability for enterprise-wide BI deployments. Its wide-scale use is to be expected given its scalability and functionality." said Nigel Pendse, author of The OLAP Survey 3.
Number of Users
The Survey also polls sites to determine the average number of named user licenses (also known as 'seats') pur- chased of each product. The average number of user seats purchased is a strong indicator of whether products are purchased for enterprise-wide BI deployments or strictly departmental applications. That is, there is a correlation between the number and range of BI applications a product can support and the number of seats purchased.
In The OLAP Survey 3 poll, MicroStrategy customers purchased more user licenses than customers of any other BI vendor. MicroStrategy customers reported purchasing almost 2 1/2 times more user licenses on average than Business Objects and Brio, indicating their comfort in deploying MicroStrategy more widely within their organizations.
Average Number of Named User Seats
Source: Chart drawn by MicroStrategy using data from The OLAP Survey 3. Key: Average number of named user seats (excluding server licenses) Figure 7: Average Number of Users
"The fact that MicroStrategy customers continue to deploy to the most users suggests that MicroStrategy continues to be purchased on an enterprise scale for a range of BI applications." said Nigel Pendse, author Top BI Tools By Market Share
While all BI products can access - to varying degrees - the major relational databases through the use of indus- try standard SQL, products differ in their ability to do so in an optimized fashion. In addition, BI vendors differ in their formal technical and marketing partnerships with each of the relational database vendors. These close working relationships are critical to ensuring a BI tool remains optimized with, and fully leverages, the relational database engines as they evolve over time. Figure 8 shows MicroStrategy continues to be the most widely used tool for Teradata databases and is one of the top 3 front end products for IBM databases. Source: Table compiled by MicroStrategy using data from The OLAP Survey 3. Key: User ratings by Figure 8: Top BI Tools by Database Market Share
"MicroStrategy's is one of the most widely used BI products for both Teradata and IBM databases due to its unique SQL optimizations for large-scale databases," said Nigel Pendse, author of The OLAP Survey 3.
"MicroStrategy should be regarded as a very serious candidate for anyone looking to build a BI solution on Teradata or IBM relational database platforms." Conclusion
The OLAP Survey 3, as the leading independent survey of real-world BI implementations, provides unique, statisti- cally significant insight into actual BI implementations and customers' experience with various BI products. The results of the Survey provide an important guide map to the product capabilities and support you can expect from the various product vendors. As BI product selection has a significant impact on BI project success, it is recom- mended that meaningful product evaluations, starting with a close review of the product benchmarks in The OLAP Survey 3, should be conducted when embarking on new BI projects. Notable in The OLAP Survey 3 results are the widely varying customer experiences and product results among the BI products. Categories in which clear vendor trends have emerged over the past several years include the following: Most Likely to Satisfy Business Goals and Provide Benefit Number of Seats Purchased and shelfware rates In several of the above categories, MicroStrategy has consistently been a leader, sometimes by a wide margin, in the three years The OLAP Survey has been conducted. Furthermore, in categories such as Average Data Volume, Web Deployment, and Extranet Deployment, the gap between MicroStrategy's leadership position and the posi- tions of the other vendors continues to grow at an increasing rate. For more information about The OLAP Survey 3, please see:

Source: http://www.microstrategy.com.br/company/Downloads/OLAP3%202004.pdf