What is Business Intelligence?

Howard Dresner of the Gartner Group popularized the following definition for Business Intelligence*:

An umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems."

Business Intelligence, as a practice, has really come to the forefront in the last fifteen years - primarily as a way for organizations to make sense of the mountains of data that have become available to them due to the proliferation of very inexpensive storage and the ability to collect rather large amounts of data via multiple input sites.

What Makes Up a Business Intelligence Solution?

Business Intelligence solutions grew out of a combination of increased globalization, competition, and pervasiveness of information systems.

The original "Business Intelligence" systems were little more than copies of transactional systems databases - these allowed reports, developed by system programmers, to be run against the data without impacting the performance of production systems.

However, any report changes or new reports required intervention by the IT group, since the data models used by transactional systems were highly complex and often somewhat cryptic, with space saving techniques often used to reduce the data storage (remember the Y2K crisis?)...and then, on top of that, it was pretty much impossible to combine information from separate systems.

The need for current and complete data to facilitate business decisions, the proliferation of systems and data, and a desire to drive the data to the business user has resulted in the explosion of Business Intelligence over the past decade.

So, what does it take to build one of these beasts? A traditional approach to Business Intelligence is usually comprised of three major components:

Data Storage

Organizations typically store the extracted and transformed data in a separate database distinct and separate from the production systems - this insulates the production systems from any "run away" queries that could be generated by the users, and it allows the organizations to control updates to the data - since there is often a desire to ensure that everyone reports from the same time series!

Extract-Transform-Load (ETL)

Extraction-Transformation-Loading is the process and tools whereby data is extracted from source or transactional systems. Typically, data is extracted and changed or transformed prior to loading into the reporting database (more commonly referred to as a Data Warehouse or Data Mart). The transformation is undertaken to accomplish three things:

• Provide data in a consistent format. For example, the same data may be represented differently depending upon the source system. Gender may be represented as Male/Female in one system, M/F in another, and even 1/2 in a third system! The ETL process supports the transformation of data to a consistent format.
• Transform data to a format easily understood by the business. Transactional systems are designed to process information as quickly as possible. As a result, the Relational Data Models employed are often very complex and difficult to understand even for seasoned programmers. A Dimensional Data Model or Star Schema model is typically employed in order to represent data in a way that is easily understood by the business and that expedites database access.
• Data may be enriched or corrected when transformed. For example, integration of multiple customer records to a single customer record may be undertaken, and rules may be applied to ensure that the data is correct. Customer records may be segregated by region or area based on the zip or postal code. This correction step more and more is being eschewed in favour of correcting data at the source level, as the corrective activities themselves can build in incorrect assumptions regarding the data.

Reporting / Analysis ("BI Tools")

There are a plethora of tools available for the surfacing and analysis of data. Most organizations will utilize multiple tools as there isn't a "one size fits all" BI tool. I often segregate their users into categories:

Consumers typically use delivered reports an/or dashboards (summarized information presented on a web page). If further analysis is required, they will have another member of the organization undertake this work. These are often senior executives within an organization who don't have the cycles to do the analysis and, more importantly, have individuals who can undertake the analysis.

Analysts will often utilize tools that allow a structured review of the data, such as configurable reports and queries or the use of data cubes, often referred to as OLAP or Online Analytical Processing, to slice and dice the data and to drill-through to details.

Explorers undertake detailed analysis of data. They will often use ad-hoc reporting tools or may program their own queries in SQL. Additionally, some users may utilize Data Mining tools to undertake advanced statistical analysis of data to drive out patterns and trends.

The Evolution of BI Solutions

Originally, specialized offerings were developed by organizations, a la the Best of Breed approach, to provide point specific functionality. Over time, larger commercial software vendors have acquired these offerings in order to provide fully integrated across the board "solutions". For example, SAP's acquisition of Business Objects, IBM's acquisition of Cognos, and Oracles acquisition of BRIO and Hyperion. And each of these organizations swallowed up other best of breed solutions prior to being acquired.

This consolidation has meant that organizations looking to implement BI solutions are faced with the choice of purchasing a very expensive consolidated solution from a large vendor rather than acquiring best of breed tools. Open Source Solutions offer organizations the opportunity to implement best of breed BI Solutions that offer everything the commercial products have in a cost effective manner and a-la-cart.

*H.P. Luhn first defined Business Intelligence in the IBM Journal in October 1958 - "A Business Intelligence System".