How IT leaders are modernizing their business intelligence systems

Here's what a state-of-the-art business intelligence (BI) system looks like: A customer service rep takes a call from a customer looking for an explanation of a medical bill. The rep taps some keys and data quickly flows into her screen. The customer gets a thorough answer and hangs up, satisfied.

That's one example of how a multi-year project to install a best-of-breed business intelligence system has unfolded at Montana State Fund, the largest provider of workers' compensation coverage in the state. "In under a minute, [the CSR] put together some parameters and pulled up a quick screen-generated report," said Al Parisian, CIO of the Helena-based company. "She had a comprehensive, authoritative answer that was compelling because it was factual."

The latest BI systems -- flexible, robust, user friendly -- are moving to tie together more data stores of more data types than ever before, even from transactional systems. They are letting business users easily query them in the course of their jobs. They are proving their value day in and day out with every business decision made, every customer satisfied.

And at many organizations, they are also far from reality.

Unlike Parisian, who basically got to start from scratch in building his BI system, many IT executives grapple with proliferating, complex BI environments. With a mix of reporting tools, data silos and technology requiring IT-built queries, these business intelligence systems can pose daunting challenges from an architectural and organizational standpoint when it comes to adding next-generation functionality.

Consider the situation at digital video recorder company TiVo Inc., where IT wants to break down information silos but isn't sure yet how to implement a cost-effective BI platform. Or Grafton School Inc., a health care nonprofit, where analysts manipulate data in Excel and await an upcoming electronic health system purchase to boost their firepower.

Their road to nirvana -- stores of clean, rich, integrated data accessed easily by those who need it -- is a journey that's barely begun.

Get the data right

Three years ago, that was also the case at Montana State Fund, where 280 employees manage insurance for some 27,000 policyholders. "We went from having literally just a few people using an old operational data store system (for which the reporting front end had broken) to having 20% of company staff using the BI system today," Parisian said.

Today's best-of-breed system, which includes components from Oracle Corp. and SAP BusinessObjects, has been in the works for 2½ years. Starting the project with a consulting firm -- Millbrook Inc., which did the company's data modeling -- was the most important decision Montana State Fund made, said Parisian, who is a member of Millbrook's Business Intelligence Executive Customer Council.

"The quality of your decisions is directly correlated to the data that goes in to them," Parisian said. "Any company around more than a couple of years has legacy data problems coming out the kazoo. You're going to have all kinds of junk."

Once it had installed a coordinated set of products as a brand-new BI suite, the team ran the bulk of its data through extract, transform and load. Now, elements of that data show up as a series of data marts that make up the BI platform. Parisian said 1,800 different elements make up the system's data, and Montana State Fund is currently adding 600 more -- for example, medical payment details for workers. "Upon completing this we think we'll have a rigorous model," he said. "It's being used every day."

Types of data vary widely in any company, as do the needs of employees using that data. But a change in data types is a key trend in business intelligence now, according to Franz Aman, vice president of intelligence platform product marketing at SAP BusinessObjects.

Aman pointed to the oft-cited statistic that 80% of an organization's information is unstructured data -- the emails, Web pages and customer phone discussions that are the lifeblood of many interactions yet not captured by BI. SAP BusinessObjects is investing in this area with what Aman calls "sentiment extraction" -- technology that mines unstructured data to enable the customer service department to gauge the mood of customers who call, for example, and from there report on their satisfaction levels.

Dyke Hensen likes to cite what he calls "high-definitional data" as driving change for business intelligence systems. Hensen, chief marketing officer at PivotLink Corp., a BI Software as a Service vendor, noted that a shoe store isn't selling just a pair of shoes, but a pair of size nine-and-a-half, blue, lace-up shoes, sold with a certain promotional code. "It's a description of an element," Hensen explained. "It's high-attributional, high-dimensional data. These problems are challenging, and they don't fit in a spreadsheet or a SQL server database. People want to report on that information."

Richard Rothschild, senior director of IT at Alviso, Calif.-based TiVo, hears that call. He said he'd like to eliminate siloed reporting and get more specific answers from customer analytics information to improve the bottom line. "I'd love to be able to see what effect a marketing plan has on revenue and retaining customers," Rothschild said, "and here's the data that says it's working."

But the hurdles, he said, are "how to get disparate pieces of data together in a more cohesive space, and technically how do you solve that and come up with a project that will get approved and implemented." Finding the right way to streamline data is a challenge under a budget, too. Rothschild said finding a project that isn't super expensive or doesn't use a lot of resources is key for TiVo, which has about 550 employees.

CIO Bill Davis at Winchester, Va.-based Grafton, which provides health care to developmentally disabled children and adults, has seen the demand for data growing, too; Grafton's analysts would like to access data about client behaviors, symptoms and goals, for instance. An electronic medical/health record system now on the boards will offer built-in reporting and BI capabilities that will take his team's capabilities beyond the Excel files now in use. "We'll have a lot more information than what we have now," Davis said. "The ideal in two to three years would be that we'd have data mapped and loaded, with a dashboard that any analyst or savvy manager could use."

But for many organizations, more systems and more data aren't what's needed.

Reduce complexity

For many, the first step to a new generation of BI is to reduce complexity. Often, that means getting all users on the same platform.
How? Parisian turned off the built-in reporting functions in the core applications at Montana State Fund. "There was a lot of screaming and howling," he acknowledged.

"But we are going to see more and more advantages to our company, as long as we show discipline not to flip on those other screens," he added. Now, all reporting gets done at the BI layer, so everyone uses the same data and does his own reporting vs. seeking custom reports. "It's reduced the amount of noise and fighting over limited resources," Parisian said. A bonus: When the company needs to replace applications, "we don't have to shop for something that has management reports as part of its infrastructure."

Grafton, with 650 employees, has reduced complexity in-house with a tried-and-true BI tool: Microsoft Excel. "We have job streams that run every night and grab data from three different systems to make two big files," Davis said. "Our managers and analysts download them and then parse them through Excel in various ways." It works well, he said, because the IT team doesn't have to generate custom reports.

Even when the electronic health system is up and live, Excel will still have its place. "We'll end up using that as the end engine, but we need something else to get it there," he said. "Say what you want to, Excel can do an awful lot for the majority of our users."

Consider cost

Indeed, Excel is perennially popular for many reasons -- cost among them. New, comprehensive BI systems can be quite expensive and therefore not an option for many businesses, even when BI is a top business priority. So if system replacement (Montana State Fund) or application-based analytics (Grafton) isn't in the cards, another way to move BI ahead is through Software as a Service (SaaS) providers.

"With the ways technology has advanced over the past few years, there's still going to be a lot of BI on premise," said Hensen of SaaS provider PivotLink. He said customers often augment existing BI implementations with cloud or SaaS tools, because it gives them the ability to mash up data, use Web 2.0 tools and reduce costs. "It doesn't mean we do everything," he said. "But the majority of BI is about better operating performance and getting managers closer to data, and that's a great opportunity for SaaS-based BI players."

Aman, of SAP BusinessObjects, said he sees a lot of customers doing hybrid BI deployments -- some pieces installed on-premise, and some available on demand. While data warehouses work better on premise, applications with small amounts of data moving back and forth might work well with a cloud or on-demand model, he said. "Some customers like the cloud for accounting purposes, because you pay as you go and get some capability quickly," Aman said.

However a company procures its BI, its ROI is tied closely to business results. When measuring the effectiveness of business intelligence programs, "The No. 1 criteria is to look at bottom-line business results," said William McKnight, practice manager at Irving, Texas-based consulting firm US-Analytics Solutions Group LLC. He acknowledges that can be difficult to measure. "Short of that, I would go by user sentiment. Is the data they need made available to them in the right format in a timely manner, and structured for their analytical requirements?"

User participation in aspects of system design is a well-known best practice. The team that chose BI for Montana State Fund, for example, included two executive sponsors, two project managers and two employee teams, each divided between the business and technology sides of the house. Increasingly, organizations are moving beyond this initial team to create a group, often called a "BI competency center," to keep the discipline top of mind on a continual basis.

It's "definitely not something that every company has -- far from it," McKnight said. "It's an idea that the technology team needs to be more flexible for its user community, and intelligently look into the future, making sure they're bringing all possible methods of BI to the table."

The future of BI is about helping users get specific information quickly, whether it comes out of email, a transaction database or a data warehouse. There are many ways to get there, but keeping the big picture in mind is key.
"If your data model is a matter of taking what you used to do and putting it into a BI data model, you're putting yourself into a small box," Parisian said. "Step back and look at all the data."