Petro outlines six keys to the future of analytics and big data in healthcare.

1. Organizations are “drowning in information, but dying of thirst” at the same time. According to Petro, one CMIO at Nuance sums up the current state of big data eloquently: “When you’re in the institution and you’re trying to figure out what’s going on and how to report on something, he says you’re dying of thirst in a sea of information,” he said. “And what he means by that is, there’s a tremendous amount of information but a big data problem, and the issue is how do we tap into that to make sense of what’s going on?” This question applies not only to the patient, Petro continued, but also to the government’s plans in regard to disease and population management. “The issue is it isn’t organized,” he said. “It’s a mixture of structured and unstructured data, and what’s going to happen over the course of next several years is the government is imposing a tremendous amount of information for folks to report.”

2. Technologies that tap into big data will become more prevalent and ubiquitous. From a patient’s perspective, Petro said, analytics and big data will aid, for example, in determining which hospital in a patient’s immediate area is the best for treating their condition. “If I have a huge number of choices, today, you [determine that] by word of mouth,” he said. “But the government wants you to be able to look at a report card for various institutions, and the way to tap into the report card is to unlock all that information and impose regulations and reporting.” At the center of that, Petro continued, are the various types of IT used to tap into unstructured information, like dashboard technologies and analytics, business intelligence technologies, clinical intelligence technologies, and revenue cycle management intelligence for institutions. “These things will become more and more prevalent and ubiquitous … and [they] will become a lot more readily available to the patient.”

3. Decision support will be easier to access. In the institutions, Petro said, evidence-based medicine and decision support will become easier to access as a result of leveraging big data and analytics. “For example, if a patient is suffering from a particular condition, there’s a high potential something is going to happen to that patient because of their history,” he said. “That stuff is going to be brought up in the front of the care cycle, and the physician will be tapped on the shoulder, so to say.” Essentially, it’s about a lot more precise information at the point of care. “These are all the things that are going to tumble out of cracking the code, so to speak, of the big data problem.”

4. Information will flow more easily. Petro looked back to his days prior to working in health IT to remember what being in a hospital is like from a patient’s perspective. “I always remember sitting in a room with someone who’s sick, and you’re wondering what the heck is going on,” he said. “Then a physician comes in, and you’re afraid to talk to them.” He said there’s a lack of information flow, and a lack of ability for both physicians and patients to make choices. “The interesting part is, that happens all over the place within the workflow,” he said. After “cracking the code” of big data, he continued, the flow of information not only will be easier for physicians, but will more easily extend to patients. “For example, I can tap on my cell phone and see there’s a 15-minute ED wait over here, and a five-minute wait over here. It’s making the availability of data more ubiquitous, and I think it’s going to come in plain and simple ways like that, and in more complicated ways, like diagnostic support and evidence-based medicine support in the workflow.”

5. Quality of care will increase to maintain revenues and drive costs down. From a cost perspective and a quality of care point of view, said Petro, there are a number of different areas that will be impacted. For example, if a patient experiences an injury while staying in a facility, the organization isn’t reimbursed for his/her care. “So the ability for a system to see that this [has the potential] to happen and alert everyone, so that type of thing doesn’t happen to me as a patient,” said Petro. “The one way the government is putting pressure on that is you won’t be compensated like the old days.” Through reporting, Petro predicts, these issues will become less and less common. “They’re going to tap into the information, and that’s just one example,” he said. “There are a whole bunch of things that could happen that are preventable and should be completely avoidable. After tapping into the information, I think that’s going to drive down the cost of healthcare.”

6. The physician-patient relationship will grow with the help of social mediaand mobile apps. And this all stems from the need for hospitals to keep patients healthy and out of their facilities, said Petro. “We have this whole notion of an ACO, and hospitals are going to start getting comped for keeping you healthy. In the old days, hospitals made money the sicker you are, and the longer they keep you there, the more they make.” Petro predicted that because of this, there will be an “explosion” of mobile applications and even social media, allowing patients to have easier access to nurses and physicians. “It’s about keeping [patients] healthy and driving down costs,” he said. “Those are the two major areas where there’s going to be a lot of stuff going on from a health information technology point of view, all underpinned by the availability of data, and tapping into that.”

The analysis report, “The Big Returns from Big Data,” involved input and anecdotes from 58 big data implementation case studies across a range of industries and verticals. Nucleus reported staggering ROI figures for enterprises that successfully fanned out big data tools and access across internal and external data sets. Maturity of big data implementations was key to returns, according to the report. Extensive additions of predictive analytics and connections of internal data sets to those massive, real-time sources such as social media, brought in, on average, 241 percent ROI for enterprises involved in the Nucleus review.

Examples of those returns included: a 942 percent ROI for a manufacturer that was able to scour large, disparate data sets from vendors for purchasing and cost information; 1,822 percent ROI from reduced labor costs by a resort that integrated shift scheduling processes with data from the National Weather Service; and an 863 percent ROI by a metropolitan police force that was able to combine various crime databases alongside predictive analytics and its department assets.

On the other side of those enterprises reviewed in case studies, 12 failed to fully pay back their implementation investment. From that group, Nucleus found that 7 stopped or became hung up at the automation phase with their large and unstructured data sets, while the other 5 “were only slightly more advanced and were using big data to support operational tasks,” says Nucleus Principal Analyst Hyoun Park.

“Companies that failed to achieve high levels of ROI typically had not taken advantage of the collaborative, contextual and predictive benefits of big data. Instead, they were simply automating big data processing for standard reports and workflows,” Park says. “None of these companies [that failed to see ROI] had spread the use of big data to an enterprise-wide level or done any significant work to align the collection of big data with specific revenue-producing activities.”

Over the next 6 to 12 months, Park says the two main challenges with big data projects will be mastering visualization of data sets for more widespread use and contextualization of data within enterprise parameters and governance. These forces combined will mark the next tier of ROI and success with the growing expectations from big data, says Park.

“Enterprises that successfully use big data will figure out how to turn millions of inputs into a few easily digestible figures that can be shared socially and made available either in the board room or on a laptop, smartphone, or tablet,” the analyst says.

Information-management.com, May 4, 2012

Justin Kern is senior editor at Information Management and can be reached atjustin.kern@sourcemedia.com. Follow him on Twitter at @IMJustinKern.

Business technology thought leader John Ladley has spent 30 years around data and IT project management, where he witnessed and pioneered in the progressions of data warehousing, business intelligence and enterprise information management. This week the author and partner of Amitech Solutions, led a series of data innovation briefings addressing emerging technologies at the Enterprise Data World 2012 conference in Atlanta that often included the topics of big data and semantic technologies.

What are you noticing in applied data technologies today versus a couple of years ago?

One thing that’s becoming a real versus a nascent idea is in the use of semantic tools, and of course a lot of people are seeing that also with the big data tools. In both cases it’s becoming real that if the organization has the right mix of data with velocity, volume and variety and the source isn’t too terribly convoluted, they can find a very short time to market with potentially powerful results. Both semantics and big data have been called “next big things” of the kind that usually take three to five years to come to fruition or just disappear. In this case, both have come to fruition and are establishing themselves very rapidly.

We hear a great deal about big data, but the buzz around semantic concepts, RDF and OWL and SPARKL seemed to appear for a few years and then went underground. You’re saying the applied use of semantics is clearer now?

We saw an interesting case just this morning from a company called Spry doing work with the Department of Defense. Also, there have been acquisitions of some of the pioneers and incorporation of the technology. RedHat bought MetaMatrix years ago, for example, and you’re beginning to see that technology emerge in areas where there is a value proposition to be applied.

You mentioned it’s kind of an abrupt appearance of technology.

It’s not unlike the mid-80s experience with relational databases for those of us with gray hair. I was hip deep in delivery in those days with what was then the relational database and everyone thought it was a great idea. But it was hard to understand, still pretty abstract and the performance wasn’t quite there. We weren’t really sure, but we kept dabbling and finding value until all of a sudden one day the world was nothing but relational databases.

You say there’s a window of opportunity for companies, but so far we have a lot more questions than constructive answers, like we’ve just invented some new kind of telescope for data that lets us see things for the first time.

That’s a really good analogy, you’re saying we can look out and see these swirly dots that aren’t just stars, they’re galaxies and we may have no idea what they are or mean or how to interpret what I am looking at.  To a certain extent that’s still an evolution that has to occur with big data. When I talk about a window of opportunity, I qualify the comment to say that as the organization finds it has a concise bounded huge set of data, there is value there. There is another 95 percent of organizations that can’t use big data. I got into a lively discussion with a bunch of analysts about a year ago on that topic. They were saying everyone should be going for big data.

As in all aboard or miss the boat?

Right, exactly.  I said absolutely not, I am putting my CEO face on, or my CIO face on. I have an organization here that culturally cannot get a transaction entered correctly without having to change it the next day, and now you’re telling me I am going to do big data? You can’t do big data on garbage. What I find interesting about the big technology, and I need to qualify this more, is it is ready to go if you’ve got the background and the data and the culture.

Is it true that the big data folks aren’t the same IT data people who had been walking the hallways, that it’s a different group?

Yes, but the point is that the products are working. In the old days of the relational world, they’d say the product works but we’re just getting no data. When you closed the door with Oracle they’d say it wasn’t working too well yet. But the Hadoop stuff and MapReduce, that stuff works. You can jam terabytes in there and it works.

Now the business is going to want a silver bullet, right?

Of course, and this was part of that discussion with the analysts. They won’t care just to grab a bunch of data and process it. What they can think about is whether they know what to do with it. They can think about whether they have the culture to react. If you don’t have the sincerity to use information as an asset or as fuel for the organization, they are still going to be wasting their time and money.

Isn’t it a little daunting to be invested midstream with BI or MDM and then be told there’s this new direction everybody is going in? It can’t make our program managers presently carrying all the project risk happy.

BIG DATA GETS ALL THE PRESS these days, but as important — and perhaps even more important — is detailed data.

That’s according to Sam Ransbotham, an assistant professor at Boston College in the Information Systems department. He’s been at BC for four years, and before that he was at the Georgia Institute of Technology, where he got his PhD in IT management and his BA in chemical engineering.

Detailed data gives companies “the opportunity to try to figure out the ways that items, such as customers, differ,” he says. And that’s not just demographically, but in their behavior. “By observing detailed, transactional data, we can actually find much more interesting things than we can by lumping them into demographic groups.”

Ransbotham’s initial research interests were in security and risk, but that led him to analytics. “What really sparked my interest is how do you make sense of that much data, since detection system logs have data in huge numbers, the billions of records magnitude. How do you spot trends and how do you figure out what’s going on in those trends?” His research also now encompasses what he calls “more positive areas” of customer service and customer reviews and how those things are being used in marketing.

In a conversation with David Kiron, executive editor of Innovation Hubs at MIT Sloan Management Review, Ransbotham explains why detailed data can tell companies not just why someone did something but why they didn’t do something else, how hospitals don’t seem to face heightened malpractice risks when they install electronic medical record systems and what companies should and should not be worried about when their customers fire off real-time comments on Twitter.

You see all the talk about what big data is doing to the competitive landscape. What do you see as “big data”? What does that mean to you?

It’s easy to talk about the number of records, just the total volume, and there’s no question that that’s increasing and is huge. But more than just the size, is also the types of data. It’s transaction-level, detail-level data — such as clickstream data versus summary data. And that’s really the more interesting of the trends. The details are what give companies the opportunity to do so much more.

What kinds of “more”? How is big data different from regular kinds of analytics?

There’s the opportunity to try to figure out the ways that items, say customers, differ. And not just how they demographically differ, which is how people have been reporting and thinking about things for forever, but how does their behavior differ. By observing detailed transactional level data, we can actually find much more interesting things than we can by lumping them into demographic groups.

Can you give us an example? What kinds of behavioral patterns can you discern with new techniques that you couldn’t before, that could help managers manage better?

One of the things that we’ve become very good at is capturing data about what people are doing. If you think about classic point-of-sale systems, they capture details of the transactions. What people buy, what time of day, whether the product was on sale.

But those systems don’t tell you what the customer didn’t do. What did he look at and not buy? What did he not look at? How did he walk around the store?

I know some researchers are monitoring where shopping carts are in the store and where people are looking when they’re in grocery stores and those sorts of things. But when we talk about the Web, we’re getting new kinds of data that do show us the kinds of things that people looked at but didn’t buy. That’s opening up a new opportunity to understand how people are going about the process. Web data, click stream data, was one of the first chances we got to look into that.

We’re still limited there, because companies tend to get just what people did on their website, and not what people did categorically. You also can’t tell if people have shopped in stores and then are buying online. But we’re gradually getting more and more data about what people are doing in the process of shopping and of buying.

Can you draw that out a little bit — what can you learn from what customers didn’t do?

Think about the example of online grocery stores: You can tell what people looked at but didn’t purchase. And you’d want to know, was it because of price? Was there some attribute of it? Was it the photo, or the text?

Companies that are savvy can start to manipulate those variables. So, they’ll pick half their customers and send them down one path and half down another path. One path might have more information, or less information, or better pictures, or more detailed pictures, or less detailed pictures. Companies can really start to understand what types of information and presentations make a difference to consumers.

Experimentation is cheap now, isn’t it?

It’s cheap and it’s fast. If you have a random way of showing people different things on your website, then you can pretty quickly, with a very small number of observations, start to figure out what’s working and what isn’t. In real time, you can begin to refine your presentation — and I’m using Web commerce as an example, just because that’s an easy way of running experiments, but experimentation can go well beyond that context.

It goes back to some things we were just talking about in terms of the difference between big data and detailed data — you really don’t have to have that much data for an experiment like that. It’s not like you need to run it for six months. These are answers you can find out with not the huge volume, not the billions of records, but with the detailed level of the records. Randomness combined with a clear decision point (such as a purchase) is powerful.

And the thing is this: if your company is not doing this, somebody else is, and they’re doing it quickly, too.

Yeah, talk about that: What are the competitive pressures to build a capability around being able to deal with a variety of analytics?

Well, I’m a technical person in general, so I don’t want to minimize the importance of having good technical people on your team, and certainly building models, understanding them and running them is important. But I think that’s really the secondary skill here, and not the one that’s most in demand.

The competitive challenge is that while it’s hard to find people that do the technical things, it’s even harder to find people who can interpret them, who can use creativity to ask provocative questions, who can think about experiments to run that would be interesting. It’s hard to have a corporate culture that encourages that sort of manipulation, experimentation and data-based decision making.

Again, there’s certainly a shortage of people who have the technical skills, but I think we’ll see, much like we have in the rest of IT, that those things move more quickly towards commodities than these managerial skills do.

How do companies deal with this expertise shortage, finding people who can blend analytic skills with business expertise?

We’re certainly seeing that employers want people with lots of technical skill coming straight out of school. That’s where they’re pulling some resources from. But for managerial skills, companies are sending people out to explore what other people are doing and trying to stimulate some thinking that way.

Let’s switch gears and talk about some of your research on security and risk with IT. Tell us about what you found looking at healthcare and malpractice lawsuits.

Sure. One of the things I’ve looked at is medical malpractice lawsuits and whether there is increased risk to hospitals that install computer systems that, as a byproduct, log what happens during patient care. Does this affect their medical malpractice, the lawsuits? Does it change anything?

Ransbotham’s research indicates that hospitals using electronic medical record systems don’t seem to face heightened malpractice risks.

This is a study I did with Eric Overby at Georgia Tech, and we looked at data in the state of Florida because it has some laws about reporting that make it public information. We also have information about who’s installing computer systems, particularly things like electronic medical records.

So on the one hand, there’s a lot of evidence out there that says that patient health care improves with the installation of these systems. They can help doctors prevent drug interactions, they can improve accuracy. Lots of positives out there. But at the same time, there’s a fear that all this detailed information can be used against hospitals in the context of medical malpractice. That if something goes wrong, then people go through an electronic discovery process and try to dig through these detailed logs and find out something that’s happened. You’re talking about lots of different people involved with a patient and there can be lots of opportunities for something to not be absolutely perfect.

It’s an inherently empirical question here of which of these tensions is stronger. Which way do things work out? To me, it’s that process that’s the most interesting, trying to figure out how the data we’re collecting can be used to answer that question.

So what did you find? Did the data increase the risk?

The net result is that we don’t see any adverse effect of installing those systems. If anything, there’s improvement. But they’re certainly not worse. So you get all the patient health care benefits, and it doesn’t seem to be hurting from a medical malpractice perspective.

That’s a fine result.

Yes. And to tie that back into what we were talking about earlier, the presence of all this data is new. It is unusual for managers who are used to making decisions the way they’ve made them all along, who are used to relying on their experience, whether or not it’s right or wrong, good or bad. The idea that you would actually look to data to answer these things is a big shift. It’s a big shift for physicians, it’s a big shift for a manager.

Do you have a perspective on what kind of help leaders need to accept that their experience may not be all they thought it’s worth, that data can supplant it — how can they make that shift in attitude?

Well, I don’t think that data completely supplants experience. Maybe to make that point stronger, I’ll say that you get billions of records out there to analyze, and we need to shift people who have that experience, who have relied on that, into guiding those questions. We’re not John Henry, the steel-driving man, fighting the machine. These are tools. We still need people to help understand what kind of experiments to run, and to understand how to shape those tools.

On the other hand, we certainly don’t need people bookkeeping by hand. That’s not a good use of people. So it’s trying to apply people where they’re most useful.

Now, your question was more about some of the cultural shifts and people skills to make those transitions, and that’s something I don’t know. I’m not sure that I’m qualified to answer that.

Ok. Let’s switch gears one more time. Your research also looks at the volume of data in social media and mobile devices, and what all that data and speed at which it’s generated means for companies. Can you talk a little about that?

So, we have all these social media tools out there, and if you think about it, what have those things done, really? At the core, they reduce transaction cost and coordination cost. They’ve made it really easy for us to share stuff.

By sharing stuff, I mean that people are creating data and providing feedback, and they’re doing it right at the time of the good or bad experience. The idea that these things are firing in real time and that they’re visible to everybody is, I think, a brave new world.

So we as customers are walking around with mobile devices that make it so easy to take a picture, to post something, to act immediately. Maybe it’s just an overall trend in society to react to things so quickly, maybe too quickly, but in either case, the devices and the infrastructure have certainly enabled that.

Here’s the question for companies: what are the risks of this? Before cell phones, if you went to a restaurant, maybe you had to wait in line for longer than you wanted to, but the food was great and it was a nice night. By the time you got home, you said, “Ah, that was a nice evening.” Whereas today, the worry is that in our modern world, you’ve already fired off Facebook updates and Twitter updates while you’re waiting in line, complaining about the restaurant. You don’t just turn to the person in line next to you and make some comment about how things are terrible; you get to broadcast that everywhere quickly.

So I did a study looking at restaurant reviews (with Nick Lurie, funded by theWharton Customer Analytics Initiative), where we had some coming from mobile users and some coming from desktop users. It’s not clear what should happen. On the one hand, mobile people might react like I said, react instantaneously and not really kind of get the holistic experience in their head. On the other hand, they don’t have problems with recall bias, and they’re more likely to be accurate the closer they are to the actual experience.

The study looked at how those reviews are different. We did a lot of text analysis and said, okay, the things that people were actually typing, how did they differ? Are there more emotional words? Are there more positive words or negative words, or are there more words that indicate future thinking or past thinking? We looked those variables and tried to explain the difference in influence of reviews written on mobile and desktops.

Have you reached a point in your analysis where you can recommend to managers how concerned to be about these immediate reviews that come from customers who post to Facebook or who tweet as the experience is happening?

I’d say some of the fears — like the fears about health care electronic medical records and litigation — are unfounded. What we saw when we were looking at the influence of reviews is that mobile reviews are less influential. Perhaps people recognize that other people are hotheads, or that the mobile experience might be jaded, and they’ll discount that. Which I think is really interesting. I think that companies don’t need to panic about this as much as they thought. Yes, get a few stories out there like the guy who made the YouTube video “United breaks guitars” and those sorts of quintessential social media explosions, but for the most part, people are discounting those things. At least in our restaurant context they seem to be.

You can go back to where does competitive advantage come from and how can you sustain it. Some of the things that we’re talking about, the data about your customers and how they behave, can really become a source of advantage for companies. Now again, the challenge is that other people are trying to do this as well at the same time, and so it may just be the kind of thing where you need to run just a little bit faster than everybody else. Companies need to figure out how to turn that into some sort of competitive advantage, a sustainable or non-ephemeral competitive advantage. Those are the things that we’re still working on.

Sam Ransbotham (Boston College), interviewed by David Kiron

April 26, 2012: MIT Sloan Management Review

If you haven’t read my friend Tom Davenport’s book Competing on Analytics (Harvard University Press, 2007), you should. If you have read it, it’s a really good time to read it again.

Why? The Big Data revolution. Or I should say, the Big Analytics revolution. (BTW, I think that “Big Analytics” isn’t a great term either. But what the heck, let’s just make it easy for the marketing folks to transition from Data to Analytics by using the same adjective.

In his book, Tom talked about organizations that were using analytics – analyzing massive amounts of data – to gain a real competitive edge in their business performance. Practitioners ranged from health care organizations and pharmaceutical companies to retailers (such as Best Buy) and the entertainment industry (Harrah’s Entertainment, whose CEO Gary Loveland, an MIT graduate, wrote the foreword to the book). And we can’t forget the sports teams – not just the Oakland A’s, famously portrayed in Michael Lewis’ book and then the movie Moneyball, but our own Boston Red Sox and New England Patriots. Professional sports is being transformed radically by analytic tools, techniques and culture.

Today, I think we’re on the edge of a secular shift in business: the ability of virtually any business – not just large and technically sophisticated businesses with big budgets – to get a competitive edge by using analytics every single day.

Big Analytics for the Rest of Us

Big Data is only part of the story. What matters more is what you do with Big Data: Big Analytics.

Big Data is a fact of life for almost every company today. It’s not something you run out and buy. You already have it, whether you want it or not – or whether you know it or not. It’s the large quantities of data that companies accumulate and save daily about their customers, their employees and their partners; plus the vast corpus of public data available to companies elsewhere on the Web.

In years past, the constraints of traditional database technology – such as first-generation relational database architectures and the outdated business models of the companies who commercialized these systems – made it difficult and expensive for people and organizations to store and access such volumes of data for analysis. Today, the popular adoption of innovative technologies such as the Hadoop distributed data file system (HDFS)/MapReduce, as well as many other “built for purpose” database systems enable data to be aggregated and available for analysis cost efficiently with extreme performance. (BTW, I believe that Hadoop/MR is way over-hyped right now: it’s great and very useful, but only one piece of the Big Data/Big Analytics puzzle.)

Some of the innovative products/companies that I’ve had the privilege to be a part of include:
Vertica*
VoltDB*
Paradigm4*
Cloudant*

There are MANY others – which is good database innovation mojo – especially compared to seven years ago when Mike Stonebraker and I started Vertica. At that point, the standard response from people when I said I was working with Mike on a new database company was “Why does the world need another database engine? Who could possibly compete with the likes of Microsoft, IBM and Oracle?” But the reality was that Oracle and the other large RDBMS vendors had significantly stifled innovation in database systems for 20+ years.

Jit Saxena and the team at Netezza deserve huge kudos for proving that, starting in the early 2000s, the time was right for innovation in large-scale commercial database system architectures. Companies were starved for database systems that were built for analytical purposes. I’m not a fan of using proprietary hardware to solve database problems (amazing how quickly people forgot about the Britton Lee experiment with “database machines”). But putting the proprietary hardware debate aside, thanks to innovators like Mike Stonebraker, Dave Dewitt, Stan Zdonik, Mitch Cherniack, Sam Madden, Dan Abadi, Jit Saxena and many others, now we’re well on our way to making up for lost time.

Some other database start-ups of note include:
NuoDB, Jim Starkey’s company
Akiban Technologies
ParElastic
Hadapt, Dan Abadi’s company
Basho, the makers of Riak
10Gen, the MongoDB company
Cassandra, not really a company (yet, I think) but a viable key value store

There are many new tools out there for managing Big Data, and new innovations are being delivered to the market every month, from big and small companies alike. I’ve actually been impressed with the progress that Microsoft has made with SQL Server of late, mostly driven by Dave DeWitt, PhD, and his new MSFT Jim Gray Systems Lab at University of Wisconsin. Most business folks don’t realize that many of the technical principles behind systems at Teradata, Greenplum, Netezza and others were based on innovations such as the Gamma parallel database system as well as the dozen+ systems that Mike Stonebraker and his vast network of database systems researchers have been churning out over the past 15+ years.

The challenge now for most commercial IT and database professionals is the process of trying to match the right new tools with the appropriate workloads. If, as Mike and his team say in their seminal paper “one size does not fit all for database systems,” then one of the hardest next steps is figuring out which database system is right for which workload (a topic for another blog post). This problem is exacerbated by the tendency to over-promote the potential applications for any one of these new systems, but hey, that’s what marketing people get paid to do.

Until just recently, however, another key element that has been missing is the focus on how data is going to be used when people implement their Big Data systems. Big Data is useless unless you architect your systems to support the questions that end users are going to ask. (Yet more fodder for another blog post.)

For many decades, there was no open, scalable, affordable way to do Big Analytics. So the kind of capabilities that Tom Davenport talks about in Competing on Analytics were available only to companies with huge financial resources – either to pay companies like Teradata (which is where the Wal-Marts and eBays of the world ended up) or hire tons of Computer Science PhDs and Stats professionals to build custom stuff at large scale (Google, Yahoo, etc.) The analytics themselves were even further restricted within those companies, to professional analysts or senior executives who had staff to make the results of these analytics digestible and available to them. These capabilities were kind of a shadow of the Executive Information Systems trend in the 1980s.

Today this is changing. Established companies and start-ups are creating technologies that “democratize” Big Analytics, making large-scale analysis affordable for even medium-sized businesses and usable by average people (instead of just business analysts or professional statisticians). A great example is Google Analytics. Ten-plus years ago, the kind of analytics you get today with Google Analytics were only available to Webmasters who had implemented specialized logging systems and customized visualization. Now, my son Jonah gets analysis of his Web site that would have cost big bucks a decade ago.

However, there are still missing pieces. I believe we need:

- Large-scale, multi-tenant analytic database as a service, similar to Cloudant and Dynamo but tuned/configured specifically for analytical workloads with the appropriate network infrastructure to support large loads

- Large-scale, multi-tenant statistics as a service – equivalent functionality to SPSS, R, SAS, but hosted and available as an affordable Web service. The best example of this right now is probably Revolution. I guess the acronym would be Statistics as a Service – or Statistics as a Utility
- Radically better visualization tools and services: I think that HTML5 has clearly enabled this and is making tools like Ben Fry’s Processing more accessible so that the masses can do “artful analytics”

Once analytic databases and statistical functionality are available as Web services, I believe we’ll see the proliferation of many new affordable and sophisticated analytic services that leverage these capabilities. One of the best examples of this that I’ve been working on is a product created by Recorded Future.* I believe it is one of the most advanced analytics companies in the world. Christopher Ahlberg, Staffan Truve and the entire amazing team at Recorded Future are making some of the most sophisticated analytics in the world available to the masses. Another example in Boston is Humedica, where my friend Paul Bleicher (founder of Phase Forward) is doing fantastic work – perhaps some of the most advanced health care analytics in the world.

At this point there are still significant technical hurdles. But in the very near future, the challenge for most companies won’t be technology: it will be people, especially those who will no longer be limited to static reporting.

To be successful, businesses will need to build analytics-driven cultures: cultures where everyone believes it’s his job to be information-seeking and to think analytically about the integrated data that can help him make better decisions and move faster every day.

The #1 Step Toward Building the Right Culture

So, how do you go about building an analytics-driven culture?

This is obviously a long discussion; Tom’s book is a great primer on this, particularly Chapter 7 where he points out that it’s analytical people that make analytics work. But I think that the most important step for businesses is to rethink the way we build systems and to respond to the call to action created by the “consumerization” of information technology in the enterprise.

Every day new analytic tools are being made available to consumers over the Internet. Those consumers then walk into their workplaces and are faced with a pitiful cast of static, mundane and difficult-to-use tools provided by their IT organizations – most of which are 10+ years behind on analytics. (Sorry if it hurts to hear this, but it’s true – and I include myself as one of the people who is under-delivering on meeting enterprise end users’ analytical expectations).

To build analytics-driven cultures, businesses need to shift IT’s emphasis from process automation/”reengineering” (popularized by the late Michael Hammer and others) to decision automation. With process automation, the average worker is treated as a programmable cog in a machine; with decision automation, the average worker is treated like an individual and an intelligent decision point.

This isn’t New-Age management theory or voodoo; there’s already a good track record for it. Southwest Airlines empowers its gate agents do things that gate agents at American Airlines can’t even think of doing. Ditto for Nordstrom and Zappos in retail, where sales representatives have broad discretion on how they satisfy each customer. These companies believe in the individual identity of every person in that organization and use data and systems to empower them. (The antithesis to these companies are companies that are stuck in post-industrial employment models – most amusingly like Charlie Chaplin’s employer in “Modern Times” [watch]. And, yes, such companies do exist even today – otherwise we wouldn’t have shows like “The Office” or movies like “Office Space.”)

Unfortunately, the technology people in most companies don’t think this way, and most of their vendors are still stuck in the process automation mindset of the 80s and 90s. If companies thought of their competitive edge as being decisions, they would expect their systems and their user experiences to be radically different. In some ways, this is the process of thinking about an organization’s systems in context of how data is going to be used/consumed instead of how the data is being created. Because we tend to build systems with a serial mindset, many systems in today’s organizations were built to “catch” the data that is being generated. But the most forward-thinking organizations are designing their systems from the desired analytics back into the data that needs to be captured/managed to support the decisions of the people in their organizations.

Ironically, there are a bunch of us artificial intelligence (AI) people from the 1970s and 1980s who experienced a technology trend called expert systems. Expert systems really involved taking AI techniques and applying them to automating decision support for key experts. Many of the tools that were pioneered back in the expert systems days are still valid and have evolved significantly.

But we need to go one step further. Today, consumer-based tools are providing data that empower people to make better decisions in their daily lives. We now need business tools that do the same: Big Analytics that enable every employee – not just the CEO or CFO or other key expert – to make better everyday business decisions.

Here’s one great example of what can happen when you do this.

Over the past decade, synthetic chemists have begun to adopt quantitative/computational chemistry methods and decision tools to make them more efficient in their wet labs. The use of tools such as Spotfire, RDKit and many others have begun to change the collaborative dynamic for chemists, by enabling them to design libraries using quantitative tools and techniques and perhaps most importantly to use these quantitative tools collaboratively.

It’s very cool to see a bunch of chemists working together to design compounds or libraries of compounds that they wouldn’t otherwise have created. Modern chemists use their remarkable intuition along with incredibly powerful computational models running on high- performance cloud infrastructure. They analyze how active or greasy a potential compound could be or how soluble, big, dense, heavy or how synthetically tractable it might be. Teams of chemists spread across the globe use this data to make better decisions about which compounds are worth synthesizing and which are not as they seek to discover therapies that make a difference in the lives of patients.

This is where the magic comes from – from being decision-oriented not process-oriented. Big Analytics can make average people junior artists – and natural artists wizards – by giving them the infrastructure to make sense of data and interact with people. It makes art and magic more repeatable.

Google Analytics is a good example of what happens when a business adopts an analytics-driven culture. By using Google Big Table with the Google filesystem, Google expressed the value of its analytics in a way that could be given to anyone who manages a Web site. Google then watched the value of the analytics get more rich, statistical, analytical.

I predict that this is what will happen in the rest of the business world as Big Analytics takes hold and analytics-driven cultures become more the norm – and expected of every enterprise system.

I have seen what can happen close up, many times. As the co-founder of Vertica, I was fortunate in having Mozilla and Zynga as two of our best early customers. Zynga thought of itself as an analytics company first and foremost. Yes, their business was providing compelling games, but their competitive edge came from making analytics-based recommendations to their customers in real time within games and about where to place ads in their games. Another company that I work with closely that is providing this type of capability is Medio Systems.* Companies like Medio are democratizing Big Analytics for many companies and users.

By rethinking how you build systems – within the context of how the data in the system will be analyzed/impactful, and thinking of every person in the company as an intelligent decision point – you’ll smooth the path to Big Analytics and 21st-century competitive analytics.

Analytics as Oxygen

In the future, analytics won’t be something that only analysts do. Rather, analytics will be like oxygen for everyone in your organization, helping them make better decisions faster to out-maneuver competition. Competing on analytics is not something to be done only in the boardroom: it’s most powerful when implemented from the bottom up.

By empowering the people who are closest to the action in their organizations, Big Analytics will have an impact that dwarfs the potential suggested by Executive Information Systems and Expert Systems. Companies that figure out how to leverage this trend will reap significant rewards – not unlike how companies like I2 and Trilogy realized the value of artificial intelligence decades after AI was perceived to have failed.

*Disclosure: I am a board member, investor or advisor to this company.
POSTED BY ANDY PALMER AT 10:46 AM

Research looking at the possible relationship between the 100 Top Hospital winners and the HIMSS Analytics Electronic Medical Records Adoption Model (EMRAM) scores was conducted in November 2011.  Hospitals included in the sample received the 100 Top Hospital award in either 2009 or 2010.

[See also: Top 100 Hospitals named by Thomson Reuters]

The research found statistically significant relationships between the 100 Top Hospitals of 2009 and 2010 and the advanced stages of the EMRAM model during the same time period. For example:

• In 2009, 14 percent of 100 Top Hospitals were in Stages 5 to 7, compared with six percent of all U.S. hospitals.
• In 2010, only 1 percent of the 100 Top Hospitals were at Stage 0 or Stage 1, compared to 17 percent of all U.S. hospitals. Additionally, 21 percent of 100 Top Hospitals were at Stage 5 or higher, compared to nearly nine percent of all U.S. hospitals.

“Objective proof that higher performance is correlated with the electronic medical record represents a major breakthrough for the hospital and IT industries,” said Jean Chenoweth, senior vice president, 100 Top Hospitals, Thomson Reuters.  “100 Top Hospitals are more than twice as likely to be in EMRAM Stages  5-7.  During the same time period, the 100 Top Hospitals national benchmarks rose. As more hospitals take advantage of government incentives to adopt the electronic medical record, performance standards in hospitals may rise much more quickly and uniformly across the country. This is very encouraging news for patients, providers, payers and the government.”

In addition, an analysis of the percent of hospitals in each stage also demonstrates there is a higher proportion of 100 Top Hospitals in Stages 3 or higher of the EMRAM model when compared to the overall U.S. population, as shown in tables three and four in the report.

[See also: Advanced EMRs reap advanced benefits]

“The very strong correlation between Thomson Reuters 100 Top Hospitals and hospitals at higher levels on the EMRAM model shows the benefits of deploying advanced clinical applications in the delivery of healthcare in U.S. hospitals,” said John P. Hoyt, executive vice president, HIMSS Analytics. “This is one of the first studies to make the connection between hospitals using advanced information technologies and quality and safety benchmarks.”

Read the study on the HIMSS Analytics website.

April 18, 2012 | Bernie Monegain, Editor HealthcareITNews

Last year, a computer named Watson (named after IBM founder Thomas J. Watson) beat the two greatest human Jeopardy!champions at the popular quiz show. Viewers tuned in to see Watson analyze a Jeopardy! clue, search through millions of documents written in human language, and return a correct answer – all in less than three seconds. It was a captivating television moment. But once the game was over and IBM engineers brought Watson back to the lab, one final question lingered. What did this new level of artificial intelligence mean for companies in the real world?

At its core, Watson is a computing system that can extract facts and understand the relationships in vast quantities of data with lightning speed. What sets Watson apart from other computing systems is its ability to understand natural, human language, which is inherently full of ambiguity. This power, combined with its ability to judge each possible answer in real time and decide which one is most likely the correct answer, marks a major leap in computing innovation.

This has enormous implications for business. Don Campbell, IBM Distinguished Engineer for Business Analytics, sees the potential. “In business analytics, we really need to enable businesses to get to the answer of a question, not just point to more resources,” says Campbell. “There are all kinds of applications where being able to surface up an answer based on a fairly complex human question is very business critical.”

Revolutionizing business analytics

Existing business analytics solutions available today recognize subtle trends and patterns in data to give insight that results in better business decisions. Business analytics has helped companies do everything from preventing high-value customers from leaving for a competitor, to up-selling to current customers, to helping develop successful products. With the daunting complexity of human language however, there are limits to what existing business analytics can do.

The complexity often lies in unstructured data, which is frequently in the form of emails, text messages, audio and video files and represents up to 80 percent of data within an organization. “You can imagine the data sources of today not just having the traditional structured capabilities that we’ve had in the past,” says Campbell, “but adding all of this new content through social media and user generated content. That’s a perfect place for Watson to add value and understand what the buzz is around your product from a marketing and customer relationship perspective.”

The promise that Watson represents is a major breakthrough for business analytics. Mark Morton, IBM Cognos Express Marketing, envisions how Watson will help companies better understand their customers. “Imagine your call center is on the line with an unhappy customer,” says Morton. “With access to things like Web 2.0 type thinking and social media, we might be able to identify that this person is not a large customer but is influential in social media. So perhaps you should make a more expensive offer to keep him happy because if he goes, he may take hundreds or thousands of Twitter followers with him.”

Making confident decisions

Having confidence that you have the right answer, of course, is critical in business analytics. In years past, companies would have total control of the information in their database and could be very confident of the integrity of their data. Today, with the wealth of data outside of company’s control, it’s a different story. “We now have pollution of information that’s coming at us from untrusted sources, but if we just ignore that information then we’ve lost any value it could bring as well,” explains Campbell.

In situations where answers may not be black or white, Watson introduces the idea of the confidence element in decision making. “By applying algorithms to deal with the system’s confidence in certain types of hypotheses and in possible answers, Watson provides a confidence level associated with the response,” explains Campbell. “That is very valuable to a decision maker who is sitting on top of both internal data and external data which has a variable amount of trust.”

“In business analytics, we really need to enable businesses to get to the answer of a question, not just point to more resources.”This surely hits a sweet spot for industries like healthcare where answers can have life or death consequences. Campbell describes a scenario in the not too distant future in which “Watson can understand more about what the patient is describing and is also able to mine through all of the medical information underneath the covers to better come up with what that patient might be suffering from.” And companies in other industries are lining up as well. Earlier this year, Citigroup announced that the bank will collaborate with IBM to explore how to best infuse Watson’s deep content analytics into the business.

Watson for midsize businesses

With the increased number of instrumented and interconnected systems generating massive amounts of data, companies are looking to business analytics to make sense of it all. According to a recent IBM study, “Inside the Midmarket: A 2011 Perspective,” 70 percent of midsize companies have plans to implement business analytics in their operations.

According to Morton however, Watson will not be the sole domain of large enterprises. In the future, smaller firms may be able to access Watson-like capabilities as well. Midsize firms, says Morton, “are the ones who are most in need of agility in response to fast changing conditions so something like this that’s going to give them that extra half step in front of everyone else is very important to them.”

In fact, to best meet the budget constraints of the midsize company, Watson may be commercialized as a highly scalable solution accessible via a cloud environment. Campbell predicts, “We could assemble the Watson pieces in a way that could answer business questions with a smaller amount of hardware, with a slower response rate that would still add value to a certain class of user.”

Getting to the right answer

On a smarter planet, the demands of business will require the same kind of real-time response and advanced analytics that Watson offers. And similar to the Jeopardy! game show, a wrong answer in a business environment can have negative financial consequences. “In your business, you have those same kinds of risk parameters,” says Campbell. “What confidence level do I have to have in order to change my pricing to a certain level? What confidence level do I have to have in order to give this specific drug to this patient? In the end, you don’t want 10 answers. You want to have one right answer.”

Originally posted: IBM.com Let’s Build a Smarter Planet: Forward View