Cloud-based Hadoop solution helps pharmaceutical company Astellas compress pieces of its market research from weeks to hours.

In the high-stakes game of promoting a brand new drug, understanding what’s working and what is not could make the variation between success and costly failure.

For top-20 pharmaceutical company Astellas, a brand new product launch last year presented the promoting analytics team with an information-velocity problem: Data from the sphere was arriving much faster than before.

“We started getting weekly patient data, in place of monthly,” Chad Dau, associate director of promoting analytics at Astellas told InformationWeek by phone. The accelerated pace meant the team couldn’t properly process and report at the data before the following batch arrived.

Adding complexity, Astellas and other drug companies increasingly are incorporating new data sets into their analyses, including unstructured text, audio, and video.

[ Read how automation has drawn in cloud enthusiasts: Private Cloud Adoptions On A Roll. ]

Dau turned to NewVantage, a Boston-based data and analytics consultancy, to pilot a cloud-based Hadoop application, leveraging its expertise in big data and optimized Hadoop configurations.

“We went from every week to 3 hours,” said Dau, adding that Astellas plans to expand its cloud Hadoop approach next year.

“Their in-house solutions took many years to run, although they’d a knowledge warehouse and analysis tools,” NewVantage managing partner Paul Barth told InformationWeek. For example, in a single case, analyzing how well patients take their medication — called “patient persistence” in medical circles — the NewVantage configuration was in a position to process the information 100 times faster.

“Their older style took per week or so as to complete,” Barth said. “We did it in under an hour.”

Nor is that sort of performance boost atypical, Barth said.

“In our experience with a dozen of those set-ups, that speed-up is pretty typical,” he said, noting that during some cases simply translating the knowledge processing logic from mainframe SQL to Hadoop leads to that sort of speed increase.

Paralleling these performance gains are the velocity of those deployments, Barth said. “Unlike data warehouses and SQL that took a year or two to determine in the event that they worked, these [solutions] inform you if they’re valuable in six weeks,” he said.

But Barth doesn’t think this inevitably drives every customer to real-time analytics.

“Some business processes are naturally slowly changing,” he said. For these businesses, it wouldn’t make sense to switch go-to-market strategies day by day. In other scenarios, reminiscent of fraud detection, time is of the essence, in fact.

A Gartner survey found 64% were investing or planning to speculate in big data technology in 2013. Within the June survey of 720 of its worldwide clients, Gartner found 55% are currently addressing enhanced customer experience using big data, while 49% are using big data to handle process efficiency.

On the alternative hand, Barth said there does look like a trend toward capturing the raw data and keeping it in an area where ad-hoc analysis is feasible, akin to a stock trading application that may access a repository with 10 years of transaction history in it.

“This permits you to see trends, patterns you hadn’t seen before,” he said.

Ellis Booker, based in Evanston, Ill., covers big data, data analytics, and education technology for InformationWeek. a well-known name inside the computer trade press, he has held senior editorial posts at top IT publications including InternetWeek, Web Week, and IDG’s Computerworld.

You can use distributed databases without putting your company’s crown jewels in peril. Here’s how. Also within the Data Scatter issue of InformationWeek: A wild-card team member with a unique skill set may also help provide an outdoor perspective that could turn big data into business innovation. (Free registration required.)

More Insights