"Advanced Analytics" - A Brief Interview

 
I recently had the pleasure of a short interview with Fern Halper, Director of Research for The Data Warehouse Institute, during the 2014 TDWI World Conference in Las Vegas. Here's a quick recap of our discussion.

 

 How do you define advanced analytics?

Unfortunately the term Advanced Analytics has become a bit amorphous. What seems  advanced to some isn’t really to others.  I think the intent is to clarify the distinctions between business intelligence, point-in-time reporting/dashboards/scorecards, and exploratory analytics. First it’s important to describe the various modes of “analytics”. I rather like Tom Davenport’s International Institute for Analytics (www.iianalytics.com) point of view;

Descriptive, Diagnostic, Predictive, and Prescriptive. That said, When I hear "advanced analytics", I tend to associate that term to the diagnostics, predictive, prescriptive modes. Let me explain;

·         Diagnostic Analytics is the mode where interesting hypothesis are formulated, using techniques and tools that enable data discovery, data mining, text mining to uncover apparent patterns, anomalies and/or behavior trends.

·         Predictive Analytics is the mode where those hypothesis are tested using predictive modeling disciplines, applying future state assumption and running various simulations to find and validate causal relationships which prove, disprove or even or morph that hypothesis.

·         Prescriptive Analytics where recommendations for change or remediation are developed. A prescription, if you will, for how the targeted business process/activity should be tweaked to achieve a higher level of optimization given the predicted future state.  The ultimate goal is to effect change to realize an improvement in process execution for a better business outcome.

 

What do you see your clients doing with the technology?  What’s new and interesting? 

The self-service data discovery tools are being used by business-community clients to do much more than rapidly create visually rich performance dashboards.

·         Clients are using this technology to flush out complex specifications for things like data quality rules and MDM match-merge-survivor rules. 

·         Clients are finding that next “a-hah” moment, discovering things about how the business is running by diagnosing why the data looks the way it does. They are using these data discovery disciplines as a divining rod to feed candidate projects into the predictive/prescriptive programs.

Virtually all of the BI platform providers are developing integration kits between their visualization platforms and the predictive modeling/scoring platforms (both open source as well as the mainstream commercial offerings). This will improve their ability to visualize the data at the macro-level with drill downs to the detailed level during model development and testing. This also improves the ability to visualize the ongoing shifts in business outcomes as a result of implementing the predictive model and acting on the prescribed changes.

Virtually all of the BI Platform providers and Data Integration Platform providers are developing integration kits and transformation objects to embrace the Hadoop eco-system for “big data”.  This opens the door for the Data Scientists to integrate more and more external, contextual data from 3^rd parties or public sources thereby enriching their models.

Another early trend I am seeing, is increased collaboration btw the internal process improvement teams and the business analytics teams. Companies with Lean or 6-sigma programs are defining process performance measures during value-stream and activity-based costing analysis. Those data points when integrated into the business analytics platforms provide business leaders visibility to how the company is performing not only on the traditional key performance lag-time indicators but also at a lower level based on these process improvement measures to give insight to how well business functions are running on a lead-time basis.

Finally, another emerging trend is the use of automated process discovery tools and workforce engagement tools that capture how processes are actually being performed and what people are actually engaged on throughout the day. Armed with this functional view of the business operations we can make better prescriptions on how to optimize those activities going forward.

 

What’s the biggest challenge companies face with analytics and how do you see them dealing with it?

The biggest challenge is definitely not the technology, rather it is the organizational alignment and the culture of the enterprise, particularly around shifting from “wisdom of experience” based decisions to embrace more “data-driven” based decisions. This arms our business leaders with facts to support their decision making. This culture shift is being led by the “Chief Analytics Officer” role.

 

 

 
 
 

 

 

 

 

 

 

 

 

 

 

 

Effective CAO’s create a culture of best practice evolution as organizations create an analytics ecosystem that prizes data-based decisions; manage the changes that analytics decisions bring about; act as a Trusted Partner across the various internal lines-of-business units; find/champion those compelling business decisions and projects on behalf of the functional line-of-business owners. A CAO type person is a gifted story teller speaking the language of the business as well as the language of the data scientist as an evangelist promoting the need to explore the possibilities. They take ownership and demonstrate confidence of Decision Outcomes; is willing to take risks and takes responsibility for the results (good and bad) by implementing a culture that creates transparency, accountability and “air-cover”.

Another key challenge is developing/acquiring the skills in the enterprise to become Advanced Analytics mission capable. According to a 2011 study by the McKinsey Global Institute, US businesses alone will need 190,000 more workers with deep analytical expertise and 1.5 Million more data-literate managers by 2018.

·         Business Managers that have the capacity for a deeper level of thoughtfulness and rigor around decision making along with a willingness to explore and modify assumptions.

·         Business analysts with expertise to mine data with an eye towards detecting patterns, anomalies or behaviors

·         Data Scientists for developing predictive models.  These Data Scientists need deep expertise in statistical model design, complex data mining and scripting in various programming languages as well as be a good story teller.