Why Can't the BI Hardware & Software Vendors Leave Good Enough Alone?

It's tough being a VP of Business Analytics these days.  The vendors are relentless. They'll call on anyone in the enterprise and convince them to acquire their technology. My business community is restless, dare I say, sometimes reckless.  They buy into these pitches and acquire products often times without my even knowing. And Oh my...The myriad of options out there are mind-boggling and seem to be growing every quarter.  Why can't I just keep my shop running as it is?

The challenge facing most providers of business analytics (BI/DW IT Shops) is their solutions take too long and typically miss the requirements mark. After a lot of hard work they end up with a largely dissatisfied consumer population.  Our heritage is to focus on getting that "single version of truth". A pristine set of curated data before we allow the business community to consume.  Our techniques and our architectural reference patterns were built upon an industry heritage and historical tradition developed on technologies that are rapidly becoming obsolete. Our understanding of the true meaning of operational data is limited and our mission has become too data centric. (We think data is the "ends", when it is merely the "means" an end.  Namely, Optimized Functional Performance through actionable insights from better business analytics)  End-result, the business community takes BI into their own hands. Wham! Spread-marts, user-marts, data-sprawl, data-anarchy,  disastrous business decisions, fines, lost licenses, jail time, workforce reductions, bankruptcy.  That is a bit unnerving!  So I get the CIO to convince his peers to slap the business' hands, allocate spend time & money to  rationalize/simplify that data mess back into IT control, recoup costs, mitigate risks, get it under control.  Whew, success... then Wham! The 2-3 year cycle starts again. Where's the innovation in that?

Over the last 7-8 years the BI tool offerings, especially the self-service data discovery tools, cut through the critical path of any data warehouse/BI project because the end-consumers who really know their data, interact WITH production data through advanced visualization techniques. In collaboration with IT the business drives out the formulation of a federated, integrated business view, complete with difficult integration, transformation and data quality rules applied against it. All validated by the business in real-time. End-result?  Usable, reliable, agile self-service solutions to enable better decision making now.   If that same set of information needs to be accessible to other consumers via other tools, the developers now have a validated set of specifications from that self-service BI tool metadata repository. This metadata export is used as the basis for implementing the more traditional ETL-->Enterprise DW-->reporting Data Mart(OLAP) architecture. This may not be for everybody. However, it behooves us to continually assess the viability new self-service enabling technologies. The sooner the better, because next on the horizon will be self-service near=time time Predictive & Prescriptive Analytics! The pristine designs that we spent years, even decades refining are built on enabling hardware and software stacks which are increasingly becoming obsolete and relatively more expensive to maintain vs the newer options in the market.  So I ask myself.  "Self? What is the acceptable life-span of good enough?" 

On the big data front. For the last 3 years that term ("big data") raised the hackles on my neck. "We've been doing big data for decades folks!" Well, while that is true, I had to reckon with the fact that a lot of investment is being poured into hadoop-alooza. It is a new architecture that is spawning a whole new revolutionary round of innovation. More importantly, there is a deluge of digital information sets floating around the world waiting to be monetized, now. This is information that previously just wasn't available.  Information properly harnessed, which can bring richer insight to the markets, competitive landscapes, prospect/customer behaviors & psyches, world events and causal relationships that can & will impact/influence how my enterprise leaders can be better informed.  “Big Data” is enabling just that. It is my responsibility as a BI solution provider to investigate how we can cost-effectively snatch that data out of the air-waves and avail it to the creative minds of our business analysts & data scientists to find that next "a-hah" moment and proactively act upon it before the competition does or before an early-stage outbreak becomes the next world epidemic.

Data Warehousing/Business Intelligence has been mainstream for a good 15-20 years. It was the radical idea back in those early adopter years and was met a lot of resistance.   Imagine the state of our world had we took the position that the truckload of green-bar reports delivered to mailroom was good enough?  As I look across the Business Analytics landscape, I see a new generation taking the helm in our business management and leadership roles.  This generation literally grew up with technology in their hands. They were weaned on gaming technology that learns overnight based on the previous days' world-wide interactions.  They are demanding their business operational systems learn how do the same.

I applaud the innovation coming out of our hardware and software provider communities. They challenge me as a Business Analytics solution provider to establish an "innovation culture".

What's good enough? A culture that comes in each morning firmly anchored in the truth that change is a constant that must be embraced. A culture excited to learn one new thing and propose one new idea to try. I challenge myself and my team to find ways to uplift our current investment value and embrace new technologies and techniques with prudent governance.