Innovation and Big Data in Corporations: A Roadmap

Big Data is all about technology and business model innovation.  Why? Because, a lot of next generation business models are DATA centric.  Almost all companies talk about the need to mine their customer and transaction data.  Using a test-and-learn approach, leading firms tend to design, execute, manage, and measure market-facing DATA experiments. Every new idea has risk and the potential to be partly wrong and partly right, the goal is to quickly iterative-and-learn before committing to an expensive scaling of the idea.

DATA centric implies a new way of thinking about what a business does, and how it does it.  However, what is not clear is the roadmap to becoming a DATA centric business. What is the complicating matters is the complexity and challenge of making phased investment decisions in a shifting technology and services landscape as the gap between the technology investment and the capture of value usually lags by several years.

 

With the influx of money, attention and entrepreneurial energy, there is a massive amount of innovation taking place to solve data centric problems in new ways.  There two clusters of “parallel” innovation: (1) technology/infrastructure centric and (2) business/problem  centric.

The interesting thing is the multiple paths of innovation that are taking along different technology stacks shown below. The innovation is happening along 4 different fronts: (1) Data Stack;  (2) the Hadoop Ecosystem Stack; (3) Enterprise Information Management; (4) BI platforms, Analytics Tools and Visualization.

Different firms are attacking different feature/functionality gaps or problems. All relevant but inter-dependent if you are a CIO or CTO.

Big Data Analytics Infrastructure

While the Big Data infrastructure is evolving, there is a tremendous amount of prototyping and piloting taking place on the business side. We’re talking about innovations in business models and practices—everything from how a company runs its own business to innovative ideas for starting new ones and creating new markets. This is happening along three fronts:  (1) point solutions;  (2) process solutions; (3) vertical specific solutions.

Business or LOB Buy

Technology centric innovation invariably creates decision dilemmas between the “status quo” (Oracle stack) and emerging (e.g., Hadoop). Dilemmas emerge when there is tensions between two apparent opposite ideas or concepts.  In business we face these dilemmas all the time:  cost vs. quality, centralization vs. decentralization, stability vs. change, short term results vs. long term competitiveness.  Dilemmas are dynamic but inevitable.  They don’t go away.  They must be managed by making choices and creating a roadmap.

The challenge for management is how to integrate these various streams of innovation into a cohesive multi-phase roadmap that is relevant for their corporation.  How to articulate and put their version of the innovation pattern together.  Also how to version this pattern so the firm can execute without getting all tangled up in complexity.

Strategy

The emerging DATA opportunity is clear to most firms. Which path you take to innovation depend on two critical philosophical questions (or perspective and context):

  • is Big Data disruptive or incremental for your infrastructure?
  • Is your approach a Platform-to-Solution (common services driven) or a Solution-to-Platform (business problem driven) approach?

We are in the early days of this virtuous cycle.  Technology innovation and business model innovation are complementary.  Technology standards and maturity will also encourage business-model innovation.  Business model innovation fuels another round of technology investments.