Stop The Alarming Trend Where KPIs Are Set As Your Data Strategy

Data strategies are not defined by your Key Performance Indicators (KPIs)

Published in

Openbridge

5 min readMay 15, 2019

At a recent event, a panelist was emphatic companies limit the use of data to only what was needed to support KPIs. All other data was simply “noise”. There seemed to be head nodding amongst others on the panel too.

Is this really the best advice to be following? Is your data strategy really a function of point-in-time KPI(s)? What if KPIs need to change? What if they are incomplete or wrong?

While KPIs are important, the advice seemed counterproductive. Narrowly working back from a KPI as a defining characteristic of a data strategy artificially limits broader data exploration and discovery. This limited field of vision could cause a business to miss signals that can drive performance.

What Is A KPI?

Key Performance Indicator (KPI) is a form of performance measurement. KPIs benchmark success of an organization or activity such as customer satisfaction programs, products attributes, marketing campaigns, sales, and many others.

What if Google search worked the way the panelist suggested KPIs should work? Imagine attempting a search where Google had to know your search prior to being able to execute it. Asking Google “Who won the Oscar for best movie in 1999?” resulted in a response of “Sorry, you have never asked about the Oscars before and we only have data about who won the Oscars in 2019”.

It would be unlikely you would get more than one search done in a week or either bother asking Google anything any more. This is not to imply that Google does not have a collection of KPIs, they do. However, Google does not work back from a fixed, limited set of KPIs to determine what they are indexing.

Lessons From Signals Intelligence?

What if the CIA, NSA, FBI…approached intelligence the same way the panelist suggested? Imagine this statement “Sorry Mr. President, we ignore digital data, we just focus on intel from traditional landline phones since this is what we have KPIs aligned against” How effective would they be in their communications intelligence? Not very.

The intelligence community, like the NSA (National Security Agency in the United States), practice what is called signals intelligence (SIGINT). Often, this involves casting a broad net across a lot of data to find signals. They look for patterns, behaviors, and trends which allow them to derive insights. The NSA describes this as SIGINT:

SIGINT is intelligence derived from electronic signals and systems used by foreign targets, such as communications systems, radars, and weapons systems. SIGINT provides a vital window for our nation into foreign adversaries’ capabilities, actions, and intentions.

The NSA has to look more broadly at data because they have a stated mission to “provide intelligence to those who are asking questions”. Sound familiar? Like Google, the NSA has KPIs, but they do not work back from KPI delivery as the mission, they are in the business of providing intelligence to those asking questions. Some may think that is a subtle difference, but if you are working back from a KPI you align all resources to the delivering this as the principal outcome. KPIs are a means to an end, but not the end.

If your data strategy is narrowly defined as a function of your KPIs, then you will be operating with both hands tied behind your back as you try to achieve strategic goals and business objectives.

Why do KPIs get cast into stone?

Part of the challenge teams have around KPIs is they often viewed as static “truths” in a dashboard. While people may advocate KPIs embrace a level of dynamism, that rarely happens. Unfortunately, the norm is KPIs suffer from analytics rigamortis. Rather than embrace a reality of changing behaviors, technologies, modalities, relationships, and economics, KPIs are treated as the gospel, unchanging over the arch of time.

So what are typical challenges for KPIs?

KPIs are prone to groupthink; published KPIs are passed around as “truth” within different industries
A narrow collection of KPIs often reflects some form of system or “thinking” constraints. This approach can be traced to a time where resources like CPU, memory, and disk were expensive or where data expertise is limited
KPIs reflect an economic decision around how much data could be processed. Each incremental traunch of new data would have a significant cost, not only hardware, but the human ability to process it.

Instead of treating the KPI as the alpha and omega, how about looking at the Scientific Method for guidance for these types of data-driven business processes.

The Scientific Method For Data Analytics Efforts

The first steps of the Scientific Method are to make an observation and ask a question. This provides you with information to develop a hypothesis, test and form a conclusion. If the conclusion warrants it, you may develop a KPI from that process.

Broader data exploration and discovery starts with these observations, questions, and hypothesis. KPIs reflect answers (or perceived answers) to a prior these explorations. The panelist was getting these confused and confusing the audience in the process by working back from a KPI in defining a data strategy.

Better advice to the attendees would have been stressing the importance of a data strategy with analytics founded in observations, questions, and hypothesis. An iterative, agile data and analytics process delivers more cycles as well as more value. This is true just as true for the NSA as it is for business.

The world is not static, nor is your data, which means a holistic data and analytics strategy can help you can achieve your goals and measure progress in your business performance.