Here’s How To Keep Your Data Project From Running Of The Rails

We were told that “data is the new oil.” The Internet of Things combined with the ability to store massive amounts of data and powerful new analytical techniques like machine learning would help derive important new insights, automate processes and transform business models. It seemed like a massive opportunity.

Yet Gartner analyst Nick Heudecker‏ estimates as many as 85% of big data projects fail, due to a lack of data skills, poor internal coordination between departments and lack of integration with line managers and staff. Implementing a big data project, it seems, is far more challenging than installing a new email system.

The news isn’t all bad. A survey by Deloitte of “aggressive adopters” of cognitive technologies found 76% believe that they will “substantially transform” their companies within the next three years. So while big data and cognitive technologies are no panacea, they can deliver value, if pursued wisely. Here’s how you can keep your data project from going off the rails.

1. Build for Purpose

In the Deloitte survey, one major concern that even leaders who are successful implementing cognitive technologies have is integration into existing processes and systems, with nearly half saying that it is a top challenge. However, Roman Stanek, CEO at GoodData, a firm that specializes in data analytics and intelligence, believes this is in large part a problem of conception.

“The first question you have to ask is what business outcome you are trying to drive,” he told me. “All too often, projects start by trying to implement a particular technical approach and not surprisingly, front line managers and employees don’t find it useful. There’s no real adoption and no ROI.”

One company that has been successful with this approach is CAVA, a fast growing restaurant chain similar to Chipotle but focused on healthy Mediterranean cuisine. It been almost fanatical about making sure line managers are involved with data projects from the start, in many cases initiating them, rather than just having new systems thrust upon them by IT staff.

“What’s been essential is that our data science team has built genuine partnerships with our other teams with the intention of extending the capabilities of every facet of our business, rather than trying to replace human talents with a set of algorithms,” CAVA CEO Brett Schulman told me.

2. Automate The Most Tedious Tasks First

While many worry that cognitive technologies will take human jobs, David Autor, an economist at MIT sees the the primary shift as one of between routine and nonroutine work. In other words, artificial intelligence is quickly automating routine cognitive processes much like industrial era machines automated physical work.

One of the most basic mistakes many firms make is to try to use cognitive technologies to replace humans to save costs rather than to augment and empower them to improve performance and deliver added value. This not only kills employee morale and slows adoption, it usually delivers worse results.

Stanek advises his customers to start with automating the most tedious tasks first. For example, in a project GoodData undertook in to automate facility management, routine decisions like ordering light bulbs and cleaning supplies were largely automated, which freed up professionals to do higher level work, like implementing large-scale projects and rooting out fraud.

Perhaps most importantly, this approach can actually improve morale. Factory workers actively collaborate with robots they program themselves to do low-level tasks. In some cases, soldiers build such strong ties with robots that do dangerous jobs that they hold funerals for them when they “die.”

3. Focus on The Right Data, Rather Than Just Big Data

Many executives groaned when the European Union announced its General Data Protection Regulation (GDPR), but Kenneth Sanford, an analytics expert at Dataiku told me that he sees it as a blueprint for better data governance and practice. As it turns out, more data isn’t always better and can be far worse.

“The big data mantra is to store everything” GoodData’s Stanek told me. “Not every data set is important. Just because you can collect data doesn’t mean you should. You can’t just collect all the data and expect someone to figure it out someday.” He also points out that using too much data increases costs and can diminish accuracy through overfitting.

“Now, you have GDPR liability over that. Companies need to learn to be more careful with the data they collect. You need to start with the business process and the personnel that are intimately involved on a daily basis with the core systems involved with those core activities. That’s how you determine what’s the right data to deliver a return on investment, Stanek says.”

As data security becomes increasingly important, organizations need to focus on data governance as much as they do on data collection. You have to know what data you have, what you’re going to use it for and how to get rid of it when it no longer serves a useful purpose.

4. Start Small And Build On Success

Probably the biggest mistake executives make is starting out too big. During the planning process, each department pushes their own “wish list” and features get added on to the project plan. As things snowball, you end up with a five-year death march to a system that’s will be irrelevant by the time it’s released.

At GoodData, they urge their customers to start small and then build on success. “\“Everything these days is done in an agile way and data projects need to be agile as well, Stanek told me. “You want to get feedback as soon as possible and be able to show results in the first 3 months. That’s how you build momentum and go on to greater things.”

For example, in one project for FCM, a travel solutions company, a proof of concept was put in place in just a few weeks, but that simple project helped build up steam and since then the effort has been expanded to include over 1300 customers and 450 projects. Where you start is not necessarily where you end up.

So instead of coming up with a grand plan to transform your business into something that resembles the bridge on the starship Enterprise, you are much better off starting off with a single business function in a single department, implementing a minimum viable project and learning as you go. You can’t expect the road to a cognitive enterprise to be a simple straight line. The important thing is to keep moving forward.

– Greg

An earlier version of this article first appeared in Inc.com

image: pxhere