By Kumar Singh, Research Director, SAPinsider
What is the true failure rate of data science projects?
During one of the conversations with a guest we were hosting during the 4th of July weekend, we started a discussion focused on guesstimating what percentage of data science projects are actually successful. The other person had an opinion that the failures are far less and few. After all, every data science professional in their network reports back success after success. Where are the failures being reported?
That reminded me of an example that I walked him through. Back in business school, we actually had a case study in our curriculum that was about a company, that had “transformed” an aspect of its transportation operations. I will not share more details on the case study here since it will give away the company name. After many years, I got an opportunity to work on a project for the same company. I could see the vast gap between the “capabilities” that this organization built in the case study and the real world in which the company’s transportation organization and associated systems and processes operated.
So was the case study a lie? No, every word of it was probably true. But it was true when it was written. And that is where the concept of true success or failure of any analytics, data science or transformation comes into play. Was the solution implemented considering the long-term impact and long-term practicality in perspective or was it just implemented? It is not about “making it stick for a while”, so that an impact can be “shown”. True success, as a plethora of successful, innovative organizations have shown, is about actually making a stable, persistent and long-term impact, by leveraging analytics levers. Otherwise, it is just about “playing the digital transformation show-off” game. And the unfortunate thing is, while this playing the game has worked for decades, the band-aids have started to fall apart. This decade will see the gap between leaders, averages, and laggards, a categorization we frequently use at SAPinsider to define capability maturity, increase significantly. Many leaders and laggards may end up investing the same amount of resources in technology, yet will fail to extract value. The temporary “cosmetic benefits” will not sustain.
So what is the missing link here? While there are obviously many ingredients, ranging from technical to organizational, what is that one element that can help increase the probability of success of data science solutions exponentially? Everyone has a different perspective. The idiot’s perspective (i.e my perspective) is- it is the lack of a design thinking approach in the world of data science. Organizations that have been successful in developing real and useful data science solutions have been able to build teams that align with design thinking. To understand this a bit more in-depth, let us explore what design thinking entails in terms of key and relevant components.
Design Thinking- The true teamwork framework
In order to understand why I believe a design thinking approach is imperative to build real-world analytics solutions, we will first explore two fundamentals concepts:
A. What are the core elements of the design thinking approach
B. What are the various persons/skills that can be part of successful teams that drive innovation
Why do we need to review these two in tandem? While this is a question we will answer in a subsequent paragraph, the one-word answer is “teamwork”. Successful data science or advanced analytics solutions are the results of many individuals, mindsets, and personas coming together, and that is a critical element of the design thinking approach. So let us first review the key elements of design thinking, as shown in the illustration below:
The five key components of the design thinking approach, as shown in the illustration above, are:
Empathy: “Imagine the world from multiple perspectives.”
Understand the pain points of the end-user of the solution and the business. Despite hundreds of solutions within the same solutions category, if you interview the end-users, they will share the same set of challenges with all those solutions. Design solutions around pain points of business users and their business objectives.
Integrative thinking: “Not only rely on analytical but also exhibit the ability to see all of the salient.”
Despite all the hype around data, the major driver of failure in my experience (and opinion), is a fixation with the “numbers” aspects of a solution. The very word “solution” means it needs to address a problem.
Experimentalism: “Pose questions and explore constraints in creative ways that proceed. in entirely new directions.”
Develop a culture where every voice is heard and there is no fear of failure when trying new initiatives. This is what drives people to feel free to experiment vs leveraging the “safe” options in terms of analytics solutions.
Collaboration: “The increasing complexity of products, services, and experiences has replaced the myth of the lone creative genius.”
This is one of my favorites. True data science solutions will NOT come from the community of programmers, a single analytics translator, or a project manager. It will be a result of ideas and efforts from people with different personalities and skillsets.
Optimism: “At least one potential solution is better than the existing alternatives.”
My version of optimism from a supply chain analytics solutions perspective is that the worst culture you can propagate is “this is how it has always been” and it applies to analytics solutions as well. In today’s world, with the right talent, you can build an analytics solution to address even minute pain points.
Combining design thinking with “The Ten Faces of Innovation”
The postulation here is that that the key for building robust design thinking, is to build teams that will infuse all the critical elements of the design thinking approach.. And to understand how to build such teams, we will leverage a framework from the book “Ten faces of innovation” by Tom Kelley and Jonathan Littman. The book is one of its kind in several aspects since it:
1. Identifies the types of employees you need in your organization to propagate innovation
2. Describes how it is a MUST to have ALL these types to drive true innovation and transformation
3. Helps you with ideas on how to structure teams using these types to create innovative teams
The book borrows its title from the ten skillset types/employee types described in the book. The ten types are shown in the illustration below.
As you can see from the illustration above, there are ten key personas that exist within an organization, under three broad buckets. And based on the type of data science solution you are building, you will be building teams that leverage many of these “faces of innovation” . You can read more about these ten faces in this link: https://www.ideo.com/post/the-ten-faces-of-innovation
I strongly suggest reviewing this link as we will leverage these personas in the second art of the article extensively.
What does this mean for SAPinsiders?
While this first article lays the foundation, in the second part of this article that will be published next week, we will understand how these two frameworks can be leveraged to build data science teams, that leverage different talents, and approaches to create teams that can build data science solutions that align with real-world nuances. The key here is to understand that an innovative data science solution is not created by data engineers, data scientists or project managers. It is a careful orchestration of skills of many different personas, augmented by technology and processes.