Continuing from my previous two posts, today I am going to discuss the last 3 reasons of failure in data science from the article "Data Science: Reality Doesn't Meet Expectations".
(5) Your impact is tough to measure — data doesn’t always translate to value
In data science, it is never easy to measure the impact, very unfortunately. Firstly, as the article mentioned, data science is usually a supportive role so likely the credit needs to be shared. Secondly, data science is about experimentation and improved iterations, ie. there are experimentation costs to it. This makes it tougher as the experimentation costs are obvious and tangibles whereas the value generated are not (How will you measure the increase in loyalty?).
To be honest I do not have a solution to this. What I will suggest is to ensure there is proper documentation of your project especially the lift, accuracy scores, etc. With these scores documented, also state down how much cost savings and additional revenue it can translate into. And if possible (with reminders perhaps), document down the final impact made, after the individual campaign finishes.
It is going to be challenging to pinpoint the exact value but it is no excuse not to justify the benefits that data science brought about in a business organization. Measure the value/impact.
(6) Data & infrastructure have serious quality problems
Most organizations do not have nice data to work with. Why? Because they did not pay attention to it until the management starts to think about what value can be gotten from the data, likely to be after attending a conference where data science/analytics was shared. There are many areas to look out for when it comes to managing data so even if a company wants to start getting the data act together, it can still take some time before data is manageable.
Similarly, not much attention is paid to the infrastructure as well. Infrastructure at the start is built with a strong focus on generating value for customers rather, without building data infrastructure into the design. So any organization that is starting work in data science face with "shitty" data, infrastructure, and what adds into the potent mix is the lack of documentation.
Solution? Pretty straightforward. Start planning! Take care of your data, even if you are not going to use it in the near future. At least you do not have to spend time to collect good quality data later. Design data infrastructure into your business IT infrastructure. Your stumbling block next maybe the lack of knowledge in how data and data science work. Seek help, start going to data communities, and have a chat with the people there. Good luck! :)
(7) Data work can be profoundly unethical. Moral courage required
Yes, data work can be profoundly unethical. I shared this before in another post, where we need to pay attention to how we use the insights from our model (Must Consider the Impact on Stakeholders).
Data Science work is constantly under the barrage of biases, in data collection, in the presentation of insights, etc. What we really can do is to have moral courage, to admit that we are not saints and things can go wrong, and when we realize things have gone wrong, we have the courage to admit and rectify the mistakes quickly.
Also, we need to start having ethics classes in data science training programs. This is to raise awareness on what data science needs to pay attention to, to keep in mind that our work has an impact on the society. This is something I have been advocating for a while now. See the post here (Ethics in Artificial Intelligence: Let's Start Now).
Conclusion
So we have concluded the possible 7 reasons why data science failed in organization. I hope what is shared in the 3 posts is going to be useful for your organization and career.
If your business is thinking of engaging an external consultant/mentor. Consider chatting with me after checking out my LinkedIn profile. Or we can keep in touch on Twitter.
If what I have discussed here or any of my articles is of value to you, consider subscribing to my newsletter. Have fun in your Data Science and Artificial Intelligence journey!