Ten years ago Harvard Business Review named data scientists as having the “sexiest job of the 21st century.” This month they said it’s “still the sexiest job” of this century. I guess that depends on one’s view of sexy.
Sure, it’s difficult to source data science talent, given a scarcity of supply. There are strategies for improving this, like looking to adjacent job functions to upskill them. But while this may fix the overall supply problem, it won’t solve the persistent problem of churn. Why would data scientists quit their richly paid jobs, you ask? Let me count the ways.
SEE: Hiring kit: Data scientist (TechRepublic Premium)
Data science is drudge work
“While many believe data science is all about using machine learning algorithms to build models and make business impact, data cleaning is also an essential part of being a data scientist,” wrote Vicky Yu.
Not only is data cleansing an essential part of data science, it’s actually where data scientists spend as much as 80% of their time. It has ever been thus. As Mike Driscoll described in 2009, such “data munging” is a “painful process of cleaning, parsing and proofing one’s data.” Super sexy!
Now add to that drudgery the very real likelihood that enterprises, as excited as they are to jump into data science, many lack “a suitable infrastructure in place to start getting value out of AI,” as Jonny Brooks has articulated:
The data scientist likely came in to write smart machine learning algorithms to drive insight but can’t do this because their first job is to sort out the data infrastructure and/or create analytic reports. In contrast, the company only wanted a chart that they could present in their board meeting each day. The company then gets frustrated because they don’t see value being driven quickly enough and all of this leads to the data scientist being unhappy in their role.
As I have written before: “Data scientists join a company to change the world through data, but quit when they realize they’re merely taking out the data garbage.” The problem is that the grass isn’t any greener at another company — or, otherwise put, the trash is just as… trashy. Which wouldn’t be so bad if the data scientist’s employers paid attention to their work.
The problem of ignored work
As the HBR authors note: “Many organizations don’t have data-driven cultures and don’t take advantage of the insights provided by data scientists.”
At least they’re paid well, right? Well… “Being hired and paid well doesn’t mean that data scientists will be able to make a difference for their employers,” the HBR authors said. “As a result, many are frustrated, leading to high turnover.”
Even as machine learning and other components of data science have gotten more sophisticated, it’s still the case that executives will sidestep data in favor of intuition. Years ago, enterprise executives were adamant that they were data-driven, but they would reanalyze data if it didn’t match intuition. Such studies are legion. We love data when it confirms our preconceived notions, but not so much when it doesn’t.
None of which is to suggest that data science is failing. Despite these issues, demand for data science, and data scientists, remains strong. But it’s probably too soon to claim that data scientist is a “sexy” job. Increasingly essential, yes. But sexy? I guess that’s in the eye of the beholder.
Disclosure: I work for MongoDB, but the views expressed herein are mine.
Subscribe to the Data Insider Newsletter
Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more.
Delivered Mondays and Thursdays
