A question of culture: how to prevent data scientist churn

Media release

Rich Pugh, Co-founder and Chief Data Scientist, Mango Solutions, explains how employers can prevent a high turnover rate of data scientists

8 June 2020 Posted in Employer News UK


In a recent study, over 56% of the UK data science professionals polled suggested they would be looking for new jobs in the next 12 months. This is at the same time that around two thirds of companies reported looking to expand their data science initiatives, with three times more postings for jobs than searches for those same roles. In particular, the shortage of applicants is centred around more senior data scientists – those with hands-on data science experience, who can apply advanced data-driven initiatives in a business context.

With this in mind, one thing is abundantly clear: finding ways to motivate and retain your existing data scientists will be critical for success in creating and maintaining a data-driven business. In order to do this, it’s important to understand the primary reasons contributing to the current risk of churn for data scientists. In broad terms, these break down into three categories: difficulties delivering (and likely proving) value, siloed working conditions and access to the right training.

1) Proving the Value

Let’s start with the most serious issues first. From a business perspective, over a quarter of data scientists referenced a lack of support from managers and leaders, almost half also expressing issues with bureaucracy as the biggest challenge they face in their day-to-day work lives. This is indicative of a business that is hiring data scientists without a fully-formed and well-planned data science initiative in place. Likely they are relying on the technology, and not the process, to deliver results against a largely open-ended brief of “delivering insights”, without specific direction.

To address this, businesses need to think critically about what they are trying to achieve with their data scientists. Any data-driven business needs to start with a common understanding of the goals of the project, all of which should be specific, measurable and have clear timelines. Importantly the initiative needs to be pioneered from the top downward rather than vice versa. By doing this, data scientists will have a clearer idea of the brief they are working to, and be better able to prove value, by demonstrating success against the initial project goals. If done properly, this should alleviate the underlying cause of needless bureaucracy and lack of managerial support – a lack of common understanding, common language and trust between managers and practitioners.

2) Working in Siloes

Secondly comes addressing siloed working conditions. Current situation aside, humans are naturally social creatures, but that is not the only advantage of collaborative work practices. Bringing together teams of complementary skills – programmers, communicators, data visualisers and so on – ensures that all the different aspects of a data science project, from brief interpretation to execution, are executed by one team who can work together to ensure cohesion and maintain the focus of the original brief. Connecting those with data science skills and interests via a common language can facilitate knowledge and best-practice sharing, which can help to nurture and grow expertise within the company. This can be achieved through a new community of practice, knowledge-sharing sessions, hackathons or other similar initiatives to challenge individuals and promote integration.

3) Training

Finally, there comes the issue of providing the right training. Over two-thirds of managers (69%) plan to upskill existing professionals to address the skills gap. However, time remains overwhelmingly the constraint for professionals seeking to learn a new skill. Similarly, not knowing where to start and a lack of funding also feature as a concern. All of these reasons filter back to a single root cause – learning and development programmes aligned to address an organisation’s skill gaps. Businesses need to find a way to assess and track the current experience of their teams, and then use this to inform decisions on future training resource (time and money), based on a prioritised list of the skills the team needs and individuals want to develop.

Software tools are available that can be used to do exactly that. By investing in the right resources to develop your existing team, both individuals and businesses stand to benefit: businesses ensure they have the right people with the right skills to deliver industry-leading results, while practitioners feel supported in their learning objectives and feel the business is investing in their progress which can aid in employee retention.

There are three constructive steps businesses can take to prevent data scientist churn, and deliver better results for their business: ensure data science projects are endorsed from the top down with clear timelines and objectives; arrange opportunities for collaboration and knowledge-sharing; and invest in the right training at the right time to support employees. By doing this, businesses will not only address the central frustrations of existing data scientists that currently drive them to seek alternative positions, but they will create better business processes that lead to better results from data science practices – and all of this has the added benefit of shared costs and reduced frustrations that occur from constantly trying to hire for a position it would take a unicorn to fill.

About the author

Richard Pugh is Mango Solutions’ Chief Data Scientist and has over 20 years’ experience of working with data. As co-founder of Mango, Richard has led a wide range of ground- breaking data science projects for some of the most forward-thinking companies in the world.