maths & statistics awareness month
Blogs home Featured Image

April marks Mathematics and Statistics awareness month #MathStatMonth in the USA, with the aim of increasing the level of interest in these subjects. Working in an increasingly data-driven world, the ability to harness meaningful insights from data is an essential business and requires specialised data science expertise.

Data science is the proactive use of data and advanced analytics to drive better decision making. This ‘proactive’ use of data is what distinguishes data science from traditional ‘statistical analysis’ and needs to be an active part of an organisation in search of insight, better decision making or improvement.

Data science as a career choice for maths, statistics, and science graduates

Many graduates from maths, statistics and science backgrounds are increasingly attracted by a career in data science. Our current graduate placement Student, Elizabeth tells us more about her early interest in data science and why it presents a natural career path for those interested in mathematics and statistics. “Data science combines the skills and applications of mathematics and statistics with the use of big data and innovating technology to solve a variety of problems. I’m particularly interested in providing solutions to real-world problems and communicating these results at a high level within a business”, says Elizabeth.

“Throughout my placement I have seen the application of using mathematics and statistics within data science projects in performing exploratory data analysis to creating statistical models. My personal interest is in different types of statistical models, and I am due to study Time Series and Bayesian statistics in the final year of my degree”.

Elizabeth has benefited from seeing how mathematics and statistics have been used to model complex situations and improve business decisions from the optimum timing of routine maintenance, saving unnecessary reactivity and costs to creating descriptive, diagnostic and predictive insights which delivered great value and significant return on investment during her time at Mango.

Growing demand for data science

With the demand for Data Scientists still on the rise into 2021, the pandemic has created an even more urgent need for rapid decision making, informed and supported by constantly changing data sets, backed by effective visualization (highlighted by the World Economic Forum (WEF) in July).

Rich Pugh, Mango’s Chief Data Scientist summarises, “Leaders increasingly understand the potential of using data to create smarter, leaner, more engaging organisations. As such, we are still seeing growing demand for “data scientists” who are able to turn that data into acumen in a repeatable and scalable way. As a multi-disciplinary practice, “data science” relies on the combination of “advanced analytics” and “computer science” skill – this, combined with an ability to creatively explore challenges that can be solved, is at the core of realising the value promised by data science”.

“At it’s core, data science relies on mathematics and statistical rigour to provide robust algorithms that can be relied upon to solve often-complex challenges. As interest in data science continues to grow, the work at the Royal Statistical Society becomes increasingly important – to drive the discussion around statistical governance, and the correct and ethical application of statistical routines”, Rich concludes.

demand forecasting this easter
Blogs home Featured Image

Using data to accurately forecast demand for chocolate this Easter

Easter is a significant holiday for many businesses especially within the UK. The holiday period brings an increase to food and drink sales, with Easter being the second most popular period of the year for chocolate consumption. Many Britons book holiday trips in this period as well, making it an important time for the leisure industry.

Data science enables businesses to effectively plan for this busy period. Analysis of historical sales data can be used to predict future demand, allowing businesses to accurately plan stock levels; real-time analysis provides visibility of the state of a product, enabling businesses to quickly resolve issues regarding the manufacturing of products like chocolate bunnies.

Accurate forecasting to match demand

Demand forecasting is a commonly used approach which allows businesses to effectively predict future sales, plan and schedule production, improve budget planning, and develop efficient pricing strategies. Predictive analysis is used to understand and forecast demand over time, helping businesses make well-informed decisions.

Adapting in line with the coronavirus pandemic

The COVID-19 pandemic has brought great challenges within businesses. Easter brings challenges itself with the date of the holiday moving each year, however in 2020, businesses were simply not prepared for the impact of the pandemic. Easter egg sales fell by £36 million with many retailers having to sell lots of eggs at discounted prices. Some retailers were also unprepared for the boom in online sales and were not able to meet demand. On the upside, many businesses are better prepared for this year’s Easter period, with many focussing on online operations. Through demand forecasting techniques and last year’s data, businesses have been able to better prepare for this year’s demand. Many, for example Hotel Chocolat, are offering a limited range of Easter eggs this year.

As well as benefitting the retail and leisure industries, demand forecasting is used by other organisations over Easter. The NHS use forecasting techniques to predict demand and capacity for their services. This has been particularly important during the pandemic. In the January peak, NHS hospitals were caring for over 34,000 COVID-19 patients in England, approximately 80% higher than the first peak in 2020. Demand forecasting and mathematical models are being used to predict hospital bed demands frequently, tailored to specific hospitals, to help the NHS and government plan for future holiday periods such as Easter.

Demand forecasting is an effective approach that is being used by many businesses to plan for this year’s Easter holiday. Data-driven businesses can make well-informed decisions for the future, and as a result many will be better prepared this Easter.

Can we help with any aspect of your demand forecasting? Read our case study to find out how we have helped other companies with this.

world water day
Blogs home Featured Image

As we celebrate World Water Day, we consider the lack of access to safe and affordable water – a dangerous reality for billions across the world. It has a damaging effect on not only the health of billions of people but also on many aspects of their lives. With climate change comes droughts, floods and scarcity of water, bringing with it social and economic devastation. For example, clean water and sanitation is an important factor in reaching equity amongst gender.

By 2030, the UN aims to achieve universal and equitable access to safe and affordable drinking water and adequate sanitation and hygiene for all. To reach these targets, it is vital that businesses create and achieve their own targets for water consumption to help make a positive change. Unfortunately, many of these targets are not being met.

Using the power of data to make a positive change

In data-driven world, data science allows us to harness the power of data to make positive change. It can be used in several ways to improve water quality and water consumption around the world:

  • The innovation of machine learning can be used to improve the way in which water is collected and transported to reduce CO2 emissions. Moreover, it can be used to improve the treatment and utilization of water.
  • Real-time monitoring gives communities the power to ensure water is safe to drink while saving on resources.
  • Data analysis allows predictions to be made about the quality of water given a number of factors like weather and pollution. This allows for better planning when it comes to supplying clean drinking water to those in need.
  • Identification of water supply issues can be significant in preventing the spread of diseases through water supplies.

Mango realises the importance of using data to help other businesses bring positive change to the world. Data science can be used to inform businesses on their water targets and make steps towards reaching them. As well as harnessing data, it is equally important to involve stakeholders in decision making to identify, understand and overcome water challenges.

Reducing water waste

Water waste is a problem that many businesses face, with more than 25% of water wasted being due to leaks. This can be significantly reduced through the use of data science and in turn help businesses reduce their water consumption.

One company, i20, provides smart network solutions designed to help water companies reduce their leaks and bursts, energy use and CO2 emissions. The company recognised that it was collecting a large amount of data but were not harnessing it. Mango were able to not only help i20 realise their data capabilities but develop a solution that greatly improved the performance of their smart network solutions, leading them into the world of AI and data analytics. This has enabled water companies to shift to conditional based maintenance and reduce the number of water leaks. One client reduced leakage by 15% in the north of their city within 2 weeks of using the solution.

Data science is a powerful tool which can be used to inform businesses and improve their water consumption as well as having world-wide applications in reaching the UN targets of providing clean, accessible water to all.

Find out more about how Mango has helped i2o harness their data.

 

Global Recycling Day
Blogs home Featured Image

It’s Global Recycling Day today and a day to raise awareness of the importance of recycling and how crucial and lasting change can help preserve the future of our planet. Recognised in the UN’s Sustainable Development Goals 2030, we are already seeing many individuals, governments and organisations taking direct action to support the global green agenda.

Founded by the Global Recycling Foundation, this year’s theme is #recyclingheros and recognises people, places and organisations that inspire us – demonstrating positive action.

From plastic pledges and recycling,  to waste targets, businesses are positively impacting the environment, and data science is being used positively to reduce environmental impact and financial costs – aligned to their corporate responsibility. There are many examples of data analytics applications that can play just a small part in decelerating the process of climate change – the more focus that organisations place on this, the brighter the outlook for our planet. We decided to take a look at some of the positive use cases.

Food waste

Think about the waste problem in supermarket fresh food sales. Many businesses are using data science to help the UK meet its target of eliminating food waste to landfill by 2030. Analytics of weather patterns can help supermarkets ensure they have the right amount of seasonal produce to meet demand for a particular weather period without wastage; and enhanced analytics of customer weekly shopping habits would mean the store could ensure it has met demand without having surplus fresh food.

Gousto, a British meal kit retailer, implemented forecasting algorithms in an effort to reduce their food waste. They were able to predict demand and analyse seasonal trends to better manage their fresh food stock. Forecast modelling allowed the business to not only predict with a high degree of confidence the number of orders they would receive in future weeks, but also predict the performance of existing and new recipes.

Plastic waste

Plastic waste posing a considerable threat to our planet, with 8 million metric tons of plastic being added annually to the world’s oceans. That is why many businesses in the UK have come together to work towards having all plastic packaging recyclable, reusable or compostable by 2025. One such company is Tesco who are rolling out collection points for soft plastic packaging in their stores. Tesco’s efforts should help the public in their efforts to recycle as well their own.

Outside of the UK, many companies are also reducing their plastic waste and increasing their recycling, with many also helping the public do so. The Gringgo Indonesia Foundation, with the help of Google, have used AI and machine learning to create an app to help better classify waste items. It can be used by businesses and the public to help improve their recycling. With the use of data science, within a year of launching the app, recycling rates were increased by 35% in their first pilot village.

Space junk

Space junk poses a danger to astronauts in orbit, the world’s network of communication and weather satellites. Luckily, data science is here to help. NASA have been developing technology to remove space junk. Using machine learning algorithms, NASA are working towards improving the detection of space junk for removal.

Clinical waste

Since the start of the COVID-19 pandemic, there has been an increase in single use plastics and clinical waste. With only 15% of clinical waste being hazardous, there is a massive opportunity to reduce and properly manage clinical waste using data science. From reducing the number of unnecessary hospital appointments to the size of some healthcare equipment, a positive change can be made.

Reducing waste and recycling is vital for the future of our world. Data science provides many tools for creating and implementing solutions, and with data-driven businesses striving to reduce their waste, the future looks bright.

To discuss any use cases to align your recycling goals, contact us.

Author: Elizabeth Brown, Professional Placement Student at Mango

British Science Week
Blogs home Featured Image

As a data science consultancy, we’d like to celebrate British Science Week #BSW21 and the innovation in science, technology, maths and engineering, as well as the diversity of these roles. Many of our graduate consultants join Mango from a Maths and Statistics background, but also equally from a science background where many of the data and analytic approaches, including R are introduced.

Can data science be classified as a Science? We asked our Consultants, and the answer was a resounding ‘yes’, as our Graduate Data Scientist, Elizabeth Brown explains. “In my opinion, Data Science is a Science. The goal of Science is to gain a better understanding of the world around us, to explain why things happen or to describe the relationship between concepts. A big part of science is taking this understanding and applying it to real world situations, whether that be making advancements in medicine as we have witnessed with the vaccine development, or a modern way of introducing scientific methods to automate processes and make more intelligent decisions – ‘innovating for the future’, just as the theme for British Science Week this year. Like a science, we make observations, come up with hypotheses and through experimentation, test our hypotheses”.

Rich Pugh, Mango’s Chief Data Scientist agrees, “When done right, data science should be based on, and resemble, the scientific method. We formulate a “hypothesis” (although the structure of this can vary across application), then we use data to test that hypothesis”.

“Data is everywhere and being able to use it effectively to improve our understanding of the world is very exciting – expanding data-driven decision making, scientific discovery and  automation”, explains Elizabeth.

The growing capabilities of AI and machine learning are paving the way for real world solutions such as self-driving cars, much needed fraud prevention and addressing climate change. Fundamentally, data science isn’t just solving business problems, as a career it can support initiatives to create a healthier, greener and kinder world.

If you are interested in a data science career with Mango, click here to find out more.

International Women's Day
Blogs home Featured Image

On International Women’s Day, we’d like to celebrate the achievement of our own talent, raise awareness against bias and take positive action for equality. At Mango we believe in creating inclusive and diverse workplaces. A workplace that offers challenges and an active learning environment.

Kasia Kulma, one of Mango’s Senior Data Scientist’s, reinforces the many benefits that diversity brings to the workplace, her opinions are shared and published in Diversity Q, FE News and Education Technology.

According to Kasia, “parity in tech companies can be hugely beneficial to the industry as a whole be it in respect to gender, ethnicity, age, sexual orientation or other. Some of them are rather pragmatic, for example, by embracing a more diverse talent pool we can address talent shortages and progress to closing the talent supply/demand gap. More fundamentally, though, diversity brings a variety of perspectives which has a knock-on effect in increased creativity and thus faster problem solving and improved products. The company culture can, benefit significantly too. By helping employees feel included, no matter their background or gender, it can break down barriers and reduce the fear of being rejected”.

“This is a great way to empower your employees”, says Kasia and “harness their ideas and thoughts and attract talent”.

Closing the diversity gap

When you increase the investment in training for women to fill the corporate need, we believe it goes a long way to the overall goal of closing the gap once and for all.

According to Kasia, “the tech industry is very dynamic and it could offer the most creative and stimulating of environments. It gives you the flexibility to work cross-industry or to specialise in one area if you like. You could work on the bleeding edge of innovation and devote your time and career to something you really care about.”

“The most important advice I could give to an aspiring women technologist would be to look for employers that offer a supportive environment and embrace diversity – this way you’ll be more likely to spread your wings and learn. There are more and more tech companies that understand the importance and value that the workplace diversity brings, so join them… and enjoy the ride!”

Happy International Women’s Day!

If you’re interested in a career in data science,  you can find out more about Mango and check out our current vacancies.

 

NHS-R Community
Blogs home Featured Image

The NHS is one of the UK’s most valued institutions and serves as the healthcare infrastructure for millions of people. Mango has had the pleasure of supporting their internal NHS-R community over the last few years, supporting the initiative from its inception and sharing our knowledge and expertise at their events as they seek to promote the wider usage and adoption of R and develop best practice solutions to NHS problems.

According to a recent survey by Udemy, 62% of organisations are focusing on closing skills gaps, essential to keeping teams competitive, up to date and armed with the relevant skills to adapt to future challenges.  For many institutions, an important first step is connecting their analytics teams and data professionals to encourage the collaboration and sharing of knowledge. With ‘Data literacy’ fast becoming the new computer literacy, workforces with strong data skills are fast realising the strength and value of such skills across the whole organisation.

As the UK’s largest employer, comprising 207 clinical commissioning groups, 135 acute non-specialist trusts and 17 acute specialist trusts in England alone, the NHS faces a particularly daunting task when it comes to connecting their data professionals, a vast group which includes clinicians as well as performance, information and health analysts.

The NHS-R community was the brainchild of Professor Mohammed Mohammed, Principal Consultant (Strategy Unit), Professor of Healthcare, Quality & Effectiveness at the University of Bradford. He argues,  “I’m pretty sure there is enough brain power in NHS to tackle any analytical challenge, but what we have to do is harness that power, promoting R as the incredible tool that it is, and one that can enable the growing NHS analytics community to work collaboratively, rather than in silos”.

Three years in and the NHS-R Community has begun to address that issue, bringing together once disparate groups and individuals to create a community, sharing insights, use cases, best practices and approaches, designed to create better outputs across the NHS with a key aim of improving patient outcomes.  Having delivered workshops at previous NHS-R conferences, Mango consultants were pleased to support the most recent virtual conference with two workshops – An Introduction to the Tidyverse and Text Analysis in R. These courses proved to be a popular choice with the conference attendees, attracting feedback such as “The workshop has developed my confidence for using R in advanced analysis” and “An easy to follow and clear introduction to the topic.”

Liz Mathews, Mango’s Head of Community, has worked with Professor Mohammed from the beginning, sharing information and learnings from our own R community work and experience.  Professor Mohammed commented:

“The NHS-R community has, from its very first conference, enjoyed support from Mango who have a wealth of experience in using R for government sector work and great insight in how to develop and support R based communities. Mango hosts the annual R in Industry conference (EARL) to which NHS-R Community members are invited and from which we have learned so much. We see Mango as a friend and a champion for the NHS-R Community.”

Blogs home Featured Image

In 2020 the EARL conference was held virtually due to the restrictions imposed by COVID-19. Although this removed the valuable networking element of the conference, the ‘VirtuEARL’ virtual approach meant we reached a geographically wider audience and ensured a successful conference. Thought leadership from academia and industry logged in to discover how R can be used in business, and over 300 data science professionals convened to join workshops or hear presenters share their novel and interesting applications of R. The flexibility of scheduling allowed talks to be picked according to personal or team interests.

The conference kicked off with workshops delivered by Mango data scientists and guest presenters, Max Kuhn of RStudio and Colin Fay from ThinkR, with topics including data visualisation, text analysis and modelling. The presentation day both began and finished with keynote presentations: Annarita Roscino from Zurich spoke about her journey from data practitioner to data & analytics leader – sharing key insights from her role as a Head of Predictive Analytics, and Max Kuhn from RStudio used his keynote to introduce tidymodels – a collection of packages for modelling and machine learning using tidyverse principles.

Between these great keynotes, EARL offered a further 11 presentations from across a range of industry sectors and topics. A snapshot of these shows just some of the ways that R is being used commercially: Eryk Walczak from the Bank of England revealed his use of text analysis in R to study financial regulations, Joe Fallon and Gavin Thompson from HMRC presented on their impressive work behind the Self Employment Income Support Scheme launched by the Government in response to the Covid-19 outbreak, Dr. Lisa Clarke from Virgin Media gave an insightful and inspiring talk on how to maximize an analytics team’s productivity, whilst Dave Goody, lead data scientist from the Department of Education, presented on using R shiny apps at scale across a team of 100 to drive operational decision making.

Long time EARL friend and aficionado, Jeremy Horne of DataCove, demonstrated how to build an engaging marketing campaign using R, and Dr Adriana De Palma from the Natural History Museum showed her use of R to predict biodiversity loss.

Charity donation 

Due to the reduced overheads of delivering the conference remotely in 2020, the Mango team decided to donate the profits of the 2020 EARL conference to Data for Black Lives. This is a great non-profit organization dedicated to using data science to create concrete and measurable improvements to the lives of Black people. They aim to use data science to fight bias, promote civic engagement and build progressive movements. We are thrilled to be able to donate just over £12,000 to this brilliant charity.

Whilst EARL 2020 was our first such virtual event, the conference was highly successful. Attendees described it as an “unintimidating and friendly conference,” with “high-quality presentations from experts in their respective fields” and were delighted to see how R and data science in general were being used commercially. One attendee best described the conference: “EARL goes beyond introducing new packages and educates attendees on how R is being used around the world to make difficult decisions”.

If you’d like to learn more about EARL 2020 or see the conference presentations in full, click here.

Mango's success - a data conversation
Blogs home Featured Image

As we approach the new year, it seems an appropriate time to look back at how Mango’s 18-year history has reflected the evolving landscape of the data industry. It’s hard to believe that founders, Matt and Rich, have been sharing the data story since 2002, long before the term ‘data science’ gained popularity, and well before most organisations had begun to recognise the value of their data.  Matt and Rich have borne witness to this data revolution, via the big data era through to the current day where data is recognised as a new class of economic asset; universities routinely offer data science courses and Government departments have adopted algorithmic decision making.

Championing transition projects focussing on productivity through data science and a move towards repeatable and scalable models, Mango’s emphasis has been on ingraining data as part of a company’s DNA and supporting the creation of a data-driven culture.

It’s easy to see how the co-founders have remained at the forefront of the industry for so long, delivering data science projects to some of the world’s best-known companies. They credit their longevity to their open, honest and outcome-focused way of doing business and their deliberate shift from analytics as a reactive tool to adding value and the insights to drive decision making.

Asked about the most notable transition they had seen over the years, Matt referenced how the world has changed: “You can’t have barriers of data within organisations. Siloed data and analytics teams were once the norm, but these create structural, cultural and technological obstacles, wasting resource and inhibiting productivity. Many of the biggest challenges associated with data are not so much analytic problems, but fundamental information integration issues. Technology has moved at a huge pace in the past decade and that continuum between software advances and a recognition of the importance of data grows ever closer.”

Secrets of Success

There have been many secrets to Mango’s success, starting with its name.  “We considered lots of options incorporating ‘Statistics’ or ‘Analytics’ but they all seemed rather dull or dry and, in retrospect, would have dated very quickly,” remembers Rich. “Whilst ‘Stats Entertainment’ was just one of Matt’s inspired suggestions, our decision to name the company Mango, after his cat, has allowed us to continue to evolve and stay relevant through all the technological changes of the past 18 years.”

The name aside, it’s the founders’ approach that has been the real secret of their success. “Data for us has always been a way of doing business”, says Matt. “Looking back, we were right to place the emphasis on using analytics to empower end users. Our business has always been about making sense of data science, building out the capability by finding the experience, looking for knowledge and focussing on skills transfer and developing autonomy and support.  We’ve always believed in making data science easier for organisations, working alongside them and helping to broaden the scope and skills of the inhouse teams”.

Matt and Rich are unanimous that a vital element in Mango’s success, has been its people. “We’ve been lucky enough to attract extremely talented people, whilst also having a very successful internal graduate programme,” confirms Matt.  “My father’s advice was always to surround yourself with the best people and that’s exactly what we’ve managed to achieve. It was a proud moment to see that this year’s DataIQ list of Top 100 data professionals featured not only Rich, but also two of our former colleagues.”

Highlights

There have been many highlights along the way, but for Matt and Rich there have been some standout memories and high points over the past eighteen years.  “Standing on the platform at Zurich train station celebrating our first major contract win was a very memorable moment,” recalls Matt. “It was the point when we realised that we really were onto something new, securing a big customer who’d been won over by our style and attitude.”

A particular Mango achievement is their work in the R Community, including the creation of EARL (Enterprise Applications of the R Language), the first commercially focused R conference. The first EARL conference was delivered in 2014 and is now a firm annual fixture for R users across the UK and Europe.  Previous iterations have also seen EARL conferences delivered across the US. The original idea for the conference came from Rich, and the event is entirely organised and run by Mango staff. “The culture and openness displayed at EARL is fantastic, with companies keen to share their knowledge and use cases and talk frankly about their R journeys” remarks Rich. “Our work within the R community and the recognition that Mango has received for our R user groups and EARL is something we are particularly proud of.”

Lessons learned

Mango’s initial work was primarily within the life sciences and financial sectors. “A lot of our early work was in highly regulated industries and the rigour of working in those environments was massively valuable”, recalls Rich. “Everything we learned in those regulated industries we now deploy across industry ensuring a robust approach and the delivery of best data science practices and real practical advice.  Whilst much of our early work was in SAS, S- Plus and R, Mango has always been agnostic about tech, working within whichever language best meets our clients’ requirements and objectives; these days much of our work is in python.”

A phrase that resonates with Mango is ‘Give a man a fish and you feed him for one day; teach a man to fish and you feed him for a lifetime’.  “We work alongside our clients, mentoring and helping to upskill their teams, leaving them able to operate independently at the end of our involvement,” states Rich. “This approach is greatly valued by our customers, irrespective of where they are in their own digital transformation journey, who recognise the value that we add.”

Teamwork is at the heart of Mango’s work, whether it’s working in internal teams or as part of a client’s team. The introduction of the Belbin framework has been enormously useful in creating a team structure and awareness of individuals’ behavioural strengths, fostering more effective communication. “We started by employing the right people”, said Rich, “but the Belbin framework and our own Trusted Consultant programme has cemented a really productive team ethos.”

“Looking back, if there was one thing that we wished we’d done earlier, it would have been to introduce a marketing presence,” mulls Matt. “We were fortunate to grow organically and benefit from recommendations and repeat business, but in the past couple of years, the work undertaken by our marketing team to promote Mango to a wider audience has resulted in awards and recognition that have really amplified our presence and message.”

Looking ahead

“We are extremely proud of the company that we have built,” attests Rich, “and today Mango is focused on facilitating the sorts of conversations that we recognised as needing to be had some 18 years ago when we first founded.  We urge businesses to embrace methodical and pragmatic data processes before they dive in at AI/ML-level but are grateful, at least, that these latter tools have finally provoked the data conversation”.

 

 

Value at the Intersection of Data and Software
Blogs home Featured Image

For the last 18 years, Mango have been helping customers deliver on the potential of data and analytics.

When we started Mango back in 2002, the wider world of data and analytics was mostly reactive, with workflows conducted by individuals who produced reports as ‘one time’ outputs. As such, while data professionals wrote code, it could largely be considered a by-product of what they did. The advent of data science, together with the increasing need for just-in-time intelligence, has driven more proactive analytic workflows underpinned by open-source technologies such as Python and R.

Working at the forefront of data science, Mango understands the vital role of technology; to allow data to be transformed into wisdom in a repeatable way and deployed to business users at the right time, to support informed decision making.

There is a clear learning here for modern technology initiatives:

Every data project is a software project, and every software project is a data project.

To realise business value, it is vital that we balance both data and software elements of technical projects around a common and clear purpose.

Every data project is a software project.

Back in 2012, Josh Wills described a data scientist as someone who is “better at statistics than any software engineer and better at software engineering than any statistician”. While modern data science incorporates a broader range of analytic approaches than statistical modelling alone, Josh’s description of data science at the intersection of analytics and software engineering still holds today.

The changing role of data and analytics from a reactive practice to a strategic approach has driven the need for advanced analytics to be combined effectively with software engineering. If analytics is now an always-on capability, we need to codify the intelligence in systems that can be properly deployed and scaled within a business.

A ‘local’ alternative is just not practical – you can’t become a true data-driven business if analytics is run by experts on their laptops. We can’t stop making intelligent decisions if a data scientist is on leave. If a consumer purchases a product on Amazon, they will not wait hours or days until a statistician crunches the data to come up with other recommended products.

To positively impact a business with data, an end-to-end analytic workflow needs to be implemented using software engineering approaches. This encompasses everything from the creation of data pipelines, the deployment of models, and the creation of user interfaces and applications that can convey insight in the right way, linked directly to operational systems to action and process outcomes.

Every software project is a data project.

Increasingly digitalisation and regulation have driven more focus on requirements regarding the role of data in software systems. We can consider 3 types of requirement regarding the treatment of data:

  • User – requirements relating to users and preferences to provide a more personalised experience
  • Governance – requirements relating to the way in which data is managed in a secure fashion to confirm with data regulations and protect confidential data
  • Provenance – requirements relating to historical system actions to provide an audit trail, or to enable rollout back to, or understanding of, previous actions
  • Beyond this, the most important consideration in the design of modern systems is the ability to leverage advances in data and analytics to create richer, more useful experiences and applications. A growing understanding of the possibilities offered by analytics allows us to strive to ask better questions – to build software tools that are truly aligned to a users’ objectives.

For example, imagine we are building a software application to be used by call centre staff when speaking with customers. Traditionally, we may have built a system that combined data from various sources to give the user a single view of the customer. Perhaps this included data on previous orders, previous interactions, demographic data etc.

With data science, we could extend the functionality for the user – perhaps to include an understanding of likely customer churn linked to suggested retention actions, or a suggested ‘next best offer’ for the customer, or suggestions around the ways in which to talk to the user. Perhaps when the customer calls the call centre they can be allocated to exactly the right person to talk to, as opposed to being randomly allocated to the next available agent.

The use of data and analytics in software can have a transformative effect on the quality and usefulness of our software systems.

In summary…

Helping customers build capabilities at the intersection of data and software is the most effective way to unlock value in an increasingly digital economy. Technology businesses like ours who want to be part of that customer journey need to be ambidextrous in their approach to data and software, agile in their execution and above all empathetic to each customer’s unique context.

We’re excited to apply our passion for data science to a wider market as we join forces with Ascent – increasing our combined ability to design and deliver ‘the big picture’ for customers that helps them compete and flourish.

Author: Rich Pugh, Chief Data Scientist