A New Economy
What do oil and data could have in common? They seem to belong to different worlds because an oil refinery is imagined as an industrial cathedral, a place of drama, power and dark recess, usually in the middle of the ocean, creating long shadows its long towers.
Despite that, the data and oil industry have much to share: they are both an essential fuel for the world economy: the former feds cars, plastics, and drugs, while the latter power all kinds of online devices and services. They also both created new infrastructure, business, monopolies and economies, but probably data did it in an even more disruptive way, by challenging current regulations and business models. Moreover, is reaching unparalleled limits: according to IDC, which is a market research firm, the data created and copied every year is going to reach 180 zettabytes (18*1021) by 2025, with an exponential rate of growth.
Nonetheless, the story is not only about volume, but it also regards quality: data collection has started as a database creation activity of names and other personal data, is now evolving towards the analysis of real quick time flows of unstructured data, that are collected from an increasing number of devices, infrastructures, vehicles, et cetera.
Not only the volume and the quality of data is changing, but also organizations are more and more improving in extracting information from the interactions registered. Not surprisingly, big tech Silicon Valley companies like Facebook and Google are developing and improving cutting edge algorithms able to learn, for instance, how to target advertisement from the like or comments that the target user has made recently. One of the most significant struggles for regulators and the public opinion is that the learning mechanism has a so-called snowball effect underneath: the more users interact, the more the algorithm are good at learning from them, the more the mechanism is effective and the firm profitable, which creates favouring condition for oligopoly’s rise (2019, The Economist).
How Data is Helping Firms Thriving
Big Data is not bringing a revolution only in Advertising and Product Marketing, but it is creating a giant leap from the past, also when internal management practice are taken into account. Having to use just a few words to describe Big Data impact on management, Peter Ducker famous quote should be mentioned: “You can’t manage what you don’t measure,” the more significant the amount of information you have on your customer, the more the decisions taken after are going to be appropriate.
At this point, it would be normal to ask whether there is a difference between the terms Big Data and Analytics, which have been a management commodity for a long while. The answer is yes. Big Data is overcoming old-fashioned analytics in at least three different aspects, that are commonly acknowledged as Big Data’s Vs:
- Volume. As mentioned before, the amount of data created every year is increasing, almost exponentially, and big organizations are facing relevant challenges regarding data collection and storage. For instance, according to estimation, Walmart collects more than 2.5 petabytes (2,5 * 10^15) of data from its customer transactions every single hour.
- Velocity. Since in many applications data has to be available and processed exceptionally quickly, real-time transmission technologies are widely adopted to ensure agility and advantage. For instance, the act of collecting financial information a few minutes before Wall Street workers may turn in substantial potential profits.
- Variety. As suggested previously, the sources of data are multiplying: it started with social networks, then smartphones, and nowadays, the so-called movement of IoT, i.e., the rise of an interrelated network of computing devices, mechanical and digital machines able to transfer data across a network without needing human-to-human or human-to-computer interaction, involves more and more typologies of devices. Moreover, the fall of the cost of computing is making data-intensive approaches cheap; the trend appears to be growing incessantly, turning human into “walking data generators”.
It may sound natural to wonder whether there is any evidence that an increase in Big Data use is strongly correlated with better business performance. A team from MIT Center for Digital Business, in partnership with McKinsey’s business technology office, tried to answer the question by conducting structured interviews with managers of 330 American public companies regarding their technology management practices, that have been then confronted with performance data coming from different reports. The answers showed a very heterogeneous variety of data-driven management practices and significantly better financial and operational results coming from organizations that defined themselves data-driven. More specifically, the firms in the top third of their industry in the use of data-driven decisions making were on average 6% more profitable and 5% more productive than their competitors, also taking into account the contributions of capital, labour, traditional IT investment and purchased services.
If someone has to name an industry where 1 minute lost matters, he would probably reply air travels, that is the reason why one not named US airline decided to take action after being confronted with the fact that 30% of the flights into its central hub used to have at least a gap of five minutes and 10% of them had a gap of at least ten minutes. The landing time of flight used to be an estimation of the pilot. Hence, the airline decided to outsource the timing estimation to PASSUR Aerospace, that thanks to its service RightETA were able to increase the accuracy by putting together publicly available data about flight schedules, weather, feeds from a network of proprietary radar stations installed near airports. In particular, these installed radars collect a wide range of information every 4.6 seconds regarding the planes it manages to perceived and send it to PASSUR data centres, that thanks to the amount of information collected in years of activity can also generate pretty accurate predictions. The intervention has worth several millions of dollars per year per airport. As said before, more data means accurate predictions and better decisions (2019, Harvard Business Review).
The Know-How Gap
The expectation at this point would be that the world has taken the road of becoming entirely data-driven, achieving digital transformation, implementing AI processes and going fully analytics. The real story is different, as the NewVantage Partners’ 2019 Big Data and AI Executive Survey tell. 64 high-level manager from several Fortune 1000 companies had been asked to evaluate their companies’ data technology, and the results sound moderately alarming:
First of all, 72% of survey participants report that they have yet to forge a data culture. Moreover, 69% report that they have not created a data-driven organization. Added to that, 53% of the respondents state that they are not yet treating data as a business asset. Furthermore, 52% admit that they are not competing on data and analytics. Lastly, in the last three year of the survey, the number of firms defying themselves as being data-driven has fallen from 37,1% to 31,0% in 2019.
As odd as it could sound, the investment in big data and AI initiative has taken the opposite direction: 88% report a greater urgency to invest in big data and AI, 92% of survey respondents reported that the speed of their big data and AI investments are accelerating. Moreover, 55% of companies reported that their investments in big data and AI now exceed $50 million, up from 40% just last year and 75% cite fear of disruption from another company as a motivating factor for big data and AI investment. Furthermore, companies are building human resources organigrams to manage their big data and AI initiatives: the appointment of Chief Data Officers has been rising from 12% (in 2012) to 68% of companies having staffed this role in the past seven years.
What are the reasons behind this scenario? Why do big firms fail to implement data-driven routines? One possible explanation could be that often managers are pushed to short-term goal achievement, putting apart significant data initiatives that are supposed to deliver benefits in the long run. Another reason may be a lack of collaboration between their organizations and the innovation labs instituted to research how to facilitate data-driven innovation.
Another critical element for driving change in an organization is the commitment from the whole management body that seems to be rare: several chief analytics and data officers reported to the authors of the survey that senior managers advocating for data-driven transformation culture are rare.
For sure, big data and AI initiative create enormous challenges, in particular, Andrew McAfee and Erik Brynjolfsson highlighted five of them (2012):
First of all, as mentioned before companies need to have leadership in charge responsible for innovating the standard internal procedure coherently: new technologies cannot be implemented as a standalone closed box, but, there is a need for leaders who can spot opportunities, think creatively, understand how the market could evolve and articulate a catchy vision in order to persuade the employers into the embracement of change.
In the second place, while the cost of data collection is falling the cost of complementary assets is rising: skilled data scientist are rare in the job market because modern data manipulation techniques are rarely thought in traditional courses. Rarely the data collected are structured and ready to use, as a consequence, it is essential to have the capability of cleaning, organising and visualising meaningfully large datasets. It is not only about data scientist, but along with software also hardware has to be managed by competent computer scientists in order to be able to handle enormous sets of data. In general, data talents should be able to reformulate business challenges in data queries.
Even if it is getting cheaper and cheaper, technologies still play a critical role. It is essential to keep in mind that these technologies require skills that are new to many data scientists, like for instance being able to integrate different internal and external source of data the company was already using.
Building on the first two concepts, an effective organisation is supposed to align information and decision making. Every organisation should continuously wonder the best way to make sure that the critical decision-makers collect the needed information at the right time and place, and at the same time how to make sure that decision power is allocated coherently near the sources of information available is.
As of last and most important, big-data and ai initiatives cannot exist without company culture. How can we define organisational culture? Organisational culture can be defined as a system of common assumptions, values, and beliefs, which governs how people behave in organisations. These shared values have a strong influence on the people in the organisation and dictate how they dress, act, and perform their jobs. Every organisation develops and maintains a unique culture, which provides guidelines and boundaries for the behaviour of the members of the organisation.
All in all, you do not need big budgets to implement data-driven strategies in your day to day operation, as I tried to explain here.
Cited in this article
- McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J., & Barton, D. (2012). Big data: the management revolution. Harvard business review, 90(10), 60-68.
- Davenport, T. H., & Bean, R. (2018). Big companies are embracing analytics, but most still don’t have a data-driven culture. Harvard Business Review, 6.
- Economist, T. (2017). Data is giving rise to a new economy. The Economist, 5.
- Economist, T. (2017). The world’s most valuable resource is no longer oil, but data. The Economist: New York, NY, USA. ISO 690”