Big data has become one of the most important aspects of supply chain management. The concept of big data refers to the massive data sets that are generated when millions of individual activities are tracked. These data sets are processed to yield insights that help inform managerial decision-making. Supply chains in particular have leveraged big data because companies have been able to develop technology to not only capture hundreds of millions of data points, but to process them in meaningful ways to eliminate waste and promote efficiency in the supply chain systems. This paper will examine the concept of big data, how it has arisen and come to dominate supply chain management, and look at the different ways big data is transforming the supply chain function. Lastly, the paper will take a closer look at the future for big data with respect to supply chain management. As it becomes easier to gather data, and as there are diminishing returns to statistical robustness as the number of data points increases, are the competitive advantages of big data going to diminish?
The field of logistics management was focused on controlling the flow of materials, in-process inventory and finished goods through a company’s system from the time that it enters the system until the time that it leaves the system (Cooper, Lambert & Pagh, 1997). As the field became more strategic in nature, it came to encompass other issues, such as sourcing materials and building in redundancy (Cooper & Ellram,1993). More than simply moving things from point A to point B, the field became holistic in nature, where the quality and price of goods were factored into purchasing decisions as well as the logistics of getting those goods to the right place at the right time. Driving this change was the move towards a globalized marketplace. Globalization increased the complexity of the supply chain, adding longer transportation routes, border wait times, currency exchange, duties and tariffs, and a host of other variables that now had to be taken into consideration – logistics has remained important but it always viewed in context with the rest of the supply chain.
The concept of big data really began to arise in the 1990s but has become increasingly important since that point. Big Data refers to the use of very large data sets to enhance managerial decision-making. The concept of big data arose as technology has developed to allow businesses to capture enormous data sets, and process them relatively easily (Boyd & Crawford, 2012). Companies have long collected data at a rudimentary level. Loyalty programs and credit cards represented an evolution in the ability of companies to collect data and distill that data into consumer spending habits. This information is then made actionable by letting companies understand more about buying patterns. Big data is similar, but with a lot more data. One of the major advantages of big data is that it allows for complex problems to be solved. A modern supply chain can be exceptionally complex, and one of the important things about this complexity is that no one person can effectively make all the decisions – decision-making tools are needed that can ensure not only consistent decision-making across the company but coordinated decision-making as well (Hult, Ketchen & Slater, 2004). It is these coordinating mechanisms where the true power of big data lies – being able to identify things and make decisions that an entire team of humans working without big data would probably never be able to identify (Fugate, Sahin & Mentzer,2005). Once big data gets to that point, a company can generate true competitive advantage. And when a company is large enough that is has a data advantage, it will be able to sustain that advantage, which is why there has been such a rush in recent years with respect to big data.
As the concept was being fleshed out in academia, businesses were just starting to learn what they could do with all of the information that they were collecting – and one of the applications was to move away from marketing and use data to make decisions about the supply chain (McAfee & Bryjolfsson, 2012).
One of the first steps that companies needed to make was to hire data scientists – the sort of people who could process these data sets and derive useful information about them. Data scientists suddenly became popular, for their ability to take vast quantities of data, and derive actionable findings from that data (Provost & Fawcett, 2013). At the heart of the drive to adopt big data is competitive advantage. Companies have invested in their data programs because they can derive significant advantage from big data under two conditions. The first is that larger companies have access to more data than smaller companies. The incremental cost of data acquisition is lower, and the company’s ability to use that data in decision-making is theoretically better. The second is that even among larger companies, there are first-mover advantages to be had. This is evident in the supply chain, especially among companies that are competing on price. Using the classic example of Wal-Mart, one of the leaders of data-driven supply chains, the company competes on offering the lowest prices, as do most of its competitors. Thus, if it can lower the cost of getting goods to its stores, it can pass those savings along to customers. There is opportunity for competitive advantage under that scenario, if cost leadership is the chosen strategy. Even when cost leadership is not the strategy, making the groundbreaking decision early puts a company in a better competitive position than its competitors (LaValle, et al, 2010).
As the largest non-oil company in the world, Wal-Mart is looked to as a leader, so the fact that they were first movers on the use of big data in supply chain management has ensured that the rest of retail – and other industries as well – have followed. Some of the technologies that Wal-Mart has adopted allow the company to track its inventory from when it leaves the supplier –if not before – all the way through the logistics channel. Once Wal-Mart takes possession of the good, that good is scanned regularly through the process. The company’s trucks are tracked via satellite. Stores use automatic re-ordering triggers to ensure that goods can be received as soon as they are needed. The goals of all this are to lower inventory holding costs by reducing the amount of inventory that stores have. Goods are turned over more quickly, because Wal-Mart receives them only days before it expects to sell them. Big data plays a significant role in ensuring that this process can be achieved. There are a couple of key areas highlighted for big data in supply chain management.
Demirkan & Delen (2013) note that data, and how a company uses its data, is one of the ways it can truly differentiate from its competitors. It can be difficult to truly and consistently attract superior talent, and it can take time to move the needle on brand image, but data has become a popular means of finding competitive advantage largely because it is new, and firms in many industries are basically in a data arms race to find innovative ways to use their data to extract competitive advantage.
The first is predictive analytics. Data science often focuses on using past events to predict future ones, and that is one of the main uses for big data in supply chain management. For example, if Wal-Mart in Smalltown, OH is running out of shovels at the end of February, and it takes twenty days to order new ones from China, including manufacturing and shipping times, three things can happen. The company can order a lot of shovels and ensure that they have supply. If spring comes, those shovels will sit in a warehouse until next November. They could also run out of shovels, but a late-season snow could leave demand on the table if the store lacks inventory. Modelling both weather patterns and local buying patterns can help the company to settle on demand. Even when weather is not a factor, the company can examine past purchasing patterns to set order quantities. The earlier it can set these quantities, the better response it can get from suppliers. Wal-Mart knows already what the normal amount of hot dogs it sells on the 4th of July, for example, so it can feed that information to its suppliers to ensure that they have those dogs at the Wal-Mart warehouse, exactly in the quantity Wal-Mart needs.
Predictive analytics is used in supply chain management to take the variability out of the system as much as possible. Inventory usage is reduced, as is the potential for waste, especially with perishable goods. The chances of disappointed customers is also reduced. It is almost impossible – and certainly it is impossible for a company like Wal-Mart – to have exactly everything delivered exactly when the customer needs it. That means that there is always room for improvement. The pathway to improvement lies with bigger data sets, better analytics, and at scale even small incremental gains in the robustness of data or the ability of the company to analyze the data can yield meaningful financial gains (Waller & Fawcett, 2013).
But using data for something like predictive analytics – managerial decision-making, essentially – requires having good data, lots of it, and the means by which to process it. This is where larger companies enjoy scale advantages in big data. First, the technology to track events is not necessarily cheap. It can involve scanners, and certain involves large amounts of servers, routers, cloud storage – a lot of hardware. Larger companies are at an advantage in buying this hardware but they also have advantage in that they have many more data points. Wal-Mart can estimate sales because it has several years’ worth of sales, and can break these down by product, store, day, or even time of day. And instead of guessing for decision-making, the company’s managers can look at the data and make the decision that on average delivers the greatest outcome. Data replaces decision-making heuristics when the data is sufficiently robust. Because the transference of big data relies on the Internet and communications technology infrastructure, that ICT infrastructure becomes a risk point for many companies but it also becomes a critical point of investment for companies that work with big data – how fast can the data collected on-site make its way to the decision-making tools matters in many businesses where time is of the essence in decision-making (Lu, et al, 2013).
Predictive analytics has more than just value in ordering; it can help businesses to identify trends more quickly. This can be critical to advantage in some industries. Think of a “fast fashion” retailer – it needs to identify trends as soon as possible to get its knock-off clothes onto the market while the fashions are still fresh. Instead of anticipating, which is fraught with error, it can react to trends that have been verified with data. By understanding buying patterns and market cycles, companies can make better choices about what they make and when. This, in turn, is important to the supply chain, because companies also need to know what they need to produce their goods, and when. If there are fluctuations in availability, of if there is any variability among suppliers, then big data has the ability to point these factors out, and give the company an opportunity to deal with them proactively (Wang et al, 2016).
When the concept of big data was first being elaborated, it promised major impact on business. Instead of guessing, firms would be able to make data-driven decisions that would reduce error, reduce waste and improve speed. As firms understand how to gather the data that they need, and to process it, they become more adept at this, big data has a bigger impact. Some leading firms have used the predictive powers of big data to help with their marketing. Amazon, for example, will recommend products to its customers based on what they have viewed and what they have purchased. Netflix does the same thing – and thereby encourages binge-watching of its shows. Both of these companies have become leaders in their respective businesses, and Netflix has done this specifically in the era of big data, by using that data to foster brand loyalty (Chen, Chiang & Storey, 2012).
If a company ends up as a first mover in big data, it will be able to gain advantage, and in many cases will make market share gains. Amazon faced a challenge from Wal-Mart a few years, ago, but has made use of big data to driver a high level of brand loyalty, while Wal-Mart fell short on its ability to use big data on the marketing side of its business. Netflix faced threat when major studios wanted to charge more for their content – so it created its own content and even more importantly used big data to improve the information architecture of its platform, allowing people to find content they want to consume. This increased the value of Netflix for many customers, thereby driving business value. Google uses data to target ads better, and charge its customers a premium. Customers are willing to pay more for a Google ad because they know that they will get more traction.
So it is important that companies understand data on a conceptual level. One of the reasons that this is so important is that data today comes from a variety of different sources. This ties back to the concept of supply chain management, where the supply chain is a highly-integrated system with many parts from one end to the other. Understanding how the different variables within this system interact so that supply chain systems can be redesign in a more optimal way. Consider the way FedEx used the hub-and-spoke model before passenger airlines thought to do so. Consider how Wal-Mart designed its entire logistics network around lowering the amount of time that it takes for stores to restock. There are different approaches, but the innovations should derive from analysis of the data that identifies areas where the company might potentially perform better. Maybe sourcing goods from a certain country is no longer the lowest cost method, given how long it takes to get those goods to market. There are different ways of conceptualizing a supply chain, and now that companies are able to use data analytics to make those decisions, it is likely that many firms will start to restructure their supply chain (Tan et al, 2015). Total cost will become more important, but so too will overall responsiveness. Sourcing locally might provide a company with the responsiveness it needs for certain products that have higher variability in demand, for example.
While there is presently a shortage of people who have strong data analysis skills, these skills are becoming increasingly in demand, and schools are starting to train more students in the use of big data. One of the important factors here is that data has become much cheaper – big data arises because the cost of acquiring any given data point is very small, and continuing to shrink. Retailers in particular have been able to reduce their cost of data acquisition dramatically (Chen, Chiang & Storey, 2012). Key to learning about the use of data is how to identify the problems that can be solved with data, how to match the data you have with the problems that you want to solve, and then developing systems to acquire the data that you do not have. At this high level of understanding, a company that thinks a good data game is in a much better position because having the right data matters just as much as knowing what to do with that data (Hazen, et al, 2014).
The cloud and the Internet of Things (IoT) are driving a lot of changes in the way companies do business, and big data is playing a significant role in this restructuring of business. Zaslavsky, Perera and Georgakopoulos (n.d.) note that data is becoming a service function, with companies preparing to offer the means by which data can be acquired as a service, and the same for data analytics. We know that data is cheap to acquire, but combine that with lowering costs of processing data and there is a business model here, as well as one that focuses on using data to enhance business. The IoT will be more engaged in the data gathering process. For example, while convention supply chain data gathering might involve devices at the store level, the IoT might drill down further, to the individual level. Ovens could know how many people are cooking a frozen pizza and this information could be sold to frozen pizza makers, so that they can get a better sense of not only the performance of their products but of their competitors as well. This is the example a hungry person thinks up, but with more devices having some internet capability, it seems likely that type of application will emerge. Tesla is already a leader in gathering data about driving from its cars (Edelstein, 2016 & Hull, 2016).
Another progressive idea is that of big data benchmarking. If it is possible to buy and sell data to the point where a company can learn about the best practices at all levels for multiple companies in an industry, that would be incredibly valuable information to any firm in that industry. With the data explosion has come a rapid pace of innovation in the gathering and use of data. With this will come firms that buy and sell data, without actually gathering their own. Until now, data has largely been proprietary in nature, as a key source of sustainable competitive advantage, but as the cost of data acquisition declines, this might not be the case much longer. Secondary markets for data are already emerging and ultimately data will become commoditized – this process might take many years but it will happen and that will make for interesting analysis about the future of data , in particular the extent to which data can continue to be a driver of competitive advantage going forward (Ghazal et al, 2013).
Finally, big data is also becoming a competitive weapon, which makes security of big data a major issue. Companies that gather and own data sets, and in particular the usable intelligence that has been gathered from those data sets, are increasingly going to be targeted with hacks. Security of big data is going to be an issue going forward. This is especially true of supply chain data, because that is powerful business intelligence. So it will be necessary, especially when using remote or cloud solutions, that data security is paid attention to, as the more that data becomes a source of competitive advantage the more at risk it will likely be.
Supply chain management had already emerged as a force in business, a holistic view of the supply chain that started with logistics but incorporated purchasing, product design and marketing as well, in order that supply chain decisions were not just based on a simply understanding of cost, but a complex one that took into account a number of different variables. Ultimately, supply chain management required significant amounts of data to be effective, and this realization occurred at just the time that managers realized they had the ability to gather, store and process data much more cheaply and easily than before. The transactional value of data grew at precisely the time that the acquisition cost declined.
Data is typically used to aid in managerial decision making. Some companies have focused on the low-level decision where they seek out incremental gains on repeatable processes, knowing that those processes and other companies have sought insight that will allow them to completely transform their supply chains. Big data has become so important because the companies that are using it tend to be the market leaders. It is apparent that there is a scale value to data, which means that the largest companies, ones that have more data and lower data acquisition costs, are going to have sustainable competitive advantage. This has driven demand for data experts, such that there is a shortage of such individuals.
Big data is going to continue to influence supply chain decision-making. There will be more points at which data is gathered, and the cost of processing data will continue to drop. There will still be a strong need, however, for talent that can conceptualize how that data should be used – after all, companies need to ask the right questions to get the most out of their data. If they can do that, they can sustain competitive advantage.
In addition to there being an increasing ability to gather data, another reality is that many companies are going to be in the business of selling data. A company like Google sells data by proxy with its advertising, but as data becomes commoditized, the market for data will become more developed. An interesting aspect of this is that competitive benchmarking will be more common with respect to data practices. Firms will need to be careful to ensure that their proprietary data is secure so that they can maintain the competitive advantages that their data is giving them. If they can, then they can gain first mover advantage for tactics that deliver incremental gains, or the complete overhaul of a system to take advantage of something gleaned from the data.
Boyd, D. & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication and Society. 15 (5) 662-679.
Chen, H., Chiang, R. & Storey, V. (2012) Business intelligence and analytics: From big data to big impact. MIS Quarterly 36 (4) 1165-1188.
Cooper, M. & Ellram, L. (1993). Characteristics of supply chain management and the implications for purchasing and logistics strategy. International Journal of Logistics Management 4 (2) 13-24.
Cooper, M., Lambert, D., & Pagh, J. (1997). Supply chain management: More than a new name for logistics. The International Journal of Logistics Management. 8 (1) 1-14.
Demirkan, H. & Delen, D. (2013) Leveraging the capabilities of service-oriented decision support systems: Putting analytics and big data in cloud. Decision Support Systems. 55 (2013) 412-421.
Edelstein, S. (2016) Tesla’s autonomous-car efforts use big data no other carmaker has. Green Car Reports. Retrieved April 1, 2017 from http://www.greencarreports.com/news/1108065_teslas-autonomous-car-efforts-use-big-data-no-other-carmaker-has
Fugate, B, Sahin, F. & Mentzer, J. (2005) Supply chain management coordination mechanisms. University of Tennessee. https://www.researchgate.net/profile/Brian_Fugate/publication/228349679_Supply_Chain_Management_Coordination_Mechanisms/links/0c96051e3eaaa0280f000000/Supply-Chain-Management-Coordination-Mechanisms.pdf
Hull, D. (2016) The Tesla advantage: 1.3 billion miles of data. Bloomberg. Retrieved April 1, 2017 from https://www.bloomberg.com/news/articles/2016-12-20/the-tesla-advantage-1-3-billion-miles-of-data
Hult, G., Ketchen, D. & Slater, S. (2004). Information processing, knowledge development and strategic supply chain performance. Academy of Management Journal. 47 (2) 241-253.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. & Kruschwitz, N. (2010). Big data, analytics and the path from insights to value. MIT Sloan Management Review. http://sloanreview.mit.edu/article/big-data-analytics-and-the-path-from-insights-to-value/
Lu, T., Guo, X., Xu, B., Zhao, L., Peng, Y., & Yang, H. (2013). Next big thing in big data: The security of the ICT supply chain. IEEE Computer Society. Retrieved April 1, 2017 from http://diyhpl.us/~nmz787/pdf/Next_Big_Think_in_Big_Data__the_Security_of_the_ICT_Supply_Chain.pdf
McAfee, A. & Brynjolfsson, E. (2012). Big data: The management revolution. Harvard Business Review. Retrieved April 1, 2017 from http://www.rosebt.com/uploads/8/1/8/1/8181762/big_data_the_management_revolution.pdf
Provost, F. & Fawcett, T. (2013) Data science and its relationship to big data and data-driven decision-making. Big Data. 1 (1) 51-59.
Tan, K., Zhan, Y., Ji, G., Ye, F. & Chang, C. (2015) Harvesting big data to enhance supply chain innovation capabilities: An analytic infrastructure based on deduction graph. International Journal of Economics. 165 (2015) 223-233.
Waller, M. & Fawcett, S. (2013). Data science, predictive analytics, and big data: A revolution that will transform supply chain design and management. Journal of Business Logistics. 34 (2) 77-84.
Wang, G., Gunasekaran, A., Ngai, E. & Papadopoulos, T. (2016). Big data analytics in logistics and supply chain management: Certain investigations for research and applications. International Journal of Production Economics. 176 (June 2016) 98-110.
Zaslavsky, A., Perera, C. & Georgakopoulos, D. (no date). Sensing as a service and big data. https://arxiv.org/ftp/arxiv/papers/1301/1301.0159.pdf
