Background
Walmart is not only the largest retailer in the world, but it is also the largest company in the world, with over two million employees and 20,000 locations located in 28 different countries.
It should not come as a surprise that a corporation of this size would see the benefits of data analytics at an early stage in their development. In 2004, when Hurricane Sandy struck the United States, it was discovered that when data was reviewed as a whole, rather than as separate individual sets, surprising insights were revealed. This discovery was made in light of the fact that when Linda Dillman attempted to forecast the demand for emergency supplies in preparation for Hurricane Sandy, she came upon some data that surprised her. As a result of the forecast for severe weather, there was a surge in demand for strawberry Pop Tarts, in addition to flashlights and other emergency supplies. During the year 2012, while Hurricane Frances was bearing down on Florida, a considerable quantity was sent to retailers that were in the line of the hurricane.
Since then, Walmart has significantly increased the size of their division devoted to Big Data and analytics, which helps to ensure that the company stays at the forefront of their industry. In 2015, the company announced their aim to construct the world’s largest private data cloud, which would have the capability to process 2.5 petabytes of data in an hour.
What kind of problems may be solved by utilising Big Data?
Supermarkets are responsible for the daily sale of millions of products to millions of customers. It is a very competitive industry that satisfies the need of a massive population in the developed world for essential items. Markets compete with one another not just on the basis of pricing, but also on the basis of the service they provide to customers and, most crucially, on the basis of how handy their locations are. Finding a way to deliver the right things to the right people at the right time presents enormous logistical hurdles. In order to price things in a manner that is competitive, close attention to every single cent is required. Customers who learn that they cannot obtain all they require in a single, accessible location are more likely to purchase elsewhere.
Applications of Big Data
In 2011, Walmart established @WalmartLabs and their Fast Big Data Team in order to investigate and deploy new data-led initiatives across the business. This was done in response to a growing awareness of how data could be used to understand customer needs and provide them with products they wanted to buy.
As a direct result of implementing this strategy, the company’s headquarters in Bentonville, Arkansas now has a state-of-the-art analytics hub called The Data Cafe. In addition to monitoring the other 200 streams of internal and external data in real time at the Cafe’, the analytics division is able to maintain track of a staggering 40 petabytes of sales data from the weeks before.
The Senior Statistical Analyst of Walmart, Naveen Peddamail, was quoted as saying, “If you can’t obtain insights until you’ve analysed your sales for a week or a month, then you’ve lost sales inside that time.” This highlights the significance of real-time data analysis in increasing the efficiency of an organisation.
We want to get the word out to our business partners as quickly as we possibly can so that they can react and reduce the amount of time it takes for the entire process to be completed. Analytics has the ability to both anticipate and react to upcoming occurrences.
Teams from all throughout the organisation are encouraged to bring their data challenges to the Cafe’, where they can work together with the analysts to figure out a solution to the problem. In addition, the organisation has a system that monitors key performance indicators and sends out automated notifications when they reach a predetermined threshold. At this point, the teams responsible for those indicators are urged to talk with the data team about ways to make things better.
Peddamail offers a team from a grocery store as an example of a group that is seeking to determine why a generally popular item has suddenly seen a decline in sales. As soon as the data from the Cafe was in the hands of the analysts, it was obvious that the decline was due to an error in pricing that had been made. After a couple of days of realising the error, it was corrected, and business went back to its usual state.
In addition, it is possible to monitor sales in many locations in real time. Peddamail recalls that during one Halloween, analysts were monitoring the sales numbers of novelty cookies and found that in some places, the cookies were not selling at all. This caught the attention of Peddamail’s employer, who decided to investigate the situation. Because of this, the merchandising staff in the stores were notified, and they realised very quickly that they hadn’t even loaded the shelves with the products. Certainly not the most complicated algorithm, but without access to real-time data, it is difficult to implement.
The Walmart Social Genome Project is another programme that monitors conversations taking place on public social media platforms in an effort to anticipate the kinds of goods that shoppers would purchase. In addition to having their very own search engine, which is known as Polaris, and being able to analyse the search terms that customers enter on their websites, they also have a service known as Shopycat, which is designed to predict how people’s shopping habits are influenced by the shopping habits of their friends (again, using social media data).
Outcome from Investigation
According to Walmart, the amount of time it takes to get from identifying an issue in the data to suggesting a solution has been cut down to something in the neighbourhood of 20 minutes thanks to the Data Cafe’ technology. In the past, completing this task would often take between two and three weeks.
Employed Information
The Data Cafe’ is driven by a database that stores 200 billion rows of transaction data, and that’s only for the data pertaining to the most recent few weeks.
In addition to this, it compiles data from two hundred additional sources, such as reports on the weather and economy, data on telecommunications and social media platforms, statistics on gas costs, and a database of events that will take place in close proximity to Walmart stores.
Notable Particulars of the Study
The real-time transaction database that Walmart uses contains 40 petabytes’ worth of information currently. In spite of the massive magnitude of this amount of data, we are just including the transactions from the most recent weeks’ worth of data because that is where the value lies in terms of real-time analysis. For the retailer, Hadoop serves as a centralised data warehouse, containing records from its physical stores, its online branches, and its corporate offices (a distributed data storage and data management system).
The goal of the strategy is to ensure that a wide variety of users have access to the information contained inside the company’s databases. As a result, CTO Jeremy King has given the plan the name “data democracy” due to the fact that it may be utilised by any member of the organisation. After the broad adoption of the distributed Hadoop architecture in 2011, there came a time when analysts started to worry that their capacity to adequately analyse the data would be hindered by the fast expanding volume of data. As a consequence of this, a plan for “intelligently managing” the collection of data was decided upon. This plan required the establishment of a number of different systems in order to clean and organise the information prior to archiving it in a permanent location. Other technologies such as Spark and Cassandra, in addition to programming languages such as R and SAS, are utilised in the construction of analytical programmes.
Challenging Circumstances
The ambitious analytics operation that was planned presented a challenge for a quickly growing firm like Walmart because it was difficult to find the right employees with the proper talents. According to a recent poll conducted by experts at Gartner, more than half of firms believe that they do not possess the essential competence to carry out Big Data analytics. This problem is not specific to Walmart in any way.
One of the resources that Walmart utilised in its search for a solution was Kaggle, a website that hosts competitions in data science.
The users of Kaggle were given a challenge in which they were asked to predict the effect that Christmas sales and stock-clearing sales would have on the sales of a number of different products. Candidates whose models showed the greatest consistency with the retailer’s actual data were sought after to fill open positions on the data science team at Walmart. One of the people who took part in the competition and afterwards found job at Walmart is Naveen Peddamail, whose observations I have integrated into this section. Naveen is currently employed by Walmart.
All new analysts are required to participate in Walmart’s Analytics Rotation Program for training. They gain an understanding of the numerous business uses of analytics by taking part in a rotation among the various analytical teams the organisation has.
According to Mandar Thakur, a senior recruiter for Walmart’s Information Systems Operation, who shared this information with me, “The Kaggle competition created a lot of interest in Walmart and our analytics group.” The most beneficial aspect was that it demonstrated how we are making strategic use of the data that Walmart creates and stores.
Conclusion and Takeaways
Supermarkets are complex creatures that are made up of a large number of distinct subsystems that all work together to function at high speeds and are subject to ongoing change. The fact that this is the case makes them an excellent candidate for the incorporation of big data analytics into their firm.
The degree of rivalry that exists between different companies is one of the most important determinants of how successful each one is. When it comes to data-driven initiatives such as loyalty and reward programmes, Walmart has always been one step ahead of the competition. By making a full commitment to the most recent developments in real-time, responsive analytics, Walmart has demonstrated their intention to maintain their position at the forefront of the retail industry.
Walmart has demonstrated that traditional brick-and-mortar retailers can reap just as many benefits from cutting-edge Big Data research as their more high-profile online competitors, such as Amazon and Alibaba. This is the case because to Walmart’s use of this technology.
It would appear that consumers, either out of habit or out of personal preference, are still willing to get in their cars and drive to shops rather than use the more convenient options that are available to them today. These options include online shopping, in-store pickup, and grocery delivery services. Which implies there is still plenty of opportunity in the market for innovative enterprises to improve efficiency and enhance the consumer experience, and those that do so will likely find significant success in the market.
