Until recently, Macy’s ran its optimization models for pricing markdowns on Sunday nights so the retailer’s discounts could be in place by Tuesday morning. Today, the giant retailer still runs the algorithms on Sunday, but the new prices are in stores a full day earlier.
“That is huge for them,” said Mike Newkirk, director of industry marketing for SAS, a global provider of business analytics based in Cary, N.C., that counts Macy’s among its retail clients. “It changes their whole business process.”
It’s a transformation brought about not only by getting better answers through big data, but also by getting those answers faster. And that’s made possible by new computing and storage capabilities that allow companies to extract value from a data universe that the McKinsey Global Institute estimates will grow 40 percent a year worldwide through 2020.
Big data is being captured, stored, mined and analyzed across industries. It’s being used for metrics, reporting and forecasting, and for management of sales, marketing, transportation, inventory and risk. Sensors, micro-cameras and embedded computers are used to gather and transmit data for proactive maintenance of everything from trucks to the human body, saving lives and untold billions of dollars.
How big is data becoming? A single jet engine can generate 10 terabytes of data in 30 minutes, and there are some 25,000 airline flights daily. By comparison, the entire U.S. Library of Congress had amassed 235 terabytes of data as of April 2011, according to the McKinsey Global Institute, or MGI.
The sheer volume, variety and velocity of data are major challenges for supply chain executives. In logistics, the data deluge is coming from mobile devices, telematics, electronic onboard recorders, enterprise resource planning systems, RFID tags, smart sensors, Web-based platforms and other sources, said Sundar Swaminathan, senior director of transportation industry strategy and marketing at Oracle.
Oracle’s approach to big data management is tactical and strategic and comprises four stages: acquisition, organization, analysis and decision-making. The latest technologies can process massive amounts of structured and unstructured data at incredible speeds. They can identify means, averages, outliers and trends that can translate into real — and potentially huge — business value.
“You have to build out scalable data acquisition mechanisms, with embedded techniques and large-scale analysis that allow you to identify what has value,” Swaminathan said.
Big data represents a potential new source of competitive advantage for shippers, carriers and third-party logistics providers. By harvesting data for enhanced insights into market trends, cost structures and demand and capacity fluctuations, service providers can operate more efficiently and improve the scalability of their enterprises. They can get ahead of the marketplace by developing new products and services based on a greater understanding of markets, consumers and competition.
Big data and analytics create value by making information transparent and usable. Through data analysis, customers can be segmented by numerous criteria, allowing companies to modify and develop products and services for targeted niche markets.
Big data and analytics drive the kind of experimentation that leads to better decision-making. Aggregated transactional data is especially useful in the logistics industry for forecasting, inventory and asset management, and developing new products and services, said Michael Chui, an MGI principal.
To capture value, companies first should be aware of what data is available. There’s a lot of it out there from many sources. Data amassed from multiple sources, including free and purchased data, tends to have more value than data from a single source. “It takes effort to understand how to capture value,” Chui said. “It’s a learning process.”
Big data isn’t easily managed and poses big challenges. Companies are paying much more for data storage even though storage costs have fallen. Analytics are easier to use, but they’re still complex, especially with streaming real-time data. Significantly, the U.S. alone faces a shortage of as many as 1.5 million analysts and managers of big data, according to MGI.
Data-based decision-making can change the way companies develop, ship, market, warehouse and sell products. As business processes and operations change, the structures of organizations also may have to change.
Are big data and analytics replacing knowledge and experience as drivers of logistics decision-making? Not entirely, Chui said. Both are valuable, but the latter need to be supplemented. “Some say experience or domain knowledge is useless in a world of big data” he said. “I think of it as adding another set of tools to the management kit.”
Data has always been big, but it has become humongous with the advent of social media and mobile technologies, Newkirk said. Data is flowing into organizations with increasing velocity, variety and variability. Big data is, or should be, fast data. Algorithms that once took days for SAS to run on mainframe computers now take minutes or even seconds, allowing companies such as Macy’s to transform their models.
The uses and value of big data vary by industry, but analytics enable companies in all sectors to better determine optimal pricing and inventory levels, recalculate risk, mine customer data for insights that drive new strategies for customer acquisition and retention, identify key customers, maximize the value of mobile technologies, analyze and incorporate data from social media, and determine root causes of errors and exceptions.
Seven of the nine largest railroads in the U.S. and Canada use SAS predictive asset maintenance and forecasting tools. Given the criticality of asset allocation in the railroad industry, advanced analytics and faster processing contribute to service improvements and significant reductions in maintenance costs.
“If railroads can improve by even half of a percent for a particular region, that incremental difference is very important,” Newkirk said.
Earlier generations of analytics products were too complex for the average user. They’ve become more user-friendly, as evidenced by the railroads and trucking companies that use them with no formal analytics training. The software can automatically determine which forecasting methods to use, singly or in combination.
Companies that turn to SAS want quantifiable results. Prior to investing in analytics, clients typically give SAS several years’ worth of historical data and compare test results with actual results.
Analytics can help reduce data acquisition and storage costs by filtering out excessive data. Truck sensors and telematics can capture more than 200 data elements, such as RPM and oil pressure. Do companies really need streaming oil pressure readings, or would five-minute intervals be enough?
SAS can identify how much data companies need for proactive maintenance, and discard the rest. They’re not alone in dumping massive amounts of data. MGI estimates that health care providers discard nearly
90 percent of the data they generate, including almost all the video feeds created during surgeries.
“Too much data can slow you down, so you don’t want to collect it all,” Newkirk said.
As valuable as big data is, you don’t want it to make decisions for you. Data must be interpreted and applied, functions for which humans with experience and good instincts are ideally suited, said Joe King, senior vice president of services and sales for JDA Software Group, a provider of supply chain solutions. “At the end of the day, a person still has to pull the trigger on how much Pepsi to order and where to put it,” he said.
Companies that consider big data mainly in terms of storage increase the likelihood they’ll overspend on hardware and software without getting desired results. Data should be evaluated to the extent that it adds value to an organization, such as leading to improvements in forecast/replenishment, asset allocation or transportation planning.
Because data resides with multiple parties, collaboration is one of the keys to big data management. JDA’s cloud-based services provide vast, low-cost storage capacity, and a collaborative platform for sharing data and analytics, in user-friendly formats that provide forward and backward looks.
Perhaps more importantly, JDA’s veteran supply chain experts are available to help customers interpret and analyze data big and small. They help customers break it down and identify what is relevant to the decision-making process.
Extracting value from big data starts with listening to customers. JDA tries to bridge the gap between technology and the business use of technology by helping customers decide which investments to make in computing power, storage, backup and network.
There’s data on the one hand, and information on the other. Data is raw and essentially useless, like unroasted coffee beans. For many companies, the challenge is in transitioning from a culture of shuttling data around to one in which data is put to work as useful information, said Greg Kefer, vice president of corporate marketing for GT Nexus, a software company that provides a cloud-based collaborative platform for carriers and shippers.
Data volume swelled with global outsourcing. As the number of supply chain partners grew and technology evolved, a problem emerged. About 80 percent of the data that companies need resides with supply chain partners. Technical connectively with suppliers can be difficult and expensive.
A key part of the solution is linkages, for supplier connectivity and big data management. GT Nexus provides a Web-based platform that allows the exchange of transactional shipping data.
The cloud has benefits beyond storage. By enabling the exchanging and rationalizing of data, cloud-based platforms bring economies of scale and help instill a multi-enterprise, collaborative mindset among supply chain partners.
Supply chain information is valuable only when it’s flowing between companies in a standardized common format, which is enabled in a cloud environment. “Your supply chain partners are doing a lot of the heavy lifting,” Kefer said. “To get them moving toward the same goals, you have to put information in the middle of the network.”
There’s plenty of room for creativity in the big data era, said Greg Smith, director of freight and logistics for Oracle’s industries business. Once data is collected and analyzed, it still must be cross-referenced and applied to any number of operations and processes. Decision-makers still must know the business.
“There are all kinds of options for applying creativity and nonlinear thinking in using massive amounts of data,” Smith said.
Online retailers were early leaders in integrating real-time social and mobile data with historical data. That data, aggregated and analyzed, is used for predictive analysis, strategic network design, asset allocation and real-time inventory routing.
The latest big data technologies take unstructured data from telematics, mobile devices and other sources and apply it to tactical and strategic decision-making in near real-time. Replacement parts for airplanes or trucks can be positioned prior to the arrival of the plane or vehicle based on data, for example — a type of move Smith said “would be impossible without these new technologies.”
Contact David Biederman at email@example.com.