Tough road ahead for freight data standardization

Tough road ahead for freight data standardization

Technology and shipping.

Data standards in logistics have long been elusive and have been only marginally more pervasive than in other sectors, despite industry-led and government-led attempts to harmonize data transfer between organizations. Photo credit:

Correction: An earlier version of this story misidentified Craig Fuller as the CEO of the Blockchain for Transport Alliance (BiTA). The co-founder stepped down from the role in the fall.  

The container shipping industry will struggle to transform momentum around blockchain and associated data standards into reality. That’s due to the sheer number of oft-competing consortiums and entities, questions around the neutrality of the applications, and broader confusion about the technology.

Over the last 15 months, the various initiatives — including ones led by Maersk/IBM, CargoSmart, APL/Accenture, and the Blockchain in Transport Alliance (BiTA) — have set bold goals, promising to bring a greater degree of reliability, transparency, and efficiency to shipping processes that shippers have been told need to be digitized. The thinking goes, if blockchain is the key to this digitization, standardization must come first.

But along the way, fissures have appeared, ranging from member frustration in standards-setting organizations to deeper concerns about the viability of the technology in the container shipping space. And the plethora of entities is also creating headaches for shippers wondering if standards are even required for blockchain (or worth waiting for), and which, if any, commercial platforms will take hold.

These broader questions strike at the heart of the spread of any technology: is adoption driven by the value of a commercial product, or by neutral standards that define the basis by which parties will and won’t compete? What’s more, the growing pains around how blockchain in logistics will evolve is compounded not just by the variety of standard-setting entities, but by most people’s foundational lack of understanding of the technology.

Data standards

Data standards in logistics have long been elusive and have been only marginally more pervasive than in other sectors, despite industry-led and government-led attempts to harmonize data transfer between organizations. That standards initiatives keep cropping up in logistics speaks to their ultimate allure — a utopian picture of an industry all conforming to a single data format.

That desire seems especially strong when it comes to blockchain’s role in logistics. As potential users of blockchain get their head around what it is and what it can do, there seems to be a parallel drive to redraw the logistics data landscape in a less complicated way through data standards.

But is that realistic? Do the scars of previously unsuccessful standards battles — some as basic as a single industry-wide bill of lading format — show that the industry is too fragmented to standardize? And more to the point, is it not the role of software providers to standardize data formats?

To be clear, logistics is already governed by data standards — whether as singular as an automatic identification system (AIS) code on a vessel or as varied as the electronic data interchange (EDI) message sets that govern the bulk of shipping processes today. The most famous standard in shipping is the container — whether 20-foot or 40-foot — but that’s less of a data standard than a physical standard, making it more analogous to railroad track gauges than blockchain data formats.

But true, broad data standards are relatively few and far between in commerce. The GS1 barcode is an example of a standard that has stuck on a global retail supply chain level, while data and transactions in the financial industry are largely governed by message sets developed by the banking cooperative the Society for Worldwide Interbank Financial Telecommunication (SWIFT).

In general, standardization processes have hit their mark when they come gradually, via growth in peer-to-peer commercial interactions as opposed to distinct standards initiatives.

In a logistics context, the lack of progress thus far in blockchain-related standards has manifested in skepticism of and frustration with the most prominent blockchain entities to emerge, namely Maersk’s and IBM’s TradeLens platform, and the BiTA. Those two entities, in a sense, represent two sides of the same coin: TradeLens a commercial platform intended to attract a wide range of stakeholders related to international shipments, and BiTA a neutral standards-setting organization designed to coalesce predominantly domestic freight interests around common data and protocol standards.


But both have hit familiar roadblocks. TradeLens has faced public and private skepticism over its ability to be a neutral platform for the shipping industry, given Maersk’s role in developing it, and other carriers’ unwillingness to join. BiTA, meanwhile, has struggled to galvanize consensus around singular standards from its wide range of 490 members, while it has also evolved its brief to develop use cases for the emergent technology rather than just standards.

“Standards will come, but blockchain standards for an insurance verification will be different than a load-tracking standard,” said Craig Fuller, a co-founder of BiTA, who stepped aside as managing director in the fall to focus on Freightwaves, told Freightwaves, an online news provider, has a contract with BiTA to manage event planning, content, and resources to assist in building the latter's community.

“We’re focusing the energy around those uses cases, with 10 to 20 companies working on insurance, finance, cargo tracking. You need specific outcomes for those verticals,” Fuller said. 

And TradeLens and BiTA are far from the only games in town.

In mid-November, five of the world’s top six ocean carriers — Maersk, Mediterranean Shipping Co., CMA CGM, Hapag-Lloyd, and Ocean Network Express — said they were discussing the joint creation of open-source information technology standards, not to develop or operate a digital platform, but instead “to ensure interoperability through standardization.” The standards are not blockchain-specific and are still in development.

Separate from that group, two other ocean carrier-led blockchain consortiums have emerged to challenge TradeLens: one led by the logistics software company CargoSmart, Oracle, and five carriers predominantly from the Ocean Alliance (CMA CGM, Cosco Shipping and subsidiary brand OOCL, Evergreen Marine, and Yang Ming), and another led by Accenture, APL, AB InBev, and Kuehne + Nagel. More pertinent to shippers, the agriculture corporations Archer Daniels Midland, Bunge, Cargill, and Louis Dreyfus in October announced plans to jointly study how blockchain technology might improve their industry, including around global shipping processes.

But some context is required. This is early days for blockchain as a technology, and it might be unfair to expect major progress yet on standards.

So while all this activity might seem like advances toward a future where critical data elements in domestic and international shipments will be better defined among all carriers and logistics service providers (LSPs), to shippers, these standards-setting initiatives can also be confusing and counterproductive.

To start, there’s the question of what exactly is being standardized. Is it the industry’s more superficial data elements (such as port abbreviations or liner service names) or the more foundational, programmatic elements of interconnected systems, referred to in software development as protocols?

Most software developers see the latter as being far more important in driving blockchain adoption — really the adoption of any technology — while standardization of terms such as port or liner service names is a more straightforward problem that could be resolved if ocean carriers agreed in principle to standards. Many software companies also see that type of data normalization as a core part of what they provide.

But some blockchain investors are encouraging developers to build tools that are even protocol agnostic, to account for the differences in the various protocols being developed in parallel (some freight-specific).

In the context of the growing interest and investment in blockchain as a viable technology, the question now is where do standards fit in that discussion? Or more precisely: is any single organization equipped to develop standards on behalf of the entire industry (even a subset of the industry); and does it even make sense to expend resources to try to develop standards for blockchain, or would the industry be better off letting commercially viable products determine what blockchain standards are organically?

A 2016 example

One only needs to look back to 2016, as the liner shipping industry wrestled with the International Maritime Organization’s (IMO’s) verified gross mass (VGM) mandate on container weights, to realize how difficult it is to get even a subset of parties to agree to a standard. In the months leading up to implementation of the rule, there was widespread uncertainty about how shippers would weigh their containers and then convey those weights to container lines in a timely manner.

It took a number of software companies, notably the ocean freight booking and documentation platform INTTRA, and shippers associations to take the reins and get the industry to agree around a workable format to comply with the rule.

VGM was an example of a government-imposed mandate that forced the industry to agree on a standard, but other, non-mandated attempts to standardize even basic elements of a shipment have largely failed. In the mid- to late 2000s, INTTRA, the most widely used platform for freight forwarders and shippers to submit booking requests and shipping instructions, tried to standardize various documents across all the carriers it touched: first a stab at standardizing the bill of lading format and then a push toward standardized e-invoicing. Neither effort ever gained sufficient buy-in from carriers to achieve standardization.

Indeed, the battlefield of past logistics standardization efforts is strewn with casualties.

For instance, there’s Bolero, an entity formed in 1998 by the TT Club and SWIFT to overhaul freight payment and letters of credit. Some veteran industry observers liken the hype around blockchain to Bolero’s introduction in the early 2000s. But while Bolero is an ongoing concern — it rolled out a (non-blockchain) electronic bill of lading product in March 2018 with Evergreen Line — it’s hardly a household name in liner shipping, much less the driver of new industry data standards.

In other corners of the freight world, it’s been similarly difficult for standards to take root. The International Air Transport Association’s (IATA’s) Cargo IQ program, designed in part to migrate paper airfreight bills to an electronic format through agreed-upon standards among airlines and freight forwarders, was originally called Cargo 2000. The program took so long, it had to be renamed.

A few logistics data standards have managed to stick: notably the 60-year-old National Motor Freight Classification (NMFC) rating system for tariff classification for the less-than-truckload (LTL) industry; and the Standard Carrier Alpha Code (SCAC), which all road transport companies, ocean carriers, and freight forwarders must maintain for government identification purposes. These standards, however, date back to an earlier era of transportation, when standards were easier to achieve because they arose within the context of a still heavily regulated industry.

Another, of course, are Incoterms, developed by the International Chamber of Commerce and revised every decade. Incoterms define the responsibilities of sellers and buyers for the delivery of goods under sales contracts and are a broadly accepted set of standards in logistics.

The most prevalent standard in freight is EDI, which governs a wide range of processes and status events across all transportation modes, not to mention processes outside of logistics, such as invoicing. Although some decry EDI as outdated, rigid, and inefficient, shippers and their service providers still use it widely and few see it going away anytime soon. Adoption of EDI as a standard got a huge boost from shippers, specifically the big three US automakers, who in the 1980s mandated the use of EDI for invoices by its suppliers.

The question looming over the blockchain discussion is which of three potential paths standards will take, said Daniel James, director of product at the enterprise collaboration software provider Sedna Systems and former director at IHS Markit (parent company of, where he led the development of blockchain products for the financial industry.

“There can be a true nonprofit group that has been bestowed the credibility and moral authority to lead,” he said. “Basically, a group that is widely believed to be neutral and have its interest aligned with the greater good of an industry. That authority can be bestowed by a neutral arbiter [such as a government].”

The second route, James said, would be “a solution that explodes in the absence of standards, a solution that is so popular that the standards it creates just become the de facto standard.”

The third route would be where small groups of companies, or single big players, publish standards and then try to use their weight to push them. “Typically when this happens I think you end up with a fragmented market and lots of different standards,” he said. 

In the context of blockchain in logistics, the effort launched in November by the five liner carriers to push broad data standards falls in the third category, the consortiums led by Maersk/IBM and Cargosmart fall into the second category, and BiTA falls into the first (although BiTA is not nonprofit and requires members to pay an annual membership fee of $2,000 to $10,000, two BiTA members have told

“The key question is what job the standards are trying to achieve,” James said. “If they are truly open and aimed at growing an industry, then they have to be led by a group with the credibility and mandate. But lots of people push standards as a wolf in sheep’s clothing to try to corner a market — and those almost always fail. Because consensus is hard.”

Ocean carriers and adoptable standards

As it relates to blockchain in ocean freight, there’s also the issue of whether shippers will even see ocean carriers as a group capable of driving adoptable standards. As one shipper representative who spent time at an ocean carrier put it to, “The carriers can’t even handle block stowage, but we suddenly believe they’ll be able to handle blockchain?”

The broader data standards dilemma and blockchain adoption question are intertwined, but there is serious disagreement about whether standards are needed for blockchain adoption to progress, or whether ponderous efforts to develop standards might hinder the advancement of blockchain solutions.

For instance, would resources now going toward setting blockchain standards be better directed toward adoption of application programming interfaces (APIs) in lieu of EDI? Software experts deem APIs a necessary foundational technology to enable blockchain, so would broader use of APIs simply negate the need to develop blockchain standards?

“Our product is built on dozens of APIs, but the truth is today most of our customers are linking up with us via EDIs, so we integrate whatever EDI they have,” Graham Parker, CEO of the subscription-based freight forwarding software provider Kontainers, said in a January podcast with the supply chain technology accelerator Dynamo. “And getting the data in and out is relatively basic. I think when we see over the next couple of years, when customers move to APIs, and we can truly move that data seamlessly in and out of dozens of systems, we can really improve the analytics, and that’s when you’ll really start to see some real improvements in supply chain.”

Some blockchain advocates, including Andrew Bruce, CEO of Data Gumbo, a Houston-based startup that develops blockchain-based supply chain smart contracts to the energy industry, said he sees standards as a hindrance. He agreed with the assertion put to him by that data standardization from different parties is part of the value proposition of software providers.

“There will always be different data formats, no matter how much we try to standardize,” he said. “We can’t wait for standards to be set.”

And a global logistics visibility software provider put it this way to, “We don’t need the ocean carriers to set data standards. We just need them to make the data available, in whatever format.”

Indeed, the carrier data standard initiative unveiled in November overlooks the reality that global logistics software providers basically take on this data normalization and system interoperability role, and that shippers are more likely to trust software providers to harmonize data than they would their liner or surface transportation carriers.

In technology terms, this is referred to as middleware, providers that act as translation service providers between systems. Middleware is inexpensive right now, Jonah McIntyre, CEO of Berlin-based trucking optimization software provider TNX Logistics, said in a mid-December LinkedIn discussion about data standards in logistics.

“Point-to-point is cheap to start,” he said. “Joining shared networks is cheap for larger companies. Standards are small-community affairs, among supply chain participants who have similar semantic views of what they are exchanging data on, and win new adherents based on business need versus premeditated ‘rightness’ for their domain.”

Charles Marrale, CEO of digital freight forwarder Exfreight, said much the same in a December conversation with, “Middleware solutions are the way to go. They’re not expensive; they provide speed to market and one cohesive messaging standard.”

Data standards granularity

Some logistics technology providers argue that data standard initiatives get too weedy and thus lack relevance for the entire industry.

“EDI has been so persistent because it gives us a common language,” Brian Glick, CEO of logistics systems integrator, said in the mid-December LinkedIn discussion. “Like English, the most common standard isn't always the easiest to use. The presence of the common language is more important than its quality.”

Glick said problems occur when parties can't agree on the meaning of words — for instance, a part number versus a style versus a stock-keeping unit (SKU). Or when parties have diverging definitions for the estimated time of arrival (ETA) of a vessel and how often that ETA should be updated.

“As a group, we should be looking for opportunities to expand our common understanding of our shared language, and we should worry less about document syntax,” he said.

From a shipper perspective, data standard initiatives can cloud an already murky pool of information.

“While I welcome the initiative, I am skeptical,” Abir Thakurta, vice president of global supply chain at the furniture retailer Havertys, wrote in a blog on Nov. 26 about the five ocean carriers working on common data standards. “Initiatives around data standards are vague; as a shipper, I want to know more specifics and a body to drive ‘common IT standards’ is non-specific. Today carriers can’t even agree on a common [bill of lading]; and we are talking about specific data standards. Today we have varying degrees of data quality on simple shipping statuses and we are talking interoperability. If your quality of data is bad, how can we trust you to harmonize it and create data standards.”

The buzz around blockchain has brought this discussion of data standards to the fore, primarily through the emergence of two organizations: BiTA and TradeLens, the entity founded by Maersk and IBM to create an ocean freight-centric platform for trade and transportation.

BiTA’s emphasis to this point has been on domestic freight. Fuller told JOC that the vast majority of its members are trucking companies, freight brokers, or domestically oriented software providers. It does have a number of the world’s biggest names in logistics and software, including UPS, FedEx, and SAP, and a handful of large beneficial cargo owners (BCOs), such as Target and Whirlpool.

But the association’s progress has been slower than many members had expected, according to roughly two dozen member companies who spoke to, most on background. The organization’s meetings have been too infrequent and have lacked focus, those sources said.

“It grew faster than we had anticipated,” Fuller said. “It took off so fast, it is taking some time to get things together. When you go from zero to 490 members, there will be some growing pains. The challenge was when membership became so diverse, it’s difficult to give everyone a voice and that’s been sort of the issue. So we’ve tried to pare it back. Standards will come, but standards for an insurance verification will be different than a load-tracking standard.”

Fuller said that the evolution in BiTA’s focus has created an emphasis on company collaboration, even outside of blockchain applications.

He said annual membership renewals are between 80 and 85 percent, although he sees the attrition as a positive, especially for those members that were poking their heads in or didn’t see value. “As big and diverse as it had become, we couldn’t have delivered value to every organization.”

He also said expectation setting from BiTA was a likely source of some of the critiques of the organization.

“If I go in thinking of having a standard for everything, that’s an unreasonable expectation, and we’ve not messaged that well,” Fuller said. “So BiTA is out to show that value can be created not just around standards, but around commercial outcomes. I think we’ve achieved that with people who want to do it. Where we have not delivered to a place of satisfaction is in explaining that you have a lot of work to do around standards, that it will take time.”

Adam Robinson, marketing manager at the transportation management systems provider and BiTA member Cerasis, said that while individual blockchain providers, including those that are members of BiTA, might drive progress toward blockchain standardization commercially, “if you want to make it work for industries, it falls short without standards.”

“That’s an intrinsic problem in supply chain; we have this tech gap,” Robinson said. “It’s a blue-collar world, not necessarily a bunch of tech heads. It needs a lot of stewardship to align the right people, to say ‘here are our goals, here is who needs to attend.'"

Many of those who spoke to said it was the fear of missing out that drove their organizations to join BiTA, the idea that others would either race ahead in their understanding of blockchain or be involved in setting standards to which they would be beholden.

The blockchain innovation pull

But some members pointed to the positives of being involved in an initiative driving how the industry thinks about the use of blockchain.

The freight brokerage GlobalTranz, for instance, touted the potential data standardization benefits of joining the association in a November 2017 press release. “Developing standard transactions using blockchain technology within our industry has potentially the same positive impact as that of X-12 EDI, EDIFACT, and other standards used today,” GlobalTranz chief technology officer Greg Carter said in a statement.

“Yes, we are getting value, but like all new organizations and technological advancements, progress takes time. I wish it all moved faster,” one freight broker told “The value is in sharing ideas with thought leaders in an open forum.”

"We’re creating this human-based group of forward thinkers pondering the art of the possible when we’re middlemen, and blockchain in theory eliminates middlemen,” said Eric Rempel, chief innovation officer of the third-party logistics provider (3PL), freight brokerage, and BiTA member Redwood Logistics. He added that BiTA has created opportunities for his company to collaborate with large entities in the industry it normally would not get. But Rempel, who sits on one of BiTA’s subcommittees, added that the number of participants on the committee’s monthly calls have been dwindling.  

In terms of whether BiTA can drive standards in logistics, Rempel was optimistic but said the industry is still in the mindset of “using Charles Schwab to trade rocks for food.”

“Every business and its data is inherently different,” he said. “Because of the complexity of what we do and how fragmented it is, it’s going to need a massive outside force for people to adopt. Market pressure, internal pressure, everything has to be aligned, and it will. But it won’t be any of the use cases for blockchain we’re thinking about right now.”

Some of the larger members of BiTA that reached out to declined to comment on the value they are getting from membership. Fuller noted that some of the larger organizations that are BiTA members have a better appreciation for the time it takes to drive change than many of the start-up members that were eager for rapid progress.

TradeLens, meanwhile, has been hindered by similar skepticism. Maersk’s rival ocean carriers have expressed public and private concern over joining a consortium centered around data collection that’s administered by a competitor.

TradeLens representatives have emphasized that they see the company as an open-source platform on top of which it expects the industry to build useful applications. In other words, TradeLens would provide the protocol, its members would provide the underlying data, and the industry would be allowed to build applications on top of the protocol.

But TradeLens officials admitted in a November interview with the blockchain-focused news outlet Coindesk that the entity needs critical mass, including participation from other carriers, to reach that ambition. For context, the five carriers in the CargoSmart-led blockchain consortium, called the Global Shipping Business Network, collectively deploy roughly one-third of total global containership capacity.

It’s worthwhile to ponder whether TradeLens and the CargoSmart consortiums are also effectively rivals to BiTA, rather than being complementary organizations. In other words, should shippers eyeing participation in either type of entity think of a platform or consortium as a commercial solution to the standards issue, whereas a standards alliance is an inorganic attempt to develop that alignment?

It’s a chicken-and-egg question that is further muddied by the extent to which the industry is grappling with blockchain on a more fundamental level. Without fully understanding what blockchain is, it is hard for many industry participants to even have an opinion on whether standards should be driven by standards bodies or by commercial solutions/middleware providers.

“I think that products become commercially viable once standards have been set,” one freight broker and BiTA member told “Historically, revolutionaries get slaughtered.”

Contact Eric Johnson at and follow him on Twitter: @LogTechEric.





Well done article defining some of the challenges in trying to create a universal technology standard for the shipping/logistics/supply chain groups. It's always been that way, nothing new. Early larger companies had home grown solutions and never tried looking at it from an industry standpoint. There is some attempt to address it now but again, along narrow lines of like the ocean carriers and not all who are part of the global logistics industry. It is such a diverse and large group of entities, from multi-billion dollar firms to mom and pop truckers, massive manufacturers to the smaller factories over store fronts in China. How do you begin to address rounding up all of those under anything resembling a standard data format? I don't think you can, to me opening the door to 3rd party "translators", black box solutions as Intra was/is but of course on a much larger and broader scale. Then the challenge of getting people to participate. Don't hold your breath for this one.

In reality, the maritime industry has made tremendous strides in data availability and richness for shipper visibility in recent years and we have have the numbers to prove it. As the article indicates, private commerce has long since stepped in to provide the industry standardization function that consensus-based bodies have been unable to address. GT Nexus (now part of Infor), a global supply chain network platform and one of the first born-in-the-cloud supply chain software vendors, has addressed data disparity and harmonization in global shipping and commerce for 20 years. Dealing with details like variation in port names, for instance, was overcome early with software design, and as a majority of ocean carriers became connected to the GT Nexus network for digital communications, specific effort was possible between shippers and those carriers to measure and improve the quality and consistency of critical supply chain data streams. The creation of our GT Nexus Shippers Council many years ago led to an annual evaluation of ocean carriers and the completeness and latency of the key milestone messages they are responsible to provide for customers. We track individual carrier performance in transmitting ETA, In Gate, On Board, Depart, Arrive, and Discharge for each of the tens of thousands of ocean shipments passing through our network each year. Since 2006, when we began tracking carrier performance for our global shipping customers, the influence of a critical mass of cargo owners working with an objective source of transportation visibility data has promoted steady improvement in data completeness and in data timeliness. The most recent assessment shows that the top 8 ocean carriers are now providing over 98% of the key milestones to our customers, and the top 4 are doing so within 24 hours for 90% of the events. These milestones are a key foundational element of global supply chain visibility. The data standardization provided by our network platform, based on the pressing needs of global commerce execution efficiency, has arguably made this steady industry improvement possible. The visibility of metrics for all stakeholders, shippers, LSPs and ocean carriers alike, has also promoted further discussion, collaboration and industry improvement that continues to move all connected parties forward. Much of the promise of blockchain is already available in the marketplace today from alternate technology platforms.