Kubermesh: Ocado’s Kubernetes Variant for Building Fault-Tolerant Infrastructure

The British online supermarket, Ocado has a long history when it comes to innovation. It was the first online-only groceries provider in the UK; it was one of the first to introduce shopping apps and it was the first U.K. retail outlet to offer shopping by Apple Watch.

With this willingness to innovate, there’s little surprise that the company is constantly looking for ways for technology to transform the whole shopping experience. The company even has its own subsidiary, Ocado Technology, an organization that has been set up to exploit opportunities in this area.

The company has been very active in many different aspects of technology, such as being particularly advanced in its use of robotics. For much of its work in this area, it’s a heavy user of the Google Cloud Platform and the container orchestration system, Kubernetes, which was developed by Google and now managed as an open source project by the Cloud Native Computing Foundation.

Kubernetes is already being used extensively within Ocado, originally used to manage large data sets running continuously. So Mike Bryant, Ocado’s network systems team leader, thought that the system could be deployed to streamline the company’s warehouses. What the team came up with, Kubermesh, is set to radically reshape the whole operation.

“What we saw in Kubernetes was a way in which we could orchestrate different components to build a resilient, fault-tolerant infrastructure,” Bryant said.

Although Kubermesh is an enhanced version of Kubernetes, it’s not a replacement. “It sits underneath providing a mesh network for our warehouses,” said Bryant. “It’s more focused on the underlying infrastructure.”

What Ocado has done with Kubermesh is to remove the need for data centers in running the warehouses. While the system hasn’t obviated the need for servers, what it has done is offer the potential to considerably cut costs. “Most people run Kubernetes on high-performance servers, we run it on PCs — it’s a very dramatic cost saving,” said Bryant. “And it’s not just the servers, we don’t need enterprise network cabling, we don’t need high-end networking kit, we save on cooling — we’re really cutting costs.”

And the economics makes sense: “In our newest warehouses, we’re looking to save a £1 million per deployment. It’s a saving that will stack up given the number of Ocado warehouses that the company has to administer.”

But the Kubermesh system isn’t just about saving money; it’s about providing resilience in an industry where speed of delivery is essential. What Kubermesh does provide is a distributed mesh network that ensures continuous service, “we run Kubermesh so even if one workstation dies, then it still running,” said Bryant.

The underlying IPv6 network uses Open Shortest Path First (OSPv3) for streamlining configuration to negate having to configure point-to-point connections. The mesh network means that servers can be randomly dotted around the warehouse, and it’s so designed that all the components work individually. This means that if a server is taken offline, the whole system will still work. It’s not just one server either: the Ocado set-up has been designed to keep on running if as much as a third of the servers were cut off.

“What we saw in Kubernetes was a way in which we could orchestrate different components to build a resilient, fault-tolerant infrastructure.” — Mike Bryant.

At the moment, the Kubermesh project is still at the pilot stage. Ocado Technology has run proof-of-concept trials to show that the system works but is not planning to rip out existing infrastructure in order to implement the new technology.

“There’s no financial advantage in replacing existing systems, but when we next build a new warehouse, we will be rolling out Kubermesh,” said Bryant. “However, there’s no timescale when this will be.”

Ocado is striking out on its own with this virtual data center for the warehouse. “We’re not aware of any other competitor that has done anything like this,” said Bryant. The company, however, has put the code for Kubermesh on GitHub, so there is the possibility that it could be deployed elsewhere.

Competitive Pressure

There aren’t many industries that have undergone such radical change as retailing. It was only 20 years ago that we all did our shopping the same way — a weekly or monthly trip to the supermarket for food with trips to individual stores for other purchases.

Not anymore: we’ve all fully embraced online shopping. According to U.S. The Centre for Retail Research, online sales in the U.S. grew by 14.4 percent to $399.53 billion in 2016. Sales are expected to hit $459.07 billion by the end of the year, an increase of 14.9 percent, while the projected increase in 2018 is 15.4 percent.

Naturally, given the size of the market and growing demand for online shopping, retailers are looking at ways of improving the shopping service.

What Ocado has done, however, is to demonstrate that retail needs to adapt even more to keep ahead of the game. The Kubermesh system is just part of the many efforts required to improve the shopping experience for the 21st century.

The Cloud Native Computing Foundation is a sponsor of The New Stack.

Feature image via Pixabay.