Software – and the data it produces – is the engine that is driving IT these days. Hardware, either in traditional datacenters or in the cloud, no longer is front and center in the mind of IT professionals. Instead those servers and storage appliances are the tools that enable them to run their applications quickly and efficiently and to move and store the massive amounts of data they are accumulating.
This shift over the past several years has rippled through the hardware world. Companies like Dell Technologies, Hewlett Packard Enterprise, Lenovo and Cisco Systems still make high-performance and increasingly secure systems, but their focus now is on transitioning to becoming solutions and services providers with plans to deliver much if not all of their portfolios as a service. Because what matters most to their end users is the software they’re running, not so much the underlying hardware.
For storage vendors, this shift has meant a change in what they are expected to do. It’s not enough to simply collect and store the data and ensure that enterprises can access that data. In an increasingly distributed world of clouds and the edge, they have to become data management companies.
Pure Storage made it bones with its all-flash arrays, but in September 2020 doled out $370 million for Portworx, a startup with a background in container storage and Kubernetes, key tools in the rapidly changing IT world. The acquisition was an indication that Pure understood that managing the data – and not simply storing it – was becoming a required capability, according to Murli Thirumale, a co-founder and former CEO of Portworx and now vice president and general manager of Pure’s Cloud Native Business Unit.
“Pure was like any other storage company,” Thirumale tells The Next Platform. “It started out in the storage business. You store things and make sure they don’t get lost, it’s reliable and all of that kind of stuff. That continues and all other storage companies do that. But the nature of these applications start to [evolve] and you get more and more workloads that are containerized.”
He estimates that 85 percent o new applications are containerized and cloud-native. This is what Thirumale calls the “post-DevOps, post-cloud era.” Everyone is using modern technologies.
“Customers and enterprises will come to companies like Pure and EMC and NetApp and IBM and say, ‘How do I make this work?’” he says. “They need to add data management because you can’t just say to them, ‘Go buy new storage.’ Nobody’s going to throw away that old storage. It’s been set up. It’s reliable. It works. Every storage company needs to start looking up the stack into this realm of middleware because that’s where a lot of data management and data protection – all these functions – are being performed. More and more in the middleware layer, not in the hardware. You can do it in the hardware layer, but it’s pricey and it’s a little bit inflexible and difficult for customers to manage.”
DATA MANAGEMENT IS FUTURE FOR STORAGE VENDORS
Pure understood data management was the way forward and that Kubernetes is the driving technology. Containers and microservices had hit the scene, but it wasn’t until Kubernetes escaped the labs at Google that order was fully in place.
Now the role of Kubernetes is expanding beyond simply orchestrating containers, Thirumale says. It’s becoming a key middleware component for managing data and is a tool for managing hardware and could find itself being used to manage uncontainerized applications. It still is used to place containers where there are available compute cycles. Polyworx’s technology creates what he calls a seamless plane of data, ensuring the data is available in every node, independent of where the data originally was. This is data management with a fast and flexible fabric at the top for the application.
Underneath is infrastructure comprising fairly inflexible and siloed hardware. The data management middleware from vendors like Portworx is provided as a software overlay that sits atop the hardware, enabling organizations to run containerized workloads without having to upgrade their hardware, he says.
EXTENSIONS OPEN UP THE FUTURE OF KUBERNETES
Google engineers and others at vendors like Portworx understood that extensions were needed to enable Kubernetes to do such jobs as manage compute allocations, data security and networking, so the CNI (container network interface) and CSI (container storage interface) were created, leading to “a new avatar for the second coming of Kubernetes,” he says.
“Kubernetes was originally – and still is, obviously – being used to manage containers,” Thirumale says. “But with these extensions of CNI, CSI and security extensions, Kubernetes can actually be used to manage data and storage and manage networking and all of that. If I were to put a Kubernetes layer in the middleware layer, looking upwards, it’s managing where the containers land. But looking down, it’s actually now managing infrastructure. There’s a whole new way of managing infrastructure. The traditional way was you had to go to the storage admin and say, ‘Give me five more nodes and give it to me in these terabytes and with this capability and all of that that,’ then they’d provision your EMC box or a Pure box or NetApp box or what have you.”
Now it’s a self-service model of code as infrastructure. IT tells Kubernetes how many nodes and terabytes it needs and what class of services, Kubernetes tells Portworx what to do and Portworx looks at the underlying storage and manages all of that to provide that automatically, he says.
“This new role of Kubernetes as an IT infrastructure management tool has taken off like wildfire,” he says. “There are many companies in the security space, in the infrastructure space and so on. In the last three to four years, that’s what caused the growth of companies like Portworx, but now every storage manufacturer is working on a version of something like the Portworx capability.”
PORTABILITY IS KEY
There are multiple ways to get this done and the industry is moving in that direction because it addresses the issue of portability, Thirumale says. Containers can run anywhere, on premises or in the cloud, and the data also needs to be seamless. Data management needs to be multicloud and solutions like those from Portworx and others operate on the premise of being able to work with any application, any Kubernetes distribution and any underlying hardware.
At one point, OpenStack was seen as the future of infrastructure management, the one set of interfaces that worked across all infrastructure. However, while it’s deployed now, it’s limited. Kubernetes is starting to fill that role. Not only is it a multicloud technology but also offers self-service capabilities. DevOps users no longer have to ask storage admins for more storage; instead they state what they need and Kubernetes makes it happen, and this allows for the rapid deployment of infrastructure. Many DevOps professionals don’t know where the storage is coming from and that’s how it should be, he says.
“Containers caused the deployment of Kubernetes, but now that Kubernetes is being deployed, people are realizing this is really the right way to manage infrastructure,” Thirumale says. “In my view, it’s going to be used to manage uncontainerized workloads in the future.”
It increasingly also will be used for stateful applications as well as stateless workloads, he says. More than 80 percent of applications in enterprises are stateful, involving databases, artificial intelligence (AI) or similar technologies. “If you excluded stateful applications, you’re not benefiting the majority of the applications that exist in an enterprise,” he says.
The use of Kubernetes will only grow, he says. A recent survey by Portworx found that 68 percent of respondents used more Kubernetes as a result of the COVID-19 pandemic, 89 percent said Kubernetes will play a larger role in how they manage infrastructure management and 84 percent have used it to test or develop AI models and applications.
Comments