UberCloud Docker & Trinity host Private clouds

“High-Performance Computing in the Cloud? That does not really work for real number crunching.” That was the general opinion for a long time in the HPC community. Then Docker came along. Its containers solved some of the problems – especially replacing heavy Virtual Machines with light weight containers, but it still had a strong feeling of “work in progress”. During the Computing Insight UK 2015, Wolfgang Gentzsch – from UberCloud – and Hans Then – from ClusterVision, jointly presented a complete HPC Cloud stack based on Docker. UberCloud provides HPC -optimised application containers based on Docker. ClusterVision’s Trinity platform allows operating an HPC cluster from a regular Cloud dashboard. Together, they showed that Trinity is capable of managing UberCloud’s highly optimised application containers. Today you can operate a cluster as a cloud without giving up on performance.

Hans Then and Wolfgang Gentzsch explained to us how they converged over 2015 with the initial concepts and how they demonstrated that building an HPC environment in this manner actually works.

Q. Before we dive into the collaboration between ClusterVision and UberCloud it would be good to focus on ClusterVision’s Trinity platform and the UberCloud application containers. Hans, could you tell us a bit about Trinity?

Hans Then: For a very long time now, many in the HPC community have been looking at Cloud as a method of gaining flexibility. For instance a growing number of national supercomputer centers only receive grants if they also offer cloud capabilities to Small and Medium Enterprises. Many in the HPC community have been looking at OpenStack as a means of providing a cloud infrastructure. So when ClusterVision was approached by our customers a few years back to see what OpenStack could do for them, we started investigating this. We soon noticed there is still a huge mismatch between OpenStack and HPC. If you take an HPC installation, and then put OpenStack on it, it will not have the features that you are looking for as a typical HPC user.

This is especially true in the academic world where users run physics applications which have low latency requirements that are very tightly coupled. OpenStack adds a layer of virtualisation that gives you more flexibility, but that is going to cost you in performance. It is not going to be efficient to manage an HPC cluster inside a Cloud. So what we did, is instead of modelling each of our HPC resources as a virtual machine inside OpenStack, we decided to build something that would model a machine cluster as one entity, as one virtual resource.

We did want to virtualise an HPC Cluster, and thus to provide the extra flexibility that comes normally from using virtual machines.We looked around and we found that in Docker. This is how Trinity originated: It is an HPC cluster which we have wrapped up inside an OpenStack resource with the help of Docker. From your OpenStack Dashboard you can manage your HPC cluster. When we did that, we noticed that Docker provides additional flexibility over virtualisation. Docker offers a very advanced form of application packet management. You can see Docker as an enhanced form of RPM. For a long time at ClusterVision we wanted to be able to put a number of applications inside containers and offer these containers directly to our customers.  A few months ago we found UberCloud was already doing this, so we were able to use UberCloud’s containers directly in our system.

Q: How do you model an HPC cluster with Docker containers inside the Trinity system? Is the cluster one resource? Or is each computer node a Docker container?

Hans Then: For Trinity it basically is one Docker container per node. But it is possible, and that is the part where we started to cooperate with UberCloud, to run separate containers for each of the applications you want to run.  If you have a ten node cluster you can use five for one application container and the other five for a different application container.

Q: So that’s how Trinity found UberCloud. But how did UberCloud find Trinity?

Wolfgang Gentzsch: That is a little bit of history. At UberCloud we have run 187 experiments since 2013. Each experiment is taking engineering applications and their data to different clouds with the sole aim to learn about the roadblocks. We identified ten major road blocks for the adoption of cloud computing by HPC engineering applications. After identifying these road blocks we started looking for a solution.

In early 2013 we participated in the first public workshop of Docker. Docker is located in San Francisco, just around the corner of our US office. Due to this we learned about Docker very early. When we sat in the Docker workshop we immediately saw that Docker, which was originally built for micro-services, is also nicely extensible towards what we now call macro-services. Services which basically consist of computer simulations for engineers and scientists, and also include the necessary data, tools and libraries, and complete workflows. We started developing Docker further into what we call “UberCloud containers”. A year ago we finished the UberCloud base container, and started to containerize applications like OpenFoam, ANSYS, STAR-CCM+ from CD-Adapco, Numeca, and others. Currently, we are working on containerizing COMSOL Multiphysics.

We found out that these UberCloud application containers remove almost all the roadblocks we had found during the 187 experiments. We discovered Trinity quite early on the ClusterVision website, and said “wow, they can handle containers as well”. We met at ISC 2015 in Frankfurt and it quickly became clear for both of us that we would do a joint project: Running our containers on top of Trinity. And that is what we did.

Q: If someone has a cluster, can they buy the UberCloud application containers and run them on their own system?

Wolfgang Gentzsch: The beauty with the UberCloud containers based on Docker, is that they come with features like packageability, portability, accessibility, usability, scalability, etc. These containers are not only well-suited for running applications in the Cloud, but they are also useful for running on in-house clusters. They run on your laptop, they run on your workstation, they run on your server, they run on private cloud, hybrid cloud, hosted cloud, they run everywhere. You port and package it once and for all in your container, with your data, with the tools, with the libraries. In short, the complete workflow that an engineer needs.  Then you can seamlessly port it everywhere. This will change the applications software landscape over the next couple of years quite dramatically.

Q: Is the licensing also included somehow in an UberCloud application container?

Wolfgang Gentzsch: Every container has its own license manager. We have a license manager which can handle different types of licenses. It really depends on how the ISV wants to handle it. In our UberCloud community we are working together with over 100 ISVs. They have different types of licensing models especially when it comes to the Cloud. In the meantime, many of them have licenses for on demand software-as-a-service. The license can be handled from inside the container. The license can sit in the container, but the container can also connect, for example, to a license server at the ISV. There are different models and the containers cover all these models.

Q: During ISC 2015 in Frankfurt the two of you sat together and discussed Trinity and the UberCloud containers. What happened next? Did you already implement and test it?

Hans Then: Following the Frankfurt meeting, Wolfgang introduced me to Burak Yenier from the Los Altos branch of UberCloud. He provided us with a number of test containers.  At ClusterVision we started experimenting with these test containers. The UberCloud containers have the capacity to completely run stand-alone, but we wanted to see whether they were capable of being integrated with our scheduling tools. Inside our Trinity HPC platform we use a standard scheduler which is very familiar to many HPC users. We wanted to be able to call these application containers from inside this scheduling environment, and did a couple of experiments. At the Computing Insight UK 2015 last week, we demonstrated that an UberCloud OpenFOAM application container can be invoked from inside the Trinity environment. We ran an application inside Trinity over multiple MPI nodes, without any changes to the OpenFOAM UberCloud container. This was a very successful experiment.

Q: What are the next steps?

Hans Then: A number of ClusterVision customers are interested. I previously mentioned the national computing centers. I think this would be an ideal use case for them. Most users in academia have their own applications which they run at the computing center. But these centers also have SME customers who would like to have access to predefined computing applications. To set up a small cluster with four nodes can be prohibitively expensive for an SME, so why not use a very professionally managed supercomputer cluster that is hosted by the national supercomputing center?  Offering predefined applications to those supercomputing centers might give them a very attractive offering to their customers.

The next step for ClusterVision is to productize this. We want to make it very easy to run any application from a default Trinity cluster which comes from the UberCloud Marketplace.

Q: What are the plans for UberCloud?

Wolfgang Gentzsch: We are seeing ourselves as becoming the “containeriser” of engineering and science applications, and – as Hans mentioned – our Marketplace will be filled more and more with containerized application software. We will strongly push this trend towards standardisation. We certainly are not the only ones who containerise applications, but we will do it in a professional and commercial way for the ISVs out there, and for the open source communities. We help the community to containerise their software.

Q: Trinity software is Open Source software? Will ClusterVision add Trinity to the offerings for its customers?

Hans Then: We offer Trinity as a standard tool for cluster management. During 2015 a growing number of the HPC solutions we delivered included Trinity. Trinity is an open source product and we ask a fee depending on the level of support required.

Q: Will there also be other system integrators using Trinity?

Hans Then: One of the main goals of offering Trinity as an open source tool is to empower our customers not to get locked into a proprietary product. This way, the customers are assured that whatever happens, they always have access to the Trinity software.  Actually we have already seen some of our customers contribute code to Trinity too.

Q: This combination of UberCloud application containers with Trinity HPC cluster management software looks very promising for HPC applications. But will it also be suited for the real large HPC applications that need hundreds of cores to run?

Wolfgang Gentzsch: Docker is built for single-node micro-services.  Recently, in a project with one of the largest US national laboratories, we enhanced Docker to a multi-node, multi-container environment for what we call “macro-services”. We ran benchmarks on up to 1.024 cores without observing any difference between the dockerised application and the bare metal application running directly on the supercomputer. This is a great breakthrough especially for high-performance computing that is not only about 16 to 32 cores but also about hundreds of cores which you only can get within multiple node HPC server environments.

Hans Then: What I noticed, is that people first see Docker as an alternative for a Virtual Machine. They think it is a kind of virtualisation technology. But over the course of our experiments with Docker we have come to realize that Docker’s advantages are not so much focused on virtualisation, but rather on flexibility in packaging, so it is a superior packaging mechanism. It is a superior kind of RPM. That realisation allowed us to design much better solutions using Docker than we have previously been able to do.

Docker is very much a packaging technology that allows you to build new applications and to manage those applications in an easy way. This is where I see the strength of this cooperation with UberCloud, because they have provided this whole repository of packaged applications which will run anywhere. Thus instead of having a CentOS machine that will need an RPM, or a Debian machine that will need a DEB package, you simply have something that can be used anywhere.

Wolfgang Gentzsch: We are seeing growing opportunities for UberCloud containers also for Private Clouds. This is currently the major area that ClusterVision is covering. We see this trend especially at large companies, because this way they have more control over their hardware and software resources. A private cloud could be in-house and still come with all the flexibility and agile features Hans mentioned. Private clouds can also be hosted clouds; hosted in a data center, which is just controlled by the company. This dedicated cluster provides a secure environment for Docker HPC container technology. The servers only belong to you. There is no multi-tenancy. Using UberCloud application containers on a Trinity managed private cloud is a highly efficient and secure way of running high-performance computing applications.

For further information please visit:

• https://clustervision.com/trinity
• https://www.theubercloud.com/store/