What exactly is the difference between Apache's Mesos and Google's Kubernetes? I understand both are server cluster management software. Can anyone elaborate where the main differences are - when would which framework be preferred?
Why would you want to use Kubernetes on top of Mesosphere?
This question is related to
kubernetes
mesosphere
Mesos and Kubernetes both are container orchestration tools.
When you say "Google Kubernetes"?
Google Kubernetes Engine provides a managed environment for deploying, managing, and scaling your containerized applications using Google infrastructure.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.” Kubernetes was built by Google based on their experience running containers in production over the last decade.
The major components in a Kubernetes cluster are:
pods — a way to group containers together replication controllers — a way to handle the lifecycle of containers labels — a way to find and query containers, and services — a set of containers performing a common function
Mesos is an open-source cluster management project by Apache, designed to scale to very large clusters, from hundreds to thousands of hosts. Mesos supports diverse kinds of workloads such as Hadoop tasks, cloud native applications etc. It gives you the ability to run both containerized, and non-containerized workloads in a distributed manner.
It was initially written as a research project at Berkeley and was later adopted by Twitter as an answer to Google’s Borg (Kubernetes’ predecessor). To combat its high degree of complexity (Mesos is super complicated and hard to manage!), Mesosphere came into the picture to try and make Mesos into something regular human beings can use.
Mesosphere supplied the superb Marathon “plugin” to Mesos, which provides users with an easy way to manage container orchestration over Mesos.
In mid-2016, DC/OS (Data Center Operating System) — an open source project backed by Mesosphere — was introduced, which simplifies Mesos even further and allows you to deploy your own Mesos cluster, with Marathon, in a matter of minutes.
Now, if we compare kubernetes and Mesos(DC/OS)
kubernetes is a cluster manager for containers while mesos is a distributed system kernel that will make your cluster look like one giant computer system to all supported frameworks and apps that are built to be run on mesos.
Mesos was born for a world where you own a lot of physical resources to create a big static computing cluster. The great thing about it is that lots of modern scalable data processing application runs very well on Mesos (Hadoop, Kafka, Spark) and it is nice because you can run them all on the same basic resource pool, along with your new age container packaged apps.
Mesos cluster also runs alongside the Marathon cluster. Marathon, created by Mesosphere, is designed to start, monitor and scale long-running applications, including cloud native apps. Clients interact with Marathon through a REST API.
Also, a point to be noted is that you can actually run Kubernetes on top of DC/OS and schedule containers with it instead of using Marathon. This implies the biggest difference of all — DC/OS, as it name suggests, is more similar to an operating system rather than an orchestration framework. You can run non-containerized, stateful workloads on it. Container scheduling is handled by the Marathon.
Both projects aim to make it easier to deploy & manage applications inside containers in your datacenter or cloud.
In order to deploy applications on top of Mesos, one can use Marathon or Kubernetes for Mesos.
Marathon is a cluster-wide init and control system for running Linux services in cgroups and Docker containers. Marathon has a number of different canary deploy features and is a very mature project.
Marathon runs on top of Mesos, which is a highly scalable, battle tested and flexible resource manager. Marathon is proven to scale and runs in many production environments.
The Mesos and Mesosphere technology stack provides a cloud-like environment for running existing Linux workloads, but it also provides a native environment for building new distributed systems.
Mesos is a distributed systems kernel, with a full API for programming directly against the datacenter. It abstracts underlying hardware (e.g. bare metal or VMs) away and just exposes the resources. It contains primitives for writing distributed applications (e.g. Spark was originally a Mesos App, Chronos, etc.) such as Message Passing, Task Execution, etc. Thus, entirely new applications are made possible. Apache Spark is one example for a new (in Mesos jargon called) framework that was built originally for Mesos. This enabled really fast development - the developers of Spark didn't have to worry about networking to distribute tasks amongst nodes as this is a core primitive in Mesos.
To my knowledge, Kubernetes is not used inside Google in production deployments today. For production, Google uses Omega/Borg, which is much more similar to the Mesos/Marathon model. However the great thing about using Mesos as the foundation is that both Kubernetes and Marathon can run on top of it.
More resources about Marathon:
Kubernetes and Mesos are a match made in heaven. Kubernetes enables the Pod (group of co-located containers) abstraction, along with Pod labels for service discovery, load-balancing, and replication control. Mesos provides the fine-grained resource allocations for pods across nodes in a cluster, and can make Kubernetes play nicely with other frameworks running on the same cluster resources.
Mesos and Kubernetes can both be used to manage a cluster of machines and abstract away the hardware.
Mesos, by design, doesn't provide you with a scheduler (to decide where and when to run processes and what to do if the process fails), you can use something like Marathon or Chronos, or write your own.
Kubernetes will do scheduling for you out of the box, and can be used as a scheduler for Mesos (please correct me if I'm wrong here!) which is where you can use them together. Mesos can have multiple schedulers sharing the same cluster, so in theory you could run kubernetes and chronos together on the same hardware.
Super simplistically: if you want control over how your containers are scheduled, go for Mesos, otherwise Kubernetes rocks.
I like this short video here mesos learning material
with bare metal clusters, you would need to spawn stacks like HDFS, SPARK, MR etc... so if you launch tasks related to these using only bare metal cluster management, there will be a lot cold starting time.
with mesos, you can install these services on top of the bare metals and you can avoid the bring up time of those base services. This is something mesos does well. and can be utilised by kubernetes building on top of it.
"I understand both are server cluster management software."
This statement isn't entirely true. Kubernetes doesn't manage server clusters, it orchestrates containers such that they work together with minimal hassle and exposure. Kubernetes allows you to define parts of your application as "pods" (one or more containers) that are delivered by "deployments" or "daemon sets" (and a few others) and exposed to the outside world via services. However, Kubernetes doesn't manage the cluster itself (there are tools that can provision, configure and scale clusters for you, but those are not part of Kubernetes itself).
Mesos on the other hand comes closer to "cluster management" in that it can control what's running where, but not just in terms of scheduling containers. Mesos also manages standalone software running on the cluster servers. Even though it's mostly used as an alternative to Kubernetes, Mesos can easily work with Kubernetes as while the functionality overlaps in many areas, Mesos can do more (but on the overlapping parts Kubernetes tends to be better).
Source: Stackoverflow.com