While I wouldn't dream of standing up k8s on a bare metal cluster without a devops team, I set up managed k8s using EKS several years ago for a client and... it just chugs along, self-healing, with essentially zero maintenance.
For my own projects I use a managed Northflank cluster on my own AWS account and likewise... just a fantastic experience. Everything that Heroku could and should have been. Yes the cluster is a bit pricey to stand up both in terms of EC2 compute and management layer costs, but once it's there, it's there. And the costs scale much more nicely than shoving side projects onto Heroku.
At this stage I consider managed k8s my default go-to unless it's something so lightweight I just want to push it to Vercel and forget about it.
If you need to go beyond what a single bare-metal server can offer, then consider it.
But don’t discount bare-metal first! I see a lot of K8s or other cluster managers being used to manage underpowered cloud VMs, and while I understand the need for an orchestrator if you’re managing dozens of VMs, I wonder - why do you need multiple VMs in the first place if their total performance can be achieved by a handful of bare-metal machines?
If PaaS or some "run container as a service" setup can work for your use case, I'd probably go with that. It takes care of many things K8s does without all the baggage. Also you are not investing into anything that doesn't port easily to K8s in the future.
On the other hand, if you are thinking of using bare VMs, then better go with managed K8s. I think in 2025 it's a draw in terms of initial setup complexity, but managed K8s doesn't require constant babysitting in my experience, unlike VMs, and you are not sinking hours into a bespoke throwaway setup.
I'll add my opinion as a DevOps engineer, not a startup, so take it with a grain of salt.
* Kubernetes is great for a lot of things, and I think there's many use cases for it where it's the best option bar none
* Particularly once you start piling on requirements - we need logging, we need metrics, we need rolling redeployments, we need HTTPS, we need a reverse proxy, we need a load balancer, we need healthchecks. Many (not all!) of these things are what mature services want, and k8s provides a standardized way to handle them.
* K8s IS complex. I won't lie. You need someone who understands it. But I do enjoy it, and I think others do too.
* The next best alternative in my opinion (if you don't want vendor lock in) is docker-compose. It's easy to deploy locally or on a server
* If you use docker-compose, but you find yourself wanting more, migrating to k8s should be straightforward
So to answer your questions, I think you can adopt k8s whenever you feel like it, assuming you have the expertise and are willing to dedicate time to maintaining it. I use it in my home network with a 1 node "cluster". The biggest pitfalls are all related to vendor lock in - managed Redis, Azure Key Vault. Hyper specific config related to your managed k8s provider that might be tough to untangle. At the same time, you can just as easily start small with docker-compose and scale up later as needed.
It depends of course, but probably Kubernetes is a solution to problems that you don't have, while it creates new problems that you don't currently have.
It seems to me the issue is not really setting up and building around K8s as an infrastructure orchestrator, afterall k8s sells itself as a cluster API which is the de facto standard nowadays. The issue starts when you need to handle very specific use-cases, e.g., security. This requires very low-level experience not only with K8s, but with the whole stack (including OS + HW) + knowledge of safe resource and application scheduling, which is hard to find talent for.
You can run docker swarm easy peasy. Its not that trendy, but anyone can manage it, and you can migrate to k8s later if you need to.
Of course it doesn't do some of the things that k8s does, and thats why its less complicated...
Edit: you can run a lot on one or two hetzner servers for almost no money. Compare €60/month, vs about $1000/month for a couple of replicated fargate services.
The answer, as with everything, is going to be that it depends on your situation and use-case. If you have a bunch of engineers who are already familiar with K8s then using a different implementation just because others told you to doesn't make much sense. But if you're choosing K8s because you want a good future foundation, in dreams of 'scale', then you should stop and really consider what it is that you need from K8s. Most people I've seen who's infrastructure succeeds with K8s only moved to it as a necessity, usually away from some monolithic structure, they didn't start there. Build what you need, and only that much, not for some future need that might not come.
If you've already done the work figuring out how to knock out a basic web app deployment of kubernetes for a project that you think will grow, then I say go for it. It's not much cheaper than buying a reasonable minimum amount of compute with a company like Digital Ocean.
For hobby projects that I don't really plan to scale, I've recently gotten back into non-containerized workloads running off of systemctl in an Ubuntu VM. It feels pretty freeing not to worry about all the cruft, but that will bite me if something ever does need to live on multiple servers.
Same question. I'm a one man band who wants to be scalable, but doesn't want to get married to a particular cloud. So Kubernetes appears to be the default recommendation. Are there better alternatives?
> I'm a one man band who wants to be scalable, but doesn't want to get married to a particular cloud.
(AWS employee, but opinions are my own)
Just pick a cloud provider and move on. All of the top-tier providers can scale. Choose the one you're most comfortable with and move on. Focus on the things that matter for your business like building the right product and getting customers. If you have regret later, it's going to be because you were so wildly successful that it is now an actual business risk and/or your customers demand it. But don't make this decision based on a problem you don't have and are unlikely to have in the next 12-24 months, if ever. Cloud agnosticism is rarely a functional or strategic requirement of any given business, and it's usually very expensive to implement--more than any savings you might achieve by pitting providers against each other (which you can't do anyway unless you are big enough to be a strategic customer).
I'm writing a public statement platform, and a core design priority is to avoid being Parlered, since human moderation is not an option. Responding to court orders is inevitable, but responding to cloud provider demands isn't quite, yet.
What’s your business? Do you have product market fit? Do you benefit enough from the three things K8S does well to pay the cost in increased complexity, reduced visibility, and increased toil? If you can’t immediately rattle off those three things, you don’t need it.
Don’t you want to just focus on the problem you’re solving for your customers and not the infrastructure that makes your app go?
Every startup I’ve seen doing k8s should not have been.
Every startup I’ve seen not using k8s didn’t need it. (Except a startup who moved from beanstalk which nobody should ever use, and they could have done better by moving to something like ecs)
I’ve seen a startup lose their entire DevOps team and successfully go a year without it because their core app was on heroku. What’s that worth in dollars? What are those dollars worth in opportunity cost?
I tried that in the past and found it extremely unreliable as a production environment. Documentation was also non-existent and I'd have to manually handle clusters, setup my own observability and log stack, etc. Any cloud provider these days gives you all that out of the box for K8s, so I'm, not sure the time one would invest on Swarm really makes sense?
k8s isn’t worth the time and money for many small teams, until they cross a complexity bar.
Even in some very non-startup enterprises, Cloud Foundry and Open Shift get adopted for a reason: some teams don’t need the overhead.
For startups there’s fly.io, render.com, and of course Heroku, but really — you can get from MVP to pretty decent scale on AWS or GCP with some scripts or Ansible.
Use k8s if you need it. It’s pretty well-proven. But it’s not something you need to have FOMO about.
in my current case - the ability to deploy on-prem for some specific customers (on-prem meaning their own AWS or GCP account usually) + per-customer/multitenancy on the main product (ideally with segregated databases)
In general - scaling up a small number of microservices + their associated infra (redis/rabbitmq/etc)
You are basically stuck with k8s but you will end up having to "roll your own" (bring your own components) if you intend for operations to be consistent across different clouds/prems/etc.
Ideally start with an existing kube stack and slowly make it your own.
Operationalizing across hetereogenous clusters will be an unfortunate source of excitement.
While I wouldn't dream of standing up k8s on a bare metal cluster without a devops team, I set up managed k8s using EKS several years ago for a client and... it just chugs along, self-healing, with essentially zero maintenance.
For my own projects I use a managed Northflank cluster on my own AWS account and likewise... just a fantastic experience. Everything that Heroku could and should have been. Yes the cluster is a bit pricey to stand up both in terms of EC2 compute and management layer costs, but once it's there, it's there. And the costs scale much more nicely than shoving side projects onto Heroku.
At this stage I consider managed k8s my default go-to unless it's something so lightweight I just want to push it to Vercel and forget about it.
If you need to go beyond what a single bare-metal server can offer, then consider it.
But don’t discount bare-metal first! I see a lot of K8s or other cluster managers being used to manage underpowered cloud VMs, and while I understand the need for an orchestrator if you’re managing dozens of VMs, I wonder - why do you need multiple VMs in the first place if their total performance can be achieved by a handful of bare-metal machines?
If PaaS or some "run container as a service" setup can work for your use case, I'd probably go with that. It takes care of many things K8s does without all the baggage. Also you are not investing into anything that doesn't port easily to K8s in the future.
On the other hand, if you are thinking of using bare VMs, then better go with managed K8s. I think in 2025 it's a draw in terms of initial setup complexity, but managed K8s doesn't require constant babysitting in my experience, unlike VMs, and you are not sinking hours into a bespoke throwaway setup.
I have no experience with AWS and the rest I always use Paas and they serve me well, I wonder what cases I'd need kubernetes and stuff?
I'll add my opinion as a DevOps engineer, not a startup, so take it with a grain of salt.
* Kubernetes is great for a lot of things, and I think there's many use cases for it where it's the best option bar none
* Particularly once you start piling on requirements - we need logging, we need metrics, we need rolling redeployments, we need HTTPS, we need a reverse proxy, we need a load balancer, we need healthchecks. Many (not all!) of these things are what mature services want, and k8s provides a standardized way to handle them.
* K8s IS complex. I won't lie. You need someone who understands it. But I do enjoy it, and I think others do too.
* The next best alternative in my opinion (if you don't want vendor lock in) is docker-compose. It's easy to deploy locally or on a server
* If you use docker-compose, but you find yourself wanting more, migrating to k8s should be straightforward
So to answer your questions, I think you can adopt k8s whenever you feel like it, assuming you have the expertise and are willing to dedicate time to maintaining it. I use it in my home network with a 1 node "cluster". The biggest pitfalls are all related to vendor lock in - managed Redis, Azure Key Vault. Hyper specific config related to your managed k8s provider that might be tough to untangle. At the same time, you can just as easily start small with docker-compose and scale up later as needed.
It depends of course, but probably Kubernetes is a solution to problems that you don't have, while it creates new problems that you don't currently have.
It seems to me the issue is not really setting up and building around K8s as an infrastructure orchestrator, afterall k8s sells itself as a cluster API which is the de facto standard nowadays. The issue starts when you need to handle very specific use-cases, e.g., security. This requires very low-level experience not only with K8s, but with the whole stack (including OS + HW) + knowledge of safe resource and application scheduling, which is hard to find talent for.
PS. Edit for clarity.
You can run docker swarm easy peasy. Its not that trendy, but anyone can manage it, and you can migrate to k8s later if you need to. Of course it doesn't do some of the things that k8s does, and thats why its less complicated...
Edit: you can run a lot on one or two hetzner servers for almost no money. Compare €60/month, vs about $1000/month for a couple of replicated fargate services.
The answer, as with everything, is going to be that it depends on your situation and use-case. If you have a bunch of engineers who are already familiar with K8s then using a different implementation just because others told you to doesn't make much sense. But if you're choosing K8s because you want a good future foundation, in dreams of 'scale', then you should stop and really consider what it is that you need from K8s. Most people I've seen who's infrastructure succeeds with K8s only moved to it as a necessity, usually away from some monolithic structure, they didn't start there. Build what you need, and only that much, not for some future need that might not come.
If you've already done the work figuring out how to knock out a basic web app deployment of kubernetes for a project that you think will grow, then I say go for it. It's not much cheaper than buying a reasonable minimum amount of compute with a company like Digital Ocean.
For hobby projects that I don't really plan to scale, I've recently gotten back into non-containerized workloads running off of systemctl in an Ubuntu VM. It feels pretty freeing not to worry about all the cruft, but that will bite me if something ever does need to live on multiple servers.
Same question. I'm a one man band who wants to be scalable, but doesn't want to get married to a particular cloud. So Kubernetes appears to be the default recommendation. Are there better alternatives?
> I'm a one man band who wants to be scalable, but doesn't want to get married to a particular cloud.
(AWS employee, but opinions are my own)
Just pick a cloud provider and move on. All of the top-tier providers can scale. Choose the one you're most comfortable with and move on. Focus on the things that matter for your business like building the right product and getting customers. If you have regret later, it's going to be because you were so wildly successful that it is now an actual business risk and/or your customers demand it. But don't make this decision based on a problem you don't have and are unlikely to have in the next 12-24 months, if ever. Cloud agnosticism is rarely a functional or strategic requirement of any given business, and it's usually very expensive to implement--more than any savings you might achieve by pitting providers against each other (which you can't do anyway unless you are big enough to be a strategic customer).
I'm writing a public statement platform, and a core design priority is to avoid being Parlered, since human moderation is not an option. Responding to court orders is inevitable, but responding to cloud provider demands isn't quite, yet.
Heroku Vercel AWS ECS
What’s your business? Do you have product market fit? Do you benefit enough from the three things K8S does well to pay the cost in increased complexity, reduced visibility, and increased toil? If you can’t immediately rattle off those three things, you don’t need it.
Don’t you want to just focus on the problem you’re solving for your customers and not the infrastructure that makes your app go? Every startup I’ve seen doing k8s should not have been. Every startup I’ve seen not using k8s didn’t need it. (Except a startup who moved from beanstalk which nobody should ever use, and they could have done better by moving to something like ecs)
I’ve seen a startup lose their entire DevOps team and successfully go a year without it because their core app was on heroku. What’s that worth in dollars? What are those dollars worth in opportunity cost?
Docker compose, docker swarm if you outgrow that.
I tried that in the past and found it extremely unreliable as a production environment. Documentation was also non-existent and I'd have to manually handle clusters, setup my own observability and log stack, etc. Any cloud provider these days gives you all that out of the box for K8s, so I'm, not sure the time one would invest on Swarm really makes sense?
You will be married to a particular cloud.
Either you go all in on someones setup or you get to do it all yourself.
That's true for any service. Either you drink the AWS/GCP/Axure koolaid or you make your own. Whether it's k8s or Swarm or whatever doesn't matter.
k8s isn’t worth the time and money for many small teams, until they cross a complexity bar.
Even in some very non-startup enterprises, Cloud Foundry and Open Shift get adopted for a reason: some teams don’t need the overhead.
For startups there’s fly.io, render.com, and of course Heroku, but really — you can get from MVP to pretty decent scale on AWS or GCP with some scripts or Ansible.
Use k8s if you need it. It’s pretty well-proven. But it’s not something you need to have FOMO about.
The biggest reason Kubernetes should be a big no-no is because you should have a much simpler architecture (monolith) that doesn't need K8s.
What does it provide you?
Maybe you need a cluster per client and k8s is the only option.
Maybe you literally only need a few docker services and swarm/ecs/etc are fine forever.
What is the problem that K8s solves for you?
in my current case - the ability to deploy on-prem for some specific customers (on-prem meaning their own AWS or GCP account usually) + per-customer/multitenancy on the main product (ideally with segregated databases)
In general - scaling up a small number of microservices + their associated infra (redis/rabbitmq/etc)
You are basically stuck with k8s but you will end up having to "roll your own" (bring your own components) if you intend for operations to be consistent across different clouds/prems/etc.
Ideally start with an existing kube stack and slowly make it your own.
Operationalizing across hetereogenous clusters will be an unfortunate source of excitement.
Those same cloud providers usually have simplified container deployment mechanisms as well. You don't need Kubernetes to deploy containers.
And then you can no longer deploy anywhere else, sounds perfect for the cloud provider!