A decade ago, I wrote about how we moved from Heroku to AWS at Plecto. Now, history has somewhat repeated itself at Fenerum, but this time with a different destination: Kubernetes.
We recently completed our migration from Heroku to Kubernetes, and I want to share our experiences, particularly focusing on how we solved the deployment challenges that come with Kubernetes.
Why Leave Heroku?
While Heroku remains an excellent platform for many use cases, we found ourselves constrained by two primary factors: costs and performance. Our growing infrastructure costs on Heroku were becoming significant, and we knew we could optimize this without compromising on quality or developer experience.
Enter Scaleway Kubernetes
After evaluating several providers, we chose Scaleway as our Kubernetes provider. They offer managed Kubernetes along with managed PostgreSQL and Redis services, which aligned perfectly with our stack. The transition has proven to be remarkably successful - we've seen both improved performance and significantly reduced costs.
The Deployment Challenge
While Kubernetes excels at container orchestration and scaling, it lacks a straightforward deployment mechanism like Heroku's git push. In my homelab setup, I previously used Keel.sh for deployments, but unfortunately, that project is no longer actively maintained.
At Fenerum, we needed a robust, scalable solution. After evaluating both ArgoCD and Spinnaker, we opted for Spinnaker - a choice influenced by my positive experience using it at Plecto, albeit for a different purpose.
Why Spinnaker?
Spinnaker, originally developed by Netflix and later open-sourced, was designed to handle continuous delivery at scale. Netflix created it to manage deployments across their massive cloud infrastructure, and it has since been adopted by many other tech companies.
What makes Spinnaker particularly powerful is its ability to handle complex deployment pipelines while maintaining a relatively simple user experience. It provides features like automated canary analysis, multi-region deployments, and rollback capabilities - all while integrating smoothly with Kubernetes.
Our Deployment Setup
We've configured Spinnaker to manage our Kubernetes manifests and handle our deployment pipelines. The setup provides us with:
- Git push to deploy functionality
- Deployment status directly visible in GitHub commits
- Automated pipeline triggers on specific branch updates
- Rollback capabilities when needed
The end result is a deployment process that's actually more robust than what we had with Heroku, while maintaining the simplicity that developers love.
Results After One Year
The migration has exceeded our expectations:
- Significant cost reduction (approximately 60% lower infrastructure costs)
- Improved application performance
- Better visibility into our infrastructure
- More flexibility in scaling different components independently
- Maintained developer productivity with automated deployments
Lessons Learned
The biggest lesson from this migration was that while Kubernetes itself can seem overwhelming, the right tooling makes all the difference. Spinnaker has proven to be that crucial piece that bridges the gap between Kubernetes' power and developers' need for simplicity.
If you're considering a similar move, I'd recommend focusing on solving the deployment workflow early in your migration planning. While Kubernetes offers great flexibility, without a solid deployment strategy, you risk losing the developer productivity that platforms like Heroku provide out of the box.
For those interested in learning more about Kubernetes in a smaller setting, check out my posts about setting up a homelab and my 10-month experience running Kubernetes at home.