Best and 2nd best way to work with Kubernetes + Minikube + Podman + AWS

I’m aiming to create a Kuberentes local dev environment on Fedora Silverblue, per the course that I’m following. Afterwards, I’ll push this up to AWS. The reason I’m not doing this with EKS directly on AWS and just using a pre-built automated system (which would be nice) within the context of this post/project is irrelevant/not worth your time to understand. With that said, the course goes through installing Kuberenetes, then Minikube and using Podman; Fedora Silverblue comes with Podman, so I’m aiming to solicit feedback on the best way to actually develop this.
For the record, getting Kubernetes up and running on Silverblue is quite easy, as I see it actually comes with Fedora based on the list of services AND per these instructions:

Getting Minikube and Podman up and running is where I’ve been having trouble both in the Ubuntu VM and just on the regular Fedora Silverblue Desktop, which is interesting b/c Fedora already comes with Podman BUT, per these instructions, I run into problems:

This needs to be developed in a way that I can take everything from one cloud provider and push it to another cloud with ease. In other words, I can’t be entirely dependent on AWS (nor could I be on Linode or DigitalOcean) and I’d need access to this locally. So please bear this in mind and I’m very much hopeful someone can point me in the direction of how this can be done properly.

Hi there. I am using a Fedora Server (tried Silverblue initially but somethig didn’t work in my context so I switched to the Server version) VM (on oracle virtualbox) in my enterprise laptop to connect to Azure Kubernetes Services. And yes, I have downloaded kubectl by curl, as described in the first kubernetes doc you linked, and also, nope, I started with minikube and then I uninstalled it because I noticed the kubectl from it was not ok for Azure or something like that. Honestly when I said it to an expert colleague of mine, he was a bit perplexed about the fact the kubectl could be possibly different, but eventually i think anyone would agree that:

  • If you have a cloud provider (e.g. Azure or Google or Amazon), it would make more sense to just configure kubectl for the cloud provider instead of running a local minikube

  • Kubernetes natively provides you a layer of abstraction from the cloud provider, e.g. a db implemented through stateful containers, statefulsets etc… would be natively portable to another cloud provider, as opposite to a db managed service on a specific cloud provider.

I haven’t gotten minikube to work on Silverblue but I was able to get Rancher k3d to work with podman with podman-docker installed and the podman socket running in systemd.

I have also gotten TalosLinux to work, but unfortunately with Moby and not podman (I also strongly prefer podman over Docker).

Hi Giulio,

Well, the second bullet point is actually a great point to make and I guess I didn’t think of it that way - thanks for highlighting that there is a layer of abstraction and that it’s portable. So if I provision something with AWS EKS, I can effectively move that to Google Cloud GKE and it’d work easily? Not that I plan on doing that ha, it’s more of how I can qualify this as an asset that I own and don’t rely on one (1) service provider to provide; meaning, I can technically do this without the “cloud provider” and move it wherever - even if I’m using EKS. This needs to be an intangible asset I guess is what i’m getting at…

1 Like

Hey Scott - yeah it’s one of those situations where I’m following a course, so any minor change in how this stuff is provisioned can have a massive impact on any given part of the course (that I haven’t gotten to yet). That last sentence is riddled with terrible logic, but the point is: I don’t know what I don’t know ha so if I get “something” working, who knows what other trouble that’ll cause in other parts of the course. I suppose I can do this on the Ubuntu VM, as I have that fresh install now as well. Thanks for your insight!

Yes, correct, indeed one typically uses Helm charts to install Kubernetes native apps (e.g. db or nginx etc.) and those Helm charts work independently of the cloud service provider, since Kubernetes native apps are not coupled to the underlying infrastructure.

On the opposite, for the other cloud provider services (e.g a managed db service, which will be generally easier to maintain than the containerized counterpart) you can put an intermediate layer of terraform (or pulumi or your building blocks implementation) but that layer will have to change, of course, with another cloud provider (e.g terraform modules are specific to the cloud provider).

Btw we ended up having both types of layers.

1 Like

Okay gotcha, well thanks for breaking that down.
1.) Just go the easy route and install with Amazon EKS → reason I was hesitant is because I need this categorized as an “intangible asset” and if the cloud provider can provision this service without me then it is not* an intangible asset; if I can do it without the cloud provider then it is; in other words, I’ll never categorize anything as “cloud native” and will push against it where I can, otherwise the big cloud providers get the intangible asset claim, and I cannot have that happen… All that to say, I think I’ll technically be good as long as it’s not one cloud provider controlling the intangible asset, although quite frankly I wouldn’t use Google Cloud or Azure anyways.
2.) Helm - started by Deis Labs (https://deislabs.io/) and donated (?nvm…) to Microsoft…;
3.) Helm Charts - a.) Install existing applications with Helm in AKS - Azure Kubernetes Service | Microsoft Learn
b.) https://helm.sh/
4.) Then there’s Kubernetes and all of it’s cousins.
Yeah, I’ll likely not use Kubernetes in this instance. I’d rather pay more, have to develop more and be able to claim the workload as an intangible asset than go through a million hoops and run the risk of not* being able to claim this as an intangible asset. All that to say, I chose Silverblue again this time - over Rocky Linux (which i’ve never tried) - because it looked like it was better/easier to develop apps. CentOS was evidently perfectly synced with Kubernetes, but I see that has been taken out back and shot by Red Hat, so IDK. Silverblue is the default at that point I guess. I don’t enjoy it haha. But my goal is to automate enough to rarely be on this computer anyways, so I suppose I’ll just keep this for what it is and use gnome-boxes VrM’s to get production workloads up. I wish I could install VirtualBox or VMWare Player on Silverblue b/c Gnome Boxes has just been terrible. It crashes all of the time, freezes literally every time I use it etc. But evidently that’s not possible. I am excited to see what comes of Silverblue though, and there’s a knowledge gap on my end so hopefully it doesn’t get any worse before I have the opp to really start diving in. As far as I can tell, one can’t go from 0 to building a company with these systems. It does the opposite actually ha.