Stupid question, but what how does this work on deck hardware, I thought framegen was only supported on newer GPUs?

I’m honestly shocked it took this long tbh. It’s obviously a liability, the only question is what site is going to be sued into oblivion first, and a lawsuit would name everyone in the chain, from the site hosting it, to the cloud provider with the data on it, to the card processors. Whoever loses will be “made an example of” too, more than likely shuttering the doors of the site.

Glad to be of help. It is the right decision, I have no regrets if migrating, but it is a long process. Just getting my first few services running was months, just so you are aware of that commitment, but it’s worth it.

Yeah, unfortunately most people don’t understand what that is, and buy devices thinking they’ll be supported long term. Us here know better (I bought Wemo early on and stopped years and years ago now because I knew this was coming, but I know I’m not the majority).

I’ll post more later (reply here to remind me), but I have your exact setup. It’s a great way to learn k8s and yes, it’s going to be an uphill battle for learning - but the payoff is worth it. Both for your professional career and your homelab. It’s the big leagues.

For your questions, no to all of them. Once you learn some of it the rest kinda falls together.

I’m going into a meeting, but I’ll post here with how I do it later. In the mean time, pick one and only one container you want to get started with. Stateless is easier to start with compared to something that needs volumes. Piece by piece brick by brick you will add more to your knowledge and understanding. Don’t try to take it all on day one. First just get a container running. Then access via a port and http. Then proxy. Then certs. Piece by piece, brick by brick. Take small victories, if you try to say “tomorrow everything will be on k8s” you’re setting yourself up for anger and frustration.

@sunoc@sh.itjust.works Edit: To help out I would do these things in these steps, note that steps are not equal in length, and they are not complete - but rather to help you get started without burning out on your journey. I recommend just taking each one, and when you get it working rather than jumping to the next one, instead taking a break, having a drink, and celebrating that you got it up and running.

0. Start documenting everything you do. The great thing about kubernetes is that you can restart from scratch if you have written everything down. I would start a new git repository with a README that contains every command you ran, what it did, and why you did it. Assume that you will be tearing down your cluster and rebuilding it - in fact I would even recommend that. Treat this first cluster as your testing grounds, and then you won’t feel crappy spinning up temporary resources. Then, you can rebuild it and know that you did a great job - and you’ll feel confident in rebuilding in case of hardware failure.

1. Get the sample nginx pod up and running with a service and deployment. Simply so you can curl the IP of your main node and port, and see the response. This I assume you have played with already.

2. Point DNS to your main node, get the nginx pod with http://your.dns.tld:PORT. This should be the same as anything you’ve done with docker before.

3. Convert the yaml to a helm chart as other have said, but don’t worry about “templating” yet, get comfortable with helm install, helm upgrade -i, and helm uninstall. Understand what each one does and how they operate. Then go back and template, upgrade-ing after each change to understand how it works. It’s pretty standard to template the image and tag for example so it’s easy to upgrade them. There’s a million examples online, but don’t go overboard, just do the basics. My (template values.yaml) usually looks like:

<<servicename>>
  name: <<servicename>>
  image:
    repository: path/to/image
    tag: v1.1.1
    network:
     port: 8888

Just keep it simple for now.

4. Decide on your proxy service. Traefik as you see comes out of the box. I personally use istio. I can go into more details why later, but I like that I can create a “VirtualService” for "$appname.my.custom.tld` and it will point to it.
5. Implement your proxy service, and get the (http only still) app set up. Set up something like nginx.your.tld and be able to curl http://nginx.your.tld and see that it routes properly to your sample nginx service. Congrats, this is a huge one.
6. Add the CertManager chart. This will set it up so you can create Certificate types in k8s. You’ll need to use the proxy in the previous step to route the /.well-known endpoints on the http port from the open web to cert-manager, for Istio this was another virtual service on the gateway - I assume Traefic would have something similar to “route all traffic on port 80 that starts with /.well-known to this service”. Then, in your nginx helm chart, add in a Certificate type for your nginx endpoint, nginx.your.tld, and wait for it to be successfully granted. With Istio, this is all I need now to finally curl https://nginx.your.tld!

At this point you have routing, ports, and https set up. Have 2 drinks after this one. You can officially deploy any stateless service at this point.

Now, the big one, stateful. Longhorn is a bear, there are a thousand caveats to it.

Step one is where are your backups going to go. This can be a simple NFS/SMB share on a local server, it can be an s3 endpoint, but seriously this is step 1. Backups are critical with longhorn. You will fuck up Longhorn - multiple times. Losing these backups means losing all configs to all of your pods, so step one is to decide on your stable backup location.

Now, read the Longhorn install guide: https://longhorn.io/docs/1.9.0/deploy/install/. Do not skip reading the install guide. There are incredibly important things in there that I regretted glossing over that would have saved me. (Like setting up backups first).

The way I use longhorn is to create a PV in longhorn, and then the PVC (you can look up what both of these are later). Then I use Helm to set what the PVC name is to attach it to my pod. Try and do this with another sample pod. You are still not ready to move production things over yet, so just attach it to nginx. exec into it, write some data into the pvc. Helm uninstall. See what happens in longhorn. Helm install. Does your PVC reattach? Exec in, is your data still there? Learn how it works. I fully expect you to ping me with questions at this point, don’t worry, I’ll be here.

Longhorn will take time in learning, give yourself grace. Also after you feel comfortable with it, you’ll need to start moving data from your old docker setup to Longhorn, and that too will be a process. You’ll get there though. Just start with some of your lower priority projects, and migrate them one by one.

After all of this, there is still more. You can automount smb/nfs shares directly into pods for media or anything. You can pass in GPUs - or I even pass in some USB devices. You can encrypt your longhorn things, you can manage secrets with your favorite secret manager. There’s thousands of things you’ll be able to do. I wish you luck, and feel free to ping me here or on Matrix (@scrubbles@halflings.chat) if you ever need an ear. Good luck!

Yeah with Amazon’s sheer size this has definitely been done before, curious what limits op is going to hit. My guess is they have a quota for submissions, and they’ll be banned from submitting tickets.

I mean go for it? They literally can’t do anything, you might as well complain that fire is hot though. It’s part of being in the Internet. They provide safety gloves, via VPCs and firewalls, but if you choose not to use them then… yeah I mean youre probably gonna get burned

Uh sorry dude, but no this isn’t a script kiddy, these are bots that scan every IP address every day for any open ports, it’s a constant thing. If you have a public IP, you have people, govs, nefarious groups scanning it. AWS will tell you the same as if you were hosting it locally, close up the ports, put it on a private network. Use a vpc and WAF in AWS’ case.

I get scanned constantly. Every hour of every day dark forced attempt to penetrate our defences.

Unprecedented is not a bad word. It means there’s no precedent. I mean we’re finally pushing back.

It seems to run on some form of electricity!

The only reason this would need to be a bill is if people are upset that they are failing the exam. Which means they qre failing the exams, to the surprise of no one.

What we should be doing instead is making our neighborhoods more accessible to those without cars. I’m sure they feel like their mobility is gone if they lose their license, but that shouldn’t be the case to begin with.

So tone deaf, and clearly they’re just trying to steer the narrative.

They call out that it’s never taken lightly and it has to happen. We know. Stop killing games just says you have to do something when you turn off the servers. Either release the server source code so it can be engineered by the community, release a self hostage server alternative, even just documents or guides on how to get started.

But they’re going to try to make it about the mean old gamers want them to go broke

I really wanted it to work, for me it made the most sense I thought, as little virtualization as I could do. VM felt like such a heavy layer in between - but it just wasn’t meant to work that way. You have to essentially run your LXC as root, meaning that it’s essentially just the host anyway so it can run docker. Then when you get down to it, you’ve lost all the benefits of the LXC vs just running docker. Not to mention that anytime there was even am minor update to proxmox something usually broke.

I’m surprised Proxmox hasn’t added straight-up support for containers, either by docker, podman, or even just containerd directly. But, we aren’t it’s target audience either.

I’m glad you can take my years of struggling to find a way to get it to work well and learn from it.

Personally I agree. I’ve seen way more startups kicking off with these waves of layoffs. It’s a silver lining, not much more, but I’m happy to see people finally realizing they don’t want the big tech solutions anymore.

Not at all. Proxmox does a great job at hosting VMs and giving a control plane for them - but it does not do containers well. LXCs are a thing, and it hosts those - but never try to do docker in an LXC. (I tried so many different ways and guides and there were just too many caveats, and you end up always essentially giving root access to your containers, so it’s not great anyway). I’d like to see proxmox offer some sort of docker-first approach will it will manage volumes at the proxmox level, but they don’t seem concerned with that, and honestly if you’re doing that then you’re nearing kubernetes anyway.

Which is what I ended up doing - k3s on proxmox VMs. Proxmox handles the instances themselves, spins up a VM on each host to run k3s, and then I run k3s from within there. Same paradigm as the major cloud providers. GKE, AKS, and EKS all run k8s within a VM on their existing compute stack, so this fits right in.

Just focus on one project at a time, break it out into small victories that you can celebrate. A project like this is going to be more than a single weekend. Just get proxmox up and running. Then a simple VM. Then a backup job. Don’t try to get everything including tailscale working all at once. The learning curve is a bit more than you’re probably used to, but if you take it slow and focus on those small steps you’ll be fine.

I think at this point I agree with the other commenter. If you’re strapped for storage it’s time to leave Synology behind, but it sounds more like it’s time to separate your app server from your storage server.

I use proxmox, and it was my primary when I got started with the same thing. I recommend build out storage in proxmox directly, that will be for VM images and container volumes. Then utilize regular backups to your Synology box. That way you have hot storage for drives and running things, cold storage for backups.

Then, inside your vms and containers you can mount things like media and other items from your Synology.

For you, I would recommend proxmox, then on top of that a big VM for running docker containers. In that VM you have all of your mounts from Synology into that VM, like Jellyfin stuff, and you pass those mounts into docker.

If you ever find yourself needing to stretch beyond the one box, then you can think about kubernetes or something, but I think that would be a good jump for now.

Oh they may call themselves Christians, but none of them uphold the ideals or beliefs of Christians. They are hypocrites. The same day they do this they take away healthcare from millions, take food out of children’s hands, and ramp up more violence to those already hurting.

I don’t care what their label is they put on the outside, none of these are Christian. They openly spit on the morals that Jesus laid down.

And when you pray, do not be like the hypocrites, for they love to pray standing in the synagogues and on the street corners to be seen by others. Truly I tell you, they have received their reward in full.

Yeah writer doesn’t mention money anywhere there, that’s the real trap. All of those sound nice but I have rent to pay.