Mailing List Archive: (Slightly OT?) K8S Platform As A Service Recommendations

(Slightly OT?) K8S Platform As A Service Recommendations

Apr 7, 2021, 8:16 AM

Post #1 of 4 (116 views)

Hello all,

I know this is primarily a networking list, but I know lots of server
admins hang out here.

Does anyone have a recommendation for a self-hosted, on premise,
platform as a service layer for k8s (specifically k3s)?

I have written up some context here:

https://github.com/TSYSGroup/docs-techops/blob/master/Applications/AppRuntimeLayerTodo.md

tl:dr : I have about 70 to 200 apps / (micro) services that will need to
run across a handful of k3s servers . I already have HA
database/networking/certificate/application load
balanacer/authentication stacks in production use, I am currently
running the actual websites/applications on a single Ubuntu LAMP server
and want to build out an HA runtime layer for all the
properties/applications and need a way to orchestrate k3s/metallb

Rancher rio has come up a few times in my
research:https://bram.dingelstad.works/blog/finding-the-right-paas-for-k8s/
<https://bram.dingelstad.works/blog/finding-the-right-paas-for-k8s/>

In addition to the web apps , I will also will be running a number of
r&d applications and CUDA enabled containers (across a mix of physical
x86/jetson/tegra machines with k3s workers).

Suggestions/comments/questions/flames welcome :)

On or off list as you prefer.

Re: (Slightly OT?) K8S Platform As A Service Recommendations [ In reply to ]

baldwinmathew at gmail

Apr 7, 2021, 4:04 PM

Post #2 of 4 (116 views)

Permalink

You could look at the combo of Tinkerbell and CAPI (ClusterAPI). Happy to
chat more off list.

-matt

On Wed, Apr 7, 2021, 10:42 AM Charles N Wyble <charles@turnsys.com> wrote:

> Hello all,
>
>
> I know this is primarily a networking list, but I know lots of server
> admins hang out here.
>
> Does anyone have a recommendation for a self-hosted, on premise, platform
> as a service layer for k8s (specifically k3s)?
>
> I have written up some context here:
>
>
> https://github.com/TSYSGroup/docs-techops/blob/master/Applications/AppRuntimeLayerTodo.md
>
> tl:dr : I have about 70 to 200 apps / (micro) services that will need to
> run across a handful of k3s servers . I already have HA
> database/networking/certificate/application load balanacer/authentication
> stacks in production use, I am currently running the actual
> websites/applications on a single Ubuntu LAMP server and want to build out
> an HA runtime layer for all the properties/applications and need a way to
> orchestrate k3s/metallb
>
> Rancher rio has come up a few times in my research:
> https://bram.dingelstad.works/blog/finding-the-right-paas-for-k8s/
> In addition to the web apps , I will also will be running a number of r&d
> applications and CUDA enabled containers (across a mix of physical
> x86/jetson/tegra machines with k3s workers).
>
> Suggestions/comments/questions/flames welcome :)
>
> On or off list as you prefer.
>

Re: (Slightly OT?) K8S Platform As A Service Recommendations [ In reply to ]

ray at oneunified

Apr 7, 2021, 9:38 PM

Post #3 of 4 (116 views)

Permalink

On 4/7/21 9:16 AM, Charles N Wyble wrote:> Does anyone have a
recommendation for a self-hosted, on premise,
> platform as a service layer for k8s (specifically k3s)?
FWIW:

Maybe you don't need kubernetes:
https://endler.dev/2019/maybe-you-dont-need-kubernetes/

Manually install a single node Kubernetes cluster on Debian
http://meta.libera.cc/2021/03/manually-install-single-node-kubernetes.html

Or run Salt or something and spin up LXC containers.

Re: (Slightly OT?) K8S Platform As A Service Recommendations [ In reply to ]

charles at turnsys

Apr 8, 2021, 6:31 AM

Post #4 of 4 (116 views)

Permalink

On 4/7/21 11:38 PM, Raymond Burkholder wrote:
> On 4/7/21 9:16 AM, Charles N Wyble wrote:> Does anyone have a
> recommendation for a self-hosted, on premise,
> > platform as a service layer for k8s (specifically k3s)?
> FWIW:
>
> Maybe you don't need kubernetes:
> https://endler.dev/2019/maybe-you-dont-need-kubernetes/

I have considered not running k8s. I didn't run it for a long time. I
kept an eye on developments and waited for it to mature.

However the amount of applications and services I am now needing to
support and the HA requirements and need for standardization etc.... I
don't know of a better option.

>
> Manually install a single node Kubernetes cluster on Debian
> http://meta.libera.cc/2021/03/manually-install-single-node-kubernetes.html
>
>
> Or run Salt or something and spin up LXC containers.
>
>

Sure.... and how do I manage IP addresses? Ports? HA? Containers
(LXC/docker) is the easy part (on a relative basis anyway!) . It's the
meta stuff around it that gets messy. The orchestration piece of the
containers is the difficult part.

As I mentioned, we already have a mature stack outside the app runtime
layer (for certs/LDAP/database etc). We just want applications/services
on k8s. Minimize the complexity/blast radius! :)