Run the master without kubelet

[toonsevrin]

Is it possible to run the master without the kubelet? I'd like to do k3s-in-k3s (eg. one cluster manages the other master), but I can't see to figure out how to disable the kubelet on the master nodes.

[brandond]

I haven't tested it myself, but there's a hidden option for that:

https://github.com/rancher/k3s/blob/master/pkg/cli/cmds/server.go#L302

[toonsevrin]

@brandond that does the job! Memory usage is still 375 Mb on a 1-node master (without any configuration) 😮

Any way to cut this down (ideally to <100MB RAM)?

[brandond]

If you can find a way to run the entire Kubernetes control plane plus datastore in less than 100mb, I think that would be newsworthy ;)

[toonsevrin]

Agreed, anyways, I've started a separate issue for this: #1944. Target is 150-200MB.

[brandond]

I have personally tried running it with as little as 256M, but it gets OOM killed pretty quick. at 384M it runs but doesn't perform well. As far as I can tell it should really have at least 512M.

[SOF3]

I haven't tested it myself, but there's a hidden option for that:

https://github.com/rancher/k3s/blob/master/pkg/cli/cmds/server.go#L302

link is obsolete, which line are you actually referring to?

Edit: it seems to be --disable-agent

k3s/pkg/cli/cmds/server.go

Lines 523 to 528 in fae8817

	cli.BoolFlag{
	Name: "disable-agent",
	Usage: "Do not run a local agent and register a local kubelet",
	Hidden: true,
	Destination: &ServerConfig.DisableAgent,
	},

But why is this hidden?

[brandond]

Because we don't technically support it. Many operators (system upgrade controller, etcd controller, and others) assume that the control-plane hosts will have nodes that can be used to track operations.

[ghost]

Much thanks @FabianKramm at Loft.sh for keeping the flag functional, ref: #4873 and #5118

It was definitely broken for a few versions, but has been fixed since this issue was originally created. When it was initially fixed, I don't think that the fix was propagated back to older releases. Not sure of the current state today as it shouldn't matter as older releases are obsoleted.

[brandond]

Well I was the one that fixed both of those, but yes thanks to everyone for reporting issues.

Both of those fixes were backported to the branches that were active at the time the issue was resolved.

[ghost]

I definitely misremembered that. Much thanks to you @brandond. Sorry about that.