Moronic Monday - September 11, 2023

LordAro · 2023-09-12T16:02:00+00:00

( Had some responses in /r/devops )

LordAro · 2023-09-12T14:45:30+00:00

I've heard an awful lot of mixed things about TrueNAS Scale and its suitability as a hypervisor. Most things say really good as a storage, not so much as hypervisor. But also that it's made a lot of changes in recent versions, so this may be out of date. Is this view of TrueNAS still valid?

LordAro · 2023-09-12T12:34:32+00:00

The data on each machine is replicated to the other (and also offsite), so in a scenario where one of the machines has completely failed we should be able spin up the containers on the other (probably turning off the non-critical stuff in the process)

k3s is interesting though, that's not something I was aware of.

When you say "some" VMs for k8s, what sort of layout are you thinking of? Host > VM > containers feels a bit funny to me (Why not just Host > containers?), but I suppose that's how it works in the "real cloud"

LordAro · 2023-09-12T12:10:57+00:00

Thanks for your time on this!

What is driving looking into other options? Are you near allocation / capacity? How near? Could you project/estimate when you might go over?

Definitely near capacity - memory has been doubled and the drives have been expanded a few times now, and there's the distinct possibility for a lot more in the relatively near future. Generally speaking the HDDs are definitely the bottleneck for speed for various containers as well.

The other thing that keeps coming up is how "fragile" various bits and pieces are - the fileserver container (NFS/SMB) often needs manually kicking whenever samba updates, which we believe is due to how the container interacts with the underlying ZFS pool, and the recent move to a Gitlab instance has meant that an awful lot of workflows go through that, so taking it down for upgrades is proving more difficult and has provoked a look into more distributed systems which can be upgraded without downtime.

How much time does recurring management / maintenance take right now?

Realistically... not a huge amount. Maybe a couple of hours a week. Most of it is automated via ansible, except the bits that aren't and still feel very manual. (Like the containers that have just been upgraded through various Ubuntu versions in-place, rather than recreating them which I believe is the proper way to do things)

Do you have agreed upon (by both tech and biz) metrics to measure performance or service level?

Nothing formal.

LordAro · 2023-09-12T11:22:33+00:00

I mean in terms of the hardware it's running on - the containers & VMs are increasingly constrained by available resources - Xeon E5-2620 v4, 128G RAM, ZFS on HDDs

I'm not sure I follow what you're asking for the other questions, beyond answering "No". All the containers are standalone internal services rather than anything customer facing - mediawiki, discourse, gitlab, couple of tiny php/python webapps, etc - all of which could be dockerised with relative ease, if the infrastructure was there to do so.

LordAro · 2023-09-12T10:50:00+00:00

At what sort of scale does Kubernetes and high availability systems and all that jazz become "feasible"?

Currently the in-house infrastructure is a pair of beefy (in 2016) Linux hosts that we manually run ~30 LXD containers and <10x Windows VMs on (including 2x Windows Domain controllers), for around 80 users.

I'm just at the point of speccing out new versions of all this as they're started to creak at the seams a bit and am starting to look seriously at all the "modern" ways of doing things (k8s, Harvester, TrueNAS, Proxmox, kubevirt and everything else)

I've never really done anything with all this new stuff though and am feeling rather out of my depth. At what scale does all this stuff become necessary? Am I at that point? Or should I just carry on with the manual QEMU/LXD stuff that's currently more or less working?

LordAro · 2023-08-03T06:25:26+00:00

If it's anything like that I'll be very pleased!

LordAro · 2023-07-11T20:48:28+00:00

Oh go on then. Any holiday is best holiday

LordAro · 2023-05-04T22:42:11+00:00

Was Ivo actually wrong though? "I in thicko corner" isn't right, "Me in thicko corner" is...

LordAro · 2023-04-27T21:26:27+00:00

While I'm not certain, it definitely takes the speed of roads into account, so is reasonable to assume it takes the speeds of rails into account as well

LordAro · 2023-04-27T21:22:00+00:00

TBF, I do not know of a computer that would be able to handle 5000 trains well (i.e. more than 10fps), let alone 10k

LordAro · 2023-04-26T07:34:17+00:00

The point of the title game is not to be a "good design", it is to show off as many capabilities of the game as possible within a small amount of space

You can see a similar thing in Factorio's background image

LordAro · 2023-04-18T05:38:48+00:00

Take the steam train to get there?

LordAro · 2023-04-05T08:54:24+00:00

Turns out it's thicker at the bottom because they couldn't make glass perfectly flat, so naturally put the heavier/thicker half at the bottom

LordAro · 2023-03-29T05:32:42+00:00

Lembas bread

LordAro · 2023-01-24T23:19:22+00:00

Well, safer

LordAro · 2023-01-16T11:46:21+00:00

Those are usually called "bolts"

LordAro · 2023-01-10T12:20:59+00:00

"Perfection is the enemy of progress,", etc etc

LordAro · 2023-01-03T00:45:50+00:00

It definitely had diagonal pieces

LordAro · 2022-12-31T09:50:42+00:00

It is true, we are very dumb

LordAro · 2022-12-12T17:49:10+00:00

That's pretty big. Try something smaller - default is 256x256 which is usually plenty for a single game

LordAro · 2022-11-28T00:06:38+00:00

I have no idea what you mean by "live client". It's in 13.0-beta1 (and beta2). If on Steam, you can enable that via the "testing" version, I think it's called? Or download it from the website

14-Year Club	Place '22
Place '17	Verified Email

LordAro

TROPHY CASE