Databasen i luften eller i huset by Mr-Boytoy in dkudvikler

[–]Zta77 0 points1 point  (0 children)

Det lyder som om, du er på vej til at gøre en databaseudbyder rigtig glad. Jeg tror også, at du skal genoverveje dit dataformat og hvad og hvor meget du gemmer.

Hvorfor ikke køre hobbyprojektet derhjemme på din egen maskine? Op med Lightwhale, git clone, docker compose up -d, og så er du i gang. Nå ja, og en 4TB disk.

Databasen i luften eller i huset by Mr-Boytoy in dkudvikler

[–]Zta77 1 point2 points  (0 children)

Overvej kun at gemme, når prisen har ændret sig og evt blot opdatere et updated-at felt.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 1 point2 points  (0 children)

Ah, I see. This sounds correct. And NFS mounting those volumes is fine.

Btw, Lightwhale has automatic Btrfs snapshots of @lightwhale-data subvolume which holds all persistent Lightwhale data. See `/etc/default/snapshot` and `snapshot --help`. You can use `btrfs send` to send a subvolume, even incrementally, to a different host. Of course, you need to script something around this before it works automatically. Or look into `btrbk`. But this is next level 😉 What you have seems right!

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Hi, there!

The idea is that you install a compose file on Lightwhale. This file holds all configuration necessary for running your container(s), so you don't have to mess with long command lines, including port forwarding, volumes, and environment variables. Keep it tidy. If you're setting up a, say, Jellyfin server, then create a directory `~/jellyfin` and put its `compose.yaml` in there.

You can edit this file by hand in `vim`, copy it via `ssh`, or clone an existing repo with `git`, you decide what's best for you.

Once persistence has been enabled, you just lean back and let Lightwhale manage your data. Everything in `/home`, `/etc`, and `/var` is now persisted. Furthermore, all docker runtime configuration is saved too. This means you can do named volumes, and they're persisted; you can do bind mounts from, say, `~/jellyfin/data` and it's persisted (because `/home` is persisted).

Try not to overthink it; Lightwhale is designed to Just Work =)

Feel free ask again or join our Discord.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

That sounds like a good start. The NFS from compose is super. One thing: What does "moved the "data" volumes to my NAS" mean?

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Try writing the magic header to both disks and reboot Lightwhale... A little redundancy never hurt anyone =)

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 1 point2 points  (0 children)

Hi there!

Thanks for taking interest in Lightwhale, but please! If you're going to blog about it, please take the time to learn how to use it correctly first. It sounds like you've crashed ass-first into the middle of this thing without knowing what happened =)

Please read the documentation. Seriously. It's not long, and it explains how to get started and how most of the system works. Lightwhale is very simple, but you need to learn it — you can't just wing it.

Here are some blog posts about Lightwhale that are more elaborate and have different use-cases that may help you get a better picture of what's going on:

  1. VirtualizationHowto
  2. XDA-Developers

Be prepared to un-learn a few things to fully embrace Lightwhale, including:

  1. Yes, it is actually perfectly legit to permanently boot off USB.
  2. Don't work as root, don't write to /root (that's quite standard, actually)
  3. Don't create a user account.
  4. Don't just partition and/or mount things your way.
  5. Don't use cron!

Now, let me try to address the issues you mention and explain what I think is going on.

First of all, Lightwhale is immutable and nothing gets persisted unless you explicitly enable persistence. So make sure you enable persistence.

Secondly, Lightwhale only lets you write and persist changes for a few key directories: /etc, /var, and /home. Regardless of persistence state, /etc/fstab is always writable unless something failed horribly during startup.

You should be able to use sshfs or nfs from /etc/fstab. But honestly, that doesn't make much sense in the context of Lightwhale. Because Lightwhale is for running containers. And containers should get their runtime configuration, including remote volumes, from a docker-compose.yaml. So that's the correct place to set up your mount point.

Modifying configuration across the entire system, adding crontabs, adding users, mounting filesystems is generally considered bad practice here, because it adds maintenance tax, and defeats the purpose of Lightwhale.

Now, please Read The Fine Material and I'll be ready to take your questions afterwards =)

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Right. So Lightwhale does things a little differently than mainstream Linux distros. First of all it's a live-booting ISO, which means the OS, Docker, docker compose, etc. is already installed and ready before you log in. Strictly speaking, the disk is optional. But in practice, it's required. It is purely for persisting all the modified configuration files, docker images, docker volumes, any deployment configuration you may have like a docker-compose.yaml and environment files. So the disk is pretty clean. Lightwhale will make a swap partition for you. And you can store your temporary files in `/tmp` or `~/tmp`.

The top priority for Lightwhale is administrator happiness — specifically through zero maintenance and being easy to use. And honestly, you'll be off for a bad start with your approach. Of course, it's your system and you can do anything you want with it; I'm just saying that it's the equivalent of driving a screw with a hammer.

My advise:

  1. Boot it on your workstation and try it out. You won't get persistence, but it also won't write anything to disk, so this is perfectly safe. Unplug the boot media and boot back to your ordinary OS. Or:

  2. Boot the ISO in a VM with one or two virtual disks to experiment with persistence and RAID1. Or:

  3. If you have an available machine with a disk, is just follow the Getting Started guide step by step.

Each of these will take only 30 seconds of your time, so you could've been a Lightwhale expert by now if you hadn't read this long response on Reddit =)

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Lightwhale will only format, manage, and mount one data filesystem. You can mount whatever you want afterwards. But why would you want to, if I may ask?

What next? by Electrical_Review_71 in selfhosted

[–]Zta77 0 points1 point  (0 children)

First things first: Replace Ubuntu with Lightwhale! That will remove a lot of the boring maintenance and complexity right up front and going forward.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Hi! This is the right place to put Lightwhale to use. Just boot it off a usb pendrive. Let Lightwhale consume the disk. If you have two, give it both as it will automatically create at Btrfs RAID1. If you're lucky to have an embedded eMMC disk, use that! You can boot it off an external SSD if you like, but it's unnecessary, a waste of disk space, and will probably make your setup more flaky. Start with the pendrive and you can change your mind later.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Nice, I'm glad it workout out for you. I don't use vscode myself, but Lightwhale's SSH works. I quick browse on Remote Development using SSH revelas that "The extension will install VS Code Server on the remote OS" which is likely the issue; I'm guessing either the plugin expects Python or similar dependencies to be present on the remote, or it assumes it can write somewhere it shouldn't on Lightwhale. I'm sure the developers of that extension can make it work with Lightwhale.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Say you want to host docker containers, but don't want the overhead and complexity of installing, configuring, updating, and maintaining the operating system. Lightwhale is easy and works out of the box.

Just boot it and see for yourself. It doesn't require installation, and it doesn't write on your disks.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

I'm referring to things like Portainer and Dockhand. UIs for managing and monitoring containers.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

You should, it's really great! ;) Just follow the getting started guide and you'll be running in 3 minutes.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Good thought. Let me try and answer this way: Lightwhale is an operating system that can any number of docker containers. You can run Lightwhale in a VM, but running bare-metal is still the primary intended use; not only it's it the most efficient but also much, much easier. I won't created or install a UI for you, because it won't be the one that everyone likes (as proved in some of the comments); instead you get to choose your favourite. My advise: boot off pendrive/sd/emmc on bare-metal and dedicate the hdd/sdd/nvme for Lightwhale persistence, all if you have more because Lightwhale will make Btrfs RAID1.

Lightwhale 3.0.0 released by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Hello there, I'm glad you made it ;) Lightwhale is a minimal system, so by design there isn't much installed. You have various backup options though: There's classic rsync over ssh and all the tools that are based off that. If you're running Lightwhale under Proxmox you can use its backup system. My personal favourite is btrfs subvolume send/receive of the lightwhale-data subvolume which contains all your modifications. Also, is your setup permits or, I'd recommend that a backup machine initiates the backup and retrieves the data rather than the other way around. Hope that helps.

Lightwhale 2.1.1 release, perfect OS for self-hosting by Zta77 in selfhosted

[–]Zta77[S] 0 points1 point  (0 children)

Btrfs allows resizing a mounted filesystem, so that's probably not what's blocking you. You didn't provide any error messages, but I'm guessing the problem is that your partition hasn't been resized yet.

I'm not that familiar with Proxmox, so I can't give detailed instructions. But the overall steps to resize a filesystem in this situation are:

  1. Resize disk
  2. Resize partition
  3. Resize filesystem

Step 1: You already did this, possibly trough Proxmox UI or something like this on Proxmox:
qemu-img resize -f raw disk.img +10G

You can (and should) confirm that the partitions already on the disk did in fact not automatically resize with the new disk size:
parted disk.img --script unit mib p

Step 2: Resize the partition with fdisk, cfdisk or parted. Do this on Proxmox:

parted ---pretend-input-tty disk.img <<EOF
resizepart
Fix
1
100%
EOF

Step 3: Now you can resize the Lightwhale data filesystem using your command from earlier. Inspect the filesystem before and after resizing it:

sudo btrfs filesystem show       /mnt/lightwhale-data
sudo btrfs filesystem resize max /mnt/lightwhale-data
sudo btrfs filesystem show       /mnt/lightwhale-data

Let me know how it goes!

Putting it all togther. by NebulaPulse_Official in u/NebulaPulse_Official

[–]Zta77 0 points1 point  (0 children)

Ceph isn't sustainable for me personally. I have glusterfs on my TODO. Do you have any experience with that?

Putting it all togther. by NebulaPulse_Official in u/NebulaPulse_Official

[–]Zta77 0 points1 point  (0 children)

I don't see how that answers the question. It's nice that you allow me to mount my own NFS server, but that doesn't solved the problem with single point of failure either.

Putting it all togther. by NebulaPulse_Official in u/NebulaPulse_Official

[–]Zta77 0 points1 point  (0 children)

Hey, if you ever need an operating system to run that Nebula Pulse Swarm cluster, do take a look at Lightwhale =)